CN107209862B - Identification device and information storage medium - Google Patents

Abstract

Provided is a recognition device capable of recognizing a numerical expression including a character string. The recognition device includes: a registration unit that determines whether or not an input stroke or a stroke string matches a specific gesture character for distinguishing between a character string and a numerical expression, and registers information on a range specified by the specific gesture character when the input stroke or the stroke string matches the specific gesture character; a distinguishing section that sets, as a stroke corresponding to the character string, a stroke included in a range specified by the determination gesture character among the inputted strokes, and sets, as a stroke corresponding to the numerical expression, a stroke not included in the range specified by the determination gesture character; and a recognition unit that recognizes the character string from a stroke set as a stroke corresponding to the character string by using a character string recognition engine, and recognizes the numerical expression from a stroke set as a stroke corresponding to the numerical expression by using a numerical expression recognition engine.

Description

Identification device and information storage medium

Technical Field

The invention relates to a program, an information storage medium, and a recognition device.

Background

Conventionally, as a method for inputting numerical expressions, a method using a specialized language such as LaTeX, a method using a numerical input editor, or the like has been used, but the former requires mastering specialized words, and the latter requires repetition of menu selection and symbol selection, which are inconvenient for the general public to use. In contrast, a method of recognizing a numerical expression handwritten on a tablet or the like by a computer has been studied for a long time. Further, the recognition rate has been improved by many years of research, and further, the possibility of penetration into the general public has been increased due to rapid spread of tablet PCs in recent years.

Documents of the prior art

Patent document

Patent document 1: japanese laid-open patent publication No. 2009-80615

Disclosure of Invention

Problems to be solved by the invention

The numerical expression is composed of a relation between a symbol (symbol) and positions and sizes of four operators, brackets, score symbols, root symbols, integral symbols, symbols representing power, and the like. In the handwritten numerical recognition system, a symbol is generally assumed to be an alphanumeric symbol, but actually, a character string other than the alphanumeric symbol is often written as a symbol, and in the conventional handwritten numerical recognition system, there is a problem that a character string included in a numerical expression cannot be recognized. Such numeric expressions containing strings are frequently found in textbooks and are also frequently written on a blackboard.

The present invention has been made in view of the above problems, and an object thereof is to provide a program, an information storage medium, and a recognition device capable of recognizing a numerical expression including a character string.

Means for solving the problems

(1) The present invention relates to a program for recognizing a numerical expression including a character string from a stroke string input by handwriting, the program causing a computer to function as a registration unit that determines whether or not an input stroke or a stroke string matches a specific gesture for distinguishing a character string from a numerical expression, and when it is determined that the input stroke or the stroke string matches the specific gesture, registers information relating to a range specified by the specific gesture, the discrimination unit setting, as a stroke corresponding to one of the character string and the numerical expression, a stroke included in the range specified by the specific gesture among the input strokes, and setting, as a stroke corresponding to the other of the character string and the numerical expression, a stroke not included in the range specified by the specific gesture, and a recognition unit that recognizes a character string from a stroke set as the one corresponding to the character string by a character string recognition engine, a numerical expression is recognized by a numerical recognition engine from strokes set as strokes corresponding to the numerical expression. The present invention also relates to a computer-readable information storage medium storing a program for causing a computer to function as each of the above-described units. The present invention also relates to an identification device including the above-described components.

According to the present invention, it is determined whether or not an input stroke matches a specific gesture character, a stroke included in a range specified by the specific gesture character is set as a stroke corresponding to one of a character string and a numerical expression, a stroke not included in the range specified by the specific gesture character is set as a stroke corresponding to the other of the character string and the numerical expression, the character string is recognized by a character string recognition engine from the stroke set as the stroke corresponding to the character string, and the numerical expression is recognized by a numerical recognition engine from the stroke set as the stroke corresponding to the numerical expression. Further, a method of automatically separating (distinguishing) an expression from a character string without using a specific gesture character may be considered. However, it is difficult to automatically distinguish between a numeric expression and a character string, and erroneous division cannot be avoided, and therefore, in the present invention, a determination gesture for distinguishing between a character string and a numeric expression is introduced.

(2) In the program, the information storage medium, and the identification device according to the present invention, the following may be used: the distinguishing unit sets, when the input stroke or the stroke string is determined to match the specific gesture character, a stroke included in a range specified by the specific gesture character as a stroke corresponding to one of a character string and a numerical expression, determines, when the input stroke is determined not to match the specific gesture character, whether the input stroke is included in the range specified by the specific gesture character, and sets, when the input stroke is included in the range specified by the specific gesture character, the input stroke as a stroke corresponding to one of the character string and the numerical expression.

According to the present invention, even when a specific gesture is written after a character string (or numerical expression) is written, or when a character string (or numerical expression) is written after a specific gesture is written, the character string and the numerical expression can be distinguished from each other, and convenience for the user can be improved.

(3) In the program, the information storage medium, and the identification device according to the present invention, the following may be used: the distinguishing unit sets, as a stroke corresponding to a character string, a stroke included in a range specified by the specific gesture character, among inputted strokes, and sets, as a stroke corresponding to a numerical expression, a stroke not included in a range specified by the specific gesture character, and the recognizing unit recognizes the numerical expression using a numerical recognition engine, with strokes included in a range specified by 1 specific gesture character being regarded as 1 mark.

According to the present invention, a stroke included in a range specified by 1 fixed gesture character (a stroke set as a stroke corresponding to a character string) is regarded as 1 mark and a numerical expression is recognized by a numerical recognition engine, whereby a numerical expression including a character string can be recognized by using a numerical recognition engine that cannot recognize a character string.

(4) In addition, the program and the information storage medium according to the present invention may be: the computer is caused to function as a display control unit that controls a display unit to display an input stroke string, and the display control unit is caused to perform control to display a stroke or a stroke string determined to match the specific gesture character as an image representing a shape corresponding to the specific gesture character.

In addition, the identification device according to the present invention may be: the display control unit performs control to display the stroke or the stroke string determined to match the specific gesture character as an image representing a shape corresponding to the specific gesture character.

According to the present invention, by displaying the stroke determined to match the fixed gesture character as the image indicating the shape corresponding to the fixed gesture character, the user can be made aware that the fixed gesture character has been correctly determined, and the convenience of the user can be improved.

(5) In the program, the information storage medium, and the identification device according to the present invention, the following may be used: the display control unit performs control to display strokes included in the range specified by the specification gesture character in a color different from strokes not included in the range specified by the specification gesture character.

According to the present invention, by displaying the strokes included in the range specified by the fixed gesture character in a different color from the strokes not included in the range specified by the fixed gesture character, the user can be made aware that the character string and the numerical expression have been correctly distinguished by the fixed gesture character, and the convenience of the user can be improved.

(6) In the program, the information storage medium, and the identification device according to the present invention, the following may be used: the login unit determines that the input stroke or the stroke string matches the specific gesture character when the input stroke or the stroke string is substantially rectangular.

Drawings

Fig. 1 is an example of a functional block diagram of the identification device according to the present embodiment.

Fig. 2A is a diagram showing an example of writing in which a numerical expression, a character string, and a fixed gesture are mixed.

Fig. 2B is a diagram showing an example of the determination gesture character table.

Fig. 3A is a diagram for explaining the order in which the determination gesture characters are written.

Fig. 3B is a diagram for explaining the order in which the gesture characters are determined by writing.

Fig. 3C is a diagram for explaining the order in which the determination gesture characters are written.

Fig. 3D is a diagram for explaining the order in which the gesture characters are determined by writing.

Fig. 4A is a diagram showing an example of writing of a rectangular fixed gesture character.

Fig. 4B is a diagram showing an example of writing of a rectangular fixed gesture character.

Fig. 5 is a flowchart showing an example of processing in the sequential method.

Fig. 6A is a diagram showing a specific display example in the successive mode.

Fig. 6B is a diagram showing a specific display example in the successive mode.

Fig. 6C is a diagram showing a specific display example in the sequential manner.

Fig. 6D is a diagram showing a specific display example in the sequential manner.

Fig. 6E is a diagram showing a specific display example in the successive mode.

Fig. 6F is a diagram showing a specific display example in the sequential mode.

Fig. 7 is a flowchart showing an example of the processing in the batch system.

Fig. 8A is a diagram showing a specific display example in the batch mode.

Fig. 8B is a diagram showing a specific display example in the batch mode.

Fig. 8C is a diagram showing a specific display example in the batch mode.

Detailed Description

The present embodiment will be described below. The embodiments described below are not intended to unduly limit the scope of the present invention set forth in the claims. The configurations described in the present embodiment are not necessarily all essential components of the present invention.

1. Form a

Fig. 1 shows an example of a functional block diagram of the identification device according to the present embodiment. Note that the recognition device of the present embodiment may be configured such that a part of the components (each part) in fig. 1 is omitted.

Thecharacter input unit 160 is used for allowing a user to input handwritten characters through a writing medium (pen, fingertip, or the like), and its function can be realized by a writing surface such as a tablet or a touch panel. Thecharacter input unit 160 detects coordinate data indicating a position of the writing medium from the time the writing medium contacts the writing surface to the time the writing medium is separated from the writing surface at regular time intervals, and outputs a detected coordinate data string (coordinate point sequence) to theprocessing unit 100 as data of strokes (strokes). Further, a vector from the end point of a stroke to the start point of the next stroke is referred to as an off-stroke (pen-motion vector), and a sequence in which the stroke and the pen-motion are continuous is referred to as a stroke string.

Thestorage unit 170 stores programs and various data for causing a computer to function as each unit of theprocessing unit 100, and functions as a work area of theprocessing unit 100, and the functions thereof can be realized by a hard disk, a RAM, or the like.

Thedisplay unit 190 is used to output the image generated by theprocessing unit 100, and its function can be realized by a display such as a touch panel, an LCD, or a CRT, which also functions as thecharacter input unit 160.

The processing unit 100 (processor) performs processing such as recognition processing and display control based on the coordinate data, program, and the like from thecharacter input unit 160. Theprocessing unit 100 performs various processes with the main storage unit in thestorage unit 170 as a work area. The functions of theprocessing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) and ASIC (gate array, etc.) or programs. Theprocessing unit 100 includes aregistration unit 110, adiscrimination unit 112, anidentification unit 114, and adisplay control unit 120.

Theregistration unit 110 determines whether or not the input stroke or stroke string matches a specific gesture character for distinguishing between a character string and a numerical expression, and if it is determined that the input stroke or stroke string matches the specific gesture character, performs a process of registering information on a range (region) specified by the specific gesture character. The registered information is stored in thestorage unit 170. Here, thelogin section 110 may determine that the input stroke or the stroke string matches the above-described specific gesture character when the stroke or the stroke string is formed substantially rectangular.

The differentiatingunit 112 sets, as a stroke corresponding to one of the character string and the numerical expression, a stroke included in the range specified by the specific gesture character among the inputted strokes, and sets, as a stroke corresponding to the other of the character string and the numerical expression, a stroke not included in the range specified by the specific gesture character. That is, the differentiatingunit 112 may set, as the stroke corresponding to the character string, the stroke included in the range specified by the above-described determination gesture character among the inputted strokes, and may set, as the stroke corresponding to the numerical expression, the stroke not included in the range specified by the above-described determination gesture character, or may reverse the setting.

Therecognition unit 114 performs the following processing: the character string is recognized by the character string recognition engine from the strokes set by the differentiatingsection 112 as the strokes corresponding to the character string, and the numerical expression is recognized by the numerical expression recognition engine from the strokes set by the differentiatingsection 112 as the strokes corresponding to the numerical expression. Here, when the distinguishingunit 112 sets the strokes included in the range specified by the fixed gesture character as the strokes corresponding to the character string, therecognition unit 114 may recognize the numerical expression using the numerical expression recognition engine, with 1 of the fixed gesture characters and the strokes included in the range specified by the fixed gesture character being regarded as 1 mark. For example, in the example shown in fig. 2A, the character strings "area", "height", and "width" may be recognized by the mathematical expression recognition engine as 1 mark such as "x", "y", and "z", respectively.

Thedisplay control unit 120 performs control to display the input stroke string on thedisplay unit 190. Here, thedisplay control unit 120 may perform control to display the stroke or the stroke string determined to match the specific gesture character as an image (for example, an image representing a rectangle) representing a shape corresponding to the specific gesture character. Thedisplay control unit 120 may perform control to display strokes included in the range specified by the specification gesture in a color different from strokes not included in the range specified by the specification gesture. Thedisplay control unit 120 performs control to display the numerical expression (recognition result) including the character string recognized by therecognition unit 114 on thedisplay unit 190.

2. Method of the present embodiment

Next, the method of the present embodiment will be described with reference to the drawings.

In the case of causing a computer to recognize a numerical expression including a character string to be handwritten, it is desirable that the numerical expression and the character string be automatically separated (distinguished) from each other and recognized from each other from an input stroke string (handwritten pattern). However, it is inherently difficult to automatically distinguish between the numerical expression and the character string, and if the numerical expression and the character string are to be distinguished separately without complete distinction, the recognition rate may be significantly reduced.

Therefore, in the method of the present embodiment, a gesture character for distinguishing a character string from a numerical expression is introduced. This gesture character is referred to as a determination gesture character. Here, the determination of the gesture symbol means the following operation: when a character to be recognized or a symbol other than a numerical expression is written, a stroke included in a range specified by the gesture character is specified, and processing corresponding to the stroke is performed.

In the method of the present embodiment, a rectangle surrounding a character string is used as the determination gesture. That is, when a stroke (or a stroke string) of the handwriting input is formed in a substantially rectangular shape, it is determined that the stroke matches the specific gesture character, a character string is recognized from strokes included in a range specified by the specific gesture character, and a numerical expression is recognized from strokes not included in the range specified by the specific gesture character. The rectangle is used as the shape for determining the gesture symbol because: in general, no rectangle appears in numerical expressions, and in addition, it is natural for humans to write a rectangle in order to specify a range. In some fields of mathematics, a square or a rectangle may be used as a marker, but in this case, a marker having another shape may be used as a gesture-determining symbol instead of the rectangle. That is, the gesture character may be determined as long as the gesture character has a unique shape (is easily distinguished from other marks and symbols) and the range indicated by the gesture character is clear. For example, the shape of the gesture character may be a shape in which one or two diagonal lines of a rectangle are written from right to left (this is because if writing from left to right, it cannot be distinguished from "X"), or a shape in which two diagonal lines of a rectangle are written continuously (a shape in which "α" is inverted). Note that, although the fixed gesture character is used to distinguish the character strings included in the numerical expressions, the fixed gesture character (for example, a rectangle surrounding the numerical expressions) may be used to distinguish the numerical expressions included in the character strings.

Fig. 2A is a diagram showing an example of writing in which a numerical expression, a character string, and a fixed gesture are mixed. Theregistration unit 110 monitors whether or not a stroke is a fixed gesture character every time a stroke is input, and determines that the stroke is a fixed gesture characterNext, the stroke order (the order of strokes corresponding to the determined gesture character) and the coordinates (the coordinates of 4 corners of the rectangle or the coordinates of 2 opposite corners of the rectangle) of the determined gesture character are registered in the determined gesture character table. Fig. 2B is a diagram showing a determination gesture symbol table corresponding to the writing example of fig. 2A. For example, in the determination gesture character table shown in FIG. 2B, the determination gesture character BG is referred to₁The coordinates of the upper left corner and the lower right corner (an example of information on the range specified by the fixed gesture character) of the character string "area" written after the character string "area" drawn 4 are registered, and the fixed gesture character BG is registered₂Register with coordinates that it is written after the 1 st "x" drawing 13 and register with its upper left and lower right corners, with respect to the determination gesture BG₃The coordinates of the upper left corner and the lower right corner thereof, which are written after the 2 nd "x" are registered. Referring to the specific gesture character table shown in fig. 2B, the differentiatingsection 112 classifies strokes surrounded by any specific gesture character into a part of a character string (strokes corresponding to the character string), and differentiates strokes other than the part of the character string into a part of a numerical expression (strokes corresponding to the numerical expression).

In the example shown in fig. 2A, the case where the definite gesture character is written in 1 stroke is shown, but it is also permissible to write the definite gesture character in multiple strokes. In this case, for convenience of processing, the maximum number of strokes (for example, 4 strokes) of the writing specification gesture symbol is defined, and it is conditioned that the stroke corresponding to the numerical expression or the character is not written from the start of writing the writing specification gesture symbol to the end of writing. If the conditions are of such a degree, the user is not greatly restricted. In addition, in the case where the specific gesture character is allowed to be written in multiple strokes, both the stroke order of the first stroke and the stroke order of the last stroke constituting the specific gesture character are registered in the specific gesture character table. Note that "order after adjustment" in the determination gesture character table of fig. 2B is described later.

In the method of the present embodiment, the process of distinguishing a character string from a numerical expression is executed regardless of the timing at which the specific gesture is written. This is because, if the order in which the gesture characters are written is designated, the user is greatly restricted, which makes the use inconvenient. As shown in fig. 3, regarding the order in which the gesture characters are determined by writing, 4 cases are considered. Fig. 3A shows a case where the specific gesture BG is initially written, and then a character string is written in the specific gesture BG, fig. 3B shows a case where a character string is initially written, and then a specific gesture BG surrounding the character string is written, fig. 3C shows a case where a character string is initially written, and then other strokes are written, and then a specific gesture BG surrounding the character string is written, and fig. 3D shows a case where a character string is additionally written in the specific gesture BG surrounding the character string that has been written.

Fig. 4 shows an example of writing of a rectangular fixed gesture character. When writing the fixed gesture BG, the user starts writing from the upper left corner of the rectangle, continues writing clockwise (see fig. 4A) or counterclockwise (see fig. 4B), and returns to the upper left corner. When at least a part of the stroke points (e.g., 1/3 or more) constituting the input stroke is included in the specific gesture character, the differentiatingsection 112 determines that the stroke is surrounded by the specific gesture character (the stroke is included in the range specified by the specific gesture character). This is to take into account that certain gestures written by the user sometimes intersect with other strokes (see FIG. 3).

In the method of the present embodiment, in order to recognize a specific gesture character with high accuracy, an online recognition method using time series information of a stroke and an offline recognition method using a feature of the stroke as an image are used in combination to recognize the specific gesture character. The recognition objects for recognizing the specific gesture symbol include numerical expressions (symbols such as english characters, operators such as four arithmetic operators, score symbols, root numbers, and integral symbols, and parentheses) and specific gesture symbols (bounding rectangles).

In the online recognition method, an MRF (Markov Random Field) model is used. Firstly, normalizing an input pattern to a standard size, and extracting characteristic points such as end points, angles and the like by using a Ramer-Douglas-Puck (Ramer-Douglas-Peucker) algorithm. Then, elastic matching is performed with the feature points of the MRF model. A threshold is used to evaluate the score and if the score is less than the threshold, the score is discarded.

In the off-line recognition method, MQDF (Modified Quadratic Discriminant Function) is used. By connecting the stroke point strings in time series order, the strokes can be easily transformed into images. Although time series information is lost when a stroke is converted into an image, by converting the stroke into an image, the influence of stroke order difference, rewriting, or the like can be eliminated. For this image, gradient features are extracted by a Sobel (Sobel) filter and projected in 8 directions. Let the original image be 64 × 64 pixels, which are divided into 8 × 8 partitions of the same size. To which a gaussian filter of 18 x 18 pixels is applied to blur it. Thus, the directional feature near the center contributes more and the peripheral contribution is less, and some contribution is obtained from the adjacent partitions, thereby resisting the position shift. As a result, 8-direction features are extracted for each 8 × 8 partition, and 512-dimensional features are obtained in total. Further, the dimensions are compressed by discriminant analysis of Fisher (Fisher) to obtain 256-dimensional features. Identification was performed using MQDF. Further, a threshold value is set, and if the distance to the rectangle of the input pattern is greater than the threshold value even when the distance is shortest, the threshold value is discarded.

And finally, integrating results of the online identification method and the offline identification method. Here, the online recognition method obtains a logarithmic value of the probability value (although it is a negative value, the larger the probability value is), and the offline recognition method obtains a distance (the smaller the probability value is, the better the distance is), so that it is necessary to make their dimensions uniform. Therefore, the score of the online recognition is multiplied by-1 to be a positive value (therefore, the smaller the score, the better), and the weighted sum is taken to calculate the total score. Composite score_combinationThe calculation is performed by the following equation.

[ number 1]

score_combination＝(-1)×w₁×score_online+w₂×score_offline (1)

Herein, score_online、score_offlineThe scores are the score of the online identification method and the score of the offline identification method, respectively. In addition, w₁、w₂Are the integrated weights that are optimized by using the training pattern. In more detail, generally, score_onlineScore of about-10 to 0_offlineTake a value of more than 0 and less than 1024. Namely, -score_onlineIs score_offlineAbout one-hundredth of (a), and therefore, in reality, 0 < w is satisfied₁＜0.1、w₂＝1-w₁The optimum value is obtained under the condition of (1). Theregistration unit 110 identifies score among objects_combinationThe smallest recognized object is recognized as the determination gesture.

3. Treatment of

Next, a flow of processing of the recognition device of the present embodiment will be described. The identification device has: an interactive device capable of displaying time-series stroke points in real time and performing feedback for recognition for each stroke, such as a tablet PC; and a non-interactive device in which a time-series stroke point is recorded by a tablet or an electronic pen having a memory, and data is read and processed by a PC or the like in a batch manner.

3-1. treatment in a sequential manner

In the sequential method employed in the interactive apparatus, a user designates a language (for example, english or japanese) for recognizing a character string, starts a system (program), and starts writing a numerical expression or a character string on an input unit such as a touch panel. The system identifies the determined gesture characters and distinguishes character strings and numerical expressions when writing strokes each time, and identifies the character strings and the numerical sequences and displays identification results after writing is finished. Fig. 5 is a flowchart showing an example of processing in the sequential method.

First, theprocessing unit 100 acquires a stroke input to the character input unit 160 (step S10). At this time, thedisplay control unit 120 performs control to display the acquired stroke on thedisplay unit 190. Next, theregistration unit 110 determines whether or not the latest stroke traced from the acquired stroke to the maximum number of strokes (for example, 4 strokes) of the fixed gesture character matches the fixed gesture character (constitutes the fixed gesture character) (step S12). For example, theregistration unit 110 calculates the total score for each of the latest 1 stroke, the latest 2-stroke string, the latest 3-stroke string, and the latest 4-stroke (maximum number of strokes)_combinationDetermining that the gesture symbol gives the smallest score among the recognized objects_combinationAnd then recognized as a determination gesture character. However, score is not calculated for strokes discarded by either of the online recognition method and the offline recognition method_combinationIn the case where each stroke is discarded, it is determined that the gesture is not consistent with the determination gesture.

When the determined gesture is matched (yes in step S12), theregistration unit 110 registers the information for specifying the gesture (the stroke order and the coordinates of the 2 opposite corners) in the determined gesture table based on the information of the stroke determined to be matched with the determined gesture (step S14). Here, theregistration unit 110 cancels the setting when it is determined that the stroke matching the determination gesture character is set as the stroke corresponding to the numerical expression before the stroke matching the determination gesture character. Further, thedisplay control unit 120 performs control to display the image of the stroke determined to match the determination gesture character as an image of a rectangle (shape corresponding to the determination gesture character) representing a color (for example, red) different from the other strokes. This enables the user to be fed back with a determination that the gesture has been correctly recognized.

Next, thediscrimination unit 112 refers to the determination gesture symbol table, and sets, as a stroke corresponding to the character string, a stroke included in the range specified by the registered determination gesture symbol (step S16). At this time, thedisplay control unit 120 performs control to change the color of the stroke (for example, from black to blue) included in the range specified by the determination gesture. This enables the user to be fed back a case where the character strings have been correctly distinguished by the specific gesture. Further, in the case where the stroke included in the range specified by the determination gesture character does not exist, the process of step S16 is skipped.

If the determined gesture does not match the determined gesture (no in step S12), thediscrimination unit 112 refers to the determined gesture list and determines whether or not the acquired stroke is included in the range specified by any of the determined gestures (step S18). When the acquired stroke is included in the range specified by any one of the fixed gestures (yes in step S18), the differentiatingunit 112 sets the acquired stroke as a stroke corresponding to the character string (step S20), and thedisplay control unit 120 performs control to change the color of the acquired stroke (for example, from black to blue). On the other hand, if the acquired stroke is not included in the range specified by the fixed gesture (no in step S18), thediscrimination unit 112 sets the acquired stroke as a stroke corresponding to the numerical expression (step S22).

Next, theprocessing unit 100 determines whether or not a stroke is input (step S24), and if a stroke is input (yes in step S24), the process proceeds to step S10, and the process from step S10 onward is repeated until writing by the user is completed.

If it is determined that no stroke has been input (no in step S24), therecognition unit 114 recognizes the character string from the strokes set in steps S16 and S20 as the strokes corresponding to the character string by the character string recognition engine, and recognizes the numerical expression from the strokes set in step S22 as the strokes corresponding to the numerical expression by the numerical expression recognition engine (step S26). Then, thedisplay control unit 120 performs control to display the recognition result (the numerical expression including the character string) on thedisplay unit 190.

Here, the numerical expression recognition engine does not understand the strokes corresponding to the identified gesture characters nor the strokes corresponding to the character strings, and cannot analyze the structure of the numerical expression if there is no information on the character strings surrounded by the identified gesture characters (that is, only information on the strokes corresponding to the numerical expression). Therefore, in the method of the present embodiment, 1 specific gesture character and a character string surrounded by the specific gesture character are regarded as 1 mark (1 symbol in the numerical expression), and the numerical expression is recognized from coordinate information of the specific gesture character regarded as the mark and information of a stroke corresponding to the numerical expression. For this purpose, the stroke order of the gesture character is adjusted. Specifically, the first stroke constituting the fixed gesture character and the earlier stroke order of the first stroke corresponding to the character string surrounded by the fixed gesture character are registered in the fixed gesture character table as the adjusted stroke order of the fixed gesture character. In the example shown in FIG. 2, BG is the determined gesture symbol₁、 BG₂、BG₃The adjusted stroke order of (2) is registered with "1", "9" and "12", respectively. In this case, the recognition unit 114 (mathematical recognition)Engine) can determine the gesture symbol BG by referring to the determined gesture symbol table₁And the character string 'area' contained in the gesture recognition device is regarded as 1 mark written in the 1 st drawing, and the gesture character BG is determined₂And the character string 'height' contained in the gesture recognition device is regarded as 1 mark written in the 9 th drawing, and the gesture character BG is determined₃And the character string "width" included in the character string are regarded as 1 mark written on the 12 th drawing, and the structure of the numerical expression is analyzed based on the relationship between the mark and the position and size of the stroke corresponding to the numerical expression.

In the sequential system, the user may have written mistakes, and therefore, it is preferable to have an Undo (Undo) function. In this case, when the user inputs the cancel command and the latest cancelled stroke is the fixed gesture, the fixed gesture and the registration information thereof are deleted, and the stroke included in the range specified by the fixed gesture (the stroke set as the stroke corresponding to the character string) is reset as the stroke corresponding to the numerical expression. On the other hand, in the case where the latest stroke that is cancelled is a stroke corresponding to a character string or a numerical expression, the stroke is simply deleted.

Fig. 6 is a diagram showing a specific display example in the sequential mode. In the example shown in fig. 6, the user first writes the character string "value" and the numerical expression, and writes the determination gesture BG enclosing the character string "value₁(FIG. 6A). Then, theregistration unit 110 recognizes the fixed gesture BG₁Thedisplay control unit 120 specifies the gesture symbol BG₁Substitution with red symbol BG₁Image BI of equally sized rectangles₁(determination of gesture character image) of the determined gesture character BG₁The color of the surrounding character string "value" changes to blue (fig. 6B). Next, the user writes a new definite gesture BG₂(FIG. 6C). Then, theregistration unit 110 recognizes the fixed gesture BG₂Thedisplay control unit 120 specifies the gesture symbol BG₂Transposing to determine gesture symbol image BI₂(FIG. 6D). Then, when the user is determining the gesture symbol image BI₂When the character string "result" is written, thedisplay control unit 120 displays the character string "result" in blue every time a stroke is acquired (fig. 6E).Finally, when the user performs an operation to instruct execution of recognition, therecognition unit 114 recognizes a character string and a numerical expression, and thedisplay control unit 120 displays the recognition result (fig. 6F). Furthermore, here, the gesture symbol image BI is to be determined₁、BI₂Removed to display the recognition result.

3-2. batch mode treatment

In the batch mode employed by non-interactive devices, the user starts the system and writes numerical expressions and character strings to an interface such as a tablet. After writing, data of the stroke string is read into a PC or the like (recognition device), and at this time, a language for recognizing the character string is specified. In the batch mode, the 3 processes of determining the recognition of the gesture character, the distinction of the character string and the number expression, the recognition of the character string and the number sequence, and the display of the recognition result are performed after the data of all the stroke strings are transferred to the PC or the like. In this case, when the input strokes are sequentially processed, the same processing as the sequential method can be performed. In the batch method, since it is not necessary to identify a gesture character and distinguish a character string from a numerical expression in real time, the following method may be adopted: after all the determined gestures are recognized, the strokes outside the determined gestures are distinguished into character strings and numerical expressions, and finally the character strings and the numerical expressions are recognized. Fig. 7 is a flowchart showing an example of processing in the latter method in the batch system.

First, theprocessing unit 100 acquires a stroke string input to the character input unit 160 (step S30). Next, theprocessing unit 100 sets the variable i to 1 (step S32). Next, theregistration unit 110 determines whether or not the latest stroke traced from the ith stroke (i-th drawing) to the maximum drawing number (for example, 4 drawings) of the determination gesture symbol matches the determination gesture symbol (step S34).

When the determined gesture is matched (yes in step S34), theregistration unit 110 registers the information for specifying the gesture (the stroke order and the coordinates of the 2 opposite corners) in the determined gesture table based on the information of the stroke determined to be matched with the determined gesture (step S36). In the case of non-coincidence with the determined gesture character (no at step S34), the i-th stroke is set as a non-gesture character stroke (a stroke not coincident with the determined gesture character) (step S38).

Next, theprocessing unit 100 increments the value of the variable i by 1 (step S40), and determines whether or not the value of the variable i is equal to or less than n (n is the total number of strokes included in the acquired stroke string) (step S42). If the value of the variable i is equal to or less than n (yes in step S42), the process proceeds to step S34, and the process from step S34 is repeated until the value of the variable i exceeds n.

When the value of the variable i exceeds n (no in step S42), theprocessing unit 100 sets the variable i to 1 (step S44). Next, thediscrimination unit 112 refers to the determination gesture symbol table, and determines whether or not the i-th non-gesture symbol stroke set in step S38 is included in the range specified by any one of the determination gesture symbols (step S46). When the i-th non-gesture character stroke is included in the range specified by any one of the determination gestures (yes in step S46), thediscrimination section 112 sets the i-th non-gesture character stroke as a stroke corresponding to the character string (step S48). On the other hand, if the i-th non-gesture character stroke is not included in the range specified by the determination gesture character (no in step S46), the differentiatingsection 112 sets the i-th non-gesture character stroke as a stroke corresponding to the numerical expression (step S50).

Next, theprocessing unit 100 increments the value of the variable i by 1 (step S52), and determines whether or not the value of the variable i is equal to or less than m (m is the total number of non-gesture character strokes) (step S54). If the value of the variable i is equal to or less than m (yes in step S54), the process proceeds to step S46, and the process from step S46 is repeated until the value of the variable i exceeds m.

When the value of the variable i exceeds m (no in step S54), therecognition unit 114 recognizes the character string from the stroke set in step S48 as the stroke corresponding to the character string using the character string recognition engine, and recognizes the numerical expression from the fixed gesture table (coordinate information of the fixed gesture regarded as a marker) and the stroke set in step S50 as the stroke corresponding to the numerical expression using the numerical expression recognition engine (step S56). Then, thedisplay control unit 120 performs control to display the recognition result (the numerical expression including the character string) on thedisplay unit 190.

Fig. 8 is a diagram showing a specific display example in the batch mode. In batchesIn the method, all strokes written by the user and taken into the recognition device are displayed (fig. 8A), and when the user performs an operation to instruct the execution of the recognition, the recognition of the specific gesture character, the distinction of the character string and the numerical expression, the recognition of the character string and the numerical sequence are executed, and the recognition result is displayed (fig. 8C). Further, after the recognition of the gesture character and the discrimination of the character string and the numerical expression are performed, and before the recognition of the character string and the numerical sequence is performed, the recognition result of the gesture character and the discrimination of the numerical expression and the character string may be displayed as shown in fig. 8B. In the example shown in FIG. 8B, the gesture character BG will be determined₁、 BG₂、BG₃Respective substitution into definite gesture symbol images BI₁、BI₂、BI₃Gesture symbol BG to be determined₁、BG₂、BG₃The color of the surrounding character strings "area", "height", and "width" is changed to blue.

According to the present embodiment, a specific gesture character for distinguishing a character string from a numerical expression is introduced, strokes surrounded by the specific gesture character are distinguished as character strings, strokes not surrounded by the specific gesture character are distinguished as numerical expressions, and recognition is performed by a character string recognition engine and a numerical expression recognition engine, respectively, whereby character strings included in the numerical expression can be clearly distinguished from the numerical expressions and recognized, respectively. In recognizing the numerical expression, the fixed gesture character and the stroke surrounded by the fixed gesture character are regarded as 1 mark, and thereby the structural analysis of the numerical expression including the character string can be performed using the numerical recognition engine.

Further, according to the present embodiment, in the sequential mode, when the fixed gesture character is written, the process of dividing the stroke surrounded by the fixed gesture character into character strings is performed (step S16 in fig. 5), and when the written stroke is surrounded by the fixed gesture character, the process of dividing the stroke into character strings is performed (step S20 in fig. 5), whereby the character strings and the numerical expressions can be distinguished regardless of the timing at which the fixed gesture character is written, and convenience for the user can be improved. That is, according to the present embodiment, the process of distinguishing a character string from a numerical expression can be performed regardless of whether the character string is written after the character string is specified (fig. 3A), the character string is written after the character string is written (fig. 3B and 3C), or the character string is added to the specified character string (fig. 3D).

The present invention is not limited to the above-described embodiments, and various modifications can be made. The present invention includes substantially the same configurations (for example, configurations having the same functions, methods, and results or configurations having the same objects and effects) as those described in the embodiments. The present invention includes a configuration in which the immaterial portions of the configurations described in the embodiments are replaced. The present invention includes a configuration that achieves the same operational effects or the same objects as those of the configuration described in the embodiment. The present invention includes a configuration in which a known technique is added to the configuration described in the embodiment.

Description of the reference numerals

100 processing part, 110 registration part, 112 distinguishing part, 114 identification part, 120 display control part, 160 character input part, 170 storage part and 190 display part.

Claims (7)

Translated fromChinese

1.一种识别装置，从手写输入的笔画串识别包含字符串的数式，其特征在于，包含：1. a recognition device, from the stroke string of handwriting input, recognizes the numerical formula that contains character string, it is characterized in that, comprises:登录部，其以逐次方式判断从上述手写输入的笔画追溯到用于区分字符串与数式的确定手势符的最大画数为止的最近的笔画是否与上述确定手势符相符，在判断为与上述确定手势符相符的情况下，登录与由上述确定手势符指定的范围相关的信息；A registration unit that sequentially judges whether or not the most recent stroke from the strokes of the handwriting input up to the maximum number of strokes of the definite gesture for distinguishing character strings from numerical expressions matches the definite gesture, If the sign matches, register the information related to the range specified by the above-mentioned determination sign;区分部，其将输入的笔画之中的包含在由上述确定手势符指定的范围内的笔画设定作为与字符串和数式中的一方对应的笔画，将未包含在由上述确定手势符指定的范围内的笔画设定作为与字符串和数式中的另一方对应的笔画；以及A distinguishing unit that sets, among the input strokes, strokes included in the range specified by the above-mentioned determination sign as a stroke corresponding to one of a character string and a numerical formula, and sets strokes not included in the range designated by the above-mentioned determination sign as a stroke corresponding to one of a character string and a numerical formula The strokes within the range are set as the strokes corresponding to the other side of the character string and the formula; and识别部，其利用字符串识别引擎从作为与字符串对应的笔画而设定的笔画来识别字符串，利用数式识别引擎从作为与数式对应的笔画而设定的笔画来识别数式。The recognition unit recognizes the character string from the strokes set as the strokes corresponding to the character string using the character string recognition engine, and recognizes the numerical expression from the strokes set as the strokes corresponding to the numerical expression using the numerical expression recognition engine.2.根据权利要求1所述的识别装置，其特征在于，2. The identification device according to claim 1, characterized in that,上述区分部在输入的笔画或者笔画串被判断为与上述确定手势符相符的情况下，将包含在由该确定手势符指定的范围内的笔画设定作为与字符串和数式中的一方对应的笔画，在输入的笔画被判断为与上述确定手势符不相符的情况下，判断该输入的笔画是否包含在由上述确定手势符指定的范围内，在该输入的笔画包含在由上述确定手势符指定的范围内的情况下，将该输入的笔画设定作为与字符串和数式中的一方对应的笔画。When the inputted stroke or stroke string is judged to match the above-mentioned specific gesture symbol, the distinguishing unit sets the stroke included in the range specified by the specific gesture symbol as the one corresponding to the character string and the numerical expression. Stroke, when the inputted stroke is judged to be inconsistent with the above-mentioned determination gesture symbol, it is judged whether the inputted stroke is included in the range specified by the above-mentioned determination gesture symbol, and the inputted stroke is included in the above-mentioned determination gesture symbol. If it is within the specified range, the input stroke is set as the stroke corresponding to one of the character string and the numerical expression.3.根据权利要求1或2所述的识别装置，其特征在于，3. The identification device according to claim 1 or 2, characterized in that,上述区分部将输入的笔画之中的包含在由上述确定手势符指定的范围内的笔画设定作为与字符串对应的笔画，将未包含在由上述确定手势符指定的范围内的笔画设定作为与数式对应的笔画，The distinguishing unit sets, among the input strokes, strokes included in the range specified by the determination symbol as strokes corresponding to the character string, and sets strokes not included in the range specified by the determination symbol As the strokes corresponding to the numbers,上述识别部将包含在由1个上述确定手势符指定的范围内的笔画视为1个标记，利用数式识别引擎来识别数式。The recognition unit regards the strokes included in the range specified by one of the specific gesture symbols as one mark, and recognizes the mathematical expression using the mathematical expression recognition engine.4.根据权利要求1或2所述的识别装置，其特征在于，4. The identification device according to claim 1 or 2, characterized in that,还包含进行使输入的笔画串显示于显示部的控制的显示控制部，also includes a display control unit that performs control to display the input stroke string on the display unit,上述显示控制部进行使被判断为与上述确定手势符相符的笔画或者笔画串作为表示与上述确定手势符对应的形状的图像来显示的控制。The display control unit performs control to display a stroke or a stroke string determined to match the definite gesture symbol as an image representing a shape corresponding to the definite gesture symbol.5.根据权利要求4所述的识别装置，其特征在于，5. The identification device according to claim 4, characterized in that,上述显示控制部进行使包含在由上述确定手势符指定的范围内的笔画以与未包含在由上述确定手势符指定的范围内的笔画不同的颜色来显示的控制。The display control unit performs control to display strokes included in the range specified by the determination symbol in a different color from strokes not included in the range specified by the determination symbol.6.根据权利要求1或2所述的识别装置，其特征在于，6. The identification device according to claim 1 or 2, characterized in that,上述登录部在输入的笔画或者笔画串形成矩形的情况下，判断为该笔画或者笔画串与上述确定手势符相符。When the inputted stroke or stroke string forms a rectangle, the registration unit determines that the stroke or stroke string matches the above-mentioned determination gesture symbol.7.一种计算机可读取的信息存储介质，其特征在于，7. A computer-readable information storage medium, characterized in that,存储有使计算机作为权利要求1至6中的任意一项所述的识别装置的上述各部发挥功能的程序。A program for causing a computer to function as the respective units of the identification device according to any one of claims 1 to 6 is stored.

Images