CN112183544A - Double-channel fused three-layer architecture mathematical formula identification method, system and storage device - Google Patents
Double-channel fused three-layer architecture mathematical formula identification method, system and storage deviceDownload PDFInfo
- Publication number
- CN112183544A CN112183544ACN202011046709.7ACN202011046709ACN112183544ACN 112183544 ACN112183544 ACN 112183544ACN 202011046709 ACN202011046709 ACN 202011046709ACN 112183544 ACN112183544 ACN 112183544A
- Authority
- CN
- China
- Prior art keywords
- layer
- attention
- channel
- input
- mathematical formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
- G06F40/111—Mathematical or scientific formatting; Subscripts; Superscripts
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Algebra (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Image Analysis (AREA)
Abstract
Translated fromChinese
Description
Technical Field
The invention relates to the technical field of mathematical formula identification, in particular to a method, a system and a storage device for identifying a three-layer architecture mathematical formula fused with two channels.
Background
Mathematical formulas are widely used in many scientific fields, and have essential functions in the aspects of explaining theoretical knowledge, describing scientific problems and the like. The mathematical formula can be input by using a mathematical tool, but the way is to input a mathematical markup language Latex to generate a corresponding mathematical formula, which requires a certain grammar basis for a user, while handwriting input can solve the input problem well, and the user can input the mathematical formula more conveniently.
Compared with the recognition of the printed mathematical formula, the recognition difficulty of the handwritten mathematical formula is far higher than that of the printed mathematical formula due to the fuzziness of the handwritten symbols, the diversity of the handwriting styles and the large amount of adhesion among the symbols.
Generally, the mathematical formula identification process can be divided into three stages: a symbol segmentation stage, a symbol identification stage and a structure analysis stage. For different ways of implementing the three phases, the current mainstream identification schemes can be divided into two types: multi-stage identification and single-stage identification.
The multi-stage solution firstly divides the symbols in the mathematical formula, then identifies the divided symbols, and finally performs the structural analysis according to the identification result and the symbol position, although the solution can realize the modularization of the identification process, it has a very serious problem: the error inherits. The error of the previous stage can be transmitted to the next stage to cause error accumulation, so that the recognition precision of the whole recognition process is influenced.
The single-stage solution realizes an end-to-end identification network by using a deep neural network, and completes three stages of formula identification at one time. The identification network usually adopts a coding and decoding structure, firstly, an encoder is used for extracting characteristics of an input mathematical formula picture and coding the characteristic, then a decoder with an attention mechanism is used for scanning the characteristics extracted by the encoder, a most relevant region is used for describing a segmented symbol, and a mathematical markup language Latex corresponding to the mathematical formula is output.
In the encoder part, because the symbol sizes of different mathematical formulas are inconsistent, in order to more effectively utilize visual features in pictures, researchers solve the problem of symbol scale inconsistency by simultaneously extracting a plurality of feature outputs with different granularities in the encoder. However, although the problem of symbol size can be solved by extracting a plurality of features with different granularities in the coding layer, the characterization force of the extracted features is insufficient, the context information of the symbols is not fully utilized, and a large number of irrelevant features are introduced.
In the attention mechanism of the decoder, the output characteristic diagram of the encoder is weighted and summed according to the attention characteristic diagram at the current moment to obtain a vector for representing the most relevant region of the current recognition character. And then decodes and outputs the serialized backward quantity. Using only the attention at the current time, symbol repetition and symbol deletion may occur in the recognition result.
Because the circular neural network has poor prediction capability in the initial stage of model training, the real label is used as the input of the neural unit in the circular neural network in the prior art, thereby preventing the training effect of the circular neural network from being influenced by the larger deviation of a certain neuron. Although the model has a better recognition effect in the training phase by using the real label instead of the input of the prediction result, the recognition accuracy is lowered due to the lack of guidance of the real label in the testing process.
Disclosure of Invention
Therefore, a double-channel fused three-layer structure mathematical formula identification method needs to be provided to solve the problem of low precision of the existing single-stage formula identification technology. The specific technical scheme is as follows:
a method for identifying a three-layer architecture mathematical formula fused with two channels comprises the following steps:
performing feature extraction on an input picture through a coding layer, wherein the features comprise: regional visual information;
capturing the context of the regional visual information through an attention layer to generate a context vector;
and decoding the context vector through a decoding layer to generate a mathematical markup language file corresponding to the formula.
Further, the "performing feature extraction on an input picture by using an encoding layer" further includes the steps of:
using DenseNet as an encoder;
extracting visual information of an input picture by using a Dense network as a backbone network;
a spatial attention module and a channel attention module are fused in the encoder.
Further, before the "extracting features of the input picture by the coding layer", the method further includes the steps of:
adding mask information in the input picture data, and additionally adding a channel to the filled part, wherein the channel is used for recording the filled information.
Further, the "performing feature extraction on an input picture by using an encoding layer" further includes the steps of:
the spatial attention module respectively performs average pooling and maximum pooling on the input feature map to obtain two feature maps with the same dimensionality, splices the two feature maps with the same dimensionality according to a channel and then obtains a spatial attention matrix through a sigmoid function, and multiplies the spatial attention matrix with the input feature map to obtain a spatial attention feature map;
the channel attention module respectively carries out global average pooling and global maximum pooling on the input feature map to obtain two feature maps, inputs the two feature maps into a shared multilayer perceptron to obtain two vectors, adds the two vectors and multiplies the two vectors by the input feature map to obtain the channel attention feature map.
Further, the attention layer is provided with a coverage vector, and the coverage vector is used for representing the accumulated sum of all attention mechanisms at past moments;
the attention calculation formula is:
expti＝fatt(αi，ht-1)
zt＝φ({αi，αti})
wherein f isattRepresenting a multi-layer perceptron, aiIs a vector in the decoder output, corresponding to a region in the picture, ht-1Is the hidden layer output of the previous unit, alphatiThe attention weight of the ith vector at the t time step is represented, c represents the coverage vector, alphalAttention weight, z, representing the ith time steptRepresents the output of the attention mechanism and denotes the application of attention weights to the image regions.
Further, the decoding layer adopts a GRU recurrent neural network for decoding.
Further, the generating of the mathematical markup language file corresponding to the formula further includes the steps of:
and selecting the sequence with the highest output score as a final output sequence by using a cluster searching algorithm on the output of the coding layer.
In order to solve the technical problem, the three-layer architecture mathematical formula recognition system fusing double channels is further provided, and the specific technical scheme is as follows:
a three-layer architecture mathematical formula recognition system fusing two channels comprises: an encoding layer, an attention layer, and a decoding layer;
the coding layer is configured to: performing feature extraction on an input picture, wherein the features comprise: regional visual information;
the attention layer is used for: capturing the context of the regional visual information to generate a context vector;
the decoding layer is used for: and decoding the context vector to generate a mathematical markup language file corresponding to the formula.
Further, DenseNet was used as the encoder;
extracting visual information of an input picture by using a Dense network as a backbone network;
a space attention module and a channel attention module are fused in the encoder;
the spatial attention module respectively performs average pooling and maximum pooling on the input feature map to obtain two feature maps with the same dimensionality, splices the two feature maps with the same dimensionality according to a channel and then obtains a spatial attention matrix through a sigmoid function, and multiplies the spatial attention matrix with the input feature map to obtain a spatial attention feature map;
the channel attention module respectively carries out global average pooling and global maximum pooling on the input feature map to obtain two feature maps, inputs the two feature maps into a shared multilayer perceptron to obtain two vectors, adds the two vectors and multiplies the two vectors by the input feature map to obtain the channel attention feature map.
In order to solve the technical problem, the storage device is further provided, and the specific technical scheme is as follows:
a storage device having stored therein a set of instructions for performing: the steps of any of the above claims.
The invention has the beneficial effects that: 1. performing feature extraction on an input picture through a coding layer, wherein the features comprise: regional visual information; capturing the context of the regional visual information through an attention layer to generate a context vector; and decoding the context vector through a decoding layer to generate a mathematical markup language file corresponding to the formula. Through the above operations of the three layers of the coding layer, the attention layer and the decoding layer, a mathematical formula with higher precision can be obtained.
2. By merging a spatial attention module and a channel attention module in the encoder. The method can lead the encoder to learn what to pay attention and where to pay attention on the channel and the spatial axis, thereby improving the expression of the interest region, leading the characteristics extracted by the encoder to have more representation power and effectively improving the recognition rate.
3. Before the feature extraction of the input picture through the coding layer, the method further comprises the following steps: adding mask information in the input picture data, and additionally adding a channel to the filled part, wherein the channel is used for recording the filled information. The information redundancy of the picture filling process can be reduced by adding the mask information.
4. A coverage vector is provided in the attention layer, which is used to represent the cumulative sum of all attention mechanisms at past times, and which tells the model which parts of the encoder's inputs are already attended to and not attended to. In order to prevent the model from paying more attention to the concerned region, the coverage vector is used as a component of the attention of the next step, so that the attention distribution generated in the next step intentionally reduces the probability of the concerned region, and the phenomena of symbol missing and symbol repetition in formula recognition are effectively reduced.
Drawings
FIG. 1 is a flow chart of a method for identifying a mathematical formula with a three-layer architecture incorporating two channels according to an embodiment;
FIG. 2 is a diagram illustrating a method for identifying a mathematical formula with a three-layer architecture incorporating two channels according to an embodiment;
FIG. 3 is a block diagram of a two-channel fused three-layer structure mathematical formula recognition system according to an embodiment;
fig. 4 is a block diagram of a storage device according to an embodiment.
Description of reference numerals:
300. a three-layer structure mathematical formula recognition system integrating two channels,
301. a layer of code to be encoded,
302. attention is paid to the layer of attention,
303. a layer of code that is decoded,
400. a storage device.
Detailed Description
To explain technical contents, structural features, and objects and effects of the technical solutions in detail, the following detailed description is given with reference to the accompanying drawings in conjunction with the embodiments.
Referring to fig. 1 to 2, in the present embodiment, a method for identifying a mathematical formula of a three-layer architecture with two channels integrated can be applied to a storage device, including but not limited to: personal computers, servers, general purpose computers, special purpose computers, network devices, embedded devices, programmable devices, intelligent mobile terminals, etc. The specific implementation is as follows:
step S101: performing feature extraction on an input picture through a coding layer, wherein the features comprise: regional visual information.
Step S102: a context vector is generated by capturing the context of the regional visual information through the attention layer.
Step S103: and decoding the context vector through a decoding layer to generate a mathematical markup language file corresponding to the formula.
Performing feature extraction on an input picture through a coding layer, wherein the features comprise: regional visual information; capturing the context of the regional visual information through an attention layer to generate a context vector; and decoding the context vector through a decoding layer to generate a mathematical markup language file corresponding to the formula. Through the above operations of the three layers of the coding layer, the attention layer and the decoding layer, a mathematical formula with higher precision can be obtained.
Wherein step S101 in this embodiment further includes the steps of: using DenseNet as an encoder; extracting visual information of an input picture by using a Dense network as a backbone network; a spatial attention module and a channel attention module are fused in the encoder. By merging a spatial attention module and a channel attention module in the encoder. The method can lead the encoder to learn what to pay attention and where to pay attention on the channel and the spatial axis, thereby improving the expression of the interest region, leading the characteristics extracted by the encoder to have more representation power and effectively improving the recognition rate.
In order to reduce the information redundancy in the picture filling process, in this embodiment, before the "performing feature extraction on an input picture by using a coding layer", the method further includes the steps of: adding mask information in the input picture data, and additionally adding a channel to the filled part, wherein the channel is used for recording the filled information.
The following specifically describes the operation of the spatial attention module and the channel attention module:
the spatial attention module respectively performs average pooling and maximum pooling on the input feature map to obtain two feature maps with the same dimensionality, splices the two feature maps with the same dimensionality according to a channel and then obtains a spatial attention matrix through a sigmoid function, and multiplies the spatial attention matrix with the input feature map to obtain a spatial attention feature map;
the channel attention module respectively carries out global average pooling and global maximum pooling on the input feature map to obtain two feature maps, inputs the two feature maps into a shared multilayer perceptron to obtain two vectors, adds the two vectors and multiplies the two vectors by the input feature map to obtain the channel attention feature map.
In this embodiment, the densnet network structure is mainly composed of a DenseBlock and a Transition, and a channel attention module and a space attention module are sequentially added between the DenseBlock and the Transition.
In this embodiment, the attention layer is provided with a coverage vector, and the coverage vector is used for representing the accumulated sum of all attention mechanisms at past time;
the attention calculation formula is:
expti＝fatt(αi，ht-1)
zt＝φ({ai，αti})
wherein f isattRepresenting a multi-layer perceptron, aiIs a vector in the decoder output, corresponding to a region in the picture, ht-1Is the hidden layer output of the previous unit, alphatiThe attention weight of the ith vector at the t time step is represented, c represents the coverage vector, alphalAttention weight, z, representing the ith time steptRepresents the output of the attention mechanism and denotes the application of attention weights to the image regions.
The coverage vector is used to represent the cumulative sum of all attention mechanisms at past times, which tells the model which parts of the encoder's inputs are already attended to and none. In order to prevent the model from paying more attention to the concerned region, the coverage vector is used as a component of the attention of the next step, so that the attention distribution generated in the next step intentionally reduces the probability of the concerned region, and the phenomena of symbol missing and symbol repetition in formula recognition are effectively reduced.
In this embodiment, the decoding layer performs decoding using a GRU recurrent neural network. The problems of gradient explosion and gradient disappearance of the circulating neural network are alleviated. And selecting a probability value in the training process to select whether the current label or the output of the previous unit is used as the input of the current unit, and finally outputting a one-hot coding set corresponding to each symbol by the coding layer. As shown in fig. 2. By randomly selecting the real label and the prediction output as the input of the coding layer recurrent neural network, the recognition capability of the recognition system under different recognition scenes can be enhanced.
Preferably, the generating of the mathematical markup language file corresponding to the formula further includes the steps of: and selecting the sequence with the highest output score as a final output sequence by using a cluster searching algorithm on the output of the coding layer.
Referring to fig. 3, in the present embodiment, an embodiment of a two-channel fused three-layer structure mathematicalformula identification system 300 is as follows:
a two-channel fused three-tier architecture mathematicalformula identification system 300, comprising: anencoding layer 301, anattention layer 302, and a decoding layer 303; thecoding layer 301 is configured to: performing feature extraction on an input picture, wherein the features comprise: regional visual information; theattention layer 302 is used to: capturing the context of the regional visual information to generate a context vector; the decoding layer 303 is configured to: and decoding the context vector to generate a mathematical markup language file corresponding to the formula.
Performing feature extraction on an input picture through thecoding layer 301, where the features include: regional visual information; capturing the context of the regional visual information through theattention layer 302 to generate a context vector; and decoding the context vector through a decoding layer 303 to generate a mathematical markup language file corresponding to the formula. By the above operations of the three layers of theencoding layer 301, theattention layer 302 and the decoding layer 303, a mathematical formula with higher accuracy can be obtained.
In theencoding layer 301, DenseNet is used as an encoder; extracting visual information of an input picture by using a Dense network as a backbone network; a spatial attention module and a channel attention module are fused in the encoder. By merging a spatial attention module and a channel attention module in the encoder. The method can lead the encoder to learn what to pay attention and where to pay attention on the channel and the spatial axis, thereby improving the expression of the interest region, leading the characteristics extracted by the encoder to have more representation power and effectively improving the recognition rate.
The following specifically describes the operation of the spatial attention module and the channel attention module:
the spatial attention module respectively performs average pooling and maximum pooling on the input feature map to obtain two feature maps with the same dimensionality, splices the two feature maps with the same dimensionality according to a channel and then obtains a spatial attention matrix through a sigmoid function, and multiplies the spatial attention matrix with the input feature map to obtain a spatial attention feature map;
the channel attention module respectively carries out global average pooling and global maximum pooling on the input feature map to obtain two feature maps, inputs the two feature maps into a shared multilayer perceptron to obtain two vectors, adds the two vectors and multiplies the two vectors by the input feature map to obtain the channel attention feature map.
In this embodiment, the densnet network structure is mainly composed of a DenseBlock and a Transition, and a channel attention module and a space attention module are sequentially added between the DenseBlock and the Transition.
In this embodiment, theattention layer 302 is provided with a coverage vector, and the coverage vector is used for representing the accumulated sum of all attention mechanisms at past time;
the attention calculation formula is:
expti＝fatt(αi，ht-1)
zt＝φ({ai，αti})
wherein f isattRepresenting a multi-layer perceptron, aiIs a vector in the decoder output, corresponding to a region in the picture, ht-1Is the hidden layer output of the previous unit, alphatiThe attention weight of the ith vector at the t time step is represented, c represents the coverage vector, alphalAttention weight, z, representing the ith time steptRepresents the output of the attention mechanism and denotes the application of attention weights to the image regions.
The coverage vector is used to represent the cumulative sum of all attention mechanisms at past times, which tells the model which parts of the encoder's inputs are already attended to and none. In order to prevent the model from paying more attention to the concerned region, the coverage vector is used as a component of the attention of the next step, so that the attention distribution generated in the next step intentionally reduces the probability of the concerned region, and the phenomena of symbol missing and symbol repetition in formula recognition are effectively reduced.
In this embodiment, the decoding layer 303 performs decoding using a GRU recurrent neural network. The problems of gradient explosion and gradient disappearance of the circulating neural network are alleviated. In the training process, a probability value is selected to select whether to use the output of the current label or the previous unit as the input of the current unit, and finally the one-hot encoding set corresponding to each symbol is output by theencoding layer 301. As shown in fig. 2. By randomly selecting the real label and the prediction output as the input of the cyclic neural network of thecoding layer 301, the recognition capability of the recognition system under different recognition scenes can be enhanced. And selecting the sequence with the highest output score as a final output sequence by using a bundle searching algorithm on the output of thecoding layer 301.
Referring to fig. 4, in the present embodiment, amemory device 400 is implemented as follows: thememory device 400 may be used to perform any of the steps of the above-mentioned two-channel fused three-layer structure mathematical formula identification method, and a repeated description thereof is omitted.
It should be noted that, although the above embodiments have been described herein, the invention is not limited thereto. Therefore, based on the innovative concepts of the present invention, the technical solutions of the present invention can be directly or indirectly applied to other related technical fields by making changes and modifications to the embodiments described herein, or by using equivalent structures or equivalent processes performed in the content of the present specification and the attached drawings, which are included in the scope of the present invention.
Claims (10)
Translated fromChinese
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011046709.7ACN112183544B (en) | 2020-09-29 | 2020-09-29 | Three-layer architecture mathematical formula identification method, system and storage device integrating double channels |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011046709.7ACN112183544B (en) | 2020-09-29 | 2020-09-29 | Three-layer architecture mathematical formula identification method, system and storage device integrating double channels |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112183544Atrue CN112183544A (en) | 2021-01-05 |
| CN112183544B CN112183544B (en) | 2024-09-13 |
Family
ID=73945817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011046709.7AActiveCN112183544B (en) | 2020-09-29 | 2020-09-29 | Three-layer architecture mathematical formula identification method, system and storage device integrating double channels |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112183544B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554151A (en)* | 2021-07-07 | 2021-10-26 | 浙江工业大学 | Attention mechanism method based on convolution interlayer relation |
| CN113743315A (en)* | 2021-09-07 | 2021-12-03 | 电子科技大学 | Handwritten elementary mathematical formula recognition method based on structure enhancement |
| CN113888551A (en)* | 2021-10-22 | 2022-01-04 | 中国人民解放军战略支援部队信息工程大学 | Liver tumor image segmentation method based on dense connection network of high-low layer feature fusion |
| CN118155221A (en)* | 2024-05-11 | 2024-06-07 | 济南大学 | A method for printed formula recognition based on multi-supervision |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060062466A1 (en)* | 2004-09-22 | 2006-03-23 | Microsoft Corporation | Mathematical expression recognition |
| US20060100872A1 (en)* | 2004-11-08 | 2006-05-11 | Kabushiki Kaisha Toshiba | Pattern recognition apparatus and pattern recognition method |
| CN109241861A (en)* | 2018-08-14 | 2019-01-18 | 科大讯飞股份有限公司 | A kind of method for identifying mathematical formula, device, equipment and storage medium |
| CN109389091A (en)* | 2018-10-22 | 2019-02-26 | 重庆邮电大学 | The character identification system and method combined based on neural network and attention mechanism |
| CN110110601A (en)* | 2019-04-04 | 2019-08-09 | 深圳久凌软件技术有限公司 | Video pedestrian weight recognizer and device based on multi-space attention model |
| CN110119765A (en)* | 2019-04-18 | 2019-08-13 | 浙江工业大学 | A kind of keyword extracting method based on Seq2seq frame |
| US20190278857A1 (en)* | 2018-03-12 | 2019-09-12 | Microsoft Technology Licensing, Llc | Sequence to Sequence Conversational Query Understanding |
| CA3050025A1 (en)* | 2018-07-19 | 2020-01-19 | Tata Consultancy Services Limited | Systems and methods for end-to-end handwritten text recognition using neural networks |
| CN111126221A (en)* | 2019-12-16 | 2020-05-08 | 华中师范大学 | Mathematical formula identification method and device integrating two-way visual attention mechanism |
| CN111199233A (en)* | 2019-12-30 | 2020-05-26 | 四川大学 | An improved deep learning method for pornographic image recognition |
| CN111562612A (en)* | 2020-05-20 | 2020-08-21 | 大连理工大学 | A deep learning microseismic event recognition method and system based on attention mechanism |
- 2020
- 2020-09-29CNCN202011046709.7Apatent/CN112183544B/enactiveActive
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060062466A1 (en)* | 2004-09-22 | 2006-03-23 | Microsoft Corporation | Mathematical expression recognition |
| US20060100872A1 (en)* | 2004-11-08 | 2006-05-11 | Kabushiki Kaisha Toshiba | Pattern recognition apparatus and pattern recognition method |
| US20190278857A1 (en)* | 2018-03-12 | 2019-09-12 | Microsoft Technology Licensing, Llc | Sequence to Sequence Conversational Query Understanding |
| CA3050025A1 (en)* | 2018-07-19 | 2020-01-19 | Tata Consultancy Services Limited | Systems and methods for end-to-end handwritten text recognition using neural networks |
| CN109241861A (en)* | 2018-08-14 | 2019-01-18 | 科大讯飞股份有限公司 | A kind of method for identifying mathematical formula, device, equipment and storage medium |
| CN109389091A (en)* | 2018-10-22 | 2019-02-26 | 重庆邮电大学 | The character identification system and method combined based on neural network and attention mechanism |
| CN110110601A (en)* | 2019-04-04 | 2019-08-09 | 深圳久凌软件技术有限公司 | Video pedestrian weight recognizer and device based on multi-space attention model |
| CN110119765A (en)* | 2019-04-18 | 2019-08-13 | 浙江工业大学 | A kind of keyword extracting method based on Seq2seq frame |
| CN111126221A (en)* | 2019-12-16 | 2020-05-08 | 华中师范大学 | Mathematical formula identification method and device integrating two-way visual attention mechanism |
| CN111199233A (en)* | 2019-12-30 | 2020-05-26 | 四川大学 | An improved deep learning method for pornographic image recognition |
| CN111562612A (en)* | 2020-05-20 | 2020-08-21 | 大连理工大学 | A deep learning microseismic event recognition method and system based on attention mechanism |
Non-Patent Citations (1)
| Title |
|---|
| 肖文斌: "基于编码器-解码器和注意力机制神经网络的数学公式识别方法", 《中国优秀硕士学位论文全文数据库》, pages 20 - 59* |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554151A (en)* | 2021-07-07 | 2021-10-26 | 浙江工业大学 | Attention mechanism method based on convolution interlayer relation |
| CN113554151B (en)* | 2021-07-07 | 2024-03-22 | 浙江工业大学 | Attention mechanism method based on convolution interlayer relation |
| CN113743315A (en)* | 2021-09-07 | 2021-12-03 | 电子科技大学 | Handwritten elementary mathematical formula recognition method based on structure enhancement |
| CN113743315B (en)* | 2021-09-07 | 2023-07-14 | 电子科技大学 | A Recognition Method for Handwritten Elementary Mathematics Formulas Based on Structural Enhancement |
| CN113888551A (en)* | 2021-10-22 | 2022-01-04 | 中国人民解放军战略支援部队信息工程大学 | Liver tumor image segmentation method based on dense connection network of high-low layer feature fusion |
| CN118155221A (en)* | 2024-05-11 | 2024-06-07 | 济南大学 | A method for printed formula recognition based on multi-supervision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112183544B (en) | 2024-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Spatial information enhancement network for 3D object detection from point cloud | |
| CN108615036B (en) | A natural scene text recognition method based on convolutional attention network | |
| CN112183544A (en) | Double-channel fused three-layer architecture mathematical formula identification method, system and storage device | |
| CN114612832B (en) | Real-time gesture detection method and device | |
| US20180114071A1 (en) | Method for analysing media content | |
| CN110210485A (en) | The image, semantic dividing method of Fusion Features is instructed based on attention mechanism | |
| CN114020891A (en) | Video question answering method and system for dual-channel semantic localization, multi-granularity attention and mutual enhancement | |
| CN113011320B (en) | Video processing method, device, electronic equipment and storage medium | |
| CN113468978B (en) | Fine-grained car body color classification method, device and equipment based on deep learning | |
| CN114283352A (en) | Video semantic segmentation device, training method and video semantic segmentation method | |
| CN117173450A (en) | Traffic scene generation type image description method | |
| Agrawal et al. | Image caption generator using attention mechanism | |
| CN117370604A (en) | Video description generation method and system based on video space-time scene graph fusion reasoning | |
| CN115082915B (en) | A visual-language navigation method for mobile robots based on multi-modal features | |
| Huang et al. | Spatial–temporal context-aware online action detection and prediction | |
| CN108389239A (en) | A kind of smile face video generation method based on condition multimode network | |
| Li et al. | Dual attention convolutional network for action recognition | |
| CN115937641A (en) | Transformer-based joint coding method, device and equipment between modalities | |
| Zou et al. | 360° image saliency prediction by embedding self-supervised proxy task | |
| CN117541668A (en) | Virtual character generation method, device, equipment and storage medium | |
| Fazry et al. | Change detection of high-resolution remote sensing images through adaptive focal modulation on hierarchical feature maps | |
| CN111753859B (en) | Sample generation method, device and equipment | |
| CN119810702A (en) | Video data processing method, device, electronic device and readable storage medium | |
| CN119832596A (en) | Pedestrian re-recognition method and training method for optimizing fine granularity feature fusion | |
| Lin et al. | Region-based context enhanced network for robust multiple face alignment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |