技术领域technical field
本发明属于图像处理技术领域，涉及一种基于深度学习的液晶数显仪表读数识别方法及系统。The invention belongs to the technical field of image processing, and relates to a method and system for recognizing readings of liquid crystal digital display instruments based on deep learning.
背景技术Background technique
液晶数显仪表因为直观性强、精度高、显示信息丰富等优点近年来被广泛应用于工业领域中，在考虑信息安全及成本等因素下，大多并不具有读取仪表数据的接口，无法实现仪表测量数据的自动采集和传输，而且改装后的数显式仪表，其密封性、精度等各项性能通常也达不到企业要求，所以用于工业领域的液晶数显仪表读数主要通过人工进行数据读取及记录。然而采用人工读数不可避免的会出现一些问题，如人的观测角度、距离及人的疲劳强度等一些人为因素会导致识别的仪表读数和真实读数有偏差，同时存在人工读数速度慢，数据记录易出错，长时间工作后工作效率会下降等缺点。Liquid crystal digital display instruments have been widely used in the industrial field in recent years because of their advantages such as strong intuitiveness, high precision, and rich display information. Considering factors such as information security and cost, most of them do not have an interface for reading instrument data and cannot be realized. The automatic collection and transmission of instrument measurement data, and the performance of the refitted digital display instrument usually does not meet the requirements of the enterprise, so the reading of the liquid crystal digital display instrument used in the industrial field is mainly done manually Data reading and recording. However, there will inevitably be some problems when using manual readings. Some human factors such as human observation angles, distances, and human fatigue will cause deviations between the recognized instrument readings and real readings. At the same time, manual readings are slow and data recording is easy. Mistakes, work efficiency will drop after working for a long time and other shortcomings.
此外，工业现场环境复杂，存在复杂背景、仪表倾斜、尺度变化、图像模糊、环境照明、面板污染等情形，因此采用传统液晶数显仪表读数识别算法对上述情形的液晶数显仪表读数识别较为困难，准确率及效率较低，且现有基于机器视觉的识别方法在仪表读数变化频率较高的情况下识别准确率较低。In addition, the industrial site environment is complex, there are situations such as complex background, instrument tilt, scale change, image blur, ambient lighting, panel pollution, etc. Therefore, it is difficult to use the traditional LCD digital display instrument reading recognition algorithm to identify the above-mentioned LCD digital display instrument readings , the accuracy and efficiency are low, and the existing recognition methods based on machine vision have low recognition accuracy when the frequency of instrument reading changes is high.
发明内容Contents of the invention
本发明的目的在于解决现有技术中的问题，提供一种基于深度学习的液晶数显仪表读数识别方法及系统，可以对多种复杂场景下的液晶数显仪表实现字符区域的快速准确检测和读数识别。The purpose of the present invention is to solve the problems in the prior art and provide a method and system for reading recognition of liquid crystal digital display instruments based on deep learning, which can realize rapid and accurate detection and detection of character areas for liquid crystal digital display instruments in various complex scenarios. Reading recognition.
为达到上述目的，本发明采用以下技术方案予以实现：In order to achieve the above object, the present invention adopts the following technical solutions to achieve:
一种基于深度学习的液晶数显仪表读数识别方法，包括以下步骤：A method for recognizing readings of liquid crystal digital display instruments based on deep learning, comprising the following steps:
采集液晶数显仪表图像，对液晶数显仪表图像进行预处理，应用透视变换图像处理技术实现液晶数显仪表图像的倾斜和旋转校正；Collect images of liquid crystal digital display instruments, preprocess the images of liquid crystal digital display instruments, and apply perspective transformation image processing technology to realize tilt and rotation correction of liquid crystal digital display instrument images;
基于倾斜和旋转校正后的液晶数显仪表图像，构建YOLOv4-tiny字符区域检测网络模型，获取字符串识别图像数据；Based on the image of the liquid crystal digital display instrument after tilt and rotation correction, a YOLOv4-tiny character area detection network model is constructed to obtain image data for character string recognition;
根据字符串识别图像数据构建CRNN字符串识别网络模型，完成液晶数显仪表读数识别。Based on the character string recognition image data, a CRNN character string recognition network model is constructed to complete the reading recognition of liquid crystal digital display instruments.
进一步的，所述液晶数显仪表图像的采集过程为：Further, the acquisition process of the liquid crystal digital display instrument image is:
调节液晶数显仪表与相机距离至所采集图像完整清晰，相机连接高性能计算机进行拍照，最后计算机对液晶数显仪表图像进行后续处理并对每幅图像给予编号。Adjust the distance between the liquid crystal digital display instrument and the camera until the collected images are complete and clear. The camera is connected to a high-performance computer to take pictures. Finally, the computer performs subsequent processing on the image of the liquid crystal digital display instrument and assigns a number to each image.
进一步的，所述液晶数显仪表图像的预处理过程为：Further, the preprocessing process of the liquid crystal digital display instrument image is:
得到液晶数显仪表图像后，按照以下公式将彩色图转为灰度图：After obtaining the image of the LCD digital display instrument, convert the color image into a grayscale image according to the following formula:
Y＝0.299R+0.587G+0.114BY＝0.299R+0.587G+0.114B
其中，Y为亮度，R、G、B分别为彩色图像红色、绿色、蓝色的分量；Among them, Y is the brightness, R, G, and B are the red, green, and blue components of the color image, respectively;
利用高斯滤波对处理过后的灰度图进行处理。The processed grayscale image is processed by Gaussian filtering.
进一步的，所述液晶数显仪表图像的倾斜和旋转校正过程为：Further, the tilt and rotation correction process of the liquid crystal digital display instrument image is:
对滤波后的液晶数显仪表图像进行Canny边缘检测，获取液晶数显仪表轮廓；Perform Canny edge detection on the filtered LCD digital display image to obtain the outline of the LCD digital display;
基于透视变换对预处理后的图像进行倾斜校正，透视变换的通用变换公式如下式所示：The preprocessed image is tilt-corrected based on the perspective transformation, and the general transformation formula of the perspective transformation is shown in the following formula:
其中，[x,y,z]T是变换前的坐标，[x′,y′,z′]T是变换后的坐标，矩阵是变换矩阵，其变换关系如下式所示：Among them, [x, y, z]T is the coordinates before transformation, [x′, y′, z′]T is the coordinates after transformation, The matrix is a transformation matrix, and its transformation relationship is shown in the following formula:
进一步的，所述获取字符串识别图像数据的过程包括：Further, the process of obtaining image data for character string recognition includes:
将倾斜和旋转校正后的液晶数显仪表图像划分为训练集和测试集，构建字符区域检测图像数据集；Divide the tilt and rotation corrected liquid crystal digital display instrument image into a training set and a test set, and construct a character area detection image data set;
构建YOLOv4-tiny字符区域检测网络；Construct YOLOv4-tiny character area detection network;
利用字符区域检测图像数据集训练YOLOv4-tiny字符区域检测网络，得到YOLOv4-tiny字符区域检测网络模型；Use the character area detection image data set to train the YOLOv4-tiny character area detection network, and obtain the YOLOv4-tiny character area detection network model;
将待检测液晶数显仪表图像输入YOLOv4-tiny字符区域检测网络模型，得到液晶数显仪表字符区域检测结果；Input the image of the liquid crystal digital display instrument to be detected into the YOLOv4-tiny character area detection network model, and obtain the character area detection result of the liquid crystal digital display instrument;
对得到的液晶数显仪表字符区域检测结果进行保存，构建字符串识别图像数据。Save the detection results of the character area of the liquid crystal digital display instrument, and construct the character string recognition image data.
进一步的，所述YOLOv4-tiny字符区域检测网络包括CSPdarknet53-tiny主干网络、FPN特征增强网络和预测特征层YOLO Head；Further, the YOLOv4-tiny character area detection network includes a CSPdarknet53-tiny backbone network, a FPN feature enhancement network and a prediction feature layer YOLO Head;
所述CSPdarknet53-tiny主干网络用于对输入的图像数据进行卷积操作，提取图像中特定目标的特征信息；The CSPdarknet53-tiny backbone network is used to perform a convolution operation on input image data to extract feature information of a specific target in the image;
所述FPN特征增强网络用于通过增加大目标的细节特征以提高后续检测过程中预测框与实际标注框的交并比；The FPN feature enhancement network is used to improve the intersection and union ratio of the prediction frame and the actual label frame in the subsequent detection process by increasing the detail features of the large target;
所述预测特征层YOLO Head用于利用得到的特征结果进行最后的预测。The prediction feature layer YOLO Head is used for final prediction using the obtained feature results.
进一步的，所述液晶数显仪表读数识别过程包括：Further, the reading recognition process of the liquid crystal digital display instrument includes:
对字符串识别图像数据进行标注，并将标注后的图像划分为训练集和测试集，构建字符串识别图像数据集；Annotate the string recognition image data, and divide the labeled image into a training set and a test set, and construct a string recognition image data set;
构建CRNN字符串识别网络，利用字符串识别图像数据集训练CRNN字符串识别网络，得到CRNN字符串识别网络模型；Construct the CRNN string recognition network, use the string recognition image data set to train the CRNN string recognition network, and obtain the CRNN string recognition network model;
将待识别液晶数显仪表字符区域图像输入CRNN字符串识别网络模型，得到液晶数显仪表字符串识别结果。Input the character area image of the liquid crystal digital display instrument to be recognized into the CRNN character string recognition network model, and obtain the character string recognition result of the liquid crystal digital display instrument.
进一步的，所述CRNN字符串识别网络包括CNN卷积层、RNN循环层和CTC转录层；Further, the CRNN character string identification network includes a CNN convolution layer, an RNN circulation layer and a CTC transcription layer;
所述CNN卷积层用于从输入图像中提取特征序列；The CNN convolutional layer is used to extract a feature sequence from an input image;
所述RNN循环层用于预测从卷积层获取的特征序列的标签分布；The RNN loop layer is used to predict the label distribution of the feature sequence obtained from the convolutional layer;
所述CTC转录层用于将从循环层获取的标签分布通过去重整合操作转换成最终的识别结果。The CTC transcription layer is used to convert the label distribution obtained from the recurrent layer into a final recognition result through a de-heavy integration operation.
一种基于深度学习的液晶数显仪表读数识别系统，包括：A reading recognition system for liquid crystal digital display meters based on deep learning, including:
图像采集模块，所述图像采集模块用于采集液晶数显仪表图像，对液晶数显仪表图像进行预处理，应用透视变换图像处理技术实现液晶数显仪表图像的倾斜和旋转校正；An image acquisition module, the image acquisition module is used to collect the image of the liquid crystal digital display instrument, preprocess the image of the liquid crystal digital display instrument, and apply the perspective transformation image processing technology to realize the tilt and rotation correction of the liquid crystal digital display instrument image;
数据获取模块，所述数据获取模块用于基于倾斜和旋转校正后的液晶数显仪表图像，构建YOLOv4-tiny字符区域检测网络模型，获取字符串识别图像数据；A data acquisition module, the data acquisition module is used to construct a YOLOv4-tiny character area detection network model based on the liquid crystal digital display instrument image after tilt and rotation correction, and obtain character string recognition image data;
读数识别模块，所述读数识别模块用于根据字符串识别图像数据构建CRNN字符串识别网络模型，完成液晶数显仪表读数识别。A reading recognition module, the reading recognition module is used to construct a CRNN character string recognition network model according to the character string recognition image data, and complete the liquid crystal digital display instrument reading recognition.
与现有技术相比，本发明具有以下有益效果：Compared with the prior art, the present invention has the following beneficial effects:
本发明提供一种基于深度学习的液晶数显仪表读数识别方法及系统，通过深度学习的深层网络模型提取液晶数显仪表字符特征，准确实现液晶数显仪表读数识别，提高目前液晶数显仪表读数识别的准确率及效率，改变传统人工检测与数据采集方式，同时减轻数据采集工作量与数据采集的人力需求，为工作人员提供仪表读数识别的有效技术支持，高效辅助工作人员完成数据采集作业。本发明与传统的液晶数显仪表读数识别算法相比，提高了液晶数显仪表读数识别的工作效率和准确率，可以对多种复杂场景下的液晶数显仪表实现字符区域的快速准确检测和读数识别。The present invention provides a method and system for recognizing the readings of liquid crystal digital display instruments based on deep learning. The character characteristics of liquid crystal digital display instruments are extracted through the deep network model of deep learning, so as to accurately realize the reading recognition of liquid crystal digital display instruments and improve the readings of current liquid crystal digital display instruments. The accuracy and efficiency of the recognition change the traditional manual detection and data collection methods, while reducing the workload of data collection and the manpower demand for data collection, providing effective technical support for the recognition of instrument readings for the staff, and efficiently assisting the staff to complete the data collection operation. Compared with the traditional liquid crystal digital display meter reading recognition algorithm, the present invention improves the working efficiency and accuracy of liquid crystal digital display meter reading recognition, and can realize rapid and accurate detection and detection of character areas for liquid crystal digital display meters in various complex scenarios. Reading recognition.
附图说明Description of drawings
为了更清楚的说明本发明实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.
图1为本发明的液晶数显仪表读数识别方法流程图。Fig. 1 is a flow chart of the reading recognition method of a liquid crystal digital display instrument of the present invention.
图2为本发明的液晶数显仪表字符区域检测流程图。Fig. 2 is a flow chart of character area detection of the liquid crystal digital display instrument of the present invention.
图3为本发明的YOLOv4-tiny模型结构图。Fig. 3 is a structure diagram of the YOLOv4-tiny model of the present invention.
图4为本发明的液晶数显仪表字符串识别流程图。Fig. 4 is a flow chart of the character string identification of the liquid crystal digital display instrument of the present invention.
图5为本发明的CRNN模型结构图。Fig. 5 is a structural diagram of the CRNN model of the present invention.
图6为本发明优选实施例基于深度学习的液晶数显仪表读数识别系统结构示意图。Fig. 6 is a schematic structural diagram of a reading recognition system for liquid crystal digital display meters based on deep learning in a preferred embodiment of the present invention.
图7为本发明优选实施例电子设备结构示意图。Fig. 7 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention.
具体实施方式Detailed ways
以下结合附图对本申请的示范性实施例做出说明，其中包括本申请实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本申请的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的全部其他实施例，都属于本申请保护的范围。Apparently, the described embodiments are some of the embodiments of the present application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.
需要说明的是，本申请实施例中所涉及的终端可以包括但不限于手机、个人数字助理(Personal Digital Assistant，PDA)、无线手持设备、平板电脑(Tablet Computer)、个人电脑(Personal Computer，PC)、MP3播放器、MP4播放器、可穿戴设备(例如，智能眼镜、智能手表、智能手环等)、智能家居设备等智能设备。It should be noted that the terminals involved in the embodiments of the present application may include, but are not limited to, mobile phones, personal digital assistants (Personal Digital Assistant, PDA), wireless handheld devices, tablet computers (Tablet Computer), personal computers (Personal Computer, PC ), MP3 players, MP4 players, wearable devices (eg, smart glasses, smart watches, smart bracelets, etc.), smart home devices and other smart devices.
另外，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。In addition, the term "and/or" in this article is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B may mean: A exists alone, A and B exist at the same time, There are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.
下面结合附图对本发明做进一步详细描述：The present invention is described in further detail below in conjunction with accompanying drawing:
本发明提供一种基于深度学习的液晶数显仪表读数识别方法，该方法能快速精准进行液晶数显仪表读数识别，将深度学习方法融入到液晶数显仪表读数识别中，由此解决传统的人工检测和数据采集强度大、效率低等问题，以及传统液晶数显仪表读数识别算法准确度及效率较低等技术问题。The invention provides a reading recognition method of liquid crystal digital display meters based on deep learning. There are problems such as high intensity and low efficiency of detection and data collection, as well as technical problems such as low accuracy and efficiency of the reading recognition algorithm of traditional liquid crystal digital display instruments.
参见图1，本发明提供的一种基于深度学习的液晶数显仪表读数识别方法，具体包括以下步骤：Referring to Fig. 1, a method for recognizing readings of liquid crystal digital display instruments based on deep learning provided by the present invention specifically includes the following steps:
步骤1：使用图像采集装置采集液晶数显仪表图像，并对每幅图像给予编号。Step 1: Use an image acquisition device to collect images of liquid crystal digital display instruments, and assign a number to each image.
图像采集装置包括CMOS高分辨率摄像头、三角支架、液晶显示屏、高性能计算机。实施例仅以图像采集装置包括CMOS高分辨率摄像头、三角支架、液晶显示屏、高性能计算机进行说明，但是其他的图像采集装置及其他执行环境能达到同样的技术效果。The image acquisition device includes a CMOS high-resolution camera, a tripod, a liquid crystal display, and a high-performance computer. The embodiment is only illustrated with the image acquisition device including a CMOS high-resolution camera, a tripod, a liquid crystal display, and a high-performance computer, but other image acquisition devices and other execution environments can achieve the same technical effect.
利用液晶显示屏模拟液晶数显仪表界面，摄像头与三角支架连接并调节至合适高度，调节液晶数显仪表界面与摄像头距离至所采集仪表图像至完整清晰，摄像头连接高性能计算机进行拍照，最后计算机对仪表图像进行后续处理。Use the LCD screen to simulate the interface of the LCD digital display instrument. The camera is connected to the tripod bracket and adjusted to a suitable height. Adjust the distance between the LCD digital display instrument interface and the camera until the collected instrument image is complete and clear. Subsequent processing of the gauge image.
步骤2：对所采集图像进行预处理，应用透视变换图像处理技术实现图像的倾斜和旋转校正。Step 2: Perform preprocessing on the collected images, and apply perspective transformation image processing technology to realize image tilt and rotation correction.
步骤2-1，得到液晶数显仪表图像后，为降低阴影和光照变化的影响，提高字符区域检测准确率，对输入的液晶数显仪表图像进行处理，按照下式将彩色图转为灰度图：Step 2-1, after obtaining the image of the LCD digital display instrument, in order to reduce the influence of shadow and light changes and improve the detection accuracy of the character area, process the input image of the LCD digital display instrument, and convert the color image into grayscale according to the following formula picture:
Y＝0.299R+0.587G+0.114BY＝0.299R+0.587G+0.114B
其中，Y为亮度，R、G、B分别为彩色图像红色、绿色、蓝色的分量；Among them, Y is the brightness, R, G, and B are the red, green, and blue components of the color image, respectively;
步骤2-2，为消除图像拍摄及分割可能的噪声，平滑图像边缘，提高图像质量，利用高斯滤波对处理过后的灰度图进行处理；Step 2-2, in order to eliminate possible noise in image capture and segmentation, smooth image edges, improve image quality, and use Gaussian filtering to process the processed grayscale image;
步骤2-3，对滤波后的图像进行Canny边缘检测，获取液晶数显仪表轮廓；Step 2-3, performing Canny edge detection on the filtered image to obtain the outline of the liquid crystal digital display instrument;
步骤2-4，基于透视变换对预处理后的图像进行倾斜校正，透视变换的通用变换公式如下式所示：In steps 2-4, the preprocessed image is tilt-corrected based on the perspective transformation, and the general transformation formula of the perspective transformation is shown in the following formula:
其中，[x,y,z]T是变换前的坐标，[x′,y′,z′]T是变换后的坐标，a矩阵是变换矩阵，其变换关系如下式所示：Among them, [x, y, z]T is the coordinates before transformation, [x′, y′, z′]T is the coordinates after transformation, a matrix is the transformation matrix, and the transformation relationship is shown in the following formula:
步骤3：基于倾斜和旋转校正后的液晶数显仪表图像，构建YOLOv4-tiny字符区域检测网络模型，获取字符串识别图像数据。Step 3: Based on the image of the liquid crystal digital display instrument after tilt and rotation correction, construct a YOLOv4-tiny character area detection network model to obtain image data for character string recognition.
步骤3-1，如图2所示，对预处理之后的液晶数显仪表图像数据进行标注，并将标注后的图像按照训练集70％、测试集30％的比例进行划分，构建字符区域检测图像数据集；Step 3-1, as shown in Figure 2, annotate the image data of the LCD digital display instrument after preprocessing, divide the annotated image according to the ratio of 70% of the training set and 30% of the test set, and construct character area detection image dataset;
步骤3-2，构建YOLOv4-tiny字符区域检测网络，包括CSPdarknet53-tiny主干网络、FPN特征增强网络和预测特征层YOLO Head；Step 3-2, construct YOLOv4-tiny character area detection network, including CSPdarknet53-tiny backbone network, FPN feature enhancement network and prediction feature layer YOLO Head;
CSPdarknet53-tiny主干网络用于对输入的图像数据进行卷积操作，提取图像中特定目标的特征信息；The CSPdarknet53-tiny backbone network is used to perform convolution operations on the input image data to extract feature information of specific targets in the image;
FPN特征增强网络用于通过增加大目标的细节特征以提高后续检测过程中预测框与实际标注框的交并比；The FPN feature enhancement network is used to improve the intersection ratio of the prediction frame and the actual label frame in the subsequent detection process by increasing the detailed features of the large target;
预测特征层YOLO Head用于利用得到的特征结果进行最后的预测。The prediction feature layer YOLO Head is used to make final predictions using the obtained feature results.
步骤3-3，利用字符区域检测图像数据集训练YOLOv4-tiny字符区域检测网络，得到YOLOv4-tiny字符区域检测网络模型，如图3所示；Step 3-3, using the character region detection image data set to train the YOLOv4-tiny character region detection network to obtain the YOLOv4-tiny character region detection network model, as shown in Figure 3;
步骤3-4，将待检测液晶数显仪表图像输入YOLOv4-tiny字符区域检测网络模型，得到液晶数显仪表字符区域检测结果；Step 3-4, input the image of the liquid crystal digital display instrument to be detected into the YOLOv4-tiny character area detection network model, and obtain the character area detection result of the liquid crystal digital display instrument;
步骤3-5，对得到的液晶数显仪表字符区域检测结果进行保存，构建字符串识别图像数据。In steps 3-5, the obtained liquid crystal digital display instrument character area detection results are saved, and character string recognition image data is constructed.
步骤4：根据字符串识别图像数据构建CRNN字符串识别网络模型，完成液晶数显仪表读数识别。Step 4: Construct the CRNN character string recognition network model based on the character string recognition image data, and complete the reading recognition of the liquid crystal digital display instrument.
步骤4-1，如图4所示，对字符串识别图像数据标注，并将标注后的图像按照训练集70％、测试集30％的比例进行划分，构建字符串识别图像数据集；Step 4-1, as shown in Figure 4, label the string recognition image data, and divide the marked images according to the ratio of 70% of the training set and 30% of the test set to construct a string recognition image data set;
步骤4-2，构建CRNN字符串识别网络，包括CNN卷积层、RNN循环层和CTC转录层；Step 4-2, constructing a CRNN character string recognition network, including a CNN convolutional layer, an RNN recurrent layer, and a CTC transcriptional layer;
CNN卷积层用于从输入图像中提取特征序列；CNN convolutional layers are used to extract feature sequences from input images;
RNN循环层用于预测从卷积层获取的特征序列的标签分布；The RNN recurrent layer is used to predict the label distribution of the feature sequence obtained from the convolutional layer;
CTC转录层用于将从循环层获取的标签分布通过去重整合操作转换成最终的识别结果。The CTC transcription layer is used to convert the label distribution obtained from the recurrent layer into the final recognition result through the de-heavy integration operation.
步骤4-3，利用字符串识别图像数据集训练CRNN字符串识别网络，得到CRNN字符串识别网络模型，如图5所示；Step 4-3, using the character string recognition image data set to train the CRNN character string recognition network to obtain the CRNN character string recognition network model, as shown in Figure 5;
步骤4-4，将待识别液晶数显仪表字符区域图像输入CRNN字符串识别网络，得到液晶数显仪表字符串识别结果，即液晶数显仪表读数。Step 4-4, input the image of the character area of the liquid crystal digital display instrument to be recognized into the CRNN character string recognition network, and obtain the character string recognition result of the liquid crystal digital display instrument, that is, the reading of the liquid crystal digital display instrument.
本发明还提供一种基于深度学习的液晶数显仪表读数识别系统，如图6所示，该系统包括：图像采集模块、数据获取模块和读数识别模块。The present invention also provides a reading recognition system for liquid crystal digital display instruments based on deep learning. As shown in FIG. 6 , the system includes: an image acquisition module, a data acquisition module and a reading recognition module.
图像采集模块，所述图像采集模块用于采集液晶数显仪表图像，对液晶数显仪表图像进行预处理，应用透视变换图像处理技术实现液晶数显仪表图像的倾斜和旋转校正；An image acquisition module, the image acquisition module is used to collect the image of the liquid crystal digital display instrument, preprocess the image of the liquid crystal digital display instrument, and apply the perspective transformation image processing technology to realize the tilt and rotation correction of the liquid crystal digital display instrument image;
数据获取模块，所述数据获取模块用于基于倾斜和旋转校正后的液晶数显仪表图像，构建YOLOv4-tiny字符区域检测网络模型，获取字符串识别图像数据；A data acquisition module, the data acquisition module is used to construct a YOLOv4-tiny character area detection network model based on the liquid crystal digital display instrument image after tilt and rotation correction, and obtain character string recognition image data;
读数识别模块，所述读数识别模块用于根据字符串识别图像数据构建CRNN字符串识别网络模型，完成液晶数显仪表读数识别。A reading recognition module, the reading recognition module is used to construct a CRNN character string recognition network model according to the character string recognition image data, and complete the liquid crystal digital display instrument reading recognition.
可以理解的是，本发明提供的基于深度学习的液晶数显仪表读数识别系统与前述提供的基于深度学习的液晶数显仪表读数识别方法相对应，基于深度学习的液晶数显仪表读数识别系统的相关技术特征可参考基于深度学习的液晶数显仪表读数识别方法的相关技术特征，在此不再赘述。It can be understood that the deep learning-based liquid crystal digital display meter reading recognition system provided by the present invention corresponds to the aforementioned deep learning-based liquid crystal digital display meter reading recognition method, and the liquid crystal digital display meter reading recognition system based on deep learning For relevant technical features, please refer to the relevant technical features of the method for recognizing readings of liquid crystal digital display meters based on deep learning, and details will not be repeated here.
如图7所示，本发明另一目的是提供一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述基于深度学习的液晶数显仪表读数识别方法的步骤。As shown in FIG. 7, another object of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, and the processor executes the The steps of the reading recognition method of liquid crystal digital display instrument based on deep learning.
所述基于深度学习的液晶数显仪表读数识别方法包括以下步骤：The liquid crystal digital display meter reading recognition method based on deep learning comprises the following steps:
采集液晶数显仪表图像，对液晶数显仪表图像进行预处理，应用透视变换图像处理技术实现液晶数显仪表图像的倾斜和旋转校正；Collect images of liquid crystal digital display instruments, preprocess the images of liquid crystal digital display instruments, and apply perspective transformation image processing technology to realize tilt and rotation correction of liquid crystal digital display instrument images;
基于倾斜和旋转校正后的液晶数显仪表图像，构建YOLOv4-tiny字符区域检测网络模型，获取字符串识别图像数据；Based on the image of the liquid crystal digital display instrument after tilt and rotation correction, a YOLOv4-tiny character area detection network model is constructed to obtain image data for character string recognition;
根据字符串识别图像数据构建CRNN字符串识别网络模型，完成液晶数显仪表读数识别。Based on the character string recognition image data, a CRNN character string recognition network model is constructed to complete the reading recognition of liquid crystal digital display instruments.
本发明第三个目的是提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现所述基于深度学习的液晶数显仪表读数识别方法的步骤。The third object of the present invention is to provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the recognition of the reading of the liquid crystal digital display instrument based on deep learning is realized method steps.
所述基于深度学习的液晶数显仪表读数识别方法包括以下步骤：The liquid crystal digital display meter reading recognition method based on deep learning comprises the following steps:
采集液晶数显仪表图像，对液晶数显仪表图像进行预处理，应用透视变换图像处理技术实现液晶数显仪表图像的倾斜和旋转校正；Collect images of liquid crystal digital display instruments, preprocess the images of liquid crystal digital display instruments, and apply perspective transformation image processing technology to realize tilt and rotation correction of liquid crystal digital display instrument images;
基于倾斜和旋转校正后的液晶数显仪表图像，构建YOLOv4-tiny字符区域检测网络模型，获取字符串识别图像数据；Based on the image of the liquid crystal digital display instrument after tilt and rotation correction, a YOLOv4-tiny character area detection network model is constructed to obtain image data for character string recognition;
根据字符串识别图像数据构建CRNN字符串识别网络模型，完成液晶数显仪表读数识别。Based on the character string recognition image data, a CRNN character string recognition network model is constructed to complete the reading recognition of liquid crystal digital display instruments.
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.
最后应当说明的是：以上实施例仅用以说明本发明的技术方案而非对其限制，尽管参照上述实施例对本发明进行了详细的说明，所属领域的普通技术人员应当理解：依然可以对本发明的具体实施方式进行修改或者等同替换，而未脱离本发明精神和范围的任何修改或者等同替换，其均应涵盖在本发明的权利要求保护范围之内。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the present invention can still be Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall fall within the protection scope of the claims of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310715202.3ACN116665227A (en) | 2023-06-15 | 2023-06-15 | A method and system for reading recognition of liquid crystal digital display instruments based on deep learning |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310715202.3ACN116665227A (en) | 2023-06-15 | 2023-06-15 | A method and system for reading recognition of liquid crystal digital display instruments based on deep learning |
| Publication Number | Publication Date |
|---|---|
| CN116665227Atrue CN116665227A (en) | 2023-08-29 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310715202.3APendingCN116665227A (en) | 2023-06-15 | 2023-06-15 | A method and system for reading recognition of liquid crystal digital display instruments based on deep learning |
| Country | Link |
|---|---|
| CN (1) | CN116665227A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117078911A (en)* | 2023-08-30 | 2023-11-17 | 广东电网有限责任公司 | A substation meter identification method, device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10262235B1 (en)* | 2018-02-26 | 2019-04-16 | Capital One Services, Llc | Dual stage neural network pipeline systems and methods |
| CN112836748A (en)* | 2021-02-02 | 2021-05-25 | 太原科技大学 | A Character Recognition Method for Casting Identification Based on CRNN-CTC |
| CN114549981A (en)* | 2022-02-11 | 2022-05-27 | 国网河南省电力公司电力科学研究院 | A deep learning-based intelligent inspection pointer meter identification and reading method |
| CN114898200A (en)* | 2022-05-25 | 2022-08-12 | 西安建筑科技大学 | A method and system for image target detection of conveyor belt workpieces based on lightweight YOLOV4-tiny |
| CN115410087A (en)* | 2022-08-30 | 2022-11-29 | 南京航空航天大学 | Transmission line foreign matter detection method based on improved YOLOv4 |
| CN115588207A (en)* | 2022-10-13 | 2023-01-10 | 成都卓视智通科技有限公司 | Monitoring video date recognition method based on OCR |
| CN116206297A (en)* | 2022-09-16 | 2023-06-02 | 上海可深信息科技有限公司 | Video stream real-time license plate recognition system and method based on cascade neural network |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10262235B1 (en)* | 2018-02-26 | 2019-04-16 | Capital One Services, Llc | Dual stage neural network pipeline systems and methods |
| CN112836748A (en)* | 2021-02-02 | 2021-05-25 | 太原科技大学 | A Character Recognition Method for Casting Identification Based on CRNN-CTC |
| CN114549981A (en)* | 2022-02-11 | 2022-05-27 | 国网河南省电力公司电力科学研究院 | A deep learning-based intelligent inspection pointer meter identification and reading method |
| CN114898200A (en)* | 2022-05-25 | 2022-08-12 | 西安建筑科技大学 | A method and system for image target detection of conveyor belt workpieces based on lightweight YOLOV4-tiny |
| CN115410087A (en)* | 2022-08-30 | 2022-11-29 | 南京航空航天大学 | Transmission line foreign matter detection method based on improved YOLOv4 |
| CN116206297A (en)* | 2022-09-16 | 2023-06-02 | 上海可深信息科技有限公司 | Video stream real-time license plate recognition system and method based on cascade neural network |
| CN115588207A (en)* | 2022-10-13 | 2023-01-10 | 成都卓视智通科技有限公司 | Monitoring video date recognition method based on OCR |
| Title |
|---|
| 张振宇;姜贺云;樊明宇;: "一种面向银行票据文字自动化识别的高效人工智能方法", 温州大学学报(自然科学版), no. 03, 31 August 2020 (2020-08-31)* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117078911A (en)* | 2023-08-30 | 2023-11-17 | 广东电网有限责任公司 | A substation meter identification method, device, electronic equipment and storage medium |
| Publication | Publication Date | Title |
|---|---|---|
| JP7331171B2 (en) | Methods and apparatus for training image recognition models, methods and apparatus for recognizing images, electronic devices, storage media, and computer programs | |
| CN112528977B (en) | Target detection method, target detection device, electronic equipment and storage medium | |
| CN111444781B (en) | Water meter reading identification method, device and storage medium | |
| WO2024002187A1 (en) | Defect detection method, defect detection device, and storage medium | |
| CN110782390B (en) | Image correction processing method and device, and electronic equipment | |
| CN109816636B (en) | A crack detection method based on intelligent terminal | |
| CN109872309A (en) | Detection system, method, apparatus, and computer-readable storage medium | |
| CN112560754A (en) | Bill information acquisition method, device, equipment and storage medium | |
| CN107895377B (en) | Foreground target extraction method, device, equipment and storage medium | |
| CN116721104B (en) | Real-life three-dimensional model defect detection method, device, electronic equipment and storage medium | |
| WO2020248848A1 (en) | Intelligent abnormal cell determination method and device, and computer readable storage medium | |
| CN115690500A (en) | Based on improve U 2 Network instrument identification method | |
| CN116071315A (en) | Product visual defect detection method and system based on machine vision | |
| CN112991218A (en) | Image processing method, device, equipment and storage medium | |
| CN110930393A (en) | Chip material pipe counting method, device and system based on machine vision | |
| CN115527089A (en) | Yolo-based target detection model training method and its application and device | |
| CN116665227A (en) | A method and system for reading recognition of liquid crystal digital display instruments based on deep learning | |
| CN104655041B (en) | A kind of industrial part contour line multi-feature extraction method of additional constraint condition | |
| CN110210401B (en) | Intelligent target detection method under weak light | |
| CN115937324A (en) | Assembly quality evaluation method, device, equipment and storage medium | |
| CN119180811B (en) | Model training and defect detection method, device, equipment and readable storage medium | |
| CN109508714B (en) | Low-cost multi-channel real-time digital instrument panel visual identification method and system | |
| CN118506495B (en) | Self-help borrowing and returning method, system, equipment and storage medium for books | |
| CN117912085B (en) | Model training method, face key point positioning method, device, equipment and medium | |
| CN110298347B (en) | Method for identifying automobile exhaust analyzer screen based on GrayWorld and PCA-CNN |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |