技术领域technical field
本发明涉及一种车牌识别系统，具体的说是一种端到端车牌识别系统及其方法，属于图像识别技术领域。The invention relates to a license plate recognition system, in particular to an end-to-end license plate recognition system and a method thereof, and belongs to the technical field of image recognition.
背景技术Background technique
车牌识别LPR(License Plate Recognition)在许多实际应用中发挥着重要作用，如自动收费，交通执法，停车场访问控制和道路交通监控等。它具有从安全性到交通控制的各种潜在应用，近年来，车牌识别技术受到了广泛的关注和研究。License Plate Recognition (LPR) plays an important role in many practical applications, such as automatic toll collection, traffic enforcement, parking lot access control, and road traffic monitoring. It has various potential applications ranging from security to traffic control, and license plate recognition technology has received extensive attention and research in recent years.
商业化方案中，几个老牌的车牌识别公司都具有较好的商业化版本，应用也较为广泛，包括北京文通科技，北京易泊时代等，文通的技术源于清华的技术团队，研究早，商业化也比较完善，在车牌识别方案上应用广泛。成都的火眼臻视公司，做停车场卡口车牌识别，包括卡口相机，配套软件等，在停车场具有较高的市场占有率。北京精英智通科技主要针对智能泊车收费，其中车牌识别方案也具有较好的识别。目前市场上的车牌识别产品，大多采用检测，分割和识别联合的方案，这种方案依赖于分割的效果，对于图像模糊粘连处理有一定难度。In the commercialization plan, several old-fashioned license plate recognition companies have better commercial versions and are widely used, including Beijing Wentong Technology, Beijing Yibo Times, etc. Wentong's technology originated from the technical team of Tsinghua University, and the research was carried out early. Commercialization is also relatively complete, and it is widely used in license plate recognition solutions. Chengdu-based Huoyan Zhenshi Co., Ltd. is engaged in car park bayonet license plate recognition, including bayonet cameras, supporting software, etc., and has a high market share in the car park. Beijing Elite Zhitong Technology mainly focuses on intelligent parking charges, among which the license plate recognition scheme also has good recognition. At present, most of the license plate recognition products on the market use a combined solution of detection, segmentation and recognition. This solution depends on the effect of segmentation, and it is difficult to deal with image blurring and adhesion.
虽然车牌识别技术已得到很大发展，但是仍然存在难点。这些难点集中体现在以下几个方面：(1)由于光照、大雾、沙尘暴等天气影响，导致采集到的图像质较低，字符受到噪声干扰，造成部分目标图像被背景图像掩盖，降低车牌字符识别的准确率。(2)我国车牌的特殊性。(a)我国车牌的字符包括中文、数字和英文字母。由于中文字符的笔画较为复杂，在对中文字符进行二值化处理后，容易造成笔画模糊，从而导致错误的识别。同时在对车牌字符识别时，需要对英文字母和数字进行混合识别，受部分英文字母和数字字符相似的影响，会造成英文字母和数字的误识。(b)我国车牌识别颜色的多样性，比如蓝色白底、黄底黑字、黑底白字等，在利用车牌颜色的特征时需要考虑多种情况，增加了额外的工作量。Although license plate recognition technology has been greatly developed, there are still difficulties. These difficulties are mainly reflected in the following aspects: (1) Due to the influence of light, heavy fog, sandstorm and other weather, the quality of the collected images is low, and the characters are disturbed by noise, causing some target images to be covered by the background image, reducing the license plate characters. recognition accuracy. (2) The particularity of my country's license plates. (a) The characters of Chinese license plates include Chinese, numbers and English letters. Because the strokes of Chinese characters are relatively complex, after binarizing the Chinese characters, it is easy to cause blurred strokes, resulting in wrong recognition. At the same time, when recognizing the characters of the license plate, it is necessary to perform mixed recognition of English letters and numbers, which will cause misrecognition of English letters and numbers due to the similarity of some English letters and numbers. (b) The diversity of license plate recognition colors in my country, such as blue and white background, yellow background black characters, black background white characters, etc., need to consider various situations when using the characteristics of license plate color, which increases the extra workload.
因此本发明利用改进的多尺度检测模块提升检测速度和精度，再利用双向门控循环单元与基于神经网络的时序类分类算法优化识别网络，加快网络的收敛速度，提升模型识别准确率。Therefore, the present invention uses the improved multi-scale detection module to improve the detection speed and accuracy, and then uses the bidirectional gated cyclic unit and the neural network-based time series classification algorithm to optimize the recognition network, speed up the convergence speed of the network, and improve the model recognition accuracy.
发明内容SUMMARY OF THE INVENTION
本发明的目的是提供一种基于深度学习的端到端车牌识别系统及其方法，以改进YOLOv3为检测网络，BGRU+CTC为识别网络构建整体的识别系统模型，通过检测网络获取目标车牌的Anchor Box，将该Anchor Box的相关坐标参数，及目标区域的内容以矩阵的形式传至识别网络，利用已经训练好的识别模型对目标区域的车牌内容进行文本识别，最终输出车牌定位的准确率，Anchor Box 角点坐标，车牌识别准确率，系统识别时间等信息The purpose of the present invention is to provide an end-to-end license plate recognition system and method based on deep learning, using improved YOLOv3 as the detection network, BGRU+CTC as the recognition network to construct an overall recognition system model, and obtain the anchor of the target license plate through the detection network. Box, transmit the relevant coordinate parameters of the Anchor Box and the content of the target area to the recognition network in the form of a matrix, use the trained recognition model to perform text recognition on the license plate content in the target area, and finally output the accuracy of the license plate location, Anchor Box corner coordinates, license plate recognition accuracy, system recognition time and other information
本发明的目的是这样实现的：一种基于深度学习的端到端车牌识别系统，包括车牌定位模块，车牌识别模块；The purpose of the present invention is achieved in this way: an end-to-end license plate recognition system based on deep learning, including a license plate positioning module and a license plate recognition module;
所述的车牌定位模块，位于系统的数据入口，利用YOLOv3对原始含有车辆的图像进行初步的多尺度检测，获取目标车牌的Anchor Box和相应的坐标参数，再将目标区域的内容以矩阵的形式传至识别网络；The license plate location module is located at the data entry of the system. It uses YOLOv3 to perform preliminary multi-scale detection on the original image containing the vehicle, obtains the anchor box of the target license plate and the corresponding coordinate parameters, and then converts the content of the target area in the form of a matrix. to the identification network;
所述的车牌识别模块，在对输入数据进行统一标准的格式化后，以检测网络返回的相关数据为基础，利用BGRU+CTC的联合方案对获取的目标车牌中的文本信息进行识别，最终输出车牌定位的准确率，Anchor Box角点坐标，车牌识别准确率，目标图像大小，系统识别时间等信息。The license plate recognition module, after the input data is formatted with a unified standard, is based on the relevant data returned by the detection network, and uses the joint scheme of BGRU+CTC to identify the text information in the acquired target license plate, and finally output License plate positioning accuracy, Anchor Box corner coordinates, license plate recognition accuracy, target image size, system recognition time and other information.
一种基于深度学习的端到端车牌识别方法，包括车牌定位和车牌识别两部分；An end-to-end license plate recognition method based on deep learning, including license plate location and license plate recognition;
所述车牌定位包括以下步骤：The license plate positioning includes the following steps:
步骤1：利用爬虫技术从互联网，或部分开源数据集等中获取训练数据 DataSet；Step 1: Use crawler technology to obtain training data DataSet from the Internet or some open source data sets;
步骤2：使用开源数据标注工具对原始训练数据进行人工标注，生成相应的数据标记文件；Step 2: Use open source data labeling tools to manually label the original training data to generate corresponding data labeling files;
步骤3：利用python脚本将数据标记文件生成相应的txt标签和list文件；Step 3: Use the python script to generate the corresponding txt label and list file from the data tag file;
步骤4：扩展YOLOv3的多尺度检测功能，提升其对小物体的定位准确率，并修改其对应的cfg文件中的网络参数；Step 4: Expand the multi-scale detection function of YOLOv3, improve its positioning accuracy for small objects, and modify the network parameters in its corresponding cfg file;
步骤5：调整YOLOv3相关的配置文件中的参数，初始化所有的数据路径，在cfg文件中开启网络的训练模式；Step 5: Adjust the parameters in the configuration file related to YOLOv3, initialize all data paths, and enable the training mode of the network in the cfg file;
步骤6：随着迭代次数的增加，在网络接近收敛，各项参数基本稳定的时候停止训练，获取检测网络的模型权重；Step 6: With the increase of the number of iterations, stop training when the network is close to convergence and the parameters are basically stable, and obtain the model weight of the detection network;
步骤7：修改darknet.py文件，在其原本的基础上增加新功能，用以获取检测网络的参数计算量Bflops和帧率Fps等，并设置相应配置文件和模型权重的路径；Step 7: Modify the darknet.py file and add new functions on the basis of its original to obtain the parameter calculation amount Bflops and frame rate Fps of the detection network, and set the path of the corresponding configuration file and model weight;
步骤8：获取车牌定位的准确率，检测速度，Anchor Box角点坐标等，并将其传至识别网络；Step 8: Obtain the license plate positioning accuracy, detection speed, anchor box corner coordinates, etc., and transmit them to the recognition network;
所述车牌识别包括以下步骤：The license plate recognition includes the following steps:
步骤1：利用训练好的YOLOv3对原始训练集DataSet进行检测，并将获取的目标车牌图像裁剪并保存至指定文件夹；Step 1: Use the trained YOLOv3 to detect the original training set DataSet, and crop and save the obtained target license plate image to the specified folder;
步骤2：对裁剪好的车牌图像数据集进行人工数据标注，生成数据集，按照 9:1的比例分割新数据集，生成对应的训练集和验证集，并编写python脚本生成对应标签文件trainLabel.txt和vaidLabel.txt；Step 2: Perform manual data annotation on the cropped license plate image data set, generate a data set, divide the new data set according to the ratio of 9:1, generate the corresponding training set and verification set, and write a python script to generate the corresponding label file trainLabel. txt and voidLabel.txt;
步骤3：以BGRU+CTC的联合方案构建车牌识别模型；Step 3: Build a license plate recognition model with the joint scheme of BGRU+CTC;
步骤4：将数据进行预处理操作后，根据识别模型入口参数对数据进行格式化；Step 4: After preprocessing the data, format the data according to the entry parameters of the recognition model;
步骤5：调整模型训练的参数，修改模型存储路径，并设置批次间隔动态保存模型；Step 5: Adjust the parameters of model training, modify the model storage path, and set the batch interval to dynamically save the model;
步骤6：待网络模型基本收敛时，停止训练，获取识别模型的权重文件；Step 6: When the network model basically converges, stop training and obtain the weight file of the recognition model;
步骤7：修改识别网络模型输入接口，与检测网络的输出接口进行匹配，进一步融合检测，识别两个模块；Step 7: Modify the input interface of the recognition network model to match the output interface of the detection network, and further integrate the detection to identify the two modules;
步骤8：利用开源数据集Open-ITS测试识别系统性能，输出车牌定位的准确率，Anchor Box角点坐标，车牌识别准确率，目标图像大小，系统识别时间等信息。Step 8: Use the open source data set Open-ITS to test the performance of the recognition system, output the accuracy of license plate positioning, the coordinates of the corner points of the Anchor Box, the accuracy of license plate recognition, the size of the target image, and the system recognition time.
本发明采用以上技术方案与现有技术相比，具有以下技术效果：本发明扩展了YOLOv3的多尺度检测功能，改进识别网络，利用BGRU+CTC实现无需车牌字符分割的识别任务，其中，(1)本发明提出的模型可以在一次前向传播中完成车牌端到端的定位与识别任务，整个网络不需要依靠人工提取特征，全部可由深度神经网络自身进行学习，并进行精细化的自动提取操作，端到端的方式节约了车牌识别的时间，极大地提升了系统的识别效率；(2)模型联合车牌定位和识别功能，无需对车牌字符预先进行分割，再进行识别操作，简化了识别过程，提升了模型的识别效率；(3)本发明改进YOLOv3网络，并扩展多尺度检测的功能，细化定位精度，提升车牌定位效率；(4)本发明利用BGRU+CTC 改进识别网络完成对已定位车牌的无字符分割的识别任务，明显缩短训练时间，提升了网络的收敛速度和识别准确率。Compared with the prior art, the present invention adopts the above technical scheme, and has the following technical effects: the present invention expands the multi-scale detection function of YOLOv3, improves the recognition network, and utilizes BGRU+CTC to realize the recognition task that does not require license plate character segmentation, wherein, (1 ) The model proposed by the present invention can complete the end-to-end positioning and recognition task of the license plate in one forward propagation, the entire network does not need to rely on manual extraction of features, and all can be learned by the deep neural network itself, and perform refined automatic extraction operations, The end-to-end method saves the time of license plate recognition and greatly improves the recognition efficiency of the system; (2) the model combines the license plate positioning and recognition functions, and there is no need to segment the license plate characters in advance, and then perform the recognition operation, which simplifies the recognition process and improves the The recognition efficiency of the model is improved; (3) the present invention improves the YOLOv3 network, expands the function of multi-scale detection, refines the positioning accuracy, and improves the license plate positioning efficiency; (4) the present invention uses BGRU+CTC to improve the recognition network to complete the positioning of the license plate. The recognition task without character segmentation significantly shortens the training time and improves the convergence speed and recognition accuracy of the network.
附图说明Description of drawings
图1为本发明系统架构示意图。FIG. 1 is a schematic diagram of the system architecture of the present invention.
图2为本发明中车牌定位模型图。FIG. 2 is a model diagram of the license plate location in the present invention.
图3为本发明中车牌识别模型图。FIG. 3 is a model diagram of license plate recognition in the present invention.
图4为本发明中车牌识别模型参数图。FIG. 4 is a parameter diagram of the license plate recognition model in the present invention.
具体实施方式Detailed ways
下面结合附图对本发明的技术方案做进一步的详细说明：Below in conjunction with accompanying drawing, the technical scheme of the present invention is described in further detail:
为了使本发明的目的，技术方案及优点更加清楚明白，便于本领域普通技术人员理解和实施本发明，以下结合附图1--系统架构示意图，附图2—车牌定位模型图，附图3-车牌识别模型图，和附图4-车牌识别模型参数图，通过具体实施例对本发明进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。In order to make the purpose, technical solutions and advantages of the present invention clearer, and to facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the following is in conjunction with accompanying drawing 1—schematic diagram of system architecture, accompanying drawing 2—license plate location model diagram, accompanying drawing 3 - The license plate recognition model diagram, and the accompanying drawing 4 - the license plate recognition model parameter diagram, the present invention is further described in detail through specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.
一种基于深度学习的端到端车牌识别系统及方法，所述车牌识别系统包括车牌定位模块100和车牌识别模块200；An end-to-end license plate recognition system and method based on deep learning, the license plate recognition system includes a license plate location module 100 and a license plate recognition module 200;
车牌定位模块100包括以下步骤：The license plate location module 100 includes the following steps:
步骤101：在互联网上采取网络爬虫技术，利用python的scrapy框架以“车牌”为关键词进行大规模多线程的爬取，并对爬取的数据做清洗，整理和标注，另外在结合开源数据集CCPD，制作训练数据集DataSet；Step 101: Adopt web crawler technology on the Internet, use python's scrapy framework to perform large-scale multi-threaded crawling with "license plate" as the keyword, and clean, organize and label the crawled data, and combine open source data. Set CCPD to make a training data set DataSet;
步骤102：使用开源数据标注工具LabelImage对原始训练数据进行人工标注，生成相应的xml数据标记文件；Step 102: Use the open source data labeling tool LabelImage to manually label the original training data to generate a corresponding xml data labeling file;
步骤3：利用python脚本对xml数据标记文件进行批量处理和转换，生成相应的txt标签和list文件，并根据8:1:1的比例制作对应的训练，验证和测试集；Step 3: Use python script to batch process and convert xml data tag files, generate corresponding txt tags and list files, and make corresponding training, verification and test sets according to the ratio of 8:1:1;
步骤4：扩展YOLOv3的多尺度检测功能，将原有的三尺度检测扩展为四尺度检测，以增强网络的特征交互，提升目标检测的精度，并修改其对应的cfg 配置文件中的网络参数，网络结构如附图2所示，在YOLOv3特征交互层处，添加卷积操作，生成104*104，75通道的feature map，然后在此进行分类和位置回归；Step 4: Expand the multi-scale detection function of YOLOv3, and expand the original three-scale detection to four-scale detection to enhance the feature interaction of the network, improve the accuracy of target detection, and modify the network parameters in the corresponding cfg configuration file. The network structure is shown in Figure 2. At the YOLOv3 feature interaction layer, a convolution operation is added to generate a 104*104, 75-channel feature map, and then classification and position regression are performed here;
步骤5：调整YOLOv3相关的配置文件，将cfg/voc.data中class设置为1，初始化train，valid，names，backup文件的相关路径，将cfg/voc.names中的类别设置为plate，cfg/yolov3-voc.cfg的配置如步骤四中所示，再定义cfg中相关参数的初始值，batch＝64，subdivisions＝16，width＝416，height＝416，channels＝3， momentum＝0.9，decay＝0.0005，angle＝0，saturation＝1.5，exposure＝1.5，hue＝.1， learning_rate＝0.001，burn_in＝1000，max_batches＝50200，policy＝steps， steps＝40000,45000，scales＝.1,.1，最后，在cfg配置文件中开启网络的训练模式；Step 5: Adjust the YOLOv3 related configuration files, set the class in cfg/voc.data to 1, initialize the relevant paths of the train, valid, names, backup files, and set the category in cfg/voc.names to plate, cfg/ The configuration of yolov3-voc.cfg is shown in step 4, and then define the initial values of related parameters in cfg, batch=64, subdivisions=16, width=416, height=416, channels=3, momentum=0.9, decay= 0.0005, angle=0, saturation=1.5, exposure=1.5, hue=.1, learning_rate=0.001, burn_in=1000, max_batches=50200, policy=steps, steps=40000, 45000, scales=.1, .1, and finally , open the training mode of the network in the cfg configuration file;
步骤6：随着迭代次数的增加，在网络模型接近收敛时，各项参数基本稳定，此时停止训练，获取检测网络的模型权重yolov3-voc.backup；Step 6: With the increase of the number of iterations, when the network model is close to convergence, the parameters are basically stable, stop training at this time, and obtain the model weight yolov3-voc.backup of the detection network;
步骤7：修改darknet.py文件，在其原本的基础上增加新功能，用以获取检测网络的参数计算量Bflops和帧率Fps，平均AP值mAP，召回率Recall等，并在net和meta位置处设置相应配置文件和模型权重的路径；Step 7: Modify the darknet.py file and add new functions on the basis of its original to obtain the parameter calculation amount of the detection network Bflops and frame rate Fps, average AP value mAP, recall rate Recall, etc., and in the net and meta positions where to set the path of the corresponding configuration file and model weights;
步骤8：运行YOLOv3检测网络，获取车牌定位的准确率，检测速度，Anchor Box角点坐标等，并将其传至识别网络；Step 8: Run the YOLOv3 detection network to obtain the license plate positioning accuracy, detection speed, anchor box corner coordinates, etc., and transmit them to the recognition network;
车牌识别模块200包括以下步骤：The license plate recognition module 200 includes the following steps:
步骤1：利用训练好的YOLOv3网络对原始训练集DataSet进行检测，将获取的目标车牌图像裁剪并保存至指定文件夹；Step 1: Use the trained YOLOv3 network to detect the original training set DataSet, crop the obtained target license plate image and save it to the specified folder;
步骤2：对裁剪好的车牌图像数据集进行人工数据标注，生成数据集，按照9:1的比例分割新数据集，生成对应的训练集，验证集，并编写python脚本生成对应标签文件trainLabel.txt和vaidLabel.txt；Step 2: Perform manual data annotation on the cropped license plate image data set, generate a data set, divide the new data set according to the ratio of 9:1, generate the corresponding training set and verification set, and write a python script to generate the corresponding label file trainLabel. txt and voidLabel.txt;
步骤3：根据附图4中模型参数图，以BGRU+CTC的联合方案构建车牌识别模型，主要的输入参数为检测模型中返回的Anchor box的角点坐标，和转化为矩阵形式的目标车牌区域的信息，整体识别模块的结构图如附图2所示，识别流程主要分为基于CNN的特征提取，基于BGRU的序列标注，和基于CTC 的序列译码三大部分；Step 3: According to the model parameter diagram in Figure 4, a license plate recognition model is constructed with the joint scheme of BGRU+CTC. The main input parameters are the corner coordinates of the Anchor box returned in the detection model, and the target license plate area converted into a matrix form. The structure diagram of the overall recognition module is shown in Figure 2. The recognition process is mainly divided into three parts: feature extraction based on CNN, sequence annotation based on BGRU, and sequence decoding based on CTC;
步骤4：用滑动窗口的方式在整个图像中提取相应的特征。首先，对图像做一定的预处理，将每一个检测到的车牌转为灰度图像，然后将车牌的大小调整为164*48，其高度与CNN模型的输入高度相同，中文车牌是由7个字符组成，其中包含1位省份汉字简称和6位字母和数字的混合体，另外中国车牌共有31 个省份，混合体中包含10个阿拉伯数字和24个英文字母(O和I不计算在内), 故而全连接层共有65个类，而最终根据所需的位数，利用Softmax层做分类。经过序列识别网络的处理，全连接层输出了18*66维(含1位空格符)的特征向量，候选车牌图像上的特征也被从左到右被依次提取，本发明中该特征图从左往右依次和7个字符信息网络层相对应，并利用Dropout进行随机 (rate＝0.25)删除一些隐藏神经元，分类层输出的7位结果对应网络的识别结果；Step 4: Extract the corresponding features in the whole image in a sliding window manner. First, do some preprocessing on the image, convert each detected license plate into a grayscale image, and then adjust the size of the license plate to 164*48, its height is the same as the input height of the CNN model, the Chinese license plate is composed of 7 It is composed of characters, including 1 province Chinese character abbreviation and a mixture of 6 letters and numbers. In addition, there are 31 provinces in China, and the mixture includes 10 Arabic numerals and 24 English letters (O and I are not counted) , Therefore, the fully connected layer has a total of 65 classes, and finally, according to the required number of bits, the Softmax layer is used for classification. After the processing of the sequence recognition network, the fully connected layer outputs a feature vector of 18*66 dimensions (including 1 space character), and the features on the candidate license plate images are also extracted sequentially from left to right. In the present invention, the feature map starts from From left to right, it corresponds to the 7 character information network layers, and uses Dropout to delete some hidden neurons randomly (rate=0.25), and the 7-bit result output by the classification layer corresponds to the recognition result of the network;
步骤5：序列标注即对特征序列中的每一层特征都用GRU递归处理，每一次都会用一个非线性的函数g(.)来更新状态ht，该函数还会同时将当前特征xt和邻接状态ht-1或ht+1作为输入；Step 5: Sequence labeling is to use GRU to recursively process each layer of features in the feature sequence. Each time, a nonlinear function g(.) is used to update the state ht , which will also update the current feature xt at the same time. and the adjacent state ht-1 or ht+1 as input;
随后是Softmax层，其将GRU的状态转换为7个类的概率分布；This is followed by the Softmax layer, which transforms the state of the GRU into a probability distribution of 7 classes;
整个特征序列被最终转换为概率估计序列P＝{p1,p2,…,pL}，其长度与输入序列相等；The entire feature sequence is finally converted into a probability estimation sequence P={p1 ,p2 ,...,pL }, whose length is equal to the input sequence;
步骤6：序列译码即通过GRU的输出序列找到具有最大概率的近似最优路径，将概率估计P的序列转换为字符串；Step 6: Sequence decoding is to find the approximate optimal path with the maximum probability through the output sequence of the GRU, and convert the sequence of the probability estimate P into a string;
步骤7：将此前准备好的数据进行预处理操作后，根据识别模型入口参数对数据进行统一格式化；Step 7: After preprocessing the previously prepared data, format the data uniformly according to the entry parameters of the recognition model;
步骤8：调整模型训练的参数，batch_size＝128，img_size＝(164,48)， num_channels＝3，label_len＝7，samples_per_epoch＝800，nb_epoch＝25,修改模型存储路径，并设置在每一个nb_epoch结束时动态保存模型；Step 8: Adjust the parameters of model training, batch_size=128, img_size=(164,48), num_channels=3, label_len=7, samples_per_epoch=800, nb_epoch=25, modify the model storage path, and set it at the end of each nb_epoch Dynamically save the model;
步骤9：在参数变化趋近于稳定，网络模型基本收敛时，停止训练，并保存识别模型的权重文件BGRU_CTC_model.h5；Step 9: When the parameter changes tend to be stable and the network model basically converges, stop training and save the weight file BGRU_CTC_model.h5 of the recognition model;
步骤10：修改整体系统识别网络模型输入接口，将其与检测网络的输出接口进行匹配，进一步融合检测，识别两个模块；Step 10: Modify the input interface of the overall system identification network model, match it with the output interface of the detection network, and further integrate the detection to identify the two modules;
步骤11：利用开源数据集Open-ITS测试识别系统性能，输出车牌定位的准确率，Anchor Box角点坐标，车牌识别准确率，目标图像大小，系统识别时间等信息。Step 11: Use the open source data set Open-ITS to test the performance of the recognition system, and output the accuracy of license plate positioning, anchor box corner coordinates, license plate recognition accuracy, target image size, system recognition time and other information.
综上所述，本发明针对传统车牌识别系统检测速度慢，识别准确率低的问题，本发明提供了一种基于深度学习的端到端车牌识别系统及其方法，扩展多尺度检测功能，以改进YOLOv3的检测网络，加强对小物体的识别，提升定位准确率，以BGRU+CTC为识别网络构建整体的识别系统模型，加快了网络训练的收敛速度，提升了识别准确率。To sum up, the present invention aims at the problems of slow detection speed and low recognition accuracy of the traditional license plate recognition system. The present invention provides an end-to-end license plate recognition system and method based on deep learning, which expands the multi-scale detection function to Improve the detection network of YOLOv3, strengthen the recognition of small objects, improve the positioning accuracy, and build an overall recognition system model with BGRU+CTC as the recognition network, which speeds up the convergence speed of network training and improves the recognition accuracy.
以上所述，仅为本发明中的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉该技术的人在本发明所揭露的技术范围内，可理解想到的变换或替换，都应涵盖在本发明的包含范围之内，因此，本发明的保护范围应该以权利要求书的保护范围为准。The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this, any person familiar with the technology can understand the transformation or replacement that comes to mind within the technical scope disclosed by the present invention, All should be included within the scope of the present invention, therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910496601.9ACN110378331B (en) | 2019-06-10 | 2019-06-10 | An end-to-end license plate recognition system and method based on deep learning |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910496601.9ACN110378331B (en) | 2019-06-10 | 2019-06-10 | An end-to-end license plate recognition system and method based on deep learning |
| Publication Number | Publication Date |
|---|---|
| CN110378331Atrue CN110378331A (en) | 2019-10-25 |
| CN110378331B CN110378331B (en) | 2022-10-04 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910496601.9AActiveCN110378331B (en) | 2019-06-10 | 2019-06-10 | An end-to-end license plate recognition system and method based on deep learning |
| Country | Link |
|---|---|
| CN (1) | CN110378331B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110796144A (en)* | 2019-11-08 | 2020-02-14 | 北京字节跳动网络技术有限公司 | License plate detection method, device, equipment and storage medium |
| CN110837838A (en)* | 2019-11-06 | 2020-02-25 | 创新奇智(重庆)科技有限公司 | End-to-end frame number identification system and method based on deep learning |
| CN110991444A (en)* | 2019-11-19 | 2020-04-10 | 复旦大学 | Complex scene-oriented license plate recognition method and device |
| CN111444916A (en)* | 2020-03-26 | 2020-07-24 | 中科海微(北京)科技有限公司 | Method and system for license plate location and recognition under unconstrained conditions |
| CN111582263A (en)* | 2020-05-12 | 2020-08-25 | 上海眼控科技股份有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN111814565A (en)* | 2020-06-11 | 2020-10-23 | 北京微播易科技股份有限公司 | Target detection method and device |
| CN112116533A (en)* | 2020-08-06 | 2020-12-22 | 南京理工大学 | Vehicle logo detection method in haze weather |
| CN112132222A (en)* | 2020-09-27 | 2020-12-25 | 上海高德威智能交通系统有限公司 | License plate category identification method and device and storage medium |
| CN112149661A (en)* | 2020-08-07 | 2020-12-29 | 珠海欧比特宇航科技股份有限公司 | License plate recognition method, device and medium |
| CN112257720A (en)* | 2020-11-20 | 2021-01-22 | 北京捍御者信息科技有限公司 | Method, device and medium for improving automatic detection and recognition rate of Chinese characters of license plate |
| CN112329881A (en)* | 2020-11-20 | 2021-02-05 | 珠海大横琴科技发展有限公司 | License plate recognition model training method, license plate recognition method and device |
| CN113205108A (en)* | 2020-11-02 | 2021-08-03 | 哈尔滨理工大学 | YOLOv 4-based multi-target vehicle detection and tracking method |
| CN113392837A (en)* | 2021-07-09 | 2021-09-14 | 超级视线科技有限公司 | License plate recognition method and device based on deep learning |
| CN113850253A (en)* | 2021-08-24 | 2021-12-28 | 中国船舶重工集团公司第七0九研究所 | License plate recognition method and system, server and medium based on deep learning |
| CN114511899A (en)* | 2021-12-30 | 2022-05-17 | 武汉光庭信息技术股份有限公司 | A method, system, electronic device and storage medium for blurring street view video |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103309982A (en)* | 2013-06-17 | 2013-09-18 | 武汉大学 | Remote sensing image retrieval method based on vision saliency point characteristics |
| CN105761266A (en)* | 2016-02-26 | 2016-07-13 | 民政部国家减灾中心 | Method of extracting rectangular building from remote sensing image |
| CN106529600A (en)* | 2016-11-16 | 2017-03-22 | 桂林理工大学 | SVM-based recognition method of building angular points in high-resolution optical image |
| CN108596103A (en)* | 2018-04-26 | 2018-09-28 | 吉林大学 | High resolution ratio satellite remote-sensing image building extracting method based on optimal spectrum Index selection |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103309982A (en)* | 2013-06-17 | 2013-09-18 | 武汉大学 | Remote sensing image retrieval method based on vision saliency point characteristics |
| CN105761266A (en)* | 2016-02-26 | 2016-07-13 | 民政部国家减灾中心 | Method of extracting rectangular building from remote sensing image |
| CN106529600A (en)* | 2016-11-16 | 2017-03-22 | 桂林理工大学 | SVM-based recognition method of building angular points in high-resolution optical image |
| CN108596103A (en)* | 2018-04-26 | 2018-09-28 | 吉林大学 | High resolution ratio satellite remote-sensing image building extracting method based on optimal spectrum Index selection |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110837838A (en)* | 2019-11-06 | 2020-02-25 | 创新奇智(重庆)科技有限公司 | End-to-end frame number identification system and method based on deep learning |
| CN110796144A (en)* | 2019-11-08 | 2020-02-14 | 北京字节跳动网络技术有限公司 | License plate detection method, device, equipment and storage medium |
| CN110991444A (en)* | 2019-11-19 | 2020-04-10 | 复旦大学 | Complex scene-oriented license plate recognition method and device |
| CN110991444B (en)* | 2019-11-19 | 2023-08-29 | 复旦大学 | License plate recognition method and device for complex scene |
| CN111444916A (en)* | 2020-03-26 | 2020-07-24 | 中科海微(北京)科技有限公司 | Method and system for license plate location and recognition under unconstrained conditions |
| CN111582263A (en)* | 2020-05-12 | 2020-08-25 | 上海眼控科技股份有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN111814565A (en)* | 2020-06-11 | 2020-10-23 | 北京微播易科技股份有限公司 | Target detection method and device |
| CN111814565B (en)* | 2020-06-11 | 2024-06-18 | 北京微播易科技股份有限公司 | Target detection method and device |
| CN112116533A (en)* | 2020-08-06 | 2020-12-22 | 南京理工大学 | Vehicle logo detection method in haze weather |
| CN112149661B (en)* | 2020-08-07 | 2024-06-21 | 珠海欧比特宇航科技股份有限公司 | License plate recognition method, license plate recognition device and license plate recognition medium |
| CN112149661A (en)* | 2020-08-07 | 2020-12-29 | 珠海欧比特宇航科技股份有限公司 | License plate recognition method, device and medium |
| CN112132222B (en)* | 2020-09-27 | 2023-02-10 | 上海高德威智能交通系统有限公司 | Type recognition method, device and storage medium of license plate |
| CN112132222A (en)* | 2020-09-27 | 2020-12-25 | 上海高德威智能交通系统有限公司 | License plate category identification method and device and storage medium |
| CN113205108A (en)* | 2020-11-02 | 2021-08-03 | 哈尔滨理工大学 | YOLOv 4-based multi-target vehicle detection and tracking method |
| CN112329881A (en)* | 2020-11-20 | 2021-02-05 | 珠海大横琴科技发展有限公司 | License plate recognition model training method, license plate recognition method and device |
| CN112257720B (en)* | 2020-11-20 | 2024-02-27 | 北京捍御者信息科技有限公司 | Method, device and medium for improving automatic detection recognition rate of Chinese characters of license plate |
| CN112257720A (en)* | 2020-11-20 | 2021-01-22 | 北京捍御者信息科技有限公司 | Method, device and medium for improving automatic detection and recognition rate of Chinese characters of license plate |
| CN113392837A (en)* | 2021-07-09 | 2021-09-14 | 超级视线科技有限公司 | License plate recognition method and device based on deep learning |
| CN113850253A (en)* | 2021-08-24 | 2021-12-28 | 中国船舶重工集团公司第七0九研究所 | License plate recognition method and system, server and medium based on deep learning |
| CN114511899A (en)* | 2021-12-30 | 2022-05-17 | 武汉光庭信息技术股份有限公司 | A method, system, electronic device and storage medium for blurring street view video |
| Publication number | Publication date |
|---|---|
| CN110378331B (en) | 2022-10-04 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110378331B (en) | An end-to-end license plate recognition system and method based on deep learning | |
| Zhang et al. | CDNet: A real-time and robust crosswalk detection network on Jetson nano based on YOLOv5 | |
| Yi et al. | An improved tiny-yolov3 pedestrian detection algorithm | |
| CN109447994B (en) | Remote Sensing Image Segmentation Method Combining Complete Residual and Feature Fusion | |
| CN101944174B (en) | Identification method of characters of licence plate | |
| CN105956626A (en) | Deep learning based vehicle license plate position insensitive vehicle license plate recognition method | |
| CN107563372A (en) | A kind of license plate locating method based on deep learning SSD frameworks | |
| CN113505640B (en) | A small-scale pedestrian detection method based on multi-scale feature fusion | |
| KR20210080291A (en) | Method, electronic device, and storage medium for recognizing license plate | |
| CN104809481A (en) | Natural scene text detection method based on adaptive color clustering | |
| Zhang et al. | Joint distribution alignment via adversarial learning for domain adaptive object detection | |
| Zhang et al. | DetReco: Object‐Text Detection and Recognition Based on Deep Neural Network | |
| CN109086778A (en) | A kind of license plate deblurring recognition methods based on convolutional neural networks | |
| Wu et al. | Traffic sign detection based on SSD combined with receptive field module and path aggregation network | |
| Jin et al. | Vehicle license plate recognition for fog‐haze environments | |
| Tariq et al. | Real time vehicle detection and colour recognition using tuned features of Faster-RCNN | |
| Fan | Research and realization of video target detection system based on deep learning | |
| CN118397250A (en) | A generative zero-shot object detection method and system based on distilled CLIP model | |
| CN117611988A (en) | An automatic identification and monitoring method and system for newly added cultivated land management and protection attributes | |
| CN111539417B (en) | Text recognition training optimization method based on deep neural network | |
| CN112750128B (en) | Image semantic segmentation method, device, terminal and readable storage medium | |
| CN110321969B (en) | A car face alignment method based on MTCNN | |
| Wu et al. | Research on asphalt pavement disease detection based on improved YOLOv5s | |
| CN115953744A (en) | A vehicle recognition and tracking method based on deep learning | |
| Oublal et al. | An advanced combination of semi-supervised Normalizing Flow & Yolo (YoloNF) to detect and recognize vehicle license plates |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |