CN112906699A

Movatterモバイル変換

Info

Publication number: CN112906699A
Application number: CN202011532814.1A
Authority: CN
Inventors: 刘毛溪; 梁添才; 赵清利; 徐天适; 潘新生
Original assignee: Shenzhen Xinyi Technology Co Ltd
Current assignee: Shenzhen Radio & Tv Xinyi Technology Co ltd
Priority date: 2020-12-23
Filing date: 2020-12-23
Publication date: 2021-06-04
Anticipated expiration: 2040-12-23
Also published as: CN112906699B

Abstract

Translated fromChinese

本发明属于智能交通领域，为车牌放大号的检测识别方法，包括：对车牌放大号所在区域进行检测定位，得到原始车牌放大号的样本图像；基于深度卷积的识别网络对车牌放大号字符进行识别，在训练阶段对原始车牌放大号的样本图像进行扩展获得训练样本集，然后构建识别网络并对实际的车牌放大号图像进行特征提取，得到最终的文本识别结果。本发明对识别网络在训练过程中文本识别部分和超分辨图像重构部分产生的损失进行加权计算，提升识别网络对低质量图像的特征表达能力，得到最优化的权重参数，降低训练样本集创建难度的同时，有效提高了车牌字符的识别效率，解决了车牌放大号的字符检测受字符大小、样式及间距不一致等影响而效果不佳的问题。

The invention belongs to the field of intelligent transportation, and relates to a method for detecting and identifying an enlarged license plate number. In the training phase, the sample image of the original license plate magnification is expanded to obtain a training sample set, and then a recognition network is constructed and the actual license plate magnification image is extracted to obtain the final text recognition result. The invention performs weighted calculation on the loss generated by the text recognition part and the super-resolution image reconstruction part in the training process of the recognition network, improves the feature expression ability of the recognition network for low-quality images, obtains optimized weight parameters, and reduces the creation of training sample sets. At the same time, it effectively improves the recognition efficiency of license plate characters, and solves the problem that the character detection of the enlarged license plate is affected by the inconsistency of character size, style and spacing, and the effect is not good.

Description

Method for detecting and identifying enlarged number of license plate

Technical Field

The invention belongs to the field of intelligent transportation, and particularly relates to a method for detecting and identifying an enlarged license plate number.

Background

The license plate recognition technology is a key and core module of an intelligent traffic system, and the detection and recognition technology of the enlarged license plate number can technically promote the existing license plate recognition technology in the following three aspects:

1. the number plate related to the existing number plate recognition technology is mainly a standard motor vehicle number plate, but relates to little or little number of the number plate amplified numbers (namely the amplified number plate numbers), and the thirteenth regulation of the national road traffic safety law enforcement regulations: the heavy and medium-sized truck and the trailer thereof, the tractor and the trailer thereof, the body or the rear part of the carriage of the tractor and the trailer thereof should be sprayed with enlarged marks, and characters should be aligned and kept clear. Therefore, the method for detecting and identifying the enlarged license plate number can supplement the existing license plate identification technology and improve the functional integrity of the license plate identification technology.

2. Due to the influences of the working environment, the driving road condition and the suspension position of the number plate of the vehicle, the situations of the number plate missing, blocking, fouling, blurring, overexposure and the like may exist in the images of the passenger and freight vehicles captured by the camera, and the detection and identification effects of the number plate identification technology on the standard number plate in the images of the passenger and freight vehicles captured by the tail are further influenced. Compared with a standard license plate, the license plate enlarged number is larger in size and more obvious in position, and the license plate enlarged number can be clearly captured even if the standard license plate is fuzzy.

3. The existing license plate recognition technology is directly applied to recognition of the enlarged license plate number, and the effect is poor. The main reasons are that: the spraying position of the license plate enlarged number is generally the rear part of a passenger and freight vehicle carriage, the background of the license plate enlarged number is not uniform, and more interference exists; the font size, font style and character spacing of the license plate enlarged number spray coating are inconsistent.

Disclosure of Invention

In order to solve the technical problems in the prior art, the invention provides a detection and recognition method for a license plate enlarged number, which effectively improves the recognition efficiency of license plate characters while reducing the difficulty in creating a training sample set by improving the sample labeling efficiency, performs weighted calculation and analysis on the losses of a text recognition part and a super-resolution image reconstruction part in the training process, and improves the feature extraction effect of a network model on low-quality images, thereby effectively solving the technical problem that the detection of the characters of the license plate enlarged number is not good in effect due to the influences of inconsistent character sizes, patterns and intervals and the like.

The invention is realized by adopting the following technical scheme: a method for detecting and identifying an enlarged number of a license plate comprises the following steps:

s1, detecting and positioning the area where the license plate enlarged number is located to obtain a sample image of the original license plate enlarged number;

s2, recognizing the license plate enlarged number characters based on a deep convolution recognition network;

step S2 includes:

s21, in the training stage of the recognition network model, expanding the sample image of the original license plate enlarged number to obtain a training sample set;

s22, constructing a recognition network based on the expanded training sample set, and performing feature extraction on the actual license plate enlarged number image by using the constructed recognition network to obtain a final text recognition result; and performing weighted calculation analysis on losses generated by the text recognition part and the super-resolution image reconstruction part of the recognition network in the training process, so as to improve the feature expression capability of the recognition network on low-quality images and obtain optimized weight parameters of the recognition network.

Compared with the prior art, the invention has the following advantages and beneficial effects:

1. the license plate is the most distinctive feature of a vehicle, compared with a standard motor vehicle license plate, the license plate enlarged number has no fixed background, the length-width ratio is greatly changed, the fonts and the sizes of characters are diversified, and the character intervals are inconsistent, so that the existing license plate recognition technology is difficult to be directly applied to recognition of the license plate enlarged number.

Aiming at the characteristics of the license plate enlarged number, the invention designs a license plate enlarged number detection and recognition algorithm based on a deep convolutional network. According to the algorithm, in the network model training process, through a mode of increasing sample image diversity, weighted calculation analysis is carried out on losses generated by a text recognition part and a super-resolution image reconstruction part in the training process, the characteristic extraction effect of the network model on low-quality images is improved, the characteristics of the license plate enlarged number can be well learned, and the algorithm has the characteristics of strong robustness and high applicability.

2. The existing mainstream license plate recognition technology mainly comprises the following processes: the three processes of license plate detection and positioning, license plate character segmentation and positioning and license plate character classification and recognition are carried out, three models of license plate detection, character segmentation and character recognition need to be constructed respectively, time and resources are consumed, and when a training sample set is established in a model training stage, single-character position labeling needs to be carried out on a complete license plate sample, so that too much labor is consumed.

In the license plate amplified number recognition process, the invention simplifies the flow of the license plate recognition technology, provides an end-to-end license plate amplified number recognition algorithm to replace the existing license plate character segmentation positioning algorithm and the existing character classification recognition algorithm, and improves the recognition efficiency; the diversity of sample images is enriched through data preprocessing, data enhancement and other modes, and the robustness of the algorithm is improved; by reducing the labeling process and improving the labeling efficiency, the difficulty in creating the training sample set is reduced.

Drawings

FIG. 1 is a schematic diagram of a license plate information detection and identification process in an embodiment of the present invention;

FIG. 2 is an illustration of an enlarged license plate position in a captured image of a vehicle according to an embodiment of the present invention;

FIG. 3 is a diagram of a license plate number amplification detection network structure in an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating a training sample image expansion process according to an embodiment of the present invention;

FIG. 5 is an exemplary diagram of a positive sample image of a license plate with an enlarged number, wherein three sub-images (a), (b), and (c) respectively illustrate one form of the positive sample image;

FIG. 6 is a schematic structural diagram of a license plate amplification number recognition network in a training phase in the embodiment of the invention;

FIG. 7 is a schematic diagram of a CNN layer structure of a license plate number amplification identification network in an embodiment of the present invention;

FIG. 8 is a schematic diagram of an RNN layer structure of a license plate number-enlarging recognition network according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of an SR layer structure of a license plate amplified number recognition network in the embodiment of the present invention;

FIG. 10 is a schematic diagram of the RG module structure of the SR layer;

FIG. 11 is a schematic diagram of the RCAB sub-module in the RG module of the SR layer;

FIG. 12 is a schematic structural diagram of a license plate amplification number recognition network in an inference stage in the embodiment of the invention.

Detailed Description

The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.

Examples

The embodiment provides a method for detecting and identifying an enlarged license plate number based on a deep neural network, which mainly comprises the steps of detecting and identifying the enlarged license plate number as shown in figure 1, wherein the position label information of the enlarged license plate number is shown in figure 2; the detailed steps are as follows:

and S1, detecting and positioning the area where the license plate enlarged number is located, and obtaining a sample image of the original license plate enlarged number.

In the step, a detection network (such as a convolutional neural network) based on deep convolution is used for detecting and positioning the enlarged license plate number, and a lightweight network architecture MobileNet-SSD is taken as an example for detailed description:

s11, firstly, according to the label sample data distribution in the training sample, calculating the generation parameters of each layer of default box by using a k-means clustering algorithm (k-means clustering, k-means) of a convolutional neural network (YOLOv 3). Because the license plate amplification number sample image generally has a large width-to-height ratio, the size of the input image of the detection network is set to be w x h, (1.5 w < h < 2 w) so as to eliminate the influence on the detection effect.

And S12, using various data enhancement methods in the training process to increase the diversity of the sample images and improve the detection performance of the detection network, including horizontal turning, cutting, zooming in and zooming out and the like.

And S13, extracting the features of the sample image by using a backbone convolutional network (MobileNet), and constructing a feature pyramid network with 6 layers for position regression and class classification.

And S14, processing the output of the multilayer characteristic pyramid network through the non-maximum suppression unit to obtain a final detection positioning result of the area where the license plate amplification number is located.

The structure of the detection network is shown in fig. 3, and includes a backbone convolutional network MobileNet, a Non-Maximum Suppression unit (NMS), and a multilayer feature pyramid network, where the backbone convolutional network MobileNet is connected to an input end of the multilayer feature pyramid network, an output end of each layer of feature pyramid network is connected to the Non-Maximum Suppression unit, and the Non-Maximum Suppression unit outputs a final detection positioning result.

Step S2, recognizing characters of the enlarged number of the license plate

The method comprises the following steps of identifying license plate enlarged number characters by a deep convolution-based identification network (such as a convolution neural network (CRNN)), specifically comprising the following steps:

and S21, expanding the training sample image to obtain a training sample set.

In the training stage of identifying the network model, the CRNN convolutional neural network uses an end-to-end (end-to-end) training mode, and needs a large number of input sample images to perform network optimization training, the invention firstly labels the sample images of the original license plate enlarged number, and then expands the labeled sample images of the original license plate enlarged number, and the expansion process is shown in fig. 4 and mainly comprises the following steps:

s211, cropping the sample image to generate area images with different sizes, as shown in (a) - (c) of fig. 5, the area images obtained after cropping specifically include the following categories:

the original license plate enlarged sample (7-8 characters): such a sample image is an enlarged-size area image of the original license plate, as shown in fig. 5 (a);

② defective license plate enlarged sample (5-7 characters): the sample image is a region image obtained by cutting after discarding the original license plate province region for short, as shown in the diagram (b) of fig. 5;

③ sample after boundary expansion: the sample image is an area image obtained by carrying out random boundary expansion on the two types of license plate amplified number area images. The extended formula is specifically as follows:

wherein l, r, u and b are the expansion sizes of the license plate magnified region image at the left, right, upper and lower boundaries respectively, w and h are the width and height of the original license plate magnified region image, and random is a random function.

Fourthly, loading samples: the samples are false detection samples of a detection network, namely non-license plate amplified number areas.

S212, image normalization processing and color transformation: before model training, the convolutional neural network CRNN needs to normalize the four types of region images obtained after the sample is cut in step S211, normalize the size to W × 32, where W is the normalized image width, and then perform color transformation; the method mainly comprises the following steps:

keeping the height h unchanged, and stretching the random width of the image to improve the recognition capability of a convolutional neural network (CRNN) on narrower characters; the formula for the random width stretch transform is:

w^*＝w*(random(0.4*w,0.8*w)+1)

wherein, w^*And w is the original image width, and random is a random function.

② judging the aspect ratio w of the image after width stretching^*Whether/h is equal to the normalized size, i.e., W/32:

1) if w is^*W/32, scaling the image to W32;

2) if w is^*W/32 is smaller than h, the image is scaled to W^***32,w^**＝w^*(32/h), then expanding the left and right image boundaries, the formula is as follows:

wherein l and r are the expansion sizes of the left and right boundaries respectively, and random is a random function. In this embodiment, the convolutional neural network CRNN has no requirement on the width of the image, and therefore, in the size normalization process, the normalized size is W × 32, but the maximum width value is set to 280 in the width stretching transformation and left and right boundary expansion processes.

3) If w is^*W/32, scaling the image to W x h^**,h^**＝h*(W/w^*) Then, the upper and lower boundaries of the image are expanded, and the formula is as follows:

wherein u and b are the extension sizes of the upper and lower boundaries respectively, and random is a random function.

And thirdly, random color space transformation is carried out, the diversity of the samples is further increased, and the sample images which are finally input into the identification network are generated.

S213, generating a sample label: and storing each license plate character of the license plate number in an array, and then generating a sample label of the license plate number according to the index value of the license plate character corresponding to the array.

The convolutional neural network CRNN needs to set a space (blank) tag, which is generally set as the first bit ("0") or the last bit ("n-1") of the tag list, where n is the length of the tag list, i.e., the number of character classes), the length of the sample tag is 8, and less than 8 bits are complemented with "0" after the tag value.

For example, if the label value of blank is set to be "0", the label value of the positive sample image with the license plate number of "87569" is "986710000", that is, the label value of the license plate character is obtained by adding 1 to the corresponding index value of the license plate character in the label list, and the label value of the license plate character is obtained by adding 1 to the corresponding index value of the character in the label list no matter whether the license plate character is a number, a letter or a Chinese character; for a negative sample image, its label value is "00000000".

S22, constructing a recognition network based on the expanded sample image, and performing feature extraction on the actual license plate enlarged number image by using the constructed recognition network.

In this embodiment, a feature extraction network is constructed as an identification network, and specifically, the feature extraction network is a deep convolutional network including a convolutional layer (CNN), a feature super-resolution branch network (SR layer), a cyclic layer (RNN), a transcription layer (CTC), and a loss function layer, where the convolutional layer is connected to the SR layer and the cyclic layer, the transcription layer is connected to the cyclic layer, the transcription layer and the feature super-resolution branch network are connected to the loss function layer, the size of an input image is W × 32, W is an image width, and 32 is an image height.

In the invention, the SR layer and the RNN layer share the characteristic sequence of the image and do not need an additional characteristic extraction network, so the number of network layers of the SR layer is less, the SR layer has a simpler structure than the existing super-resolution network, the occupied video memory of a video card is less in the training process, and the training time is shorter.

S221, a feature sequence is extracted from the input image by the convolutional layer (CNN).

Taking a dense convolutional network (DenseNet) as an example, when constructing a feature extraction network, connecting CNN layers in series by using 3 DenseNet blocks, wherein the depth of each DenseNet block is d, the feature map growth rate is r, connecting convolution layers with the kernel size of k × k and random inactivation layers (dropouts) between every two DenseNet blocks, setting the proportion of the random inactivation layers dropouts as ratio, finally connecting a pooling layer with the kernel size of m × N, and outputting a feature map with dimensions of N × C × H × W, wherein N, C, H and W are the batch processing size, the feature map channel number, the feature map height and the feature map width respectively.

S222, in the training stage, the feature expression capability of the CNN layer is improved through a feature super-resolution branch network (SR layer), and a super-resolution image is reconstructed and output.

The purpose of the feature super-resolution branch network is to obtain high-resolution image features using low-resolution images. Due to the influence of hardware conditions, working environments and driving road conditions, the camera can often acquire a large amount of low-quality license plate amplified number images, and the recognition result is influenced. Therefore, in the training process, the characteristic super-resolution branch network is added to improve the characteristic expression capability of the CNN layer, namely, the characteristic sequence obtained by the CNN layer is input into the SR layer to reconstruct the super-resolution image, so that the low-resolution characteristics are restored into the corresponding super-resolution image.

Because the license plate amplification number identification data set does not distinguish high-resolution images and low-resolution images, in the training process, the invention uses two image expansion modes of Gaussian blur processing and 4-time up-down sampling to perform online expansion preprocessing on the original image to generate the low-resolution images so as to enrich the diversity of sample images in the training data set; and after the generated low-resolution image is subjected to feature sequence extraction by the convolutional layer, the low-resolution image is input to a SR layer of a feature super-resolution branch network and is reconstructed into a super-resolution image. In this embodiment, the image after the "gaussian blur processing" and the "4-fold up-down sampling" processing is represented as:

wherein, I_blurFor processed low resolution images, f_d-uAnd f_gauRespectively representing 4 times of up-down sampling and Gaussian blur processing, O is an original image, p₁And p₂Are two random parameters and alpha is a threshold.

The SR layer is mainly implemented by 2 super-resolution base units based on a residual network structure (Resnet) and an upsampling unit (UpSample), where the super-resolution base unit is a residual channel attention block RG, the RCAB is a sub-module of the residual channel attention block RG, and two RCAB sub-blocks constitute a residual attention module RG.

SR layer uses characteristic sequence F output by CNN layer_CNNPerforming super-resolution reconstruction, and outputting deeper features through two RG layers, namely:

F_RG＝H_RG(H_RG(F_CNN))

wherein, F_RGFeatures processed by two layers of RG modules, H_RGCorresponding operation for the RG module; then using the upsampling layer UpSample, convolution operation pair F_RGAnd processing the characteristics to obtain a super-resolution reconstructed image O with the same size as the input image.

Wherein, F_UPFor the feature processed by the UpSample module at the up-sampling layer, H_UPFor the corresponding operation of UpSample Module, H_ConvCorresponding operations for the convolution module. And finally, the original high-resolution image in the training sample set is used as a real sample label, the loss of the reconstructed super-resolution image is calculated by using the super-resolution loss function of S225, and the reconstruction effect of the super-resolution image is judged and evaluated according to the loss value.

S223, the tag value distribution, i.e., the true value distribution of the feature sequence obtained from the convolutional layer (CNN) is predicted by the cycle layer (RNN).

The circulation layer RNN comprises two bidirectional long and short term memory networks (BilSTM), the features extracted by the convolution layer CNN are converted by the circulation layer to obtain T x N x M dimensional features, the T is the time sequence length of the circulation layer RNN, N is the batch processing size, M is the input feature length, then T x N x N dimensional label distribution results are obtained by the full connection layer, and N is the length of a label list (character category number); the loop layer RNN may be expressed as y ═ R_w(x) Where x is the input, w is the RNN layer parameter, and y is the output.

S224, the tag value distribution obtained from the cycle layer RNN is converted into the final recognition result by the transcription layer (CTC) through operations such as de-duplication integration.

A blank mechanism is introduced into CTC in a transcription layer, and the purpose is to obtain a final predicted text sequence through operations such as de-duplication integration. Taking the "-" symbol representing blank as an example, the CTC in the transcription layer considers that "continuous repeated characters without blank intervals" are the same character, deletes "continuous repeated characters without blank intervals" for the character sequence, and then deletes all "-" characters from the path to obtain the final predicted text sequence.

For the input x given by the cycle layer RNN, the probability that the transcription layer outputs the correct license plate is as follows:

wherein, pi ∈ B^-1(l) Representing all paths of which the result is correct license plate L after B conversion (namely after cycle layer RNN processing), and L is a prediction output sequence (namely a predicted license plate number); for any path π is:

where L' is all paths. In the training process, the training target of the CTC of the transcription layer is essentially through the gradient

Adjusting the parameter w of the loop layer RNN such that for an input sample, pi ∈ B^-1(l) The probability p (l | x) of the correct license plate is the greatest.

And S225, calculating and identifying the total loss of the network through a loss function.

In the training process, the loss function simultaneously contains the loss of the text recognition part and the loss of the super-resolution branch network part, so that the feature sequence extracted by the CNN layer simultaneously contains the information of the recognition part and the super-resolution branch network part, the feature expression capability of the recognition network on low-quality images is improved, and the feature extraction effect of the recognition network on the low-quality images is improved.

That is, in the present invention, the total loss of recognition network is the text recognition loss L generated by the transcription layer CTC_recAnd super-resolution image loss L generated by super-resolution branch network_srSumming and using a hyper-parameter lambda to the super-resolution image loss L_srThe weights of (a) are adjusted, i.e. weighted summation; the loss function can be described as:

wherein O is the original image, O_i,jIs the pixel value of the original image at the (I, j) position, I_i,jThe pixel value of a super-resolution image output by a SR layer of the characteristic super-resolution branch network at the (i, j) position is represented by x, S is a training sample set and z is a sample real label. And reducing the total loss of the recognition network through training to obtain the optimized weight parameter of the recognition network.

The five stages of steps S221 to S225 form a training stage of the recognition network, and refer to fig. 6 to 11 in detail.

S226, reasoning output stage

Using the trained recognition network model to perform inference output, specifically referring to fig. 12, the main process includes: directly inputting an actual license plate amplification number image into a CNN layer without image preprocessing such as Gaussian blur processing or up-down sampling to obtain a corresponding characteristic sequence; directly inputting the characteristic sequence output by the CNN layer into the RNN layer to obtain the probability distribution of all character types of each time step; inputting the character type probability distribution output by the RNN layer into a CTC layer, taking the characters with the maximum probability distribution in all the character types of each time step as the output characters of the time step by the CTC layer, splicing the output characters with all the time steps to obtain a sequence path as the maximum probability path, and finally obtaining the final text recognition result by using a blank mechanism in the CTC layer.

That is, in the training stage, the SR layer continuously updates the network weights through iterative training to minimize the loss function, so as to obtain the optimized weight parameters. In the inference stage, the input of the CNN layer is an actually acquired license plate amplification number image, and image preprocessing such as Gaussian blur processing or up-down sampling is not performed; and the recognition network does not use the SR layer any more, and directly uses the trained weight parameters, and because the output result of the SR layer is a super-resolution image and is useless for reasoning of the recognition network, the SR layer is discarded in the reasoning stage, and the character recognition result cannot be influenced.

The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims

1. A method for detecting and identifying an enlarged number of a license plate is characterized by comprising the following steps:

step S2 includes:

2. The method for detecting and identifying the enlarged license plate number of claim 1, wherein the step S21 includes:

s211, cutting the sample image to generate area images with different sizes;

s212, before training of the recognition network model, normalizing the multi-class area image obtained after the sample is cut in the step S211, and then performing color transformation; normalizing the size to W32, W being the normalized image width;

s213, storing each license plate character of the license plate number in an array, and then generating a sample label of the license plate number according to the corresponding index value of the license plate character in the stored array.

3. The method for detecting and identifying the enlarged license plate number according to claim 2, wherein the area image obtained after clipping in step S211 includes the following categories:

the original license plate amplified number sample is an area image of the original license plate amplified number;

the defective license plate enlarged number sample is an area image obtained by cutting after discarding the original license plate province, namely the area;

the sample after the boundary expansion is a regional image obtained by carrying out random boundary expansion on the original license plate amplified number sample and the defective license plate amplified number sample;

and the negative sample is a false detection sample of the detection network, namely a non-license plate amplified number area.

4. The method for detecting and identifying the enlarged license plate number of claim 2, wherein step S212 comprises:

(1) keeping the height h unchanged, and stretching the image in random width;

(2) determining the aspect ratio w of the width-stretched image^*Whether/h is equal to W/32; if yes, scaling the image to W32; if w is^*W/32 is smaller than h, the image is scaled to W^***32,w^**＝w^*(32/h), then expand the left and right image borders:

wherein l and r are the expansion sizes of the left and right boundaries respectively, and random is a random function; if w is^*W/32, scaling the image to W x h^**,h^**＝h*(W/w^*) Then is aligned withAnd (3) expanding the upper and lower boundaries of the image:

wherein u and b are respectively the expansion sizes of the upper and lower boundaries, and random is a random function;

(3) and carrying out random color space transformation to generate a sample image which is finally input into the identification network.

5. The method for detecting and identifying an enlarged license plate number of claim 1, wherein the identification network constructed in step S22 is a deep convolutional network comprising a convolutional layer, a characteristic super-resolution tributary network, a cyclic layer, a transcription layer, and a loss function layer, wherein the convolutional layer is connected to the characteristic super-resolution tributary network and the cyclic layer, the transcription layer is connected to the cyclic layer, and the transcription layer and the characteristic super-resolution tributary network are connected to the loss function layer.

6. The method for detecting and identifying the enlarged license plate number of claim 5, wherein the step S22 includes:

s221, extracting a characteristic sequence from the input image through the convolution layer;

s222, in the training stage, the original image is expanded to generate a low-resolution image, and the generated low-resolution image is input into a characteristic super-resolution branch network to be reconstructed into a super-resolution image after a characteristic sequence is extracted from a convolutional layer; meanwhile, a characteristic sequence extracted from the convolutional layer of the original high-resolution image in the training sample set is used as a real sample label, and the loss of the reconstructed super-resolution image is calculated;

s223, predicting the label value distribution of the characteristic sequence acquired from the convolutional layer through the loop layer;

s224, converting the label value distribution obtained from the circulation layer into a final recognition result through a duplication elimination integration operation by the transcription layer;

s225, calculating and identifying the total loss of the network through a loss function, and transcribing the layersResulting text recognition loss L_recAnd super-resolution image loss L generated by super-resolution branch network_srWeighted summation as the total loss of the identified network, where the super-resolution image loses L_srThe weight of (a) is adjusted by a hyperparameter λ; the total loss of the recognition network is reduced through training, and the optimized weight parameter of the recognition network is obtained;

s226, reasoning and outputting by using the recognition network model obtained after training, and directly inputting the actual license plate enlarged number image into the convolutional layer to obtain a corresponding characteristic sequence; inputting the characteristic sequence into a circulation layer to obtain the probability distribution of all character types of each time step; inputting the character type probability distribution output by the circulation layer into the transcription layer, taking the character with the maximum probability distribution in all the character types of each time step as the output character of the time step by the transcription layer, splicing the output characters with all the time steps to obtain a sequence path as the maximum probability path, and finally obtaining the final text recognition result by using a blank mechanism in the transcription layer.

7. The method for detecting and identifying enlarged license plate number of claim 5, wherein step S222 is performed by expanding the original image through Gaussian blur processing and multiple up-down sampling to generate a low-resolution image, and setting I_blurFor processed low resolution images, f_d-uAnd f_gauRespectively representing multiple up-down sampling and Gaussian blur processing, wherein O is an original image, and the image after multiple up-down sampling and Gaussian blur processing is represented as follows:

wherein p is₁And p₂Are two random parameters and alpha is a threshold.

8. The method for detecting and identifying the enlarged license plate number of claim 7, wherein the loss function of step S225 is described as:

wherein O is the original image, O_i,jIs the pixel value of the original image at the (I, j) position, I_i,jAnd (3) outputting the pixel value of the super-resolution image at the (i, j) position by the characteristic super-resolution branch network, wherein x is input, S is a training sample set, and z is a sample real label.

9. The method for detecting and identifying an enlarged license plate according to claim 6, wherein the loop layer in step S223 includes two bidirectional long-short term memory networks, the features extracted from the convolution layer are transformed by the loop layer to obtain T × N × M dimensional features, and the T × N × M dimensional features are input into the loop layer, where T is the length of the time sequence of the loop layer, N is the size of batch processing, M is the length of the input features, and then T × N dimensional label distribution results are obtained through the fully connected layer, and N is the length of the label list; the cyclic layer is denoted by y ═ R_w(x) Where x is the input, w is the parameter of the loop layer, and y is the output.

10. The method for detecting and identifying the enlarged license plate number according to claim 9, wherein in step S224, for the input x given by the loop layer, the probability that the transcription layer outputs the correct license plate is:

wherein, pi ∈ B^-1(l) Representing all paths of which the result is the correct license plate L after the conversion of the circulation layer, wherein L is a prediction output sequence; for any path π is:

wherein L' is all paths; in the training process, the transcription layerBy gradient of training target

Adjusting the parameter w of the loop layer so that for the input sample, pi ∈ B^-1(l) The probability p (l | x) of the correct license plate is the greatest.