Vehicle color recognition method under white light conditionTechnical Field
The invention belongs to the technical field of image processing, and particularly relates to a vehicle color recognition method under a white light condition.
Background
In recent years, with the rapid development of computer and internet technologies, the number of various motor vehicles has been rapidly increased, and various information including traffic-related information has been in an explosive growth state. In order to be able to manage such information more safely and efficiently, intelligent transportation systems have been developed. The intelligent traffic system can show strong effects in the aspects of toll collection, parking lots, criminal tracing and the like. The vehicle information in the video image comprises license plate numbers, vehicle colors, vehicle models and the like, wherein the vehicle color identification plays a role in vehicle road monitoring and is an indispensable part in the vehicle information. When the license plate number cannot be identified or distinguished, other vehicle information such as vehicle color becomes the basis for distinguishing vehicles. For example, in the case of illegal license plate vehicle identification, the same license plate vehicles must be distinguished by non-license plate information, and the vehicle color information is preferred.
The current vehicle color recognition methods mainly include the following:
(1) The method [1] of direct classification, firstly detecting the vehicle position, then directly extracting the color characteristics of the vehicle region, and finally classifying by using a classifier. The method cannot find out the area to be identified of the vehicle, so that the result is greatly influenced by the window, the tire, the background and the like.
(2) A method [2] [3] of processing vehicle color region selection based on an image. Firstly, the position and the size of a license plate are positioned, then a vehicle color recognition area is initially selected according to priori knowledge, then the vehicle color recognition area is selected according to information such as gradient and brightness, and finally the vehicle color recognition is carried out in the area. The method requires a great deal of manual experience, has low robustness and is greatly influenced by the environment.
(3) Based on a method of segmenting a vehicle color region [4]. Marking a large amount of vehicle color effective area data, detecting an effective area for vehicle color identification by adopting a semantic segmentation method, and then carrying out vehicle color identification in the area. The method has huge data labeling workload, and huge algorithm calculation amount, and recognizes the need of semantic segmentation.
(4) The gray level eliminating method (5) eliminates the interference areas of window glass, vehicle shadow, etc. by the method of the difference between the maximum value and the minimum value of RGB space, and distinguishes color vehicles and black-white vehicles by the ratio of the number of pixels. The method is based on image processing, is easily influenced by ambient light, and has low robustness
(5) And (3) based on the scene classification method [6], firstly classifying the scene of the vehicle to be identified, and then calling a color classification model of the scene to classify the colors of the vehicle. This method requires a large amount of data, and at the same time cannot find out the area to be recognized of the vehicle.
In summary, the prior art has the problems that (1) regarding the vehicle color, only partial areas can be actually used for vehicle color recognition based on the intuition of human eyes, for example, windows, lamps, tires and the like do not belong to the areas for vehicle color judgment, and the vehicle color effective areas are generally areas such as vehicle engine covers, vehicle doors and the like. If the algorithm is not able to accurately and effectively find the effective area, it will result in a reduced recognition rate.
(2) Due to the influence of factors such as different illumination, different weather, different angles and the like in a natural scene, even if the vehicle is the same color, the image often shows phenomena such as color deviation, reflection and the like, so that the algorithm recognition rate is reduced.
The phase references are as follows:
[1] Vehicle color recognition method and device, publication number CN 107844745A.
[2] A method and a device for positioning a vehicle body color recognition area are disclosed, wherein the publication number is CN 106529553B.
[3] A vehicle body color recognition method under a monitoring scene is disclosed with a publication number CN 109741406A.
[4] Automatic identification method, electronic device, computer equipment and medium for vehicle color, publication number CN 111325211A.
[5] A vehicle body color recognition method, publication number CN 105005766A.
[6] Vehicle color classification model training method and device and vehicle color recognition method, publication number CN 110348505A.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a vehicle color correction method under the white light condition, which combines the advantages of RetineX and DARKCHANNEL methods, can effectively relieve the influence caused by low illumination, backlight and other phenomena, and reduces the problem of reduced recognition rate caused by adverse environmental factors. Meanwhile, a network model based on a spatial attention mechanism is designed, the model can enable the network model to automatically focus on an effective area of vehicle colors, weight of the effective area of the vehicle colors is improved, weight of an ineffective area of the vehicle colors is reduced, and then the vehicle colors are accurately identified by adopting a deep neural network.
In order to achieve the above purpose, the technical scheme provided by the invention is that the vehicle color identification method under the white light condition comprises the following steps:
Step 1, collecting a large amount of vehicle image data collected under the white light condition, and marking the vehicle color of each vehicle;
step 2, designing a vehicle color recognition network model;
the vehicle color recognition model comprises a backlight module, attention modules, a BAP module, an FC module and a softmax module;
step 3, providing the collected vehicle data for a vehicle color recognition network model for repeated iterative training;
step4, detecting vehicle position information in the image by using a vehicle detector;
Step 5, cutting out the vehicle image from the original image through the detected vehicle position;
Step 6, carrying out color correction on the cut vehicle image;
And 7, sending the vehicle image subjected to color correction to a vehicle color recognition network, and predicting the vehicle color type.
Further, the backup module in the vehicle color recognition model in step 2 adopts the first 4 residual structures of mobilenetV2, the attention module obtains a space weight mask from the original features extracted by the backup module and is used for weighting different positions, the BAP module weights the original features extracted by the backup module by the weight mask obtained by the attention module to obtain weighted new features, the FC module is used for matching the new feature dimension with the number of output categories, and the softmax module is used for outputting the probability of each category.
Further, the specific processing procedure of the Attention module is as follows;
Performing convolution operation on feature_raw extracted by a back bone module by using M1 x1 convolution kernels to obtain Mx14x14 mask graphs, and normalizing each mask to be 0-1;
Wherein the normalization formula is:
Wherein maskSrcj denotes the original jth mask, maskDstj denotes the normalized jth mask,
Respectively representing the ith point of the original jth mask and the ith point of the normalized j mask;
min (maskSrcj),max(maskSrcj) represents the minimum and maximum values of the original jth mask, respectively.
Further, the BAP module performs dot product operation on M mask patterns in the attention module and feature_raw extracted by the backup module to obtain 64xMx14x14 feature patterns, and then performs average pooling operation on each feature pattern to obtain 64xMx1x1 feature patterns.
Further, the Softmax module performs Softmax operation on the N prediction results output by the FC module, so as to predict the probability of each class, where the Softmax formula is as follows: Where zi represents the predicted value of the i-th class and pi represents the predicted probability of the i-th class.
Further, the specific implementation process of the step 6 is as follows;
(1) Cutting out the detected vehicle image block, and marking as P;
(2) Calculating the maximum and minimum values, vmax and Vmin of P;
(3) Linearly stretching the P obtained in the step (2) to obtain p1=255 (P-Vmin)/(Vmax-Vmin);
(4) Transforming P1 obtained in step (3) to the logarithmic domain to obtain p2=log (p1+1)/log (256);
(5) According to the P2 obtained in the step (4), calculating a dark primary color channel of the P2 to obtain P3;
(6) Calculating the brightness map of each color channel of P2 according to the P2 obtained in the step (4), and obtaining P4;
(7) According to the step (5), P3 and P4 obtained in the step (6) calculate color correction images to obtain P5= (P2-P3)/(P4-P3);
(8) And (3) restoring the color correction chart obtained in the step (7) to a normal image to obtain p6=p5×255.
Further, the way to calculate the dark primary channel of P2 in step (5) is:
Wherein Jdark is called a dark primary channel of the image, Ω (x) represents a local block region centered on x, c represents one of the three channels r, g, b, Jc is a c channel of the input image, x is a pixel point on the logarithmic domain image P2, and y is a pixel point in the local block region Ω (x).
Further, in the step (6), the brightness map of each color channel of P2 is calculated by:
Wherein the method comprises the steps ofThe luminance map called image c channel, Ω (x) represents a local block region centered on x, c represents one color channel of r, g, b, Jc is the c channel of the input image, x is one pixel point on the logarithmic domain image P2, and y is one pixel point in the local block region Ω (x).
Compared with the prior art, the invention has the advantages that (1) the method for correcting the vehicle color under the white light condition is provided, and the method can reduce the influence of the reduction of the recognition rate caused by the low-illumination and backlight phenomena. (2) A network model based on a spatial attention mechanism is designed, and the model can automatically focus on the color effective area of a vehicle body without relying on manual experience to position an area to be identified.
Drawings
FIG. 1 is a flow chart of the overall process of the present invention.
Fig. 2 is a flow chart of the color correction of the present invention.
Fig. 3 is a graph showing the effect of color correction according to the present invention.
FIG. 4 is a diagram of a vehicle color classification network model according to the present invention.
Detailed Description
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
As shown in fig. 1, the method for identifying the vehicle color under the white light condition provided by the invention comprises the following steps:
And 1, collecting a large amount of vehicle image data collected under the white light condition, and marking the vehicle color of each vehicle.
In the step 1, a large amount of vehicle image data under the white light condition is collected through a bayonet and a camera of a parking lot, then the position information of a vehicle in an image is detected through a vehicle detector, wherein the position information comprises (x 1, y1, x2, y 2), x1, y1, x2 and y2 respectively represent the horizontal and vertical coordinates of the upper left corner and the lower right corner of the vehicle, then a vehicle image block is cut out from the image data according to the position information of the vehicle, and finally the vehicle color in the vehicle image block is marked in an artificial mode.
And 2, designing a vehicle color recognition network model.
The vehicle color recognition model designed in the step 2 is shown in fig. 4, and includes modules such as a backup, attention, BAP, FC, softmax, and the like. The backup adopts the first 4 residual structures of mobilenetV < 2 >, attention is a space weight mask obtained from original features extracted from the backup and used for weighting different positions, BAP is obtained by weighting the original features extracted from the backup by the weight mask obtained by attention to obtain weighted new features, the FC layer is used for matching new feature dimensions with the number of output categories, and softmax is used for outputting the probability of each category.
The vehicle color recognition network is characterized in that a spatial attention mechanism is added on the basis of a traditional classification network model, and the spatial attention module is used for automatically distributing different weights to different areas in a vehicle image, increasing the weight of a vehicle color effective area and reducing the weight of a vehicle ineffective area so as to improve the classification performance of the network model. The whole network structure is formed by combining modules of backbones, attention, BAP, FC, softmax and the like, and the network model structure is shown in figure 4.
The invention adopts the first 4 residual structures of mobilenetV to stack, the input is 3x224x224, the output is 64x14x14, and the output is feature_raw.
The invention adopts M1 x1 convolution kernels to carry out convolution operation on feature_raw extracted by a backup to obtain Mx14x14 mask graphs, normalizes each mask to be between 0 and 1, and M default is 16. The normalization formula is: Wherein maskSrcj denotes the original jth mask, maskDstj denotes the normalized jth mask,Respectively representing the ith point of the original jth mask and the ith point of the normalized j mask. min (maskSrcj),max(maskSrcj) represents the minimum and maximum values of the original jth mask, respectively.
BAP module (Bilinear Attention Pooling) the invention adopts M mask images in attention and feature_raw extracted by a backup to perform dot product operation to obtain 64xMx14x14 feature images, and then performs average pooling operation on each feature image to obtain 64xMx1x1 feature images, which are marked as feature_matrix.
And the FC module is used for fully connecting the feature_matrix extracted by the BAP as the FC of N classes and outputting the prediction result of each class.
The invention provides a Softmax module, which performs Softmax operation on N prediction results output by FC, and aims to predict the probability of each class. Wherein the softmax formula is: Where zi represents the predicted value of the i-th class and pi represents the predicted probability of the i-th class.
And step 3, providing the collected vehicle data for a vehicle color recognition network model to carry out repeated iterative training.
The parameters of the network training in the step 3 are lr=0.001, movement=0.9, weight_decay=1e-5, epoch=50, batch size=64, the network optimization adopts an SGD gradient descent mode, the learning rate adopts fixed compensation attenuation, every 2 epochs are multiplied by 0.9, wherein lr is the learning rate, movement is momentum, weight_decay is the attenuation coefficient, epoch is the training lumped iteration number, batchsize is the iteration sample number.
And 4, detecting the vehicle position information in the image by using a vehicle detector.
The vehicle detector in the step 4 is obtained by training a yolov framework and a COCO+VOC combined data set.
And 5, cutting the vehicle image from the original image through the detected vehicle position.
And 6, carrying out color correction on the cut vehicle image.
The color correction flow chart in step 6 is shown in fig. 2, and includes the steps of linear stretching, logarithmic domain transformation, dark primary color channel extraction, brightness map extraction of each color channel, correction of each color channel and the like, and the original vehicle image color is corrected through the steps. The invention combines the advantages of RetineX and DARKCHANNEL, and has certain superiority in correcting vehicle color under the condition of abuse through experiments, the flow chart is shown in figure 2, the effect is shown in figure 3, the left side is the original diagram, the right side is the diagram after correcting the color, the steps are as follows:
(1) The detected vehicle image block is cut out and is marked as P.
(2) And calculating the maximum and minimum values of P, vmax and Vmin.
(3) And (3) linearly stretching the P obtained in the step (2) to obtain p1=255 (P-Vmin)/(Vmax-Vmin).
(4) And (3) converting the P1 obtained in the step (3) into a logarithmic domain to obtain P2=log (P1+1)/log (256).
(5) And (3) calculating a dark primary color channel of the P2 according to the P2 obtained in the step (4), wherein the process of obtaining the P3 is that the minimum component in the R, G and B channels of each pixel point of the original image is extracted to obtain a single-channel dark primary color image, and then a quick algorithm of MARCEL VAN HERK is adopted to carry out local area minimum filtering, namely gray scale corrosion operation on the dark primary color image to obtain the dark primary color channel.
(6) According to the P2 obtained in the step (4), calculating the brightness map of each color channel of the P2 to obtain P4, wherein the calculation process of the brightness map of each color channel comprises the steps of decomposing an original image in an RGB space to obtain three channel maps of R, G and B, and then carrying out local area maximum filtering, namely gray scale expansion operation on the three channel maps of R, G and B by adopting a quick algorithm of MARCEL VAN HERK to obtain the brightness map of each channel.
(7) According to the step (5), the P3 and P4 obtained in the step (6) calculate color correction images to obtain p5= (P2-P3)/(P4-P3).
(8) And (3) restoring the color correction chart obtained in the step (7) to a normal image to obtain p6=p5×255.
The way to calculate the dark primary channel of P2 in step (5) is: Where Jdark is referred to as the dark primary channel of the image, Ω (x) represents the local block region centered on x, c represents one of the three channels r, g, b, and Jc is the c-channel of the input image.
In the step (6), the brightness map of each color channel of P2 is calculated by:
Wherein the method comprises the steps ofThe luminance map, called the c-channel of the image, Ω (x) represents a local block region centered on x, c represents one of the three r, g, b color channels, Jc being the c-channel of the input image.
And 7, sending the vehicle image block subjected to color correction to a vehicle color recognition network, and predicting the vehicle color type.
The specific embodiments described herein are merely illustrative of the present invention. Those skilled in the art may make various modifications or additions to the described embodiments or substitutions thereof without departing from the scope of the invention as defined in the accompanying claims.