Vehicle color identification method under white light conditionTechnical Field
The invention belongs to the technical field of image processing, and particularly relates to a vehicle color identification method under a white light condition.
Background
In recent years, with the rapid development of computer and internet technologies, the number of various automobiles is rapidly increased, and various types of information including information related to traffic are in an explosive growth situation. In order to manage the information more safely and efficiently, the intelligent transportation system is developed. The intelligent transportation system can show powerful effect in the aspect of charge bayonet, parking area, criminal tracking etc.. The vehicle information in the video image comprises a license plate number, a vehicle color, a vehicle model and the like, wherein the vehicle color identification plays a significant role in vehicle road monitoring and is an indispensable part in the vehicle information. When the license plate number cannot be identified or distinguished, other vehicle information such as vehicle color and the like becomes a basis for distinguishing the vehicles. For example, in illegal fake-licensed vehicle identification, the same-license vehicles must be distinguished by non-license information, and vehicle color information is preferred.
The current vehicle color identification methods mainly comprise the following methods:
(1) the direct classification method [1] firstly detects the position of the vehicle, then directly extracts the color characteristics of the vehicle region, and finally adopts a classifier for classification. The method cannot find out the area to be identified of the vehicle, so that windows, tires, backgrounds and the like can have great influence on results.
(2) Method for vehicle color zone selection based on image processing [2] [3 ]. The method comprises the steps of firstly positioning the position and the size of a license plate, then initially selecting a vehicle color identification region according to priori knowledge, then selecting the vehicle color identification region according to information such as gradient and brightness, and finally carrying out vehicle color identification in the region. The method needs a large amount of manual experience, is not strong in robustness and is greatly influenced by the environment.
(3) Based on a method of segmenting vehicle color regions [4 ]. And marking a large amount of vehicle color effective area data, detecting an effective area for vehicle color identification by adopting a semantic segmentation method, and identifying the vehicle color in the area. The method has huge workload of data annotation, semantic segmentation is needed for identification, and the algorithm has huge calculation amount.
(4) And (5) a gray level elimination method [5], wherein interference areas such as window glass, vehicle shadow and the like are eliminated by a method of the difference between the maximum value and the minimum value of the RGB space, and color vehicles and black and white vehicles are distinguished by a pixel number ratio. The method is based on image processing, is easily influenced by ambient light, and has low robustness
(5) Based on the method [6] of scene classification, the vehicle to be recognized is firstly subjected to scene classification, and then a color classification model carried out in the field is called to classify the vehicle color. The method requires a large amount of data and cannot find out the area to be identified of the vehicle.
In summary, the following problems in the prior art need to be solved: (1) for the color of the vehicle, only a part of the regions can be actually used for the color recognition of the vehicle based on the intuition of human eyes, for example, windows, lamps, tires, etc. do not belong to the regions for the color judgment of the vehicle, and need to be excluded, and the effective regions of the color of the vehicle are generally the regions of a hood, a door, etc. of the vehicle. If the algorithm can not accurately and effectively find the effective area, the recognition rate is reduced.
(2) Due to the influence of factors such as different illumination, different weather and different angles in a natural scene, even if the vehicle is of the same color, the image often shows phenomena such as color cast and light reflection, and the algorithm recognition rate is reduced.
The phase references are as follows:
[1] vehicle color identification method and apparatus, publication No. CN 107844745 a.
[2] A method and a device for positioning a vehicle body color identification area are disclosed in the publication number CN 106529553B.
[3] A method for recognizing the color of a car body in a monitoring scene is disclosed as CN 109741406A.
[4] Automatic recognition method of vehicle color, electronic device, computer apparatus and medium, publication No. CN 111325211 a.
[5] A vehicle body color identification method is disclosed in publication number CN 105005766A.
[6] A vehicle color classification model training method, a device and a vehicle color identification method are disclosed in the publication number CN 110348505A.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method for correcting the color of a vehicle under the white light condition, which combines the advantages of RetineX and dark channel methods, can effectively relieve the influence caused by the phenomena of low illumination, backlight and the like, and reduces the problem of low recognition rate caused by adverse environmental factors. Meanwhile, a network model based on a space attention mechanism is designed, the network model can automatically focus on an effective area of vehicle color, the weight of the effective area of the vehicle body color is improved, the weight of an ineffective area of the vehicle color is reduced, and then the vehicle color is accurately identified by adopting a deep neural network.
In order to achieve the purpose, the invention provides the technical scheme that: a vehicle color recognition method under white light conditions comprises the following steps:
step 1, collecting a large amount of vehicle image data collected under a white light condition, and marking the color of each vehicle;
step 2, designing a vehicle color identification network model;
the vehicle color recognition model comprises a backbone module, an attention module, a BAP module, an FC module and a softmax module;
step 3, providing the collected vehicle data to a vehicle color recognition network model for repeated iterative training;
step 4, detecting vehicle position information in the image by using a vehicle detector;
step 5, cutting the vehicle image from the original image according to the detected vehicle position;
step 6, performing color correction on the cut vehicle image;
and 7, sending the vehicle image subjected to color correction to a vehicle color identification network, and predicting the vehicle color category.
Further, the backbone module in the vehicle color identification model in step 2 adopts the first 4 residual structures of mobilenetV 2; the attention module obtains a spatial weight mask from the original features extracted by the backbone module and is used for weighting different positions; the BAP module weights the original features extracted by the backbone module by the weight mask obtained by the attention module to obtain weighted new features; the FC module is used for matching the new feature dimension with the output category number; the role of the softmax module is to output the probability for each category.
Further, the specific processing procedure of the Attention module is as follows;
performing convolution operation on feature _ raw extracted by a backbone module by adopting M convolution kernels of 1x1 to obtain Mx14x14 mask graphs, and normalizing each mask to be 0-1;
wherein the normalization formula is:
in the formula, maskSrcjRepresents the original jth mask, mask DstjRepresents the normalized jth mask,
respectively representing the ith point of the original jth mask and the ith point of the normalized jth mask;
min(maskSrcj),max(maskSrcj) Respectively representing the minimum value and the maximum value of the original jth mask.
Further, the BAP module performs a dot product operation on the M mask masks in the attribute module and the feature _ raw extracted by the background module to obtain 64xMx14x14 feature maps, and then performs an average pooling operation on each feature map to obtain 64xMx1x1 feature maps.
Further, the Softmax module performs Softmax operation on the N prediction results output by the FC module, so as to predict the probability of each class, where the Softmax formula is:
wherein z is
iIndicating a prediction value, p, of the i-th class
iRepresenting the prediction probability of the i-th class.
Further, the specific implementation process of step 6 is as follows;
(1) cutting the detected image block of the vehicle, and marking as P;
(2) calculating the maximum value, the minimum value, Vmax and Vmin of P;
(3) linearly stretching P obtained in step (2) to obtain P1 ═ 255 × (P-Vmin)/(Vmax-Vmin);
(4) transforming the P1 obtained in the step (3) into a logarithmic domain to obtain P2 ═ log (P1+1)/log (256);
(5) calculating a dark primary color channel of P2 according to the P2 obtained in the step (4) to obtain P3;
(6) calculating a brightness map of each color channel of P2 according to the P2 obtained in the step (4) to obtain P4;
(7) calculating a color corrected image according to P3 and P4 obtained in step (5) and step (6) to obtain P5 ═ P2-P3)/(P4-P3;
(8) and (4) restoring the color correction map obtained in the step (7) to a normal image to obtain P6-P5-255.
Further, the way of calculating the dark primary channel of P2 in step (5) is as follows:
wherein J
darkCalled the dark primary channel of the image, omega (x) denotes the local block area centered on x, c denotes one of the three channels r, g, b, J
cC channel which is an input image; x is a pixel point on the log domain image P2, and y is a pixel point in the local block region Ω (x).
Further, the calculation method of the luminance map of each color channel of P2 in step (6) is as follows:
wherein
Luminance map called c-channel of image, Ω (x) denotes a local block area centered on x, c denotes one of r, g, b channels, J
cC channel which is an input image; x is a pixel point on the log domain image P2, and y is a local block regionOne pixel in Ω (x).
Compared with the prior art, the invention has the advantages and beneficial effects that: (1) a method for correcting the color of car under white light condition is disclosed, which can reduce the influence of low-light and back-light phenomena on the recognition rate. (2) A network model based on a space attention mechanism is designed, the model can automatically focus on the effective area of the color of the car body, and the area to be identified does not need to be positioned by means of manual experience.
Drawings
FIG. 1 is an overall flow chart of the method of the present invention.
FIG. 2 is a flow chart of the color correction of the present invention.
FIG. 3 is a diagram of the color correction effect of the present invention.
FIG. 4 is a diagram of a vehicle color classification network model according to the present invention.
Detailed Description
The technical solution of the present invention is further explained with reference to the drawings and the embodiments.
As shown in fig. 1, the present invention provides a method for identifying vehicle color under white light condition, comprising the following steps:
step 1, collecting a large amount of vehicle image data collected under the white light condition, and marking the color of each vehicle.
In the step 1, vehicle image data under a large amount of white light conditions are collected through cameras of a bayonet and a parking lot, then the position information of a vehicle in the image is detected through a vehicle detector, wherein the position information comprises (x1, y1, x2 and y2), wherein x1, y1, x2 and y2 respectively represent horizontal and vertical coordinates of the upper left corner and the lower right corner of the vehicle, then a vehicle image block is cut out from the image data according to the position information of the vehicle, and finally the vehicle color in the vehicle image block is marked in a manual mode.
And 2, designing a vehicle color identification network model.
The vehicle color recognition model designed in step 2 is shown in fig. 4, and includes modules such as a backbone, an attribute, a BAP, an FC, and a softmax. Wherein the backbone adopts the first 4 residual structures of mobilenetV 2; the attention is to obtain a spatial weight mask from the original features extracted from the backbone, and is used for weighting different positions; BAP is to weight the original features extracted from the backbone by the weight mask obtained by the attention to obtain new weighted features; the FC layer is used for matching the new feature dimension with the output class number; the role of softmax is to output the probability for each category.
The vehicle color identification network of the invention adds a space attention mechanism on the basis of the traditional classification network model, and the space attention module is used for automatically distributing different weights to different areas in a vehicle image, increasing the weight of a vehicle color effective area and reducing the weight of a vehicle ineffective area so as to improve the classification performance of the network model. The whole network structure is formed by combining the modules of backbone, attack, BAP, FC, softmax and the like, and the network model structure is shown in FIG. 4.
A Backbone module: the invention is formed by stacking the first 4 residual error structures of mobilenetV2, and the input is as follows: 3x224x224, with the output: 64x14x14, and the output is denoted feature _ raw.
An Attention module: according to the method, M convolution kernels of 1x1 are adopted to perform convolution operation on feature _ raw extracted from a backbone to obtain Mx14x14 mask graphs, each mask is normalized to be between 0 and 1, and M is 16 by default. The normalized formula is:
wherein the mask src
jRepresents the original jth mask, mask Dst
jRepresents the normalized jth mask,
respectively representing the ith point of the original jth mask and the ith point of the normalized jth mask. min (mask src)
j),max(maskSrc
j) Respectively representing the minimum value and the maximum value of the original jth mask.
BAP module (biliiner attachment position): the method comprises the steps of performing dot product operation on M mask graphs in the attribute and feature _ raw extracted from a backbone to obtain 64xMx14x14 feature graphs, and performing average pooling operation on each feature graph to obtain 64xMx1x1 feature graphs which are marked as feature _ matrix.
An FC module: the invention makes the feature _ matrix extracted by BAP as N-type FC full connection for outputting the prediction result of each type.
The invention relates to a Softmax module, which performs Softmax operation on N prediction results output by FC and aims to predict the probability of each class. Wherein softmax is the formula:
wherein z is
iIndicating a prediction value, p, of the i-th class
iRepresenting the prediction probability of the i-th class.
And 3, providing the collected vehicle data to a vehicle color recognition network model for repeated iterative training.
The parameters of the network training in the step 3 are as follows: the lr is 0.001, the moment is 0.9, the weight _ decay is 1e-5, the epoch is 50, the batch size is 64, the network optimization adopts an SGD gradient descent mode, the learning rate adopts fixed compensation attenuation, and every 2 epochs are multiplied by 0.9; wherein lr: learning rate, moment: momentum, weight _ decay: attenuation coefficient, epoch: training the number of iterations of the lump, batch size: the number of samples of one iteration.
And 4, detecting the vehicle position information in the image by adopting a vehicle detector.
The vehicle detector in the step 4 is obtained by adopting yolov5 framework and COCO + VOC combined data set training.
And 5, cutting the vehicle image from the original image according to the detected vehicle position.
And 6, performing color correction on the cut vehicle image.
The flow chart of the color correction in step 6 is shown in fig. 2, and includes the steps of: linear stretching, logarithmic domain transformation, extraction of a dark primary color channel, extraction of a brightness map of each color channel, correction of each color channel and the like, and the original vehicle image color is corrected through the steps. The invention combines the advantages of RetineX and dark channel, and finds that the effect has certain superiority to the vehicle color correction in a hostile environment through experiments, the flow chart is shown as figure 2, the effect is shown as figure 3, the left side is an original graph, the right side is a graph after color correction, and the method comprises the following steps:
(1) and cutting the detected image block of the vehicle, and recording as P.
(2) And calculating the maximum value, the minimum value, Vmax and Vmin of the P.
(3) Linear stretching of P obtained in step (2) gave P1 ═ 255 × (P-Vmin)/(Vmax-Vmin).
(4) And (3) transforming the P1 obtained in the step (3) into a logarithmic domain to obtain P2 which is log (P1+1)/log (256).
(5) Calculating a dark primary color channel of P2 according to the P2 obtained in the step (4) to obtain P3, wherein the dark primary color channel is obtained by the following process: extracting the minimum component in three channels of R, G and B of each pixel point of the original image to obtain a single-channel dark primary color image, and then performing local area minimum filtering, namely gray corrosion operation, on the dark primary color image by adopting a Marcel van Herk rapid algorithm to obtain a dark primary color channel.
(6) Calculating the brightness map of each color channel of P2 according to the P2 obtained in the step (4) to obtain P4, wherein the obtaining process of the brightness map of each color channel comprises the following steps: decomposing the original image in an RGB space to obtain three channel images of R, G and B, and then respectively carrying out local area maximum filtering, namely gray expansion operation on the three channel images of R, G and B by adopting a Marcel van Herk fast algorithm to obtain brightness images of all channels.
(7) The color corrected image is calculated from P3 and P4 obtained in step (5) and step (6), and P5 is (P2-P3)/(P4-P3).
(8) And (4) restoring the color correction map obtained in the step (7) to a normal image to obtain P6-P5-255.
The way of calculating the dark primary channel of P2 in step (5) is:
wherein J
darkCalled the dark primary channel of the image, omega (x) denotes the local block area centered on x, c denotes one of the three channels r, g, b, J
cIs the c channel of the input image.
The calculation method for calculating the luminance map of each color channel of P2 in step (6) is as follows:
wherein
Luminance map called c-channel of image, Ω (x) denotes a local block area centered on x, c denotes one of r, g, b channels, J
cIs the c channel of the input image.
And 7, sending the vehicle image blocks subjected to color correction to a vehicle color identification network, and predicting the vehicle color types.
The specific embodiments described herein are merely illustrative of the present invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the scope of the invention as defined in the appending claims.