Disclosure of Invention
In view of the above, an object of the embodiments of the present invention is to provide a method for detecting a surface of a metal cap based on deep learning. According to the detection method of the metal blank cap surface based on deep learning, provided by the embodiment of the invention, the machine vision is adopted to replace human eyes to detect and judge the defects, relevant information is firstly extracted from an image of an objective target, then the relevant information is processed through an intelligent algorithm, and the final processing result is applied to the detection of an actual product.
In order to achieve the above object, an embodiment of the present invention provides the following technical solutions: a detection method of metal blank cap surface defect based on deep learning comprises step S1, analyzing normal surface of metal blank cap and defect surface of metal blank cap, and determining defect type and defect characteristic of metal blank cap surface to be detected; step S2: acquiring an image of the surface of the metal blank cap to be detected by adopting a camera, and processing the image to obtain an image data set; step S3: and constructing a detection model of the surface defects of the metal blank cap according to the image data set, extracting features of different layers of the image by adopting an improved deep convolution network, and learning and obtaining a defect type and range detection model based on the features, wherein the defect type and range detection model can detect the positions of the surface defects in the image.
As a further improvement of the present invention, in step S1, the normal surface of the metal blank cap is taken as a positive sample, and the defective surface of the metal blank cap is taken as a negative sample.
As a further improvement of the present invention, the defect types include scratches, gouges and stains.
As a further improvement of the present invention, the step S2 specifically includes: and labeling the defect type and the defect area of the image on the surface of the metal blank cap to be detected acquired by the camera by using labeImg software, labeling each image to obtain a corresponding XML (extensive makeup language) labeled file, and storing the labeled image and the labeled XML labeled file respectively.
As a further improvement of the present invention, the image capturing manner in step S2 is: on putting the thing platform, keep the detection surface up and place in putting thing platform central point with the metal stifle, set up the light source and play a face structure light in the perpendicular top of metal stifle, set up the camera and shoot the collection image in the perpendicular top of metal stifle.
As a further improvement of the invention, the improved deep convolutional network in the step S3 is a Yolov3 network, and the Yolov3 network is composed of a Darknet-53 structure and a multi-scale bounding box prediction structure, wherein the Darknet-53 structure is composed of a series of 1 × 1 and 3 × 3 convolutional layers.
As a further improvement of the invention, said Darknet-53 structure consists of a series of1X 1 and3X 3 convolutional layers, each followed by a BN layer and a LeakyReLU layer, there being a total of 53 (2+1X 2+1+2X 2+1+8X 2+1+4X 2+1) convolutional layers in Darknet-53.
As a further improvement of the invention, the multi-scale bounding box prediction structure is used for performing the prediction of the bounding box on three feature layers with different sizes of 52 × 52, 26 × 26 and 13 × 13 respectively.
As a further improvement of the method, the target boundary frames are predicted on three feature layers with different sizes, convolution prediction is carried out through (4+1+ c) x k convolution kernels with the size of 1 x 1, k is the number of preset boundary frames, k is taken asdefault 3, c is the number of classes of the predicted target, 4k parameters are responsible for predicting the offset of the target boundary frames, k parameters are responsible for predicting the probability that the target is contained in the target boundary frames, and ck parameters are responsible for predicting the probability that the k preset boundary frames correspond to the c target classes.
The invention has the following advantages:
according to the detection method of the metal blank cap surface based on deep learning, provided by the embodiment of the invention, the machine vision is adopted to replace human eyes to detect and judge the defects, relevant information is firstly extracted from an image of an objective target, then the relevant information is processed through an intelligent algorithm, and the final processing result is applied to the detection of an actual product. The detection method of the metal blank cap surface based on deep learning provided by the embodiment of the invention solves the problems of low detection speed and low precision of the traditional manual visual defect detection, and also solves the problems of limited application fields and production scenes of other traditional defect detection.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 and fig. 2, a schematic flow chart of a detection method for a metal blank cap surface based on deep learning is provided. In this embodiment, the detection method of the metal blank cap surface based on deep learning includes three steps, and the details of each step are as follows.
And step S1, analyzing the normal surface of the metal blank cap and the defect surface of the metal blank cap, and determining the defect type and defect characteristics of the surface of the metal blank cap to be detected. The surface of the metal blank cap is a circular part with a certain radian on the surface. The modern manufacturing industry has more and more strict requirements on the surface quality of processed parts, and the quality of the surface quality not only affects the appearance and shape of the product, but also more likely affects the functions of the product.
In the steps of the embodiment, a manual visual inspection method is adopted to select the surface of the metal blank cap to manufacture a data set, and the metal blank cap is divided into a qualified product and a defective product by comparing the normal surface and the defective surface of the metal blank cap, wherein the qualified product is a positive sample, and the defective product is a negative sample, and various defects needing to be detected exist on the surface of the defective product. As shown in fig. 3, the types of blank cap surface defects include: scratches, abrasions, stains; the parameters include "x, y, w, h", x being the abscissa of the predicted bounding box center position, y being the ordinate of the predicted bounding box center position, w being the width of the predicted bounding box, and h being the height of the predicted bounding box.
Step S2: and acquiring an image of the surface of the metal blank cap to be detected by adopting a camera, and processing the image to obtain an image data set. The metal blank cap part is small in size, the surface defects of the metal blank cap part are different in size, and in order to ensure that the quality of the collected image meets the detection requirement, a blank cap surface picture needs to be obtained through a specific data collection mode. In this embodiment, an image capture mode based on an industrial camera is defined: on putting the thing platform, keep the detection surface up and place in putting thing platform central point with the metal stifle, set up the light source and play a face structure light in the perpendicular top of metal stifle, set up the camera and shoot the collection image in the perpendicular top of metal stifle.
The acquired images are input into a convolutional neural network for training, label defect types and defect areas of the images of the surface of the metal blank cap to be detected acquired by a camera by using labeImg software, obtain corresponding XML label files after each image is labeled, and respectively store the labeled images and the XML label files. The XML labeling file is provided with parameters of the center coordinates (x, y), the width (w) and the height (h) of a labeling frame; in order to prevent the image from being distorted due to the difference of the aspect ratio during the resampling process, the acquired rectangular-shaped image resize is composed of 1:1 images with the resolution of 416 × 416, and then the resampling is performed.
Step S3: and constructing a detection model of the surface defects of the metal blank cap according to the image data set, extracting features of different layers of the image by adopting an improved deep convolution network, and learning and obtaining a defect type and range detection model based on the features, wherein the defect type and range detection model can detect the positions of the surface defects in the image. Taking the marked image and the marked file obtained by the marking in the step S2 as a training data set; calling an API (application program interface) for Google target detection to train; before training, modifying the configuration file of Yolov3, wherein the main part of modification is the path and the iteration number of the data set, and in this embodiment, the set iteration number is 100000; and finally, performing model training through a command line to obtain a metal blank cap surface defect detection model.
The integral structure of the Yolov3 model is shown in fig. 4. The Yolov3 model is an end-to-end network and is mainly divided into two parts: the first part is Darknet-53 network structure, the network structure is mainly composed of a series of 1 × 1 and 3 × 3 convolutional layers (each convolutional layer is followed by a BN layer and a LeakyReLU layer), the network comprises 53 resonant layers including the last Connected layer, the size of the network initial input image is 416 × 416, and the size of the final output feature map is 13 × 13; the second part is a multiscale prediction bounding box network structure, a group of convolutions are respectively carried out on feature layers with the sizes of 52 x 52, 26 x 26 and 13 x 13 obtained by convolution in a Darknet-53 network structure, the convolution operation sequence is 1 x 1 convolution kernel convolution, 3 x 3 convolution kernel convolution, 1 x 1 convolution kernel convolution, 3 x 3 convolution kernel convolution and 1 x 1 convolution kernel convolution, and finally 3 feature layers with different sizes are obtained.
After three feature layers with different sizes are obtained, convolution prediction is carried out on the feature layers through (4+1+ c) x k convolution kernels with the size of 1 x 1, k is the number of preset bounding boxes (bounding box priors), 3 is taken as a default k, and c is the number of classes of a predicted target, wherein 4k parameters are responsible for predicting the offset of the target bounding boxes, k parameters are responsible for predicting the probability that the target is contained in the target bounding boxes, and ck parameters are responsible for predicting the probability that the k preset bounding boxes correspond to the c target classes. As shown in fig. 4, the dotted rectangle is a preset boundary, and the solid rectangle is a predicted boundary calculated from the offset of the network prediction. Wherein (c)x,cy) For the center coordinates of the preset bounding box on the feature map, (p)w,ph) To preset the width and height of the bounding box on the feature map, (t)x,th,tw,th) Are respectively netsBounding box center offset (t) of the envelope predictionx,ty) And aspect ratio (t)w,th),(bx,by,bw,bh) Is the final predicted target bounding box.
In order to accelerate network convergence, the left side of the center of a preset bounding box needs to be fixed in a cell, a sigmoid function is used to scale the prediction offset between (0, 1), and the scaling formula is shown as formula 1:
bx=(tx)+cx
by=(ty)+cy
the bounding box size is preset on each feature map as shown in table 1:
TABLE 1 Preset bounding Box
Target confidence may be understood as the probability of predicting the presence of a target within a rectangular box of the target, the target confidence loss L
conf(o, c) use is made of Binary Cross Entropy losses (Binary Cross Entropy), where o
iE {0, 1} represents whether the target really exists in the predicted target bounding box i, 0 represents not existing, and 1 represents existing.
Sigmoid probability representing whether or not there is an object within the predicted object rectangular box i, whichThe formula is shown in formula 2:
Target class penalty L
cla(o, c) also used are binary cross-entropy losses, where o
ijE {0, 1} represents whether the jth class target really exists in the predicted target bounding box i, 0 represents not existing, and 1 represents existing.
The Sigmoid probability of the j-th class target in the network prediction target boundary box i is represented, and the formula is shown in formula 3:
Loss of target location L
loc(l, g) using the sum of squares of the difference between the true deviation value and the predicted deviation value, wherein
Indicating the predicted rectangular box coordinate offset,
indicating the coordinate offset between the bounding box and the default box that matches it, (b)
x,b
y,b
w,b
h) For the predicted target rectangle frame parameter, (c)
x,c
y,c
w,c
h) As default rectangular box parameter, (g)
x,g
y,g
w,g
h) For the matched real target rectangular frame parameter, the formula is as followsFormula 4:
According to the detection method of the metal blank cap surface based on deep learning, provided by the embodiment of the invention, the machine vision is adopted to replace human eyes to detect and judge the defects, relevant information is firstly extracted from an image of an objective target, then the relevant information is processed through an intelligent algorithm, and the final processing result is applied to the detection of an actual product. The detection method of the metal blank cap surface based on deep learning provided by the embodiment of the invention solves the problems of low detection speed and low precision of the traditional manual visual defect detection, and also solves the problems of limited application fields and production scenes of other traditional defect detection.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.