The defect inspection method on the irregular shape intermetallic composite coating surface based on deep learningTechnical field
The present invention relates to a kind of intermetallic composite coating detection method of surface flaw, more particularly, to it is a kind of based on deep learning notThe defect inspection method on regular shape intermetallic composite coating surface.
Background technique
It is an essential process in metal works production and manufacture field, defects detection；Due to producing work at presentThe complexity of part structure carries out defects detection to a certain extent or by human resources.Mainstream is artificial in the prior artVisual detection mode not only inefficiency, and examination criteria subjective factor is big, affects the entire production line to a certain extentAutomated process.In addition, the management to company human resource is also a kind of test with the continuous improvement of human cost.In recent yearsComing, the automation defect inspection method based on machine vision is of interest by numerous researchers, also increasingly by the favor of manufacturer,But current methods detection accuracy is low and time-consuming, is not able to satisfy real-time detection demand, becomes and restricts machine substitution mankind's progressThe principal element of defects detection.
Method used in the prior art and the test object being directed to all are not quite similar, and image source acquisition aspect includes usingLaser sensor, ultrasonic sensor, electromagnetic sensor etc.；The object of detection generally compares the surface for being confined to certain objects,Such as metal decking, metal tube surface etc..For image, there are the diversity of shape edges on intermetallic composite coating surface, merelyIt is unsatisfactory that detection effect is carried out using traditional image processing method.And template is used for the finished surface of regular shapeThe robustness that matched method carries out defects detection is not strong, and to the more demanding of imaging circumstances, application scenarios are relatively fixed,For test object it is more single, there is limitation, and cannot classify well to the defect detected.In addition, rightFor intermetallic composite coating workpiece, the station of detection is more, and surface is also more complicated, and the prior art does not reach requirement and has higherThe processing time, unsuitable real-time detecting system builds.Therefore to realize the promotion of productivity in addition to the precision in detectionIt is upper it is more demanding other than, also there is very high requirement to the time of detection processing.
Summary of the invention
It is an object of the present invention to overcome the above-mentioned drawbacks of the prior art and provide one kind to be based on deep learningIrregular shape intermetallic composite coating surface defect inspection method.
The purpose of the present invention can be achieved through the following technical solutions:
The defect inspection method on the irregular shape intermetallic composite coating surface based on deep learning, this method include following stepIt is rapid:
S1: the original image on acquisition irregular shape intermetallic composite coating surface.
S2: pre-processing collected original image, obtains ROI (region of interest, region of interestDomain).The step specifically includes:
201) piecemeal is numbered after being cut original image；
202) image after cutting is filtered and edge detection process；
203) Morphological scale-space is carried out to image, obtains multiple connected regions, connected region is marked and is classified, it is rightSorted connected region carries out the calculating of average pixel value；
204) average pixel value of calculating is judged, the highest connected region of selection average brightness, which is used as, to be needed to extractFeature detection zone；
205) the connected region degree of comparing of selection is enhanced, obtains ROI.
Preferably, the image after cutting is filtered using Gaussian filter.
Preferably, edge detection process is carried out to the image after cutting using Canny operator extraction method.
S3: using pretreated ROI as initial data set, the artificial mark of priori knowledge is carried out to initial data set,Each frames images is taken out into defect using rectangle frame, and marks actual defect classification, obtains the sample infused with defective collimation markThis.Then sample is expanded by data enhancing, using the sample after expansion as the data set of training network.
Data enhancing includes the one or more of following operation:
A. image is overturn, including flip horizontal and spun upside down；
B. Illumination adjusting is carried out to image, so that image becomes brighter or darker；
C. noise addition is carried out to image；
D. affine transformation, including translation, rotation, change of scale are carried out to image.
S4: deep learning detection network is built based on YOLOv3.Specific steps are as follows:
401) deep learning detection network is built using 53 layers of convolutional network, feature extraction is carried out to ROI；
402) feature of extraction is sent into classifier and returns device, obtain the size and location of defect；
402) each image is compared with the obtained result of device is returned with true value by classifier, according to comparingAs a result the parameter of percentage regulation study detection network.
Preferably, residual error network knot is interspersed among the convolutional layer and pond layer of 53 layers of the convolutional networkStructure.
S5: network is detected using the sample training deep learning of the defective collimation mark note of the enhanced band of data, obtains defectDetection model.
S6: the defects detection model that the acquisition image input training for being used to detect is obtained obtains defeated with rectangle box formDefective locations and confidence level out work back to the defective locations of output in original image according to the number of image, obtain originalThe defects of image position.
The data format of each rectangle frame includes the coordinate and rectangle frame of the coordinate of upper left corner angle point, lower right corner angle pointLength and width.
Compared with prior art, the invention has the following advantages that
1, the present invention has carried out pretreatment, image enhancement to the image on the irregular shape intermetallic composite coating surface of acquisition, andDeep learning detection network is built based on YOLOv3 and carries out failure prediction, the candidate region of target detection is generated, feature mentionsIt takes, classify, within four basic steps unifications to the same depth network frame of position refine, all calculating do not repeat, greatlyThe speed of service is improved greatly；
2, the pretreatment that the present invention has carried out including subregion cutting, filtering processing, edge detection original image comesProminent features detection zone is obtained, subregion cutting is exaggerated original image, increases the ratio of defect on the image, makes subsequent lackSunken processing is more accurate；In addition, interference region is filtered by filtering processing, edge detection, reduce the processing of down-streamAmount, improves recall rate, reduces error rate；
3, the present invention enhances technology pair using the data for including the operations such as overturning, illumination variation, addition noise, affine transformationTraining sample carries out data enhancing, has expanded the scale of data set, has solved the problems, such as the sample scarcity of actual production, and enhanceThe robustness and generalization ability of model；
4, the present invention uses deep learning frame, has broken traditional based on the manual extractions feature such as edge, profile, gray scaleDetection method parameter setting puzzlement；In addition, each picture by classifier and is returned the obtained result of device and trueValue is relatively adjusted the parameter of network, and the traditional method for manually adjusting parameter for the model ratio for obtaining training has moreStrong robustness；
5, the present invention is realized using image variants to the defects detection on irregular shape intermetallic composite coating surface, is comparedIn the defects detection for manually carrying out workpiece, human cost is saved, production efficiency is improved；
6, the present invention builds deep learning detection network using YOLOv3 and carries out failure prediction, can be with by adjusting threshold valueDetect different degrees of defect, suitable for multiplicity examination criteria, for different standards can by retraining to model intoRow perfect, strong applicability and using flexible；
7, the present invention is based on the detection frameworks of YOLOv3, have built a set of deep learning net suitable for metal surface detectionNetwork improves the rate of detection under the premise of guaranteeing defect detection accuracy, realizes real-time defects detection, further improveThe efficiency of detection；
8, the present invention has very high real-time on the basis of realizing high-precision detection defect, can satisfy actual productionRequirement of the middle defects detection production line to time and efficiency, is conducive to building for later period entire detection system.
Detailed description of the invention
Fig. 1 is the flow diagram of the method for the present invention；
Fig. 2 is the extraction flow chart of the characteristic area in the method for the present invention；
Fig. 3 is the schematic diagram of the bounding box prediction in YOLOv3.
Specific embodiment
The present invention is described in detail with specific embodiment below in conjunction with the accompanying drawings.
Embodiment
The defect inspection method on the present invention relates to a kind of irregular shape intermetallic composite coating surface based on deep learning is such as schemedShown in 1, this method includes the following steps:
Step 1: the original image on acquisition irregular shape intermetallic composite coating surface.
Step 2: carrying out pretreatment to collected original image obtains ROI, and ROI is cut into the identical side of sizeShape subgraph, to adapt to the input requirements of deep learning network.
Acquired image is the reflected image under same angle light source, but due to the profile and shape of metal surface to be measuredShape is not quite similar, so also will include the information on some non-surfaces to be measured sometimes.Therefore it needs to carry out centainly original imagePretreatment, prominent features detection zone and filters out interference region, and will treated that image carries out dimension normalization, it is unified deepSpend the input picture of network.On the one hand the treating capacity of down-stream can be reduced, detection speed is improved, on the other hand to extractingRegion has carried out partial enlargement, and it is more prominent to make that there may be the positions of defect, i.e., defect characteristic is more prominent, for subsequent spySign extraction step is ready, and the final recall rate that improves reduces error rate.When carrying out the training of model using deep learning in this wayThe consistency that can guarantee background type and graphical rule, enables model preferably to restrain, and improves the accuracy of defects detection.
Acquired image region may include large-scale workpiece surface region and perimeter, and due to workpieceThe profile on surface is different, and large-scale characteristic area extracts the accuracy that will affect extraction.Based on this, as shown in Fig. 2, of the inventionPiecemeal is numbered after original image is cut first.By designing good imaging scheme, can make collected originalImage has the differentiation of apparent background and prospect, wherein highlighted region is the intermetallic composite coating surface detected,The relatively clear protrusion of its profile.
It is then filtered, so that image is more smooth, while the distracter of dash area being desalinated.FilteringMethod has many kinds, the preferred Gaussian filter of the present embodiment；Then edge detection is carried out to image, the edge inspection often occurredMeasuring and calculating attached bag includes the first order differential operator including Robert operator, Sobel operator and the second order including Laplace operator is micro-Divide operator.The above operator mentions image edge detailss by the method completion of gray level image and a certain size mask convolutionIt takes, it is easy to accomplish, there is good real-time, but be easily disturbed by noise, it is easily lost edge details.The present invention usesCanny operator extraction edge.Canny edge detection algorithm be it is a kind of will optimize thought be applied to image procossing algorithm, withConventional differential operator is compared, and not only possesses higher noise output ratio, also has quite reliable precision.
Edge is obtained later by carrying out the connected region that morphologic operation obtains multiple sealings to image, then to evenLogical region is marked and classifies, and asks calculation to sorted connected region progress average pixel value, judges average brightness mostThe feature detection zone that high connected region as needs to extract.It can will be to be detected by choosing the connected regionWorkpiece surface region is split from original image, other connected domains directly fill black replacement, can greatly reduce background pairThe interference of defect recognition.
Final step is connected region degree of the comparing enhancing to extraction.Between different piecemeals, due to the light of imagingSource problem may have different Luminance Distributions.Therefore the enhancing to degree of comparing in connected region is also needed, is lacked with prominentSunken feature obtains ROI (region of interest, area-of-interest).The present invention selects histogram equalization, this methodThe enhancing to realize contrast is adjusted to gray value by using aggregation function." the middle thought of histogram equalization processingThink " it is that the grey level histogram of original image is become in whole tonal ranges uniform from some gray scale interval for comparing concentrationDistribution.The defect details in the insufficient region of illumination can also be protruded in this way, while also can be thin by the defect for exposing excessive regionSection highlights.
Step 3: carrying out handmarking to sample, the sample of tape label is obtained.
Using pretreated ROI as initial data set, need to carry out the mark of priori knowledge.For each inputIts major defect progress frame is taken by rectangle frame, and marks actual defect classification by image.The data format of one templateFor the coordinate in the upper left corner and lower right corner angle point, respectively (x1, y1), (x2, y2), the length and width of rectangle frame.For each knowledgeOther frame can all have corresponding label, the classification of the tag representation defect.
Step 4: carrying out data extending to initial data set using data enhancing technology, trained sample more abundant is obtainedThis.
To guarantee that model has very strong generalization ability, it is contemplated that the actually detected environment of metal processing piece in the factory,From scale, rotation, translation, illumination plus makes an uproar etc. data enhancing is carried out to image training sample.Particular content includes:
1, image is overturn, including flip horizontal and spun upside down.
2, Illumination adjusting is carried out to image, so that image becomes brighter or darker.
3, noise addition is carried out to image, the noise of the present embodiment addition is Gaussian noise.
4, affine transformation, including translation, rotation, change of scale are carried out to image.
It should be noted that above-mentioned data enhancement operations can be used in combination with each other, type available in this way is more, lacksRicher image data is fallen into, the training of model is conducive to.The identification frame wherein marked can keep opposite position with various transformationIt sets constant, is equivalent to have obtained a greater variety of defect sample from the point of view of sample.
Step 5: building deep learning detection network based on YOLOv3.
One 53 layers of convolutional network (Darknet-53) has been used in deep learning frame of the invention, the network number of pliesIncrease, may make the performance enhancement of network, prediction effect is more preferable.During deep neural network is built, in convolutional layer andResidual error network structure is interted among the layer of pond, network number of plies increase can have been solved, the accuracy rate of training set is caused to be saturated even declineThe problem of.
It is pre- that bounding box prediction uses dimension cluster (dimension clusters) to come as anchor point (anchor boxes)Bounding box is surveyed, the output of network is that each bounding box predicts 4 coordinates, and the meaning of 4 coordinates is respectively the central point predictedHorizontal, ordinate bx、byAnd the width b of bounding boxw, height bh, calculate as shown in Figure 3.cx、cyRespectively refer to the grid dividedThe distance in the ranks direction in the range image upper left corner unit (grid cell), pw、phRespectively refer to the width and height of priori bounding boxDegree, σ () function refer to sigmoid function, high by the position of grid cell and the width of priori bounding box, calculate prediction sideThe center of boundary's frame and width, height.
YOLOv3 predicts the score of each bounding box using logistic regression.If the degree of overlapping of priori bounding box and true frameWill be good than any other bounding box before, then the value should be 1.If priori bounding box is not best, but really withThe overlapping of real object is more than some threshold value (the present embodiment chooses 0.5), then ignores current prediction.YOLOv3 is only each trueObject distributes a bounding box, if priori bounding box is misfitted with real object, will not generate coordinate or class prediction damageIt loses, can only generate object prediction loss.
The YOLOv3 network architecture carries out prediction task from the characteristic spectrum of three kinds of different scales.It is obtained in Darknet-53Characteristic pattern on the basis of, obtain first characteristic spectrum by 7 convolution, done on this characteristic spectrum for the first time predict.Then the output for obtaining 3rd convolutional layer reciprocal from back to front carries out x2 up-sampling of a convolution, will up-sampling feature with43rd convolution feature connection, obtains second characteristic spectrum by 7 convolution, and second is done on this characteristic spectrum in advanceIt surveys.Then the output for obtaining the 3rd convolutional layer reciprocal from back to front carries out x2 up-sampling of a convolution, and up-sampling is specialSign is connect with the 26th convolution feature, obtains third characteristic spectrum by 7 convolution, and third time is done on this characteristic spectrumPrediction.The feature sizes that each prediction task obtains are N × N × [3* (4+1+80)], and N is grid size, and 3 be each gridObtained bounding box quantity, 4 be bounding box coordinates quantity, and 1 is target prediction value, and 80 be categorical measure.
After carrying out feature extraction to ROI using 53 layers of convolutional networks, the feature extracted is sent into classifier and recurrenceDevice, wherein classifier effect is that judgement has whether the picture of a certain feature belongs to defect, and the effect for returning device is to judge certainThe size of defect included by kind feature and position.Each picture by classifier and is returned into the obtained result of device and trueReal value is compared, the parameter of adjustable network, to achieve the purpose that e-learning and evolution, finally make network output with it is trueValue as close as.
The entire data set obtained after data enhancing is divided into training data and test data: will be in labeled data collection80% be used as training dataset, 20% be used as validation data set.Trained parameter is adjusted simultaneously, in total the number of iterationsIt is set as 50000 times to be trained, GPU (Graphics Processing Unit, graphics processor) can be used to improve operationRate.The data set of the tape label of building carries out the training of deep learning network as network inputs, obtains defects detection model,For carrying out defects detection to every tension position picture.
Step 6: carrying out defect inspection to collected intermetallic composite coating surface image using the defects detection model that training obtainsIt surveys, obtains the result of defects detection.
The defects detection model that the acquisition image input training for being used to detect is obtained.The position of defect is with rectangle frame(coordinate of rectangle upper left angle point and the length and width of rectangle frame) that form provides, then according to the number of image, by outputDefective locations work back in original image, to guarantee the defective locations of final system output as the position in original image.It is practicalApplication requirement in, can defect the length and width of rectangle frame less than 0.1 centimetre ignore.When statistical shortcomings, if rectangleThe length and width of frame is respectively less than 0.1 centimetre, then will not output it.In addition, the result that model judges is saved in text, saveThe defective state of content (YES indicate retain defect, NG expression ignore the defect), defective locations and confidence level.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, anyThe staff for being familiar with the art in the technical scope disclosed by the present invention, can readily occur in various equivalent modifications or replaceIt changes, these modifications or substitutions should be covered by the protection scope of the present invention.Therefore, protection scope of the present invention should be with rightIt is required that protection scope subject to.