Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a wire harness crimping defect detection system based on machine vision, which comprises:
the multispectral light source control module is used for adopting an annular LED light source and coaxial light for mixed illumination, and adapting to different wire beam surface materials by dynamically adjusting the angle and the intensity of the light source;
The multidimensional sensing acquisition module is used for acquiring harness image data and crimping geometric data in real time based on the dynamically adjusted angle and intensity of the light source;
The visual enhancement fusion module is used for denoising the harness image data by utilizing a non-local mean denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor, superposing a multi-scale Retinex algorithm and enhancing the texture contrast of the harness image data through local contrast;
The system comprises a crimping defect detection module, a wire harness crimping detection module and a wire harness crimping detection module, wherein the crimping defect detection module is used for taking a multimodal characteristic diagram of a crimping part as input of a crimping defect detection model, predicting to obtain crimping defect probability;
The defect grade classification module automatically collects crimping defect data if the wire harness crimping is defective, inputs the crimping defect data into a trained defect grade prediction model, and predicts the defect grade of the wire harness crimping;
The real-time alarm module is used for classifying alarm grades according to the defect grades of the wire harness crimping, triggering audible and visual alarm to the defect detection terminal, and the modules are connected in a wired and/or wireless mode.
Preferably, the method for dynamically adjusting the light source and adapting to different wire beam surface materials by adopting the mixed illumination of the annular LED light source and the coaxial light comprises the following steps:
acquiring harness image data through a camera, classifying harness surface materials, and presetting the harness surface materials as follows, wherein,Representing a copper core; Representing the coating; representing the insulating layer, considering the reflectivity of the wire harness surface material, dividing the wire harness surface material into a high-reflectivity material, a medium-reflectivity material and a low-reflectivity material, wherein the reflectivity of the wire harness surface material is defined as: Wherein, the method comprises the steps of,The reflectivity of the material of the surface of the wire harness; is the intensity of reflected light; is the intensity of incident light;
Presetting a first threshold value of reflectivityAnd a second threshold value of reflectivityIf (if)Judging the surface material of the wire harness to be a high-reflectivity material, if soJudging the surface material of the wire harness to be a medium reflectivity material, if soSelecting different light source combinations according to different imaging effects of the wire harness surface material under different light sources, wherein the different light source combinations compriseWherein, the method comprises the steps of,Is an annular LED light source; Is a coaxial light source;
after the material of the surface of the wire harness is identified and different light source combinations are selected, the light source angle and the light source intensity are dynamically adjusted according to the current range contrast of the wire harness image data by a light source angle adjusting formula and a light source intensity adjusting formula, wherein the range contrast is as follows: Wherein, the method comprises the steps of,The maximum brightness value in the harness image data; The minimum brightness value in the harness image data;
the light source angle adjusting formula is: Wherein, the method comprises the steps of,The adjusted light source angle is obtained; is the initial light source angle; is a light source angle adjusting factor; very poor contrast for the target; is the current very poor contrast;
the light source intensity adjustment formula is: Wherein, the method comprises the steps of,The intensity of the light source after being regulated; Is the initial light source intensity; is a reflectivity factor; the reflectivity of the surface material of the current wire harness; Is a light source intensity adjustment factor;
the reflectivity factor is limited and restrained through a reflectivity factor limiting formula, and the reflectivity factor limiting formula is as follows: Wherein, the method comprises the steps of,The number of the material categories of the surface of the wire harness; A weight factor for the current very poor contrast; and a weight factor for the number of the wire harness surface material categories.
Preferably, the harness image data includes a harness surface image, a crimp terminal image, and a crimp location image, and the crimp geometry data includes a height, a volume, a crimp angle, a thickness, and a diameter of the crimp location.
Preferably, the method for denoising the beam image data by using a non-local mean denoising algorithm and controlling the similarity measurement range through dynamic adjustment of a smoothing factor comprises the following steps:
For each pixel point in the harness image dataComputing its edge strength using an edge detection algorithmWherein, the method comprises the steps of,Is a measure of edge strength; for each pixel pointPixel values at; an abscissa for each pixel; An ordinate for each pixel;
For any pair of pixel points in an imageAndCalculating an adaptive similarity measure through a similarity measure formula;
the similarity measurement formula is: Wherein, the method comprises the steps of,Representing pixel pointsAndA similarity measure between; Is a pixel pointPixel values of (2); Is a pixel pointPixel values of (2); Is a pixel pointGradient information at; Is a pixel pointGradient information at; adjusting factors for controlling gray scale difference and gradient information weight; A smoothing factor for controlling the similarity measure range;
smoothing factor for controlling similarity measurement range through smoothing factor limiting formulaAnd carrying out dynamic restriction, wherein a smoothing factor restriction formula is as follows: Wherein, the method comprises the steps of,The total pixel number of the harness image data; is the sum of the edge intensities; An adjustment factor for controlling the influence of the total pixel number of the harness image data on the smoothing factor; An adjustment factor for controlling the influence of the edge strength on the smoothing factor;
based on edge strengthModifying the similarity measure by modifying the similarity measure formula to enhance the weight of the edge region;
The modified similarity metric formula is: Wherein, the method comprises the steps of,Is a pixel pointAndEuclidean distance between them; Is a pixel pointEdge strength at; an adjustment factor for controlling the effect of edge weighting;
And carrying out weighted average according to the similarity weight of each pixel point and the neighborhood pixels thereof by using a non-local mean denoising formula so as to remove noise, wherein the non-local mean denoising formula is as follows: Wherein, the method comprises the steps of,Representing pixels in denoised harness image dataIs a value of (2); Representing the pixel location currently being processed; Representing pixelsEach pixel in the neighborhood of (2);Representing neighborhood pixelsIs a pixel value of (a).
Preferably, the method for enhancing texture contrast of harness image data by local contrast by superimposing a multi-scale Retinex algorithm comprises the following steps;
blurring the harness image data by different scale Gaussian filters to obtain a multi-scale image, each scaleBlurring of the harness image data using convolution operations: Wherein, theRepresenting the passing scaleThe blurred image processed by the Gaussian filter is processed in the pixel pointPixel values on; Representing a gaussian kernel function; representing a convolution operation;
For each scaleEach pixel point is calculatedPixel value atAnd (3) withRatio of (1): Wherein, the method comprises the steps of,Is of a scaleA lower ratio image;
based on the ratio calculation, the image with the comparison value is adjusted through a local contrast enhancement formula, wherein the local contrast enhancement formula is as follows: Wherein, the method comprises the steps of,Is an image subjected to local contrast enhancement; to adjust the parameter factor of the enhanced intensity;
the ratio images from different scales are weighted and summed, and enhancement information of each scale is fused to obtain a multi-scale enhancement image, wherein the multi-scale enhancement image is as follows: Wherein, the method comprises the steps of,Is a multi-scale enhanced image; Is the number of dimensions; Is of a scaleA lower ratio image; Is of a scaleWeight coefficient of (2); Is the firstA dimension; Index for scale;
The inverse logarithmic mapping is used for restoring the dynamic range of the multi-scale enhanced image, and the multi-scale enhanced image is restored to the brightness range of the harness image data by performing exponential mapping on the multi-scale enhanced image.
Preferably, the method for fusing harness image data with crimping geometry data to construct a crimping position multi-modal feature map comprises the following steps:
Performing point cloud alignment on the wire harness image data and the crimping geometric data in an iterative mode by applying an ICP algorithm, continuously adjusting the relative position and the rotation angle between the two point clouds of the wire harness image data and the crimping geometric data, and searching for the optimal transformation for minimizing the distance between the two groups of point clouds;
And (3) presetting an error threshold, in each iteration process, calculating the distance errors of two groups of point clouds under the current transformation by an ICP algorithm, judging whether the preset error threshold or the maximum iteration number is reached, ending the iteration if the preset error threshold or the maximum iteration number is reached, and integrating the point clouds of the wire harness image data and the crimping geometric data into a whole after the alignment is completed, so as to generate a crimping part multi-mode feature map.
Preferably, the method for constructing the crimping defect detection model comprises the following steps:
The method comprises the steps of dividing a data set into a training set, a verification set and a test set, constructing a crimping defect detection model, wherein a sample set is a subset of the data set, and each sample set comprises a historical crimping part multi-mode characteristic map and corresponding crimping defect probability;
The crimping defect detection model comprises an input layer, a convolution layer, a pooling layer, a full-connection layer and an output layer, wherein the input layer is a multi-mode characteristic diagram of a historical crimping part, and the output layer is crimping defect probability;
Using the mean square error as a loss function, and measuring the error between the predicted value and the actual value of the model; training the dangerous probability prediction model by using a training set, and updating model parameters by using a back propagation algorithm and a gradient descent method to minimize a loss function;
And evaluating the performance of the model in a prediction task by using a test set, and predicting a multi-mode feature map of the current crimping part by using a trained crimping defect detection model to obtain crimping defect probability.
Preferably, the method for judging whether the wire harness crimping is defective according to the predicted crimping defect probability comprises the following steps:
presetting a crimping defect probability threshold, and comparing the predicted crimping defect probability with the preset crimping defect probability threshold;
If the predicted crimping defect probability is smaller than a preset crimping defect probability threshold, judging that the wire harness crimping is not defective;
if the predicted crimping defect probability is greater than or equal to a preset crimping defect probability threshold, judging that the wire harness crimping is defective.
Preferably, the training method of the defect level prediction model includes:
Dividing a data set into a training set, a testing set and a verification set, and constructing a defect level prediction model, wherein the defect level prediction model comprises an input layer, a hidden layer and an output layer, the input layer of the model is used for inputting historical crimping defect data, the output layer is used for outputting defect levels of wire harness crimping, the output layer is provided with neurons with the same number as the defect levels of wire harness crimping, each neuron corresponds to the prediction probability of a defect level type of wire harness crimping, and a softmax function is used as an activation function;
the method comprises the steps of using multi-classification cross entropy as a loss function of a model, measuring the difference between a predicted value and an actual value of the model, using a training set to train an error type identification model, updating model parameters through a back propagation algorithm to minimize the loss function, using a verification set to evaluate the performance of a defect level prediction model through calculating an accuracy index;
and evaluating the performance of the model in a prediction task by using a test set, and predicting the current crimping defect data by using a trained defect level prediction model to obtain the defect level of the wire harness crimping.
A wire harness crimping defect detection method based on machine vision comprises the following steps:
S1, adopting an annular LED light source and coaxial light for mixed illumination, and adapting to different wire beam surface materials by dynamically adjusting the angle and the intensity of the light source;
S2, acquiring harness image data and crimping geometric data in real time based on the dynamically adjusted angle and intensity of the light source;
S3, denoising the harness image data by utilizing a non-local mean denoising algorithm and controlling a similarity measurement range through dynamic adjustment of a smoothing factor, superposing a multi-scale Retinex algorithm, enhancing texture contrast of the harness image data through local contrast, fusing the harness image data with crimping geometric data, and constructing a crimping part multi-mode feature map;
S4, taking the multi-mode feature map of the crimping part as input of a crimping defect detection model, and predicting to obtain crimping defect probability;
S5, if the wire harness crimping is defective, automatically collecting crimping defect data, inputting the crimping defect data into a trained defect grade prediction model, and predicting to obtain the defect grade of the wire harness crimping;
And S6, classifying alarm grades according to the defect grades of the wire harness crimping, and triggering audible and visual alarm to the defect detection terminal.
Compared with the prior art, the invention has the following beneficial effects:
According to the invention, the angle and the intensity of the light source are dynamically adjusted, so that the wire harness surface materials with different reflectivities can obtain the optimal illumination condition, clear and accurate image data can be obtained no matter the wire harness surface materials are made of high-reflectivity materials, medium-reflectivity materials or low-reflectivity materials, and image blurring or detail loss caused by too strong or too weak illumination is avoided; the angle and the intensity of the light source are regulated according to the extremely poor contrast of the harness image data, the dynamic range of the image can be optimized, the details of the image are enhanced, particularly in the transition area between the high-reflection surface and the low-reflection surface, the contrast of the image is improved, defects on the harness surface are more obvious, subsequent defect detection is facilitated, and the illumination effect in the image is adapted to the requirements of different materials by accurately regulating the angle and the intensity of the light source, so that the presentation precision of the details of the image is improved. The system has a self-adaptive light source adjusting function, can automatically adjust light source parameters according to different wire harness surface materials and the reflection characteristics of the current image, reduces manual intervention, adopts a mixed illumination technology of an annular LED light source and a coaxial light source, can combine the advantages of the two light sources, and adapts to wire harnesses with different reflectivity materials;
By dynamically adjusting the smoothing factors in the similarity measurement and combining the weight of the edge strength, the weight of the edge area is enhanced, so that the edge information can be better reserved in the denoising process. The method solves the problem that the details of the image, particularly the edge details, are often blurred in the traditional denoising method, so that the quality of the denoised image is improved, the edge definition of the image is enhanced, and the similarity measurement is adjusted according to the gray values and gradient information of different pixels by adopting a self-adaptive similarity measurement formula and controlling the weights of gray difference and gradient information, so that the noise area and the detail area in the image are more accurately identified. The noise in the image is removed by carrying out weighted average on the similarity weight of each pixel point and the neighborhood pixels thereof based on a non-local mean value denoising formula, and meanwhile, the detail part of the image is kept, and particularly, the reservation of high-frequency signals such as textures and edge parts is more outstanding;
The method effectively enhances details of different levels in an image through the introduction of a multi-scale Retinex algorithm, combines local contrast enhancement, further improves texture contrast of a harness image, enables tiny details in the image to be more prominent, is beneficial to subsequent defect detection and quality assessment, can adapt to details of different scales through processing the harness image under multiple scales, achieves more comprehensive and accurate enhancement, can solve the problems of illumination difference and local feature protrusion under different scales, improves definition and contrast of the image under each scale, can finely adjust the contrast of a local area of the image through the introduction of a local contrast enhancement formula, improves detail performance, simultaneously avoids overadjustment of an overall brightness range, and can be more balanced through weighting and summing image information of different scales, and adjusting weights of the scales in a final image according to contribution values of different scales.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
Referring to fig. 1, a machine vision-based wire harness crimping defect detection system according to the present invention is further described, and includes:
With the development of industrial automation and intellectualization, the wire harness is widely applied to various electronic devices, especially in the fields of automobiles, aviation, electronic products and the like, and the quality of the wire harness directly influences the safety and stability of the devices. The crimp quality of the wire harness is critical to the reliability of the electrical connection, and therefore it is particularly important to accurately and rapidly detect the crimp defect of the wire harness.
Currently, conventional wire harness crimping defect detection methods rely mostly on manual visual inspection or automatic detection based on conventional image processing. The traditional automatic detection method, especially the detection based on image processing, is often influenced by factors such as illumination conditions, material differences of wire harness surfaces, image noise and the like, so that detection results are inaccurate, even missed detection or false detection cannot meet the requirements of high precision and high efficiency.
In the prior art, although some detection systems based on machine vision exist, the following problems are still faced:
The adaptability of the light source is poor, the reflection characteristics of the light of different wire beam surface materials (such as copper cores, plating layers, insulating layers and the like) are different, the traditional light source cannot be flexibly adapted, the image quality is poor, and the accuracy of subsequent detection is affected.
Noise interference, namely noise generated by factors such as uneven illumination, environmental noise and the like in an image possibly has adverse effects on image quality, and the traditional image denoising method has limited effect and is difficult to effectively remove noise under a complex background.
The detection precision is low, the traditional defect detection algorithm is generally based on simple feature extraction and comparison, and is difficult to accurately identify the fine crimping defects, and particularly under the condition of complex crimping positions and changeable harness surface materials, missed detection or false detection is easy to occur.
In the environment of a high-efficiency production line, the traditional wire harness defect detection method is slow in processing speed, cannot meet the requirement of real-time production monitoring, and increases the risk in the production process.
In order to effectively solve the above problems, the present invention provides a wire harness crimping defect detection system based on machine vision, comprising:
the multispectral light source control module is used for adopting an annular LED light source and coaxial light for mixed illumination, and adapting to different wire beam surface materials by dynamically adjusting the angle and the intensity of the light source;
The multidimensional sensing acquisition module is used for acquiring harness image data and crimping geometric data in real time based on the dynamically adjusted angle and intensity of the light source;
The visual enhancement fusion module is used for denoising the harness image data by utilizing a non-local mean denoising algorithm and controlling the similarity measurement range through dynamic adjustment of the smoothing factor, superposing a multi-scale Retinex algorithm and enhancing the texture contrast of the harness image data through local contrast;
The system comprises a crimping defect detection module, a wire harness crimping detection module and a wire harness crimping detection module, wherein the crimping defect detection module is used for taking a multimodal characteristic diagram of a crimping part as input of a crimping defect detection model, predicting to obtain crimping defect probability;
The defect grade classification module automatically collects crimping defect data if the wire harness crimping is defective, inputs the crimping defect data into a trained defect grade prediction model, and predicts the defect grade of the wire harness crimping;
The real-time alarm module is used for classifying alarm grades according to the defect grades of the wire harness crimping, triggering audible and visual alarm to the defect detection terminal, and the modules are connected in a wired and/or wireless mode.
The method for dynamically adjusting the light source and adapting to different wire beam surface materials by adopting the mixed illumination of the annular LED light source and the coaxial light comprises the following steps:
acquiring harness image data through a camera, classifying harness surface materials, and presetting the harness surface materials as follows, wherein,Representing a copper core; Representing the coating; representing the insulating layer, considering the reflectivity of the wire harness surface material, dividing the wire harness surface material into a high-reflectivity material, a medium-reflectivity material and a low-reflectivity material, wherein the reflectivity of the wire harness surface material is defined as: Wherein, the method comprises the steps of,The reflectivity of the material of the surface of the wire harness; is the intensity of reflected light; is the intensity of incident light;
Presetting a first threshold value of reflectivityAnd a second threshold value of reflectivityIf (if)Judging the surface material of the wire harness to be a high-reflectivity material, if soJudging the surface material of the wire harness to be a medium reflectivity material, if soSelecting different light source combinations according to different imaging effects of the wire harness surface material under different light sources, wherein the different light source combinations compriseWherein, the method comprises the steps of,Is an annular LED light source; The coaxial light source is a light source illumination mode, wherein the light source is collinear with the optical axis of the camera lens. That is, the light propagates along the same axis as the viewing angle of the camera. The illumination mode has the advantages that the interference of reflected light on the surface of the object can be reduced, so that the surface texture and details are clearer, and the illumination mode is particularly suitable for imaging tasks needing to highlight the surface details of the object;
after the material of the surface of the wire harness is identified and different light source combinations are selected, the light source angle and the light source intensity are dynamically adjusted according to the current range contrast of the wire harness image data by a light source angle adjusting formula and a light source intensity adjusting formula, wherein the range contrast is as follows: Wherein, the method comprises the steps of,The maximum brightness value in the harness image data; The minimum brightness value in the harness image data;
the light source angle adjusting formula is: Wherein, the method comprises the steps of,The adjusted light source angle is obtained; for initial light source angle (based on the surface material class of the wire harnessPresetting); the angle of the light source is mainly adjusted as the angle adjusting factor of the light source, namely, how to adjust the angle of the light source relative to the wire harness so as to achieve the optimal imaging effect; very poor contrast for the target; is the current very poor contrast;
the light source intensity adjustment formula is: Wherein, the method comprises the steps of,The intensity of the light source after being regulated; For initial light source intensity (based on the surface material class of the wire harnessPresetting); is a reflectivity factor for controlling the effect of reflectivity on the light source intensity; the reflectivity of the surface material of the current wire harness; The intensity of the light source is mainly regulated as a light source intensity regulating factor, namely, the extremely poor contrast of the image is improved by regulating the brightness of the light source;
the reflectivity factor is limited and restrained through a reflectivity factor limiting formula, and the reflectivity factor limiting formula is as follows: Wherein, the method comprises the steps of,The number of the material categories of the surface of the wire harness; A weight factor for the current very poor contrast; A weight factor for the number of the wire harness surface material categories;
When the reflectivity is high, the intensity of light reflected by the surface is high, and the intensity of the light source may need to be reduced to avoid overexposure;
the amount of the type of material (e.g., copper, plating, insulating layer, etc.) on the surface of the wire harness affects the reflectivity and quality of the final image. Under the influence of different materials, the light source intensity needs to be adjusted differently, the challenges of adjusting the light source intensity by various materials can be solved by dynamically adjusting the number of the materials and the weight factors of the extremely poor contrast, the adaptation of each material is ensured, the extremely poor contrast of the image (namely the brightness difference of the image) also has an influence on the adjustment of the light source intensity, a higher contrast may mean that strong light source adjustment is not needed, and a lower contrast may require the increase of the light source intensity.
The adjustment range of the light source intensity should be limited appropriately under different reflectivity and image contrast conditions. If the reflectance factor is too large or too small, it may cause the image to be overexposed or too dark, affecting quality. By setting the constraint of the reflectivity factor, excessive or insufficient adjustment can be avoided, and the intensity of the light source is ensured to be always in a reasonable range;
for example, the current range contrast10 Number of surface material categories of wire harnessWeight factor of 3 for current range contrastWeight factor of 0.6 for the number of wire harness surface material categories0.4, Then the reflectance factor after confinement is。
The wire harness image data comprises a wire harness surface image, a crimping terminal image and a crimping position image, and the crimping geometric data comprises the height, the volume, the crimping angle, the thickness and the diameter of the terminal of the crimping position.
The method for denoising the beam image data by utilizing a non-local mean denoising algorithm and controlling the similarity measurement range through dynamic adjustment of a smoothing factor comprises the following steps:
For each pixel point in the harness image dataComputing its edge strength using an edge detection algorithmWherein, the method comprises the steps of,Is a measure of edge strength; for each pixel pointPixel value at, i.e., brightness or color intensity of the point in the image; an abscissa for each pixel; An ordinate for each pixel;
For any pair of pixel points in an imageAndCalculating self-adaptive similarity measurement through a similarity measurement formula, and considering pixel value difference and gradient information;
the similarity measurement formula is: Wherein, the method comprises the steps of,Representing pixel pointsAndAnd the similarity measure between the two pixel points represents the similarity of the two pixel points. The larger the value, the more similar the two pixels are, the smaller the difference is. In the denoising process, the similarity between pixel points determines the weight of the pixel points in the weighted average process; Is a pixel pointPixel values of (2); Is a pixel pointPixel values of (2); Is a pixel pointGradient information at; Is a pixel pointGradient information at; adjusting factors for controlling gray scale difference and gradient information weight; A smoothing factor for controlling the similarity measure range;
smoothing factor for controlling similarity measurement range through smoothing factor limiting formulaAnd carrying out dynamic restriction, wherein a smoothing factor restriction formula is as follows: Wherein, the method comprises the steps of,The total pixel number of the harness image data; is the sum of the edge intensities; An adjustment factor for controlling the influence of the total pixel number of the harness image data on the smoothing factor; An adjustment factor for controlling the influence of the edge strength on the smoothing factor;
For example, the total number of pixels of the harness image data100000 Sum of edge intensities20000, A control factor for controlling an influence of a total pixel count of harness image data on a smoothing factorA regulating factor for controlling the influence of edge intensity on the smoothing factor of 0.00011, Then the smoothing factor after restriction is。
Based on edge strengthModifying the similarity measure by modifying the similarity measure formula to enhance the weight of the edge region, thereby ensuring that the noise of the edge region is reduced and the details are preserved;
The modified similarity metric formula is: Wherein, the method comprises the steps of,Is a pixel pointAndEuclidean distance between them; Is a pixel pointEdge strength at; an adjustment factor for controlling the effect of edge weighting;
And carrying out weighted average according to the similarity weight of each pixel point and the neighborhood pixels thereof by using a non-local mean denoising formula so as to remove noise, wherein the non-local mean denoising formula is as follows: Wherein, the method comprises the steps of,Representing pixels in denoised harness image dataBy applying to the pixelsThe weighted average of surrounding neighborhood pixels is obtained, so as to eliminate noise; representing the current pixel position, meaning a pixel point in the image, representing the position in the image, can be considered as a point in the image coordinate system (but in the formula, specific coordinate values are not written, directly usedRepresenting the position); Representing pixelsEach pixel in the neighborhood of (2);Representing neighborhood pixelsIs a pixel value of (a).
The method for enhancing the texture contrast of the harness image data through the local contrast by superposing a multi-scale Retinex algorithm comprises the following steps;
blurring the harness image data by different scale Gaussian filters to obtain a multi-scale image, each scaleBlurring of the harness image data using convolution operations: Wherein, theRepresenting the passing scaleThe blurred image processed by the Gaussian filter is processed in the pixel pointPixel values on; Representing a gaussian kernel function, which is a two-dimensional gaussian function for blurring an image; representing a convolution operation;
For each scaleEach pixel point is calculatedPixel value atAnd (3) withRatio of (1): Wherein, the method comprises the steps of,Is of a scaleThe lower ratio image shows the enhancement effect of the image under the scale;
on the basis of ratio calculation, the high-frequency details in the image are enhanced by adjusting the image with a local contrast enhancement formula: Wherein, the method comprises the steps of,Is an image subjected to local contrast enhancement; to adjust the parameter factor of the enhanced intensity;
the ratio images from different scales are weighted and summed, and enhancement information of each scale is fused to obtain a multi-scale enhancement image, wherein the multi-scale enhancement image is as follows: Wherein, the method comprises the steps of,For multiscale enhanced images, i.e. ratio images of different scales are summed by weightingThe obtained enhanced image; Is the number of dimensions; Is of a scaleA lower ratio image; Is of a scaleWeight coefficient of (2); Is the firstA dimension; for index of scale, the value ranges from 1 to;
The inverse logarithmic mapping is used for restoring the dynamic range of the multi-scale enhanced image, and the multi-scale enhanced image is restored to the brightness range of the harness image data by performing exponential mapping on the multi-scale enhanced image.
The method for fusing the harness image data with the crimping geometry data and constructing the crimping part multi-mode characteristic map comprises the following steps of:
Performing point cloud alignment on the wire harness image data and the crimping geometric data in an iterative mode by applying an ICP algorithm, continuously adjusting the relative position and the rotation angle between the two point clouds of the wire harness image data and the crimping geometric data, and searching for the optimal transformation for minimizing the distance between the two groups of point clouds;
And (3) presetting an error threshold, in each iteration process, calculating the distance errors of two groups of point clouds under the current transformation by an ICP algorithm, judging whether the preset error threshold or the maximum iteration number is reached, ending the iteration if the preset error threshold or the maximum iteration number is reached, and integrating the point clouds of the wire harness image data and the crimping geometric data into a whole after the alignment is completed, so as to generate a crimping part multi-mode feature map.
The method for constructing the crimping defect detection model comprises the following steps:
The method comprises the steps of dividing a data set into a training set, a verification set and a test set, constructing a crimping defect detection model, wherein a sample set is a subset of the data set, and each sample set comprises a historical crimping part multi-mode characteristic map and corresponding crimping defect probability;
The crimping defect detection model comprises an input layer, a convolution layer, a pooling layer, a full-connection layer and an output layer, wherein the input layer is a multi-mode characteristic diagram of a historical crimping part, and the output layer is crimping defect probability;
Using the mean square error as a loss function, and measuring the error between the predicted value and the actual value of the model; training the dangerous probability prediction model by using a training set, and updating model parameters by using a back propagation algorithm and a gradient descent method to minimize a loss function;
And evaluating the performance of the model in a prediction task by using a test set, and predicting a multi-mode feature map of the current crimping part by using a trained crimping defect detection model to obtain crimping defect probability.
The method for judging whether the wire harness crimping is defective or not according to the predicted crimping defect probability comprises the following steps:
presetting a crimping defect probability threshold, and comparing the predicted crimping defect probability with the preset crimping defect probability threshold;
If the predicted crimping defect probability is smaller than a preset crimping defect probability threshold, judging that the wire harness crimping is not defective;
if the predicted crimping defect probability is greater than or equal to a preset crimping defect probability threshold, judging that the wire harness crimping is defective.
The training method of the defect level prediction model comprises the following steps:
Dividing a data set into a training set, a testing set and a verification set, and constructing a defect level prediction model, wherein the defect level prediction model comprises an input layer, a hidden layer and an output layer, the input layer of the model is used for inputting historical crimping defect data, the output layer is used for outputting defect levels of wire harness crimping, the output layer is provided with neurons with the same number as the defect levels of wire harness crimping, each neuron corresponds to the prediction probability of a defect level type of wire harness crimping, and a softmax function is used as an activation function;
the method comprises the steps of using multi-classification cross entropy as a loss function of a model, measuring the difference between a predicted value and an actual value of the model, using a training set to train an error type identification model, updating model parameters through a back propagation algorithm to minimize the loss function, using a verification set to evaluate the performance of a defect level prediction model through calculating an accuracy index;
and evaluating the performance of the model in a prediction task by using a test set, and predicting the current crimping defect data by using a trained defect level prediction model to obtain the defect level of the wire harness crimping.
The method for classifying the alarm grades according to the defect grades of the wire harness crimping comprises the steps of setting corresponding alarm grades according to the defect grades, wherein each defect grade corresponds to one alarm grade, for example, a slight defect corresponds to a low-grade alarm, a medium defect corresponds to a medium-grade alarm and a serious defect corresponds to a high-grade alarm, and once a defect is detected, the system compares the defect grade with a preset alarm grade.
And when the alarm condition is met, the system transmits alarm information to the defect detection terminal through the audible and visual alarm device. The audible and visual alarm can be performed by sounding, flashing lights or screen prompts.
The preset crimping defect probability threshold is set by a worker, different crimping defect probabilities are acquired through a defect detection terminal, the average value of a plurality of crimping defect probabilities is taken as the preset crimping defect probability threshold, and the preset error threshold is set in the same way.
According to the embodiment, the angle and the intensity of the light source are dynamically adjusted, so that the wire harness surface materials with different reflectivities can obtain optimal illumination conditions, clear and accurate image data can be obtained no matter the wire harness surface materials are made of high-reflectivity materials, medium-reflectivity materials or low-reflectivity materials, image blurring or detail loss caused by over-strong illumination or over-weak illumination is avoided, the dynamic range of an image can be optimized, image details are enhanced, particularly the transition area between high-reflectivity and low-reflectivity surfaces is enhanced, the contrast of the image is improved, defects on the wire harness surfaces are more obvious, subsequent defect detection is facilitated, the angle and the intensity of the light source are accurately adjusted, the illumination effect in the image is adapted to the requirements of different materials, and the presentation precision of the image details is improved. The system has a self-adaptive light source adjusting function, can automatically adjust light source parameters according to different wire harness surface materials and the reflection characteristics of the current image, reduces manual intervention, adopts a mixed illumination technology of an annular LED light source and a coaxial light source, can combine the advantages of the two light sources, and adapts to wire harnesses with different reflectivity materials;
By dynamically adjusting the smoothing factors in the similarity measurement and combining the weight of the edge strength, the weight of the edge area is enhanced, so that the edge information can be better reserved in the denoising process. The method solves the problem that the details of the image, particularly the edge details, are often blurred in the traditional denoising method, so that the quality of the denoised image is improved, the edge definition of the image is enhanced, and the similarity measurement is adjusted according to the gray values and gradient information of different pixels by adopting a self-adaptive similarity measurement formula and controlling the weights of gray difference and gradient information, so that the noise area and the detail area in the image are more accurately identified. The noise in the image is removed by carrying out weighted average on the similarity weight of each pixel point and the neighborhood pixels thereof based on a non-local mean value denoising formula, and meanwhile, the detail part of the image is kept, and particularly, the reservation of high-frequency signals such as textures and edge parts is more outstanding;
The method effectively enhances details of different levels in an image through the introduction of a multi-scale Retinex algorithm, combines local contrast enhancement, further improves texture contrast of a harness image, enables tiny details in the image to be more prominent, is beneficial to subsequent defect detection and quality assessment, can adapt to details of different scales through processing the harness image under multiple scales, achieves more comprehensive and accurate enhancement, can solve the problems of illumination difference and local feature protrusion under different scales, improves definition and contrast of the image under each scale, can finely adjust the contrast of a local area of the image through the introduction of a local contrast enhancement formula, improves detail performance, simultaneously avoids overadjustment of an overall brightness range, and can be more balanced through weighting and summing image information of different scales, and adjusting weights of the scales in a final image according to contribution values of different scales.
Example 2
Referring to fig. 2, the embodiment is not described in detail in embodiment 1, and provides a method for detecting a wire harness crimping defect based on machine vision, which includes:
S1, adopting an annular LED light source and coaxial light for mixed illumination, and adapting to different wire beam surface materials by dynamically adjusting the angle and the intensity of the light source;
S2, acquiring harness image data and crimping geometric data in real time based on the dynamically adjusted angle and intensity of the light source;
S3, denoising the harness image data by utilizing a non-local mean denoising algorithm and controlling a similarity measurement range through dynamic adjustment of a smoothing factor, superposing a multi-scale Retinex algorithm, enhancing texture contrast of the harness image data through local contrast, fusing the harness image data with crimping geometric data, and constructing a crimping part multi-mode feature map;
S4, taking the multi-mode feature map of the crimping part as input of a crimping defect detection model, and predicting to obtain crimping defect probability;
S5, if the wire harness crimping is defective, automatically collecting crimping defect data, inputting the crimping defect data into a trained defect grade prediction model, and predicting to obtain the defect grade of the wire harness crimping;
And S6, classifying alarm grades according to the defect grades of the wire harness crimping, and triggering audible and visual alarm to the defect detection terminal.
Since the electronic device described in this embodiment is an electronic device used for implementing the machine vision-based wire harness crimping defect detection system and method according to the embodiment of the present application, a specific implementation manner of the electronic device and various variations thereof can be known to those skilled in the art based on the machine vision-based wire harness crimping defect detection system and method described in the embodiment of the present application, so that a detailed description of how the electronic device is implemented in this embodiment of the present application is omitted herein. As long as the person skilled in the art implements the electronic device adopted by the system and the method for detecting the wire harness crimping defect based on the machine vision in the embodiment of the application, the electronic device belongs to the protection scope of the application.
The above formulas are all formulas with dimensionality removed and numerical calculation, the formulas are formulas with the latest real situation obtained by software simulation through collecting a large amount of data, and preset parameters and threshold selection in the formulas are set by those skilled in the art according to the actual situation.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to those skilled in the art without departing from the principles of the present invention are intended to be comprehended within the scope of the present invention.