Disclosure of Invention
In order to solve the problem of inaccurate detection results of abnormal mold cavities, the invention provides a machine vision-based mold cavity abnormality detection method.
The invention discloses a machine vision-based die cavity anomaly detection method which comprises the steps of obtaining a preprocessed die cavity surface gray level diagram, inputting the die cavity surface gray level diagram into a preset neural network, outputting probability values, wherein the probability values comprise normal probability, residual probability, defect probability and reflection probability, obtaining a die cavity type according to the probability values, constructing a target gray level diagram according to the die cavity type, extracting a region of interest (ROI) of the target gray level diagram, conducting grid division on the region of interest (ROI) according to the probability values to obtain a plurality of subareas, calculating gradient consistency of the region of interest (ROI), and finishing die cavity anomaly detection, wherein the obtaining of the subareas comprises the steps of obtaining initial grid size of grid division, calculating updated grid size, wherein the grid size comprises length of grids and width of grids, and the updated grid size meets a relational expression:, representing the size of the grid after the update,The probability of reflection of light is indicated,And dividing the ROI area according to the updated grid size to obtain a plurality of subareas.
By preprocessing the cavity surface gray level map and outputting different probability values using the neural network, it is possible to accurately identify the cavity type and construct a target gray level map, which is an image having a large light reflection probability, that is, an image having a large possibility of containing an abnormality. The ROI area of the target gray level image is extracted, and the grid size is dynamically adjusted according to the reflection probability, so that the ROI area can be finely divided, and the consistency and the accuracy of detection can be improved.
Preferably, the neural network is a convolutional neural network, the convolutional neural network performs feature extraction on the gray level map of the surface of the die cavity to obtain image features, and classifies the image features to obtain probability values.
The training process of the neural network comprises the steps of taking a gray level diagram of the surface of a die cavity of a die in a history as input information and taking a true value of the type of the die cavity of the die as a label to obtain a group of training data, inputting the training data into the neural network to obtain an output result, calculating a loss value of the neural network through the output result and the label, using cross entropy loss by a loss function, reversely transmitting error signals according to the loss value, updating network parameters of the neural network to enable the loss value to be smaller, iteratively updating the network parameters of the neural network, and stopping updating when the neural network reaches a set maximum training frequency or the loss value is smaller than the set loss value to obtain the trained neural network.
Preferably, the method for acquiring the target gray level map comprises the steps of acquiring a corresponding mold cavity type when any one of probability values is greater than 0.5, wherein the mold cavity type is normal, mold cavity residues, mold cavity defects and mold cavity light reflection, and taking a mold cavity surface gray level map corresponding to a mold with the mold cavity light reflection and a mold cavity surface gray level map corresponding to a mold with the probability value not greater than 0.5 as the target gray level map.
The specific die cavity type can be obtained through the probability value output by the neural network, the die cavity reflecting the light of the die cavity can be found more quickly from the determined die cavity types, and the die cavity which cannot be determined through the probability value is taken as the target gray level graph, so that the die cavity which needs special attention or processing can be quickly identified, the production parameters can be timely adjusted or maintained, and the generation of defective products can be reduced.
Preferably, the obtaining of the target gray level map further comprises the steps of calculating Euclidean distances of normal probability and residual probability, euclidean distances of normal probability and defect probability, euclidean distances of normal probability and reflection probability respectively for any one of the cavity surface gray level maps, adding the three Euclidean distances and using a result of negative correlation mapping as a difficult classification degree, and using the cavity surface gray level map with the difficult classification degree being larger than a preset threshold value as the target gray level map.
By adding these distances to obtain the degree of difficulty classification, the degree of difference between the cavity image and the center point of each class can be quantified. The larger the calculation result of the Euclidean distance, which reflects the degree of deviation between the characteristic of the cavity image and each class of typical characteristic, the larger the characteristic difference between the characteristic of the cavity image and the normal state, which usually means that the cavity has some abnormality.
Preferably, the gradient consistency comprises the steps of carrying out weighted summation on the gradient directions according to the gradient value of each pixel point in each sub-region for any sub-region to obtain the total gradient direction of the sub-region, traversing to obtain the total gradient direction of each sub-region in the ROI region, constructing a gradient co-occurrence matrix of the ROI region, and taking the modulus of the gradient co-occurrence matrix as the gradient consistency.
Gradient uniformity effectively quantifies the uniformity of the mold cavity surface. The gradient consistency is realized by analyzing the gradient value of each pixel point in the subarea and carrying out weighted summation on the gradient direction, so that the fine change of the surface of the die cavity can be captured, and the sensitivity and the accuracy of detection are improved. In addition, the recognition capability of the abnormal surface of the model cavity is further enhanced by constructing the gradient co-occurrence matrix and calculating the model thereof as a measure of gradient consistency.
Preferably, the gradient consistency further comprises the steps of calculating the gradient direction of each pixel point in each sub-region for any sub-region, drawing a gradient histogram to obtain the main gradient direction and the sub-gradient direction of the sub-region, respectively constructing a main gradient symbiotic matrix and a sub-gradient symbiotic matrix of the ROI region, calculating a main gradient feature vector according to the main gradient symbiotic matrix, and calculating a sub-gradient feature vector according to the sub-gradient symbiotic matrix, wherein the gradient consistency satisfies the relation: wherein, the method comprises the steps of, wherein,The consistency of the gradient is indicated,Representing the principal gradient feature vector,Representing a transpose of the sub-gradient feature vector,A modulus representing the principal gradient feature vector,Representing the modulus of the sub-gradient feature vector.
The direction information of the local change of the image is utilized, which reflects the microstructure and potentially abnormal features of the cavity surface. The primary gradient direction generally corresponds to the direction of most significant change in the image, i.e. the primary texture direction in the sub-region of the mold cavity, while the secondary gradient direction provides additional information about the direction of change, i.e. the secondary important texture direction in the sub-region of the mold cavity.
Preferably, the gradient consistency further comprises the steps of calculating the gradient direction of each pixel point in each sub-region, drawing a gradient histogram to obtain the main gradient direction and the secondary gradient direction of the sub-region, respectively constructing a main gradient co-occurrence matrix and a secondary gradient co-occurrence matrix of the ROI region, calculating a main gradient feature vector according to the main gradient co-occurrence matrix, calculating a secondary gradient feature vector according to the secondary gradient co-occurrence matrix, and taking the similarity of the main gradient feature vector and the secondary gradient feature vector as the gradient consistency.
The invention has the beneficial effects that:
1. according to the method, the gray level map of the surface of the die cavity is preprocessed and is input into the convolutional neural network, the network can output probability values reflecting different states, the probability values comprehensively reflect the current state of the die cavity, and the method can automatically extract the characteristics from the image data and classify the characteristics, so that manual intervention is reduced, and the detection efficiency and accuracy are improved.
2. According to the invention, the accuracy and reliability of the abnormal detection of the die cavity are remarkably improved by comprehensively utilizing the gradient direction information, gradient consistency calculation and the grid size of the reflection probability adjustment. The method not only can identify the tiny change of the surface of the die cavity, but also can adapt to different detection conditions, such as the influence of reflection and shadow, thereby providing a powerful tool for the quality control of the die cavity.
3. According to the invention, through automatic feature extraction and quantitative anomaly detection, the accuracy and reliability of the detection of the cavity anomaly are obviously improved. The method can identify the tiny change of the surface of the die cavity, can adapt to different detection conditions, such as the influence of reflection and shadow, can avoid potential quality problems by timely finding the abnormality of the surface of the die cavity, improves the consistency and reliability of products, reduces the production cost and improves the production efficiency.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that when the terms "first," "second," and the like are used in the claims, the specification and the drawings of the present invention, they are used merely for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising" when used in the specification and claims of the present invention are taken to specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The invention provides a machine vision-based die cavity anomaly detection method. As shown in fig. 1, the machine vision-based mold cavity abnormality detection method includes steps S1 to S2, which are described in detail below.
S1, acquiring a preprocessed die cavity surface gray level map, inputting the die cavity surface gray level map into a preset neural network, outputting a probability value, and acquiring a die cavity type according to the probability value.
In one embodiment, an industrial camera is utilized to capture a cavity RGB (Red Green Blue) image of a mold, and the cavity RGB image is grayed to obtain a gray scale image that can simplify image data, reduce computational complexity, while retaining sufficient information for subsequent feature extraction and analysis.
The preprocessing includes, first, converting the RGB image into a gray scale by a graying process, which can reduce complexity and calculation amount of image processing. Then, histogram equalization techniques can be used to enhance the contrast of the image, highlighting details, and making the texture and structure of the cavity surface clearer. In addition, low-pass filtering methods such as mean filtering, median filtering and Gaussian filtering can be used for removing image noise, smoothing image edges and reducing high-frequency parts of image details so as to inhibit noise and bright spots.
Morphological operations such as erosion, dilation, open and close operations can further improve image quality, remove small noise or impulse noise, and enhance edge information of objects in the image. These operations help to more accurately identify and locate anomalies in the mold cavity surface in subsequent steps.
Inputting the gray level map of the surface of the die cavity into a preset neural network, wherein the neural network is a convolutional neural network, the convolutional neural network performs feature extraction on the gray level map of the surface of the die cavity to obtain image features, and classifies the image features to obtain probability values.
It should be noted that, the convolutional neural network is CNN (Convolutional Neural Networks), which includes an input layer, a convolutional layer, an active layer, a pooling layer, and a full-connection layer (or an output layer). The input layer is used for receiving the cavity surface gray level map, the convolution layer is a layer comprising a plurality of convolution operations, each convolution operation uses a different convolution kernel (or filter) to extract the feature map of the cavity surface gray level map, the activation layer usually follows the convolution layer, introduces nonlinearities so that the neural network can learn complex features, the pooling layer is used for reducing the space size of the feature map, reducing the calculation amount and extracting important features, and the full connection layer is used for mapping the extracted features to final output, namely probability values, after the convolution layer and the pooling layer.
The training process of the neural network comprises the steps of taking a gray level diagram of the surface of a die cavity of a die in a history as input information and taking a true value of the type of the die cavity of the die as a label to obtain a group of training data, inputting the training data into the neural network to obtain an output result, calculating a loss value of the neural network through the output result and the label, using cross entropy loss by a loss function, reversely transmitting error signals according to the loss value, updating network parameters of the neural network to enable the loss value to be smaller, iteratively updating the network parameters of the neural network, and stopping updating when the neural network reaches the set maximum training times or the loss value is smaller than the set loss value to obtain the trained neural network.
Illustratively, updating is stopped when the neural network training times reach 200 times or the loss value is less than 0.0001.
The output of the neural network is a probability value, and when any one of the probability values is larger than a probability threshold of the network, namely, a die cavity corresponding to the gray level map of the surface of the die cavity is the corresponding die cavity type. The mold cavity types are normal, mold cavity residual, mold cavity defect and mold cavity light reflection.
For example, the input normal probability, residual probability, defect probability and reflection probability are sequentially 0.3, 0.6, 0.05 and 0.05, the probability threshold of the network is generally set to be 0.5, and then the mold cavity corresponding to the gray level map of the mold cavity surface is the mold cavity residual.
S2, constructing a target gray scale map according to the type of the die cavity, extracting an ROI (region of interest) of the target gray scale map, dividing the ROI into a plurality of subregions according to the probability value, calculating the gradient consistency of the ROI, and finishing die cavity anomaly detection.
It should be noted that, in the mold manufacturing process, an image anomaly caused by reflection of light from the surface of the mold cavity is often encountered, and the reflection is usually only a pseudo anomaly caused by reflection of light, so that the product quality is not substantially affected. However, if other types of anomalies, such as residues and imperfections, are just hidden in the retroreflective regions, the original retroreflective regions are transformed into true anomaly regions, which require a high-precision detection system to distinguish and identify the true anomalies that are mixed in the retroreflection to ensure that the quality of the mold cavity product is not compromised.
In one embodiment, the type of the mold cavity obtained according to step S1 further includes a case where the probability value is not greater than the probability threshold of the network.
And taking the gray level map of the mold cavity surface corresponding to the mold with the mold cavity reflection and the gray level map of the mold cavity surface corresponding to the mold with the probability value not more than 0.5 as target gray level maps.
It should be noted that, when the probability value is not greater than 0.5, the probability value indicates that the residual probability and the defect probability are relatively close to the reflection probability, the gray level map of the mold cavity surface belongs to a difficult classification, and at this time, further analysis of the gray level map of the mold cavity surface is required.
Extracting the ROI area of the target gray map includes extracting directly using a rectangular frame if the position of the ROI in the image is known, extracting using a mask for irregular ROI areas, acquiring by contour detection for ROIs of complex shape, creating a mask using inRange functions by converting the image to HSV color space, and extracting ROI areas of a specific color.
Calculating updated grid dimensions including the length of the grid and the width of the grid, wherein the updated grid dimensions satisfy the relation:
, representing the size of the grid after the update,The probability of reflection of light is indicated,Representing the initial mesh size.
And dividing the ROI area according to the updated grid size to obtain a plurality of sub-areas.
The main object of study is a gray scale image of the cavity surface with high probability of reflection, and the higher the probability of reflection, the greater the possibility of containing other abnormalities, and the smaller the mesh size of the ROI region needs to be meshed.
And for any one sub-region, carrying out weighted summation on the gradient directions according to the gradient value of each pixel point in the sub-region to obtain the total gradient direction of the sub-region.
Traversing to obtain the total gradient direction of each sub-region in the ROI region, constructing a gradient co-occurrence matrix of the ROI region, and taking the modulus of the gradient co-occurrence matrix as gradient consistency.
It should be noted that, on the surface of the mold cavity, the foreign matters or defects tend to break the original gradient consistency, so that the gradient direction and the mold value of the local area are significantly changed. A high gradient uniformity means that the direction of gray scale variation is more uniform across the region, which generally indicates that the edges and texture features of the region are more sharp. In cavity inspection, sharp edges and textures help to more accurately identify foreign objects or defects. And gradient uniformity may also reflect the flatness of the cavity surface. If the gradient uniformity of a region is low, this may mean that the surface of the region is uneven or there is a roughness, which may be caused by manufacturing defects in the mold cavity itself or by wear during use.
And comparing the gradient consistency with a preset gradient threshold, and generating and sending an alarm to remind a worker to overhaul the die cavity of the die when the gradient consistency is smaller than the preset gradient threshold.
In one embodiment, the method further comprises the steps of calculating Euclidean distances of normal probability and residual probability, euclidean distances of normal probability and defect probability and Euclidean distances of normal probability and reflection probability respectively for any cavity surface gray level map, adding the three Euclidean distances and using a result of negative correlation mapping as a difficult classification degree, and using the cavity surface gray level map with the difficult classification degree larger than a preset threshold value as the target gray level map.
In one embodiment, the gradient consistency further comprises the steps of calculating the gradient direction of each pixel point in each sub-region for any sub-region, drawing a gradient histogram to obtain the main gradient direction and the secondary gradient direction of the sub-region, respectively constructing a main gradient co-occurrence matrix and a secondary gradient co-occurrence matrix of the ROI region, calculating a main gradient feature vector according to the main gradient co-occurrence matrix, and calculating a secondary gradient feature vector according to the secondary gradient co-occurrence matrix.
Calculating gradient consistency, and satisfying the relation:
wherein, the method comprises the steps of, wherein,The consistency of the gradient is indicated,Representing the principal gradient feature vector,Representing a transpose of the sub-gradient feature vector,A modulus representing the principal gradient feature vector,Representing the modulus of the sub-gradient feature vector.
In one embodiment, the gradient consistency further comprises the steps of calculating the gradient direction of each pixel point in each sub-region, drawing a gradient histogram to obtain the main gradient direction and the secondary gradient direction of the sub-region, respectively constructing a main gradient co-occurrence matrix and a secondary gradient co-occurrence matrix of the ROI region, calculating a main gradient feature vector according to the main gradient co-occurrence matrix, calculating a secondary gradient feature vector according to the secondary gradient co-occurrence matrix, and taking the similarity of the main gradient feature vector and the secondary gradient feature vector as the gradient consistency.
Preferably, the similarity may use cosine similarity.
While various embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Many modifications, changes, and substitutions will now occur to those skilled in the art without departing from the spirit and scope of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention.
The above embodiments are not intended to limit the scope of the invention, so that the equivalent changes of the structure, shape and principle of the invention are covered by the scope of the invention.