Disclosure of Invention
The embodiment of the application solves the problems of long laser welding reasoning time and inaccurate data in the prior art by providing the laser welding quality monitoring method and the system, and realizes the effective shortening of the reasoning time and more accurate data reasoning.
The embodiment of the application provides a laser welding quality monitoring method, which comprises the following steps:
S1, measuring laser beams through a photodiode and a high-speed camera, focusing the laser beams onto the surface of a workpiece by using a scanner, recording the voltage of a photodiode amplifier at each sampling time, recording the image of the high-speed camera, and completing data acquisition;
s2, preprocessing data based on data acquisition, cutting out high-speed camera images, distributing each cut high-speed camera image to a high-speed camera sample, and performing rotating and overturning data enhancement operation on the high-speed camera images through block-by-block marking;
S3, a characteristic photodiode block, a characteristic high-speed camera block, a classification photodiode block and a classification high-speed camera block are combined through a cascading system, and laser welding quality monitoring is carried out through the four blocks;
S4, a prediction result is made for laser welding quality monitoring through a machine learning method of feature engineering and decision trees or by a deep neural network.
Further, a fiber laser with the infrared wavelength of 1070nm is used for measuring light with the wavelength of 300-950 nm in a welding area, the light beam is focused on the surface of a workpiece through a scanner, the voltage of a photodiode amplifier is recorded at each sampling time, and the image of a high-speed camera is recorded;
the photodiode has a time-series sampling rate of 250 kHz, records one sample every 4 μs, and the high-speed camera has a sampling rate of 20kHz, producing one image every 50 μs.
Further, the cascade system CS is defined as follows:
;
Is provided withIs the output of the classifier that makes a pre-decision based on the photodiode signal,
When p is less than 0.5, the classifier selects abnormality, when p is more than or equal to 0.5, the reference is selected, and the closer p is to 0 or 1, the higher the decision confidence of the classifier is;
Is provided withIs a fixed threshold if p < r orThe decision of the classifier is determined;
if the first condition is satisfied, the first condition is thatThe classifier's judgment of anomalies is considered acceptable if a second condition is met, the second condition beingThe classifier's judgment of the reference is considered acceptable and if the classifier's result is uncertain, a final decision is made based on the high-speed camera data in the next step.
Further, seven statistical features and median values on the photodiode signal block are calculated for each high speed camera image;
Calculating, for each high-speed camera image, a binary mask image having the same size as the high-speed camera image, the mask assuming a 1 value at those positions in the high-speed camera image where the pixel value is equal to or greater than h and a 0 value at the position where the high-speed camera pixel value is smaller;
According to h, the binary mask may contain information about the weld size, shape, or spatter, extracting 11 geometric features of area, number of regions, area of maximum region, ratio of maximum region to area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of fitted ellipse, length of ellipse, and width of ellipse;
By different empirical thresholdsEach image may have several geometric features;
after the characteristics are extracted, classifying by using DT, and measuring the quality of cracks by using CART algorithm;
Selecting features by using a base index in an algorithm, randomly selecting two samples from a data set, calculating the probability of inconsistent categories, and selecting the features with the minimum base index;
The data set is divided into different subsets according to the selected features, and a decision tree is recursively generated for each subset.
Further, the characteristic photodiode blocks and the characteristic high-speed camera blocks are composed of convolution layers, and the classified photodiode blocks and the classified high-speed camera blocks are composed of full connection layers;
The feature block is composed ofThe composition of the composite material comprises the components,Is a 2D convolution layer with the filter size of 3 multiplied by 3, then carries out batch normalization and maximum pooling, wherein ReLU is taken as an activation function, and k is the number of filters;
The classification block consists of a flattening layer and a full-connection layer DI, wherein D represents a weight matrix of the full-connection layer, and l is a neuron;
a ReLU activation function is used when 1+.1, a Sigmoid activation function is used when 1=1, and the discard rate of the discard layer is 0.5;
In the training process of the neural network, binary cross entropy is used as a loss function and initialized with random weights.
A laser welding quality monitoring system, comprising:
the data acquisition module is used for measuring laser beams through the photodiodes and the high-speed cameras, focusing the laser beams on the surface of a workpiece by using the scanner, recording the voltage of the photodiode amplifier at each sampling time, recording the image of the high-speed cameras and finishing data acquisition;
The data preprocessing module is used for preprocessing data, cutting out high-speed camera images, distributing each cut high-speed camera image to a high-speed camera sample, and carrying out data enhancement operation of rotation and overturning on the high-speed camera images through block-by-block marking;
The quality monitoring module is used for carrying out laser welding quality monitoring through the cascade system, wherein the characteristic photodiode blocks, the characteristic high-speed camera blocks, the classified photodiode blocks and the classified high-speed camera blocks are combined by the multiple sensors;
and the prediction result module is used for making a prediction result for laser welding quality monitoring through a machine learning method of a feature engineering and decision tree or by a deep neural network.
Further, a fiber laser with the infrared wavelength of 1070nm is used for measuring light with the wavelength of 300-950 nm in a welding area, the light beam is focused on the surface of a workpiece through a scanner, the voltage of a photodiode amplifier is recorded at each sampling time, and the image of a high-speed camera is recorded;
the photodiode has a time-series sampling rate of 250 kHz, records one sample every 4 μs, and the high-speed camera has a sampling rate of 20kHz, producing one image every 50 μs.
Further, the cascade system CS is defined as follows:
;
Is provided withIs the output of the classifier that makes a pre-decision based on the photodiode signal,
When p is less than 0.5, the classifier selects abnormality, when p is more than or equal to 0.5, the reference is selected, and the closer p is to 0 or 1, the higher the decision confidence of the classifier is;
Is provided withIs a fixed threshold if p < r orThe decision of the classifier is determined;
if the first condition is satisfied, the first condition is thatThe classifier's judgment of anomalies is considered acceptable if a second condition is met, the second condition beingThe classifier's judgment of the reference is considered acceptable and if the classifier's result is uncertain, a final decision is made based on the high-speed camera data in the next step.
Further, seven statistical features and median values on the photodiode signal block are calculated for each high speed camera image;
Calculating, for each high-speed camera image, a binary mask image having the same size as the high-speed camera image, the mask assuming a 1 value at those positions in the high-speed camera image where the pixel value is equal to or greater than h and a 0 value at the position where the high-speed camera pixel value is smaller;
According to h, the binary mask may contain information about the weld size, shape, or spatter, extracting 11 geometric features of area, number of regions, area of maximum region, ratio of maximum region to area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of fitted ellipse, length of ellipse, and width of ellipse;
By different empirical thresholdsEach image may have several geometric features;
after the characteristics are extracted, classifying by using DT, and measuring the quality of cracks by using CART algorithm;
Selecting features by using a base index in an algorithm, randomly selecting two samples from a data set, calculating the probability of inconsistent categories, and selecting the features with the minimum base index;
The data set is divided into different subsets according to the selected features, and a decision tree is recursively generated for each subset.
Further, the characteristic photodiode blocks and the characteristic high-speed camera blocks are composed of convolution layers, and the classified photodiode blocks and the classified high-speed camera blocks are composed of full connection layers;
The feature block is composed ofThe composition of the composite material comprises the components,Is a 2D convolution layer with the filter size of 3 multiplied by 3, then carries out batch normalization and maximum pooling, wherein ReLU is taken as an activation function, and k is the number of filters;
The classification block consists of a flattening layer, a fully connected layer DI, where D represents the weight matrix of the fully connected layer,Is a neuron;
When (when)When using ReLU activation function, whenWhen the Sigmoid activation function is used, the discarding rate of the discarding layer is 0.5;
In the training process of the neural network, binary cross entropy is used as a loss function and initialized with random weights.
One or more technical solutions provided in the embodiments of the present application at least have the following technical effects or advantages:
the quality monitoring method is characterized in that a cascade system is adopted, quality monitoring can be rapidly and accurately carried out through a two-stage structure, the first-stage structure carries out determined classification on some welding seams through simple data such as characteristic engineering and classical machine learning inspection analysis time sequence of DT, and the final decision is made on the basis of image data through a convolution network in the second-stage structure in the uncertain field. The practice proves that CS can be superior to SSS in accuracy and reasoning time, and compared with MSS, data of different sensors in CS are not required to be transmitted to general hardware, so that the reasoning time is effectively reduced, and the productivity is effectively improved.
Detailed Description
In order to better understand the above technical solutions, the following detailed description will refer to the accompanying drawings and specific embodiments.
Referring to fig. 1, s1. Measuring a laser beam by a photodiode and a high-speed camera, focusing the beam onto a workpiece surface by a scanner, recording a voltage of a photodiode amplifier at each sampling time, recording an image of the high-speed camera, and completing data acquisition;
Specifically, the actual apparatus of the entire system is as shown in fig. 2, and a 2D galvanometer scanner using two mirrors directs a laser beam generated in a laser onto two thin metal plates. The data acquisition device uses two measurement systems, a Photodiode (PD) and a High Speed Camera (HSC). The light with the wavelength of 300-950 nm is measured in the welding area by using a fiber laser with the infrared wavelength of 1070nm, and then the light beam is focused on the surface of the workpiece by using a scanner, and the voltage of a photodiode amplifier is recorded at each sampling time, and the image of a high-speed camera is recorded.
The photodiode has a time-series sampling rate of 250 kHz, records one sample every 4 μs, and the high-speed camera has a sampling rate of 20kHz, producing one image every 50 μs. The higher sampling rate of the photodiodes allows for shorter anomalies to be detected, the amount of raw data is smaller compared to high speed camera images, and faster processing is possible. The high speed camera image provides geometric information not present in the photodiode signal, with spatial resolution.
S2, preprocessing data, cutting out high-speed camera images, distributing each cut-out high-speed camera image to a high-speed camera sample, and performing rotating and overturning data enhancement operation on the high-speed camera images through block-by-block marking;
Specifically, the high-speed camera sample represents a block, a label is allocated to each block, whether the label reference is abnormal or not is judged, defects can be positioned along a welding path through block-by-block marking, and data enhancement operation of rotation and overturning is performed on the high-speed camera image.
The data is preprocessed for use in subsequent systems. The high speed camera image is first cropped to a size of 100 x 100 pixels and scaled into the value range of 0, 1. Each high speed camera image is then assigned to 13 photodiode samples, representing a block, and each block is assigned a label, i.e., a reference or anomaly. The anomaly refers to a position of an anomaly such as a gap or a splash, and the reference refers to a position at which neither an anomaly nor a visible anomaly is caused in a recorded photodiode signal or a high-speed camera image. By marking block by block, defects can be located along the weld path. Finally, for robustness of the model, a data enhancement operation of rotation and flipping is performed on the high-speed camera image.
S3, a characteristic photodiode block, a characteristic high-speed camera block, a classification photodiode block and a classification high-speed camera block are combined through a cascading system, and laser welding quality monitoring is carried out through the four blocks;
In particular, three quality monitoring methods, namely a Single Sensor System (SSS), a multi-sensor system (MSS) and a Cascade System (CS), are described later, and the measured values are generally measuredMapping to quality-related quantitiesAnd (3) upper part. Label (Label)Wherein 0 represents an anomaly and 1 represents a reference. Each of the n photodiode time series is defined asWherein the first indexRepresenting the block number and the second index represents the sample number within the block. The image definition of a high-speed camera is
;
The data set S is made up of n triples,。
The single sensor system SSS performs process monitoring based on data from one sensor, and for photodiode and high speed camera data as inputs, SSS is defined as follows:
;
fig. 3 shows two SSSs, the left with photodiode signals as input, the right with high speed camera images as input,AndRespectively representing the optimized predictive models. Each prediction model consists of two blocks, one is a feature block that extracts important features and the other is a classification block that determines weld quality.
The multi-sensor system MSS uses data from a plurality of sensors and an MSS composed of photodiodes and high-speed camera data as inputs is defined as follows:
;
Fig. 4 shows an MSS and its prediction model, the MSS is composed of a classification block and two feature blocks, i.e. one feature block per sensor data, where features are fused and processed for prediction. The classification block has a high-speed camera algorithm structure like SSS, except that the dimensions of the photodiode feature will be processed into dimensions of the high-speed camera. The advantage of the MSS over SSS is a more comprehensive quality assessment, however the reasoning time is longer.
The cascade system CS combines the advantages of both systems, which provides the possibility to use multiple sensors as MSS < multi-sensor system >, selecting only part of the valid data to be analyzed compared to MSS to obtain welding quality, not all data, speeds up training,
Referring to fig. 5, a two-stage CS is shown. Is provided withThe output of the classifier is pre-decided based on the photodiode signals, when p is less than 0.5, the classifier selects abnormality, and when p is more than or equal to 0.5, the classifier selects reference. The closer p is to 0 or 1, the higher the decision confidence of the classifier. Is provided withIs a fixed threshold if p < r orThe decision of the classifier is determined. If the first condition is satisfied, the first condition is thatThe classifier's judgment of anomalies is considered acceptable if a second condition is met, the second condition beingThe classifier's judgment of the reference is considered acceptable and if the classifier's result is uncertain, a final decision is made based on the high-speed camera data in the next step. Formally, the cascade system CS is defined as follows:
;
All three quality monitoring methods described above are created by combining four blocks of a feature photodiode block, a feature high-speed camera block, a classification photodiode block, and a classification high-speed camera block, each of which may be made up by a machine learning method of feature engineering and Decision Tree (DT) or by a deep Neural Network (NN). For unified experimentation, when one block uses a machine learning or deep learning method, the other blocks are also used.
S4, a prediction result is made for laser welding quality monitoring through a machine learning method of feature engineering and decision trees or by a deep neural network.
Specific feature engineering and decision trees. The features of the photodiode signal may be extracted manually or automatically tsfresh (time series feature extraction based on scalable hypothesis testing), and 7 statistical features of mean, standard deviation, maximum, minimum, distance between maximum and minimum, kurtosis and skewness are calculated on each of the 13 samples of the photodiode data block to obtain several time series features. Features of high-speed camera images are classified into statistical and geometric features. First, seven statistical features calculated on the photodiode signal block and the median value are calculated for each high-speed camera image. Next, a binary mask image having the same size as the high-speed camera image is calculated for each high-speed camera image. Pixel values of mask in high speed camera imageIs considered a1 value at those positions where the high speed camera pixel value is small and a 0 value at those positions where the high speed camera pixel value is small. According to h, the binary mask may contain information about the weld size, shape, or spatter, extracting 11 geometric features of area, number of regions, area of maximum region, ratio of maximum region to area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of fitting ellipse, length of ellipse, and width of ellipse. By different empirical thresholdsEach image may have several geometric features.
After feature extraction, classification was performed with DT and crack quality was measured using CART algorithm. The CART algorithm is a classification regression algorithm, features are selected by using a radix index in the algorithm, two samples are randomly selected from a data set, the probability of inconsistent categories is calculated, and the feature with the smallest radix index is selected. The data set is divided into different subsets according to the selected features, and a decision tree is recursively generated for each subset. While the subtree is being generated, the base index is used to select features and the majority voting method is used at the leaf nodes to determine classification results. To prevent overfitting, post pruning is used to prune the decision tree.
Deep neural networks. The characteristic photodiode and the characteristic high-speed camera are composed of convolution layers, and the classification photodiode and the high-speed camera are composed of full-connection layers. The feature block is composed ofThe composition of the composite material comprises the components,Is a 2D convolution layer with a filter size of 3 x 3, then performs batch normalization, max pooling, reLU as an activation function, and k is the number of filters. The classification block consists of a flattening layer, a fully connected layer DI, where D represents the weight matrix of the fully connected layer,Is a neuron, and the fully connected layer calculates the output by multiplying the input vector with a weight matrix and adding the bias vector. "D" here is representative of that weight matrix, whose dimensions determine the connection and transformation between the input and output neurons (here "l" for neurons). Through training, the value of the weight matrix 'D' can be continuously adjusted, so that the model can better complete tasks such as classification and the like. When (when)When using ReLU activation function, whenWhen the Sigmoid activation function is used, the discarding rate of the discarding layer is 0.5. In the training process of the neural network, binary cross entropy is used as a loss function and initialized with random weights.
The technical scheme provided by the embodiment of the application at least has the following technical effects or advantages:
the quality monitoring method is characterized in that a cascade system is adopted, quality monitoring can be rapidly and accurately carried out through a two-stage structure, the first-stage structure carries out determined classification on some welding seams through simple data such as characteristic engineering and classical machine learning inspection analysis time sequence of DT, and the final decision is made on the basis of image data through a convolution network in the second-stage structure in the uncertain field. The practice proves that CS can be superior to SSS in accuracy and reasoning time, and compared with MSS, data of different sensors in CS are not required to be transmitted to general hardware, so that the reasoning time is effectively reduced, and the productivity is effectively improved.
A laser welding quality monitoring system, comprising:
the data acquisition module is used for measuring laser beams through the photodiodes and the high-speed cameras, focusing the laser beams on the surface of a workpiece by using the scanner, recording the voltage of the photodiode amplifier at each sampling time, recording the image of the high-speed cameras and finishing data acquisition;
The data preprocessing module is used for preprocessing data, cutting out high-speed camera images, distributing each cut high-speed camera image to a high-speed camera sample, and carrying out data enhancement operation of rotation and overturning on the high-speed camera images through block-by-block marking;
the high-speed camera sample represents a block, a label is allocated to each block, whether the label reference is abnormal or not is judged, defects can be positioned along a welding path through block-by-block marking, and data enhancement operation of rotation and overturning is carried out on the high-speed camera image;
The quality monitoring module is used for carrying out laser welding quality monitoring through the cascade system, wherein the characteristic photodiode blocks, the characteristic high-speed camera blocks, the classified photodiode blocks and the classified high-speed camera blocks are combined by the multiple sensors;
and the prediction result module is used for making a prediction result for laser welding quality monitoring through a machine learning method of a feature engineering and decision tree or by a deep neural network.
Further, a fiber laser with the infrared wavelength of 1070nm is used for measuring light with the wavelength of 300-950 nm in a welding area, the light beam is focused on the surface of a workpiece through a scanner, the voltage of a photodiode amplifier is recorded at each sampling time, and the image of a high-speed camera is recorded;
the photodiode has a time-series sampling rate of 250 kHz, records one sample every 4 μs, and the high-speed camera has a sampling rate of 20kHz, producing one image every 50 μs.
Further, the cascade system CS is defined as follows:
;
Is provided withIs the output of the classifier that makes a pre-decision based on the photodiode signal,
When p is less than 0.5, the classifier selects abnormality, when p is more than or equal to 0.5, the reference is selected, and the closer p is to 0 or 1, the higher the decision confidence of the classifier is;
Is provided withIs a fixed threshold if p < r orThe decision of the classifier is determined;
if the first condition is met, the classifier's judgment of anomalies is considered acceptable, if the second condition is met, the classifier's judgment of references is considered acceptable, if the classifier's result is uncertain, a final decision is made in the next step based on the high-speed camera data.
Further, seven statistical features and median values on the photodiode signal block are calculated for each high speed camera image;
Calculating, for each high-speed camera image, a binary mask image having the same size as the high-speed camera image, the mask assuming a 1 value at those positions in the high-speed camera image where the pixel value is equal to or greater than h and a 0 value at the position where the high-speed camera pixel value is smaller;
According to h, the binary mask may contain information about the weld size, shape, or spatter, extracting 11 geometric features of area, number of regions, area of maximum region, ratio of maximum region to area, convex hull, ratio of area to convex hull, perimeter, ratio of perimeter to area, area of fitted ellipse, length of ellipse, and width of ellipse;
By different empirical thresholdsEach image may have several geometric features;
after the characteristics are extracted, classifying by using DT, and measuring the quality of cracks by using CART algorithm;
Selecting features by using a base index in an algorithm, randomly selecting two samples from a data set, calculating the probability of inconsistent categories, and selecting the features with the minimum base index;
The data set is divided into different subsets according to the selected features, and a decision tree is recursively generated for each subset.
Further, the characteristic photodiode blocks and the characteristic high-speed camera blocks are composed of convolution layers, and the classified photodiode blocks and the classified high-speed camera blocks are composed of full connection layers;
The feature block is composed ofThe composition of the composite material comprises the components,Is a 2D convolution layer with the filter size of 3 multiplied by 3, then carries out batch normalization and maximum pooling, wherein ReLU is taken as an activation function, and k is the number of filters;
The classification block consists of a flattening layer and a fully-connected layer DI, wherein D represents a weight matrix of the fully-connected layer, and l is a neuron;
When (when)When using ReLU activation function, whenWhen the Sigmoid activation function is used, the discarding rate of the discarding layer is 0.5;
In the training process of the neural network, binary cross entropy is used as a loss function and initialized with random weights.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.