CN111091555A

Movatterモバイル変換

Info

Publication number: CN111091555A
Application number: CN201911278224.8A
Authority: CN
Inventors: 孙晶
Original assignee: Harbin Kejia General Mechanical and Electrical Co Ltd
Current assignee: Harbin Kejia General Mechanical and Electrical Co Ltd
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2020-05-01
Anticipated expiration: 2039-12-12
Also published as: CN111091555B

Abstract

The invention discloses a brake shoe breakage target detection method, and relates to a fault detection method for a railway wagon. The invention aims to solve the problems of high cost, low efficiency and low accuracy of the existing method for checking the train brake shoe image. The process is as follows: firstly, acquiring a linear array image; secondly, coarse positioning; thirdly, generating a countermeasure network DCGAN, and generating a fault image based on the countermeasure network DCGAN; the countermeasure network consists of a discrimination model and a generation model, wherein the discriminator adopts the convolution of down sampling, and the generator adopts the convolution of up sampling; the specific process is as follows: a third step: constructing a DCGAN discrimination model of the confrontation network; and III: constructing an antagonistic network DCGAN generation model; fourthly, establishing a deep learning training data set; fifthly, dividing a fault target; and sixthly, predicting based on the trained segmentation network model to obtain the information of the fault component. The invention has the beneficial effects that: the invention is used for the field of fault detection of rail wagons.

Description

Brake shoe breaking target detection method

Technical Field

The invention relates to a fault detection method for a railway wagon.

Background

The rail wagon industry, which is developing at a high speed, has no doubt the most important for safe transport. The railway wagon brake device comprises an air brake, a basic brake and a hand brake, which organically form the whole railway wagon brake device. The foundation brake device consists of a brake cylinder piston push rod, a brake shoe and a series of levers, pull rods, brake beams, brake shoe supports, brake shoes and the like in the period from the rear to the front, and has the function of averagely transmitting thrust on the brake cylinder piston push rod to each brake shoe after increasing several times when a railway wagon brakes so that the brake shoes tightly hold the wheel to the tread. The brake shoe comprises brake shoe back and brake shoe face, and the brake shoe back is used for fixing the brake shoe on the binding tile holds in the palm, and the brake shoe face is used for taking place the friction with the wheel pair tread.

Brake shoes and other braking devices are important parts which seriously affect the safety of a train, the states of the parts cannot be guaranteed to be all the time only by manual inspection of an inspection station, and due to the fact that manual inspection is low in efficiency, conditions of missing parts, misinformation and the like easily occur, brake shoe faults can be better detected by adopting a deep learning automatic identification technology, and the driving safety of a truck is guaranteed.

Disclosure of Invention

The invention aims to solve the problems of high cost, low efficiency and low accuracy of the existing method for checking the train brake shoe image, and provides a brake shoe breakage target detection method.

The brake shoe breaking target detection method comprises the following specific processes:

firstly, acquiring a linear array image;

step two, coarse positioning;

step three, generating a countermeasure network DCGAN, and generating a fault image based on the countermeasure network DCGAN;

the countermeasure network consists of a discrimination model and a generation model, wherein the discriminator adopts the convolution of down sampling, and the generator adopts the convolution of up sampling; the specific process is as follows:

step three, firstly: constructing a DCGAN discrimination model of the confrontation network;

step three: constructing an antagonistic network DCGAN generation model;

step four, establishing a deep learning training data set;

step five, dividing a fault target;

and step six, predicting based on the trained segmentation network model to obtain the information of the fault component.

The invention has the beneficial effects that:

the automatic fault detection of the truck has important significance. The method comprises the steps of shooting a running truck to obtain a whole-truck image. And combining the knowledge in the fields of image processing, pattern recognition, deep learning and the like. The automatic fault identification and alarm are realized, the alarm result is manually confirmed, the conversion from the manual inspection operation to the mechanical inspection operation is finally realized, the labor cost of equipment using units is effectively saved, and the operation quality and the operation efficiency are improved.

The invention uses the fixed equipment to carry a camera or a video camera, shoots the truck moving at high speed, and shoots the whole truck image at the upper part, two sides and the bottom of the truck. And obtaining a coarse positioning area containing the side brake shoe component according to the wheel base information and the prior information of the position of the component on the large online array image. Generating a countermeasure network DCGAN (DeepConvolvulatory general adaptive networks) by utilizing depth convolution to generate a polymorphic brake shoe fault image, and performing image fusion and image blurring operation by using the generated fault image and a coarse positioning image to construct a multi-fault-morphological neural network training set. And (4) building a neural network structure, training the neural network for many times until the model converges, and obtaining the parameter weight. In an actual test, loading a neural network weight, predicting a shot component image by adopting a segmentation network, judging whether the component image is a fault image, and alarming a fault area if the component image is a fault.

The neural network data set of the countermeasure network generated in multiple fault modes is generated by utilizing the deep convolution, and the accuracy of brake shoe breakage detection can be improved. Compared with the traditional machine vision detection method of manual standard feature extraction, the fault detection method based on deep learning has high flexibility, accuracy and robustness. The manual work is not needed to browse pictures one by one, the vehicle detection operation can be completed only by manually confirming the faults of the alarm pictures, a large number of dynamic vehicle detection personnel can be saved, and the operation efficiency is improved.

Drawings

FIG. 1 is an overall flow chart of the present invention;

FIG. 2 is an overall flow chart of the DCGAN of the present invention;

FIG. 3 is a diagram of a discriminator model according to the invention;

FIG. 4 is a diagram of a generator model of the present invention;

FIG. 5 is a flow chart of image fusion according to the present invention;

FIG. 6a is a DCGAN generated failure image 1 diagram;

FIG. 6b is a DCGAN generated failure image 2 diagram;

FIG. 6c is a DCGAN generated failure image 3 diagram;

FIG. 6d is a DCGAN generated failure image 4 diagram;

FIG. 6e is a diagram of a fused failure image 1;

FIG. 6f is a fused failure image 2 diagram;

FIG. 6g is a fused failure image 3;

FIG. 6h is a fused failure image 4;

fig. 7 is a diagram of a split network architecture according to the present invention.

Detailed Description

The first embodiment is as follows: the brake shoe breaking target detection method of the embodiment comprises the following specific processes:

firstly, acquiring a linear array image;

step two, coarse positioning;

because the collected brake shoe breakage real fault forms are few, and the better polymorphic fault identification cannot be realized, a depth generation countermeasure network is required to generate polymorphic fault images. As shown in the schematic diagram of the generative countermeasure network structure of fig. 2, the generative model generates pseudo fault data according to random noise, then the fault data and the pseudo fault data are put into a discriminant model for discrimination, and the discriminant model cooperates with the generative model to adjust the generated data, and the operation is repeated until a clear fault image is generated. The method is characterized in that corresponding probability distribution is learned from original brake shoe fault data samples, so that more generated fault samples are obtained according to a probability distribution function to realize data amplification.

step three, firstly: constructing a DCGAN discrimination model of the confrontation network (as shown in FIG. 3);

step three: constructing an antagonistic network DCGAN generation model;

step four, establishing a deep learning training data set;

step five, dividing a fault target;

The second embodiment is as follows: the first embodiment is different from the first embodiment in that the line array image is obtained in the first step; the specific process is as follows:

utilizing fixed equipment to carry a camera or a video camera, shooting the railway wagon moving at high speed, and shooting the whole images of the upper part, two sides and the bottom of the railway wagon; only one line of the railway wagon is scanned each time, seamless splicing can be realized, and a two-dimensional image with a large visual field and high precision is generated.

Other steps and parameters are the same as those in the first embodiment.

The third concrete implementation mode: the difference between the second embodiment and the first or second embodiment is that the second step is performed by coarse positioning; the specific process is as follows:

according to the prior knowledge of the wheelbase information of hardware, the position of the part and the like, the area of the brake shoe part to be identified is cut out from the image information of the whole vehicle, so that the calculation amount is reduced, and the identification speed is improved.

Other steps and parameters are the same as those in the first or second embodiment.

The fourth concrete implementation mode: the difference between this embodiment and the first to third embodiments is that, in the first step, a confrontation network DCGAN discriminant model is constructed (as shown in fig. 3); the specific process is as follows:

the countermeasure network DCGAN discrimination model comprises 7 groups of convolution units, wherein each group of convolution units comprises a convolution layer, a batch standard processing layer and an activation function layer; convolution layer with convolution size 4x4 and step size 2; performing feature extraction, and meanwhile, using batch standard processing, wherein a hidden layer is activated by adopting Leaky ReLu (Leaky Rectified Linear Units), the Leaky ReLu is a special version of a corrected Linear Unit (ReLU), and when the hidden layer is not activated, the Leaky ReLu still has a nonzero output value, so that a small gradient is obtained, and the phenomenon of neuron death possibly occurring in the ReLU is avoided;

wherein f (x) is the output of the activation function, λ is the weight set to avoid gradient death, and the value is generally 0 to 1, and x is the input of the activation function;

step three is one: the method comprises the steps of normalizing original breaking fault data containing brake shoe breaking faults into data with the size of 256x256, and using 8 pieces of original fault data and pseudo fault data as input of a group of discrimination models;

step three, step two: inputting data into a built discrimination model, performing feature extraction by multi-round convolution, simultaneously using Batch Normalization (BN), and activating a hidden layer by using a Leaky ReLu (Leaky RectifiedLinear Units);

step three, one step and three steps: multiplying the final convolution layer output vector by the weight vector to convert the final convolution layer output vector into a 8x1 vector, classifying the final convolution layer output vector by adopting a nonlinear classifier Sigmod function, and outputting a result of two classifications, namely an original fault data and a generated fault data judgment result;

step three, step four: repeating the process from step three one to step three one until the fault data generated by the DCGAN discrimination model of the countermeasure network is clear and has M (more) form fault transformations;

m takes a value of 100;

other steps and parameters are the same as those in one of the first to third embodiments.

The fifth concrete implementation mode: the difference between this embodiment and one of the first to the fourth embodiments is that, in the third step, an antagonistic network DCGAN generation model is constructed; the specific process is as follows:

the structure of the antagonistic network generation model is shown in fig. 4, and the design requirements of the antagonistic network generation model are as follows: when the discrimination model is fixed, the distribution characteristic of the generated data is as large as possible to be consistent with the natural data, namely, the generated fault image is close to the original fault image. The generated model structure comprises 7 groups of transposition convolution units, wherein each group of transposition convolution units comprises a transposition convolution layer with the convolution kernel size of 4x4 and the step size of 2, a batch normalization processing (BN) layer and an activation function ReLu layer;

the process is as follows:

step three, step two and step one: converting 100-dimensional noise data into a two-dimensional vector through matrix operation to serve as input of a generation model;

step three, step two: filling the input transposed convolutional layers with the convolutional kernel size of 4x4 and the step length of 2, and outputting and adding a BN layer and a ReLu layer into each layer of transposed convolutional layer;

step three, step two and step three: finally, activating the output of the transposition convolution by adopting a Tanh function, and normalizing the pixels of the output image to-1 to form pseudo fault data;

step three, step two and step four: and outputting the generated fault image, transmitting the generated fault image into a countermeasure network DCGAN discrimination model, judging whether the generated fault image is the same as (similar to) real data, if not, generating a model updating weight, generating a fault image again, transmitting the fault image into the countermeasure network DCGAN discrimination model until the generated fault image is the same as the real data, and obtaining the optimized countermeasure network discrimination model and the optimized countermeasure model.

Other steps and parameters are the same as in one of the first to fourth embodiments.

The sixth specific implementation mode: the difference between this embodiment and the first to the fifth embodiment is that a deep learning training data set is established in the fourth step; the specific process is as follows:

and generating a countermeasure network through the depth of the third step, generating a polymorphic brake shoe fault image, performing image fusion by using the original image to generate a fault image, and forming a data set of a subsequent segmentation network.

Step four, firstly: building a raw data set

Obtaining an interested area image of the part to be identified according to the rough positioning in the step two, and establishing an original data set;

step four and step two: fusing images;

as shown in fig. 5, image fusion is performed by using the countermeasure network DCGAN generation model and the original data set obtained in the first step, and a Blur (Blur) operation in image processing is added in the image fusion, so that the quality of the fusion of the image with DCGAN generation failure and the original image can be improved by the Blur operation, and the stability of the segmentation failure model in the later stage is increased;

wherein, the fuzzy operation adopts a Gaussian kernel, and the size of the kernel is 3;

step four and step three: data set creation and data tagging:

the fused image data is used as shown in fig. 6a, 6b, 6c, 6d, 6e, 6f, 6g and 6h to form a new training data set and mark the data set, so as to obtain a mark mask gt (ground truth) image corresponding to the original image.

Other steps and parameters are the same as those in one of the first to fifth embodiments.

The seventh embodiment: the difference between this embodiment and one of the first to sixth embodiments is that the fault target is divided in the fifth step; the specific process is as follows:

the invention improves the U-Net segmentation network structure and constructs an encoder-decoder network, as shown in FIG. 7. Because the real-time performance and the precision requirement of detection are higher in automatic truck identification detection, in order to meet the precision requirement of detection, the ResNet structure is adopted to replace a VGG structure as a reference network, ResNet34 is adopted from the aspect of real-time performance, and Skip connection is utilized to connect corresponding feature layers, so that the feature graph has more texture information of the bottom layer, and the detection precision is ensured.

The training process of the deep learning of the multi-feature map comprises the following steps:

step five, first: constructing an encoder network of a U-Net segmentation model; the specific process is as follows:

the encoder performs feature extraction by taking a residual error network (Resnet) as a reference network, and because the size of a part to be identified is small, 4 downsampling encoding units are adopted, wherein each downsampling unit consists of N groups of Resnet residual error units (wherein N is 3, 4, 6 and 3);

the Resnet residual unit consists of two 1 × 1 convolution layers and one 3 × 3 convolution layer and adopts a shortcut (shortcut) connection mode; the feature graph dimension is effectively reduced or expanded by using 1x1 convolution, so that the number of feature channels of our 3x3 convolution is not influenced by the input of the previous layer, and the design is to reduce the time required by the whole model training by saving the calculation time and has no influence on the final model precision;

step five two: constructing a decoder network of a U-Net segmentation model; the specific process is as follows:

the decoder adopts 4 same up-sampling decoding units, as shown in the decoding unit of fig. 7, each decoding unit comprises two convolution layers with convolution kernel size 1 × 1, one transposed convolution layer with convolution kernel size 4 × 4 performs up-sampling, and three Batch Normalization processing (BN) layers, wherein the first 1 × 1 convolution layer has 1/4 channels as input channels, which can effectively increase the calculation speed and extract more feature information;

step five and step three: joining a Skip connection (Skip connection) unit to connect the encoder and decoder feature maps;

and connecting the encoder characteristic diagram with the corresponding decoder characteristic diagram by adopting Skip connection so as to acquire more texture information of the bottom layer with brake shoe breaking faults and strengthen the proportion of the bottom layer characteristics in the final decision.

Step five and four: normalizing the data obtained in the fourth step and the label image to 256x256 with a fixed size, and inputting the data and the label image into the constructed segmentation network (step five, step five and step five are the construction of the segmentation network);

step five: a prediction (predict) image and a true value (GT) image of an original mark (true value image obtained by marking in the fourth step and the third step) output by the image data through a segmentation network are subjected to a cross entropy Loss function (BCE Loss), so that the predicted data distribution learned by the segmentation network model approaches to the real data distribution;

where P is a prediction (Predict) image, GT is a true (GT) image, P and GT have the same width W and height H, and GT_ijIs a true value (GT) of a pixel value of the image, P_ijIs a pixel value in a prediction (Predict) image;

step five and step six: carrying out error back propagation according to the loss function, and updating the parameters of the segmentation network;

step five and seven: and repeating the fifth four to the fifth six, training the training data set obtained in the fourth three for multiple times until the cross entropy loss is gradually converged, increasing the confidence coefficient to a stable value, and determining the currently learned model parameters as the trained model parameters so as to obtain the trained segmentation network model, wherein the trained model parameters comprise the parameters of the segmentation network model.

Other steps and parameters are the same as those in one of the first to sixth embodiments.

The specific implementation mode is eight: the difference between the present embodiment and one of the first to seventh embodiments is that, in the sixth step, prediction is performed based on the trained segmentation network model to obtain information of the faulty component; the specific process is as follows:

step six: obtaining a vehicle passing image and a rough positioning component area image, loading the trained segmentation network model weight, inputting the rough positioning component area image into the trained segmentation network model, and predicting a segmentation neural network to obtain a neural network segmentation target result;

step six and two: according to the result of the neural network segmentation target, information such as perimeter, length-width ratio, area size and the like of the part to be segmented is obtained by an image processing method, and fault judgment is carried out according to the artificial prior position relation of brake shoe insert support, brake shoe and brake shoe breaking fault;

step six and three: and D, acquiring fault information according to the manual prior rule, and uploading the information of the fault component acquired in the step two to an alarm platform.

Other steps and parameters are the same as those in one of the first to seventh embodiments.

The present invention is capable of other embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the present invention.

Claims

1. The brake shoe breaking target detection method is characterized by comprising the following steps: the method comprises the following specific processes:

firstly, acquiring a linear array image;

step two, coarse positioning;

step three: constructing an antagonistic network DCGAN generation model;

step four, establishing a deep learning training data set;

step five, dividing a fault target;

2. The method for detecting a brake shoe breakage target according to claim 1, characterized in that: acquiring a linear array image in the first step; the specific process is as follows:

utilizing fixed equipment to carry a camera or a video camera, shooting moving railway wagons, and shooting full-car images of the upper parts, two sides and the bottom of the railway wagons; only one line of the rail wagon is scanned each time, and seamless splicing is achieved.

3. The method for detecting a brake shoe breakage target according to claim 1 or 2, characterized in that: coarse positioning in the second step; the specific process is as follows:

and cutting out the areas of the brake shoe components to be identified from the whole vehicle image information according to the wheel base information of hardware and the position prior knowledge of the components.

4. The method for detecting a brake shoe breakage target according to claim 3, characterized in that: constructing a DCGAN discrimination model of the confrontation network in the third step; the specific process is as follows:

the countermeasure network DCGAN discrimination model comprises 7 groups of convolution units, wherein each group of convolution units comprises a convolution layer, a batch standard processing layer and an activation function layer; convolution layer with convolution size 4x4 and step size 2; performing feature extraction, and meanwhile, performing batch standard processing, wherein the hidden layer is activated by using Leaky ReLu, and when the hidden layer is not activated, the Leaky ReLu still has a nonzero output value;

wherein f (x) is the activation function output, λ is the weight set to avoid gradient death, and x is the activation function input;

step three, step two: inputting data into a built discrimination model, extracting features, performing batch standard processing, and activating a hidden layer by using Leaky ReLu;

step three, step four: repeating the process from step three one to step three one until the fault data generated by the DCGAN discrimination model of the countermeasure network is clear and has M types of form fault transformation;

m takes the value of 100.

5. The method for detecting a brake shoe breakage target according to claim 4, wherein: in the third step, an antagonistic network DCGAN generation model is constructed; the specific process is as follows:

the generated model structure comprises 7 groups of transposition convolution units, wherein each group of transposition convolution units comprises a transposition convolution layer with the convolution kernel size of 4x4 and the step length of 2, a batch standard processing layer and an activation function ReLu layer;

step three, step two and step four: outputting the generated fault image, transmitting the generated fault image into a countermeasure network DCGAN discrimination model, judging whether the generated fault image is the same as real data or not, if not, generating a model updating weight, generating a fault image again, transmitting the fault image into the countermeasure network DCGAN discrimination model until the generated fault image is the same as the real data, and obtaining an optimized countermeasure network discrimination model and a generation model.

6. The method for detecting a brake shoe breakage target according to claim 5, wherein: establishing a deep learning training data set in the fourth step; the specific process is as follows:

step four, firstly: building a raw data set

step four and step two: fusing images;

performing image fusion by using the countermeasure network DCGAN generation model and the original data set obtained in the step four, wherein the image fusion is added with the fuzzy operation in the image processing;

step four and step three: data set creation and data tagging:

and forming a new training data set by using the fused image data and marking the data set to obtain a marked mask GT image corresponding to the original image.

7. The method for detecting a brake shoe breakage target according to claim 6, wherein: dividing a fault target in the step five; the specific process is as follows:

the encoder takes a residual error network as a reference network to extract features, 4 downsampling coding units are adopted, and each downsampling unit consists of N groups of Resnet residual error units;

the Resnet residual unit consists of two 1 × 1 convolution layers and one 3 × 3 convolution layer and adopts a shortcut connection mode;

the decoder adopts 4 same up-sampling decoding units, each decoding unit comprises two convolution layers with convolution kernel size of 1 × 1, a transposed convolution layer with convolution kernel size of 4 × 4 for up-sampling, and three batch standard processing layers, wherein the first 1 × 1 convolution layer has the channel number of 1/4 of the input channel;

step five and step three: joining a jumping connection unit to connect the characteristic diagrams of the encoder and the decoder;

step five and four: normalizing the data obtained in the step four and the label image to be 256 × 256 in fixed size, and inputting the data and the label image into the constructed segmentation network;

step five: a predicted image output by the image data through a segmentation network and a true value image of an original mark are subjected to a cross entropy loss function;

wherein P is a predicted imageGT is a true image, P and GT have the same width W and height H, GT_ijOne pixel value, P, of a true value image_ijIs a pixel value in the predicted image;

step five and seven: and repeating the fifth step, the fourth step, the fifth step and the sixth step, training the training data set obtained in the fourth step and the third step until the cross entropy loss is gradually converged, increasing the confidence coefficient to a stable value, and determining the currently learned model parameters as the trained model parameters so as to obtain the trained segmentation network model, wherein the trained model parameters comprise the parameters of the segmentation network model.

8. The method for detecting a brake shoe breakage target according to claim 7, characterized in that: predicting based on the trained segmentation network model in the sixth step to obtain information of the fault component; the specific process is as follows:

step six and two: according to the result of the neural network segmentation target, obtaining the information of the perimeter, the length-width ratio and the area size of the part to be segmented by using an image processing method, and judging the fault according to the prior position relation of the brake shoe insert support, the brake shoe and the brake shoe breaking fault;

step six and three: and D, uploading the information of the fault component obtained in the step two to an alarm platform.