CN113298059A - Pantograph foreign matter intrusion detection method, device, computer equipment, system and storage medium - Google Patents

Abstract

The invention relates to the technical field of rail transit, and discloses a pantograph foreign matter intrusion detection method, a device, computer equipment, a system and a storage medium, wherein a video image acquired at the top of a vehicle in real time is firstly identified and positioned based on a region of interest ROI positioning technology, a pantograph region is extracted, a pantograph gray image is extracted, then on one hand, a Gaussian mixture model is used for segmenting the pantograph gray image into a foreground and a background to obtain a foreground object binary image, on the other hand, a three-frame difference method is used for detecting a moving object of the pantograph gray image to obtain a moving object binary image, finally, the two binary images are subjected to logic and operation results, and whether intrusion foreign matters exist in the operation results is determined by combining a contour area threshold value, so that the pantograph state can be monitored online in real time in the running process of the locomotive, and automatically and accurately finding whether foreign matters invade the working area of the pantograph or not, and providing guarantee for the safe and stable operation of the electric locomotive.

Description

Pantograph foreign matter intrusion detection method, device, computer equipment, system and storage medium

Technical Field

The invention belongs to the technical field of rail transit, and particularly relates to a pantograph foreign matter intrusion detection method, a pantograph foreign matter intrusion detection device, computer equipment, a pantograph foreign matter intrusion detection system and a pantograph foreign matter intrusion detection storage medium.

Background

In recent years, inter-city electric locomotives have been rapidly developed in China due to the characteristics of rapidness, safety, stability, timekeeping, comfort, energy conservation and the like. The electric locomotive acquires electric energy by current collection through the sliding contact of a pantograph device arranged at the top of the electric locomotive and a contact net arranged along a railway, and provides basic kinetic energy for the operation of the locomotive. With the increase of the operating load of the electric locomotive, the increase of the operating mileage and the increase of the operating speed, the pantograph is easy to have fault defects such as abnormal impact, abrasion and the like. According to statistics, in the power failure and outage accidents of the electrified railway system in China, the pantograph-catenary accidents account for about 80 percent of the total accidents. Therefore, for the driving safety of the electric locomotive, it is necessary to detect the pantograph fault in time to ensure the normal operation of the pantograph-catenary system.

At present, the fault detection of the pantograph mainly comprises a manual detection mode and an image detection mode, wherein the former needs to drive an electric locomotive into a locomotive service section, and after the locomotive is stopped, the pantograph is lowered and is powered off, a field engineer carries out roof climbing to check the fault, and whether the pantograph has a fault or not can be judged by virtue of abundant practical experience, so that the pantograph can be supported to continue operation. In summary, the manual detection method has the disadvantages of large workload, low efficiency and potential safety hazard, and cannot detect the real working state of the pantograph of the electric locomotive in the running process.

Although the image detection mode is used as a technical means of non-contact measurement, the method has the characteristics of rapidness, stability, safety, effectiveness and visual measurement result, and can be used for independently and objectively monitoring and recording the pantograph and the action relationship between the pantograph and a contact network. However, most of the current vehicle-mounted pantograph image monitoring devices are installed on an electric locomotive in operation, then a remote wireless fixed-point monitoring system is used for shooting and recording the condition of the pantograph, and then collected data are remotely transmitted back, so that relevant technicians judge the condition of the pantograph by observing pictures and playing back the recorded data, and the judgment is still carried out based on the experience of the technicians, and the problems of misjudgment or missed judgment and the like easily occur. In addition, in the image detection method, related personnel perform off-line analysis on the shot image, and the analysis result cannot be timely transmitted to the guidance operator on the vehicle-mounted device for decision making, so that hidden danger is buried in the safe and stable operation of the electric locomotive.

In addition, most of image detection systems applied to electric locomotives monitor pantograph slide plates, and no research on foreign matter invasion of the whole pantograph-catenary contact area is involved, so that the invasion of foreign matters into the pantograph-catenary action area has serious consequences and even threatens driving safety. Therefore, how to perform real-time online monitoring on the pantograph state in the operation process of the electric locomotive and automatically and accurately find whether foreign matters invade a pantograph working area is a subject which needs to be researched urgently by technical personnel in the field so as to prompt alarm information in time and effectively avoid unexpected driving safety accidents.

Disclosure of Invention

In order to solve the problems of poor automation and real-time performance of the conventional pantograph fault detection method and incapability of carrying out foreign matter intrusion detection on the whole pantograph-catenary contact area, the invention aims to provide a pantograph foreign matter intrusion detection method, device, computer equipment, system and storage medium, which can be applied to an electric locomotive, can carry out real-time online monitoring on the state of a pantograph in the locomotive running process, can automatically and accurately find whether foreign matters intrude into a pantograph working area, can be favorable for carrying out omnibearing online monitoring on the pantograph working area, and can provide guarantee for safe and stable running of the electric locomotive.

In a first aspect, the present invention provides a pantograph foreign object intrusion detection method, including:

acquiring a video image acquired by a monitoring camera in real time, wherein the monitoring camera is mounted on the roof of the vehicle and enables the view field of a lens to cover the area where the pantograph is located;

carrying out ROI (region of interest) positioning processing on the video image, and extracting a pantograph gray image;

segmenting the foreground and the background of the pantograph gray image by using a Gaussian mixture model to obtain a foreground object binary image, wherein the gray values of foreground pixel points in the foreground object binary image are uniform non-zero values, and the gray values of background pixel points in the foreground object binary image are zero values;

carrying out moving target detection processing on three frames of pantograph gray images with continuous collection time sequence by using a three-frame difference method to obtain a moving target binary image, wherein the three frames of pantograph gray images comprise the pantograph gray images and two frames of pantograph gray images with the collection time sequences positioned in front of the pantograph gray images, the gray values of moving target pixel points in the moving target binary image are uniform non-zero values, and the gray values of background pixel points in the moving target binary image are zero values;

carrying out image logic and operation processing on the foreground object binary image and the moving target binary image to obtain a new binary image;

and carrying out contour detection processing on the new binary image, and determining a target contour which is obtained by detection and has an enclosing area exceeding a preset area threshold value in the contour as an invading foreign body contour, wherein the target contour refers to a closed contour enclosed by a plurality of adjacent edge pixel points in the new binary image, and the edge pixel points refer to pixel points which have non-zero gray values and at least one adjacent pixel point in eight adjacent pixel points around and have a gray value of zero.

Based on the content of the invention, a video image acquired in real time on the roof can be identified and positioned based on a region-of-interest ROI positioning technology to extract a pantograph region, then on one hand, a Gaussian mixture model is used for segmenting the foreground and the background of the pantograph gray image to obtain a foreground object binary image, on the other hand, a three-frame difference method is used for detecting a moving target of the pantograph gray image to obtain a moving target binary image, and finally, the logic and operation results are carried out on the two binary images, and whether an invasive foreign object exists in the operation result is determined by combining a contour area threshold value, so that the method can be applied to an electric locomotive, and can carry out real-time online monitoring on the state of the pantograph in the running process of the locomotive, automatically and accurately find whether the foreign object invades a pantograph working region, and further can be beneficial to carrying out omnibearing online monitoring on the pantograph region, and an alarm prompt is sent out in time, so that the safe and stable operation of the electric locomotive is guaranteed.

In one possible design, performing ROI positioning on the video image to extract a pantograph gray image includes:

sliding screenshot windows in the transverse direction and the longitudinal direction of the video image respectively by preset step lengths, and intercepting to obtain a plurality of video sub-images with the standard size of the sample;

respectively extracting corresponding HOG (histogram of oriented gradient) features of the HOG for each video sub-image in the plurality of video sub-images;

for each video subimage in the multiple video subimages, introducing the HOG feature of the corresponding direction gradient histogram into a Support Vector Machine (SVM) classification model which is trained based on the HOG feature of a positive sample and a negative sample, judging whether the corresponding video subimage contains a pantograph graph or not, and recording the position of the area of the video subimage in the video image when the video subimage is judged to contain the pantograph graph, wherein the positive sample in the positive sample and the negative sample refers to a sample image which has a sample standard size and contains the pantograph graph, and the negative sample in the positive sample and the negative sample refers to a sample image which has a sample standard size and does not contain the pantograph graph;

according to the position of the area, intercepting a pantograph image with a sample standard size from the video image;

and carrying out graying processing on the pantograph image to obtain the pantograph grayscale image.

In one possible design, for each of the plurality of video sub-images, extracting a corresponding histogram of oriented gradients HOG feature respectively includes:

carrying out gamma correction processing on the video sub-image to obtain a new video sub-image;

respectively calculating to obtain corresponding transverse gradient components and longitudinal gradient components aiming at each pixel point in the new video subimage;

respectively calculating corresponding gradient amplitude and gradient direction angle according to corresponding transverse gradient component and longitudinal gradient component aiming at each pixel point in the new video subimage;

dividing the new video subimage into a plurality of cell units;

respectively performing histogram statistical processing on all corresponding pixel points on each angle subinterval in a q-dimension subinterval to obtain a corresponding q-dimension feature vector for each cell unit in the plurality of cell units, wherein q represents a positive integer greater than 5, and the q-dimension subinterval is an angle interval [ -90 ]^o,90^o]Performing q equal division on all angle subintervals, and performing statistics on the histogram in a mode of accumulating the gradient amplitude of a pixel point when the gradient direction angle of the pixel point belongs to the angle subintervals;

splicing p adjacent cell units in the cell units to form a block, and connecting q-dimensional feature vectors of all the cell units in the p cell units in series to form p multiplied by q-dimensional feature vectors of the block;

scanning in the horizontal direction and the vertical direction of the video image by taking the size of the cell unit as a step length respectively to obtain a plurality of blocks and p × q-dimensional feature vectors corresponding to the blocks in the plurality of blocks;

and connecting p × q dimensional feature vectors of the blocks in series to form HOG features corresponding to the video sub-image.

In one possible design, performing foreground and background segmentation processing on the pantograph gray level image by using a gaussian mixture model to obtain a foreground object binary image, including:

the acquisition timing sequence is positioned before the gray level image of the pantograph and is continuous

A frame pantograph gray scale image, wherein,

represents a positive integer greater than 10;

according to the above

Frame pantograph gray image, extracting gray value of pixel point at each pixel position

Wherein, in the step (A),

indicates that is between

A positive integer between (a) and (b),

is shown in

Frame pantograph gray scaleThe first in the image arranged from the morning to the evening according to the acquisition time sequence

The acquisition time of the frame pantograph gray image,

is shown in the first

In the frame pantograph gray image

The gray value of the pixel point at each pixel location,

indicates that is between

A positive integer between (a) and (b),

is shown in

Total number of pixel positions in the frame pantograph gray image;

to the said

Each pixel position in the frame pantograph gray image updates the corresponding gaussian mixture model according to the following steps S331 to S338:

s331. pair

And is in the range of

Each Gaussian distributed in the Gaussian mixture model corresponding to each pixel positionThe weight, the mean, and the variance are initialized, respectively, and then a step S332 is performed, in which,

is initialized to 1, and the weight is initialized to

，

Is shown in the specification and

the total number of Gaussian distributions in the Gaussian mixture model corresponding to each pixel position;

s332, judging a first matching condition

If yes, go to step S333, otherwise go to step S334, wherein,

is represented by the second

The first pixel position corresponds to

The current mean of the gaussian distribution of the number,

represents the first

The current variance of the gaussian distribution is calculated,

indicates that is between

A positive integer between (a) and (b),

represents a positive number greater than or equal to 2.5;

s333, updating the second step according to the following formula

Weight, mean and variance of the individual gaussian distributions:

in the formula (I), the compound is shown in the specification,

represents the first

The updated weights of the gaussian distributions,

represents the first

The current weight of the gaussian distribution is given,

represents the first

The updated mean of the individual gaussian distributions,

represents the first

The updated variance of the gaussian distribution,

a preset learning update rate of the weight value is represented,

a learning update rate representing the mean value and having

，

Indicating a matrix transpose symbol, and then performing step S337;

s334, updating the first step according to the following formula

Weight of each gaussian distribution:

in the formula (I), the compound is shown in the specification,

represents the first

The updated weights of the gaussian distributions,

a preset learning update rate representing the weight value, and then step S335 is performed;

s335, judging whether the first matching condition is matched with the second matching condition

In a Gaussian mixture model corresponding to individual pixel positions

If the Gaussian distributions are not satisfied, executing step S336 if the Gaussian distributions are not satisfied, otherwise executing step S337；

S336. rejecting the

The Gaussian distribution with the minimum current weight in the Gaussian distributions is added with a new Gaussian distribution to obtain a new Gaussian distribution

Gaussian distribution, and then performing step S337 in which the mean value of the new gaussian distribution is initialized to

;

S337, in the pair of

Normalizing the weight values of the Gaussian distributions in the Gaussian mixture model corresponding to each pixel position to enable the sum of the new weight values of all the Gaussian distributions to be equal to 1, and then executing the step S338;

s338, judgment

Is equal to

If yes, the updating of the Gaussian mixture model is ended, otherwise, the method is right

Updating by adding 1, and then returning to execute the step S332;

for each pixel position, according to a ratio

All the Gaussian distributions in the corresponding Gaussian mixture models are sequentially arranged from small to large to obtain corresponding Gaussian distribution queues, wherein,

the current weight value representing the gaussian distribution,

representing the current variance of the gaussian distribution;

aiming at each pixel position, determining the corresponding and front-ranked pixel position in the corresponding Gaussian distribution queue

A gaussian distribution in which, among others,

taking values according to the following formula:

in the formula (I), the compound is shown in the specification,

expressed in equation in parentheses

Reach more than the preset weight threshold

Is on the positive integer variable

The value of the process is taken as the value,

indicates that is between

A positive integer between (a) and (b),

indicates that the queue number in the corresponding Gaussian distribution queue is

The current weight of the gaussian distribution;

determining a second matching condition for each pixel position

For before corresponding

Whether any Gaussian distribution in the Gaussian distributions is established or not is judged, if yes, the corresponding pixel point in the pantograph gray level image is judged to be a background pixel point, otherwise, the corresponding pixel point is judged to be a foreground pixel point, wherein,

representing the gray values of corresponding pixel points in the pantograph gray image,

is shown before corresponding

The current mean of the gaussian distribution in the gaussian distribution,

is shown before corresponding

A current variance of the Gaussian distribution in the Gaussian distribution;

and performing image binarization processing on the pantograph gray level image according to the judgment result of the pixel point type to obtain a foreground object binarization image.

In one possible design, the method for detecting a moving object by using a three-frame difference method to acquire three continuous-time-sequence three-frame pantograph gray level images to obtain a moving object binary image comprises the following steps:

respectively calculating to obtain a first difference image of a first two-frame pantograph gray image in the three-frame pantograph gray image and a second difference image of a second two-frame pantograph gray image in the three-frame pantograph gray image, wherein the gray value of each pixel point in the first difference image is the absolute value of the gray value difference of the corresponding pixel point in the first two-frame pantograph gray image, and the gray value of each pixel point in the second difference image is the absolute value of the gray value difference of the corresponding pixel point in the second two-frame pantograph gray image;

respectively carrying out image binarization processing on the first differential image and the second differential image according to the comparison result of the gray value of the pixel point and a preset threshold value to obtain corresponding differential binarization images;

and performing image logic and operation processing on the two differential binarization images obtained through the image binarization processing to obtain the moving target binarization image.

In one possible design, before performing image logical and operation processing on the foreground object binary image and the moving target binary image, the method further includes:

and carrying out morphological open operation processing on the foreground object binary image and/or the moving target binary image to obtain a new moving target binary image and/or a new moving target binary image for carrying out the logic and operation processing.

In a second aspect, the invention provides a pantograph foreign matter intrusion detection device, which comprises an image acquisition module, an image positioning module, a foreground segmentation module, a motion detection module, an operation processing module and a contour detection module;

the image acquisition module is used for acquiring a video image acquired by a monitoring camera in real time, wherein the monitoring camera is arranged on the roof of the vehicle and enables the view field of the lens to cover the area where the pantograph is located;

the image positioning module is in communication connection with the image acquisition module and is used for performing ROI (region of interest) positioning processing on the video image and extracting a pantograph gray image;

the foreground segmentation module is in communication connection with the image positioning module and is used for performing foreground and background segmentation processing on the pantograph gray level image by using a Gaussian mixture model to obtain a foreground object binary image, wherein gray values of foreground pixel points in the foreground object binary image are uniform non-zero values, and gray values of background pixel points in the foreground object binary image are zero values;

the motion detection module is in communication connection with the image positioning module and is used for performing motion target detection processing on three frames of pantograph gray level images with continuous collection time sequence by using a three-frame difference method to obtain a motion target binary image, wherein the three frames of pantograph gray level images comprise the pantograph gray level images and two frames of pantograph gray level images with collection time sequences positioned in front of the pantograph gray level images, the gray level values of motion target pixel points in the motion target binary image are uniform non-zero values, and the gray level value of background pixel points in the motion target binary image is zero;

the operation processing module is respectively in communication connection with the foreground segmentation module and the motion detection module and is used for carrying out image logic and operation processing on the foreground object binary image and the motion target binary image to obtain a new binary image;

the contour detection module is in communication connection with the operation processing module and is used for performing contour detection processing on the new binary image and determining a target contour which is obtained through detection and has an enclosing area exceeding a preset area threshold value in the contour as an invading foreign body contour, wherein the target contour refers to a closed contour which is enclosed by a plurality of adjacent edge pixel points in the new binary image, and the edge pixel points refer to pixel points which have a gray value of non-zero value and at least one adjacent pixel point in eight adjacent pixel points around and have a gray value of zero value.

In a third aspect, the present invention provides a computer device, comprising a memory, a processor and a transceiver, which are sequentially connected in communication, wherein the memory is used for storing a computer program, the transceiver is used for transceiving data, and the processor is used for reading the computer program and executing the method according to the first aspect or any one of the possible designs of the first aspect.

In a fourth aspect, the invention provides a pantograph foreign matter intrusion detection system, which comprises a monitoring camera, a processing device, a vehicle-mounted host, a display and an alarm prompting device, wherein the monitoring camera is in communication connection with the processing device, the processing device is in communication connection with the vehicle-mounted host, and the vehicle-mounted host is in communication connection with the display and the alarm prompting device respectively;

the monitoring camera is used for being installed on the roof of the vehicle, enabling the lens view to cover the area where the pantograph is located, and transmitting the video image acquired in real time to the processing equipment;

the processing device is configured to execute the method according to any one of the first aspect or possible designs of the first aspect, obtain a foreign object intrusion detection result, and transmit the video image and the foreign object intrusion detection result to the vehicle-mounted host;

the vehicle-mounted host is used for storing the video image and the foreign object intrusion detection result, distributing and transmitting the video image and the foreign object intrusion detection result to the display, and sending an alarm trigger instruction to the alarm prompting device when judging that a foreign object intrusion condition is met according to the foreign object intrusion detection result;

the display is used for displaying the video image and the foreign object intrusion detection result;

and the alarm prompting device is used for starting an alarm prompting action when the alarm triggering instruction is received.

In a fifth aspect, the present invention provides a storage medium having stored thereon instructions for performing the method as described above in the first aspect or any one of the possible designs of the first aspect when the instructions are run on a computer.

In a sixth aspect, the present invention provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method as described above in the first aspect or any one of the possible designs of the first aspect.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flow chart of a pantograph foreign matter intrusion detection method provided by the present invention.

Fig. 2 is an exemplary diagram of a video image captured by a monitoring camera provided by the present invention.

Fig. 3 is a schematic flow chart illustrating a foreground and a background segmentation process performed in the pantograph foreign object intrusion detection method according to the present invention.

Fig. 4 is a schematic structural diagram of a pantograph foreign matter intrusion detection device provided by the present invention.

Fig. 5 is a schematic structural diagram of a computer device provided by the present invention.

Fig. 6 is a schematic structural diagram of a pantograph foreign object intrusion detection system provided by the present invention.

In the above drawings: 1-a monitoring camera; 2-a processing device; 3-vehicle host computer; 4-a display; 5-an alarm prompting device; 61-a roof power supply; 62-an in-vehicle power supply; 7-a light supplement lamp; 100-pantograph.

Detailed Description

The invention is further described with reference to the following figures and specific embodiments. It should be noted that the description of the embodiments is provided to help understanding of the present invention, but the present invention is not limited thereto. Specific structural and functional details disclosed herein are merely representative of exemplary embodiments of the invention. This invention may, however, be embodied in many alternate forms and should not be construed as limited to the embodiments set forth herein.

It will be understood that, although the terms first, second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another. For example, a first object may be referred to as a second object, and similarly, a second object may be referred to as a first object, without departing from the scope of example embodiments of the present invention.

It should be understood that, for the term "and/or" as may appear herein, it is merely an associative relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, B exists alone or A and B exist at the same time; for the term "/and" as may appear herein, which describes another associative object relationship, it means that two relationships may exist, e.g., a/and B, may mean: a exists singly or A and B exist simultaneously; in addition, for the character "/" that may appear herein, it generally means that the former and latter associated objects are in an "or" relationship.

As shown in fig. 1 to 3, the pantograph foreign object intrusion detection method provided in the first aspect of the present embodiment may be, but is not limited to being, executed by a computer device disposed on an electric locomotive and communicatively connected to a roof monitoring camera. The method for detecting intrusion of a pantograph foreign object may include, but is not limited to, the following steps S1 to S6.

S1, acquiring a video image acquired by a monitoring camera in real time, wherein the monitoring camera is installed on the roof of the vehicle and enables the view field of a lens to cover the area where the pantograph is located.

In step S1, since the lens view covers the area where the pantograph is located, a complete pantograph image can be acquired in the obtained video image, as shown in fig. 2. In addition, the computer equipment is in communication connection with the monitoring camera, so that the video images can be transmitted in real time after being collected.

And S2, carrying out Region Of Interest (ROI) positioning processing on the video image, namely, in machine vision or image processing, delineating a Region to be processed in a mode Of a square frame, a circle, an ellipse or an irregular polygon and the like aiming at the processed image, namely, the Region is called as a Region Of Interest), and extracting a pantograph gray image.

In step S2, the ROI positioning process is to identify a pantograph pattern in the video image, outline a region of the pantograph pattern in a square frame, a circle, an ellipse, or an irregular polygon, and finally intercept the pantograph grayscale image from the video image according to a positioning result of the region. Preferably, the ROI positioning process is performed on the video image to extract the pantograph gray scale image, which includes, but is not limited to, the following steps S21 to S25.

And S21, sliding screenshot windows in the horizontal direction and the longitudinal direction of the video image respectively by preset step lengths, and intercepting to obtain a plurality of video sub-images with the standard size of the sample.

In step S21, the horizontal direction refers to an X-axis direction on an XOY coordinate plane of the video image, and the vertical direction refers to a Y-axis direction on the XOY coordinate plane of the video image, and the sliding steps of the screenshot window in the horizontal direction and the vertical direction may be the same or different. The screenshot window can be square or rectangular, the size of the screenshot window is equal to the standard size of the sample, so that a video subimage with the standard size of the sample can be obtained by means of interception, the size of the video subimage is consistent with the size of positive and negative samples of a Support Vector Machine (SVM) classification model after subsequent training is finished, and accuracy of a classification result is guaranteed.

And S22, respectively extracting corresponding HOG (histogram of oriented gradient) features of the direction gradient aiming at each video sub-image in the plurality of video sub-images.

In step S22, the histogram of hog (histogram of Oriented gradient) is a feature descriptor used for object detection in computer vision and image processing. The HOG features are formed by calculating and counting a gradient direction histogram of a local area of an image, are commonly used features in the field of computer vision and recognition at present, and are mainly used for describing the distribution condition of the gradient intensity and the direction of the image. Further preferably, the steps of extracting the corresponding histogram of oriented gradients HOG features for each of the plurality of video sub-images include, but are not limited to, the following steps S221 to S228.

S221, carrying out gamma correction processing on the video sub-image to obtain a new video sub-image.

In step S221, the gamma correction, also called gamma non-linearization or gamma encoding, is used to perform non-linear operation or inverse operation on the luminance of light or tristimulus values in a film or image system, so as to make the gray-level values of the pixels have a smaller storage range (0-255) and a more balanced ratio of bright and dark portions, and thus the new video sub-image beneficial to subsequent HOG feature extraction can be obtained through the conventional gamma correction processing. Specifically, the gamma correction process may be performed according to the following formula:

in the formula (I), the compound is shown in the specification,

indicating a pixel position in said video sub-picture

The gray value of the upper pixel point is,

indicating a pixel position in said new video sub-image

The gray value of the upper pixel point is,

a lateral coordinate value indicating a pixel position,

a longitudinal coordinate value indicating a pixel position,

represents the gamma correction parameters, which can be taken as

。

And S222, respectively calculating to obtain corresponding transverse gradient components and longitudinal gradient components aiming at each pixel point in the new video subimage.

In the step S222, the calculation method of the transverse gradient component and the longitudinal gradient component is the conventional method, i.e. for the position of the pixel

Pixel of (2), its corresponding transverse gradient component

And a longitudinal gradient component

The calculation can be made according to the following formula:

in the formula (I), the compound is shown in the specification,

indicating a pixel position in said new video sub-image

The gray value of the upper pixel point is,

indicating a pixel position in said new video sub-image

The gray value of the upper pixel point is,

indicating a pixel position in said new video sub-image

The gray value of the upper pixel point is,

indicating a pixel position in said new video sub-image

The gray value of the pixel point.

And S223, aiming at each pixel point in the new video subimage, respectively calculating to obtain a corresponding gradient amplitude and a corresponding gradient direction angle according to the corresponding transverse gradient component and the corresponding longitudinal gradient component.

In step S223, the gradient magnitude and gradient direction angle are calculated by the conventional method, i.e. for the pixel position

The corresponding gradient amplitude of the pixel point

And angle of gradient direction

The calculation can be made according to the following formula:

in the formula (I), the compound is shown in the specification,

representing the arctan function.

S224, dividing the new video subimage into a plurality of cell units.

In step S224, the cell unit is a unit image, and the shape of the cell unit may be a square or a rectangle.

S225, aiming at each cell unit in the cell units, respectively carrying out histogram statistical processing on all corresponding pixel points in each angle subinterval in a q-dimension subinterval to obtain a corresponding q-dimension feature vector, wherein q represents a positive integer greater than 5, and the q-dimension subinterval is an angle interval [ -90 ]^o,90^o]And performing q equal division on all the angle subintervals, and performing statistics on the histogram in a mode of accumulating the gradient amplitude of the pixel point when the gradient direction angle of the pixel point belongs to the angle subintervals.

In step S225, q may preferably take a value of 9, that is, the angle range of each angle subinterval is 20^oThe q angle subintervals are respectively [ -90 ]^o,-70^o]、[-70^o,-50^o]、[-50^o,-30^o]、[-30^o,-10^o]、[-10^o,10^o]、[10^o,30^o]、[30^o,50^o]、[50^o,70^o]And [70 ]^o,90^o]Thus, for each cell unit, a corresponding one of the 9-dimensional feature vectors is obtained. In addition, the histogram statistical processing may be exemplified by: for the 6 th-dimension vector in the 9-dimension feature vectors, the corresponding angle subinterval is [10 ]^o,30^o]If the gradient direction angle of 356 pixels belongs to the angular sub-interval [10 ]^o,30^o]And taking the accumulated result of the gradient amplitudes of the 356 pixel points as the value on the 6 th-dimension vector.

S226, splicing p adjacent cell units in the cell units into a block, and connecting the q-dimensional feature vectors of the cell units in the p cell units in series to form the p × q-dimensional feature vector of the block.

In step S226, the value of p may also be preferably 9, that is, the upper left cell unit, the upper right cell unit, the lower left cell unit, the right left cell unit, and the central cell unit are spliced to form the block, and the 9-dimensional eigenvectors of the nine cell units are connected in series from top to bottom in the matrix to form the 9 × 9-dimensional eigenvector.

And S227, scanning in the horizontal direction and the vertical direction of the video image by taking the size of the cell unit as a step length respectively to obtain a plurality of blocks and p × q-dimensional feature vectors corresponding to the blocks in the plurality of blocks.

In step S227, a specific manner of obtaining the p × q-dimensional feature vector of each of the blocks may refer to step S226, which is not described herein again.

And S228, connecting p × q dimensional feature vectors of the blocks in series to form HOG features corresponding to the video sub-image.

And S23, aiming at each video sub-image in the plurality of video sub-images, introducing the corresponding HOG feature of the direction gradient histogram into a SVM classification model trained based on the HOG feature of a positive sample and a negative sample, judging whether the corresponding video sub-image contains a pantograph graph or not, and recording the position of the area of the video sub-image in the video image when the video sub-image contains the pantograph graph, wherein the positive sample in the positive sample and the negative sample refers to a sample image with a sample standard size and containing the pantograph graph, and the negative sample in the positive sample and the negative sample refers to a sample image with a sample standard size and containing no pantograph graph.

In the step S23, the way of acquiring the HOG features of each sample image in the positive and negative samples may refer to the aforementioned steps S221 to S228, which are not described herein again. The more the number of positive samples and the number of negative samples in the positive and negative samples are, the better, and the balance is required as much as possible. An SVM (support Vector machine) is a generalized linear classifier for binary classification of data according to a supervised learning mode, a decision boundary of the SVM is a maximum margin hyperplane solved for a learning sample, and the SVM is a general method for machine learning under a limited training sample. Furthermore, the sample images of the positive and negative samples are preferably from a video segment captured by the surveillance camera.

And S24, intercepting a pantograph image with a sample standard size from the video image according to the position of the area.

In the step S24, if it is determined in the step S23 that there are a plurality of location area positions, a pantograph image having a sample standard size may be cut out from the video image with a center position of the plurality of location area positions being a cut-out center point of the pantograph image.

And S25, carrying out gray processing on the pantograph image to obtain the pantograph gray image.

In step S25, if the video image is a grayscale image, the graying process may be skipped, and the pantograph grayscale image may be directly extracted from the video image.

And S3, segmenting the foreground and the background of the pantograph gray image by using a Gaussian mixture model to obtain a foreground object binary image, wherein the gray value of a foreground pixel point in the foreground object binary image is a uniform non-zero value, and the gray value of a background pixel point in the foreground object binary image is a zero value.

In step S3, the Gaussian Mixed Model (GMM) refers to a linear combination of multiple Gaussian distribution functions, and thus includes multiple Gaussian distributions. The segmentation processing idea of the foreground and the background is to use a Gaussian mixture model GMM to perform background modeling on the pantograph gray level image so as to realize the segmentation of the foreground and the background of the image, namely, a background representation method based on pixel sample statistical information is used, the background is represented by using statistical information such as probability density of a large number of sample values of pixels in a long time, and a statistical difference criterion is used for judging a pixel target so as to separate the foreground from the background. The Gaussian mixture model is a classical description of statistical information of pixel samples, and in the Gaussian mixture model, color information among pixels is considered to be irrelevant, and processing of each pixel point is independent. Therefore, for each pixel point in the video image, the change of the value in the sequence image (i.e. the multi-frame video image with continuous acquisition time sequence) can be regarded as a random process which continuously generates the pixel value, i.e. the color rendering rule of each pixel point is described by gaussian distribution. For each pixel point in the image, the gaussian mixture model can represent the characteristics of each pixel point in the image by superposition of a plurality of gaussian distributions with different weights, wherein the probability formula of each pixel point can be expressed as follows:

in the formula (I), the compound is shown in the specification,

indicates that is between

A positive integer between (a) and (b),

representing the total number of pixel locations in the image,

the representation is located at the second place in the image

The gray value of the pixel point at each pixel location,

is indicated to be located at the second

The gray value of the pixel point at each pixel position is

The probability of (a) of (b) being,

represents the total number of gaussian distributions in the gaussian mixture model,

indicates that is between

A positive integer between (a) and (b),

is represented by the second

Pixel point correspondences at individual pixel positions and in the Gaussian mixture model

The weight of each of the gaussian distributions is calculated,

represents the first

The mean of the number of gaussian distributions,

represents the first

A covariance matrix of Gaussian distribution

，

Represents the first

The variance of the individual gaussian distributions,

the unit matrix is represented by a matrix of units,

represents the first

A probability density function of a Gaussian distribution having

In the formula (I), the compound is shown in the specification,

to represent

The dimension(s) of (a) is,

the base of the natural logarithm is represented,

representing the matrix transpose symbol. Therefore, for the Gaussian mixture model, the model can be built according to superposition of a plurality of Gaussian distributions of different weights for each pixel point in the image, each Gaussian distribution corresponds to a state which can possibly generate the color presented by the pixel point, and the weight and the distribution parameters of each Gaussian distribution can be updated along with time so as to gradually model the background。

In the step S3, it is preferable that the segmentation process of the foreground and the background is performed on the pantograph gray scale image by using a gaussian mixture model to obtain a binary image of the foreground object, including, but not limited to, the following steps S31 to S37.

S31, acquiring a continuous acquisition time sequence positioned before the pantograph gray level image

A frame pantograph gray scale image, wherein,

representing a positive integer greater than 10.

In the step S31, the

The manner of acquiring each frame of pantograph gray scale image in the frame pantograph gray scale image can be referred to in the foregoing steps S1 to S2, which are all from the real-time video collected by the monitoring camera.

S32, according to the

Frame pantograph gray image, extracting gray value of pixel point at each pixel position

Wherein, in the step (A),

indicates that is between

A positive integer between (a) and (b),

is shown in

The first arranged in the gray level image of the frame pantograph from the morning to the evening according to the acquisition time sequence

The acquisition time of the frame pantograph gray image,

is shown in the first

In the frame pantograph gray image

The gray value of the pixel point at each pixel location,

indicates that is between

A positive integer between (a) and (b),

is shown in

Total number of pixel locations in the frame pantograph gray scale image.

S33, aiming at

Each pixel position in the frame pantograph gray scale image is updated with the corresponding gaussian mixture model in the following steps S331 to S338, as shown in fig. 3.

S331. pair

And is in the range of

Personal portraitThe weight, mean and variance of each gaussian distribution in the gaussian mixture model corresponding to the element position are initialized, respectively, and then step S332 is performed, wherein,

is initialized to 1, and the weight is initialized to

，

Is shown in the specification and

and the total number of Gaussian distributions in the Gaussian mixture model corresponding to each pixel position.

In the step S331, a mean value of each gaussian distribution may be initialized to the value

The gray value of any pixel point in the frame pantograph gray image, and the variance of each Gaussian distribution can be initialized to a larger value, such as the above

The difference between the maximum gray value and the minimum gray value in the frame pantograph gray image.

S332, judging a first matching condition

If yes, go to step S333, otherwise go to step S334, wherein,

is represented by the second

The first pixel position corresponds to

The current mean of the gaussian distribution of the number,

represents the first

The current variance of the gaussian distribution is calculated,

indicates that is between

A positive integer between (a) and (b),

represents a positive number greater than or equal to 2.5.

In the step S332, the first matching condition

For judging

Whether or not to the second

A gaussian distribution match, wherein,

the value can be specifically between 2.5 and 3.5, for example, 2.5.

S333, updating the second step according to the following formula

Weight, mean and variance of the individual gaussian distributions:

in the formula (I), the compound is shown in the specification,

represents the first

The updated weights of the gaussian distributions,

represents the first

The current weight of the gaussian distribution is given,

represents the first

The updated mean of the individual gaussian distributions,

represents the first

The updated variance of the gaussian distribution,

a preset learning update rate of the weight value is represented,

a learning update rate representing the mean value and having

，

Indicating that the matrix transposes the symbols, then step S337 is performed.

S334, according to the following stepsFormula updates the second

Weight of each gaussian distribution:

in the formula (I), the compound is shown in the specification,

represents the first

The updated weights of the gaussian distributions,

a preset learning update rate indicating the weight value, and then step S335 is performed.

S335, judging whether the first matching condition is matched with the second matching condition

In a Gaussian mixture model corresponding to individual pixel positions

If the gaussian distributions are not satisfied, if yes, step S336 is executed, otherwise step S337 is executed.

S336. rejecting the

The Gaussian distribution with the minimum current weight in the Gaussian distributions is added with a new Gaussian distribution to obtain a new Gaussian distribution

Gaussian distribution, and then performing step S337 in which the mean value of the new gaussian distribution is initialized to

。

In step S336, the new weight of the Gaussian distribution is initialized to a smaller value, for example

The variance of the new Gaussian distribution may be initialized to a larger value, such as the one described

The difference between the maximum gray value and the minimum gray value in the frame pantograph gray image.

S337, in the pair of

The weights of the gaussian distributions in the gaussian mixture model corresponding to each pixel position are normalized to make the sum of the new weights of all the gaussian distributions equal to 1, and then step S338 is executed.

S338, judgment

Is equal to

If yes, the updating of the Gaussian mixture model is ended, otherwise, the method is right

The self-added 1 update is performed, and then the execution returns to step S332.

S34, aiming at each pixel position, according to a ratio

All the Gaussian distributions in the corresponding Gaussian mixture models are sequentially arranged from small to large to obtain corresponding Gaussian distribution queues, wherein,

the current weight value representing the gaussian distribution,

representing the current variance of the gaussian distribution.

S35, aiming at each pixel position, determining corresponding and front-ranked Gaussian distribution queues in corresponding Gaussian distribution queues

A gaussian distribution in which, among others,

taking values according to the following formula:

in the formula (I), the compound is shown in the specification,

expressed in equation in parentheses

Reach more than the preset weight threshold

Is on the positive integer variable

The value of the process is taken as the value,

indicates that is between

A positive integer between (a) and (b),

indicates that the queue number in the corresponding Gaussian distribution queue is

Is calculated based on the current weight of the gaussian distribution.

S36, aiming at each pixel position, judging a second matching condition

For before corresponding

Whether any Gaussian distribution in the Gaussian distributions is established or not is judged, if yes, the corresponding pixel point in the pantograph gray level image is judged to be a background pixel point, otherwise, the corresponding pixel point is judged to be a foreground pixel point, wherein,

representing the gray values of corresponding pixel points in the pantograph gray image,

is shown before corresponding

The current mean of the gaussian distribution in the gaussian distribution,

is shown before corresponding

The current variance of the gaussian.

In the step S36, the front

The Gaussian distribution is a background model which corresponds to the pixel position and is gradually improved after a long time, and the second matching condition is

For judging

Whether the background model is matched with any Gaussian distribution in the background model or not, if so, the correspondence can be indicated

The pixel points of (1) are background pixel points, otherwise, the pixel points can be judged as foreground pixel points.

And S37, carrying out image binarization processing on the pantograph gray level image according to the judgment result of the pixel point type to obtain a foreground object binarization image.

In the step S37, the specific manner of the image binarization processing may be, but is not limited to, assigning the gray value of the foreground pixel to be 255 (i.e. a uniform non-zero value), and assigning the gray value of the background pixel to be 0 (i.e. a zero value).

And S4, carrying out moving target detection processing on three frames of pantograph gray level images with continuous collection time sequence by using a three-frame difference method to obtain a moving target binary image, wherein the three frames of pantograph gray level images comprise the pantograph gray level images and two frames of pantograph gray level images with collection time sequences positioned in front of the pantograph gray level images, the gray level values of moving target pixel points in the moving target binary image are uniform non-zero values, and the gray level values of background pixel points in the moving target binary image are zero values.

In step S4, the three-frame difference method is an improved method of the adjacent two-frame difference algorithm, which selects three consecutive frames of video images to perform difference operation, so as to eliminate the background influence revealed by motion, thereby extracting accurate contour information of the moving object. The basic principle of the three-frame difference method is that continuous three frames of images in a video image sequence are selected, difference images of two adjacent frames are calculated respectively, then binarization processing is carried out on the difference images by selecting a proper threshold value to obtain a binarization image, and finally logic and operation are carried out on the binarization images obtained at each pixel point to obtain a common part, so that contour information of a moving target is obtained. Therefore, preferably, the motion target detection processing is performed on three frames of pantograph gray-scale images with continuous acquisition time sequence by using a three-frame difference method to obtain a motion target binary image, including, but not limited to, the following steps S41 to S43.

And S41, respectively calculating to obtain a first difference image of the previous two frames of pantograph gray images in the three frames of pantograph gray images and a second difference image of the next two frames of pantograph gray images in the three frames of pantograph gray images, wherein the gray value of each pixel point in the first difference image is the absolute value of the gray value difference of the corresponding pixel point in the previous two frames of pantograph gray images, and the gray value of each pixel point in the second difference image is the absolute value of the gray value difference of the corresponding pixel point in the next two frames of pantograph gray images.

And S42, respectively carrying out image binarization processing on the first differential image and the second differential image according to the comparison result of the gray value of the pixel point and a preset threshold value to obtain corresponding differential binarization images.

In the step S42, the specific manner of the image binarization processing may be, but is not limited to: for the pixels with the gray values greater than or equal to the preset threshold, the pixels are determined as moving target pixels (namely foreground pixels), the corresponding gray values are updated to 255 (namely a uniform non-zero value), and for the pixels with the gray values less than the preset threshold, the pixels are determined as non-moving target pixels (namely background pixels), and the corresponding gray values are updated to 0 (namely 0 zero value).

And S43, carrying out image logic and operation processing on the two differential binarization images obtained through image binarization processing to obtain the moving target binarization image.

And S5, carrying out image logic and operation processing on the foreground object binary image and the moving target binary image to obtain a new binary image.

Before the step S5, in order to remove the smaller isolated interference region and the noise point in advance, preferably, the method further includes: and carrying out morphological open operation processing on the foreground object binary image and/or the moving target binary image to obtain a new moving target binary image and/or a new moving target binary image for carrying out the logic and operation processing.

S6, carrying out contour detection processing on the new binary image, and determining a target contour which is obtained through detection and has an enclosing area exceeding a preset area threshold value in the contour as an invading foreign body contour, wherein the target contour refers to a closed contour which is enclosed by a plurality of adjacent edge pixel points in the new binary image, and the edge pixel points refer to pixel points which have a gray value of non-zero value and at least one adjacent pixel point of which the gray value is zero in eight adjacent pixel points (namely, an upper left pixel point, an upper right pixel point, a right pixel point, a lower left pixel point, a left pixel point and the like) around.

In step S6, the enclosing area in the outline may be but is not limited to the total number of the enclosed pixels, the larger the number is, the larger the area is, the more obvious the outline is, and the higher the accuracy of the invading foreign object is, so that the small target interference can be filtered by combining with the preset area threshold, and the correctness of the invading foreign object determination is ensured. In addition, if no contour is detected or no target contour with an enclosing area within the contour exceeding the preset area threshold is detected, it can be determined that no foreign object intrusion event is currently present.

Therefore, through the pantograph foreign matter intrusion detection scheme described in detail in the foregoing steps S1 to S6, the video image obtained in real time on the roof of the vehicle is identified and positioned based on the ROI positioning technique, a pantograph region is extracted, a pantograph gray-scale image is extracted, on one hand, the foreground and background segmentation is performed on the pantograph gray-scale image by using the gaussian mixture model to obtain a foreground object binary image, on the other hand, the moving object detection is performed on the pantograph gray-scale image by using the three-frame difference method to obtain a moving object binary image, and finally, the logic and operation result is performed on the two binary images, and the presence or absence of an intruding foreign matter in the operation result is determined by combining the contour area threshold, so that the pantograph foreign matter intrusion detection scheme can be applied to an electric locomotive, perform real-time online monitoring on the pantograph state during the locomotive operation process, and automatically and accurately find out whether a foreign matter intrudes into the pantograph working region, therefore, the pantograph region can be monitored online in an all-around mode, an alarm prompt is sent out in time, and the safe and stable operation of the electric locomotive is guaranteed.

As shown in fig. 4, a second aspect of this embodiment provides a virtual device for implementing the method for detecting intrusion of a pantograph foreign object according to any one of the first aspect or the first aspect, including an image acquisition module, an image positioning module, a foreground segmentation module, a motion detection module, an operation processing module, and a contour detection module;

the image acquisition module is used for acquiring a video image acquired by a monitoring camera in real time, wherein the monitoring camera is arranged on the roof of the vehicle and enables the view field of the lens to cover the area where the pantograph is located;

the image positioning module is in communication connection with the image acquisition module and is used for performing ROI (region of interest) positioning processing on the video image and extracting a pantograph gray image;

the foreground segmentation module is in communication connection with the image positioning module and is used for performing foreground and background segmentation processing on the pantograph gray level image by using a Gaussian mixture model to obtain a foreground object binary image, wherein gray values of foreground pixel points in the foreground object binary image are uniform non-zero values, and gray values of background pixel points in the foreground object binary image are zero values;

the motion detection module is in communication connection with the image positioning module and is used for performing motion target detection processing on three frames of pantograph gray level images with continuous collection time sequence by using a three-frame difference method to obtain a motion target binary image, wherein the three frames of pantograph gray level images comprise the pantograph gray level images and two frames of pantograph gray level images with collection time sequences positioned in front of the pantograph gray level images, the gray level values of motion target pixel points in the motion target binary image are uniform non-zero values, and the gray level value of background pixel points in the motion target binary image is zero;

the operation processing module is respectively in communication connection with the foreground segmentation module and the motion detection module and is used for carrying out image logic and operation processing on the foreground object binary image and the motion target binary image to obtain a new binary image;

the contour detection module is in communication connection with the operation processing module and is used for performing contour detection processing on the new binary image and determining a target contour which is obtained through detection and has an enclosing area exceeding a preset area threshold value in the contour as an invading foreign body contour, wherein the target contour refers to a closed contour which is enclosed by a plurality of adjacent edge pixel points in the new binary image, and the edge pixel points refer to pixel points which have a gray value of non-zero value and at least one adjacent pixel point in eight adjacent pixel points around and have a gray value of zero value.

For the working process, working details and technical effects of the foregoing apparatus provided in the second aspect of this embodiment, reference may be made to the method described in the first aspect or any one of the possible designs of the first aspect, which is not described herein again.

As shown in fig. 5, a third aspect of the present embodiment provides a computer device for executing the pantograph foreign object intrusion detection method according to any one of the possible designs of the first aspect or the first aspect, and the computer device includes a memory, a processor and a transceiver, which are sequentially and communicatively connected, where the memory is used for storing a computer program, the transceiver is used for transceiving data, and the processor is used for reading the computer program and executing the pantograph foreign object intrusion detection method according to any one of the possible designs of the first aspect or the first aspect. For example, the Memory may include, but is not limited to, a Random-Access Memory (RAM), a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a First-in First-out (FIFO), and/or a First-in Last-out (FILO), and the like; the processor may be, but is not limited to, a microprocessor of the model number STM32F105 family. In addition, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

For the working process, working details, and technical effects of the foregoing computer device provided in the third aspect of this embodiment, reference may be made to the method in the first aspect or any one of the possible designs in the first aspect, which is not described herein again.

A fourth aspect of the present invention provides a detection system applying any one of the first aspect or the first aspect, which may be designed to the pantograph foreign object intrusion detection method, that is, as shown in fig. 6, the detection system includes a monitoring camera 1, a processing device 2, an on-board host 3, a display 4, and an alarm prompting device 5, where the monitoring camera 1 is communicatively connected to the processing device 2, the processing device 2 is communicatively connected to the on-board host 3, and the on-board host 3 is communicatively connected to the display 4 and the alarm prompting device 5, respectively; the monitoring camera 1 is used for being installed on the roof of a vehicle, enabling the lens view to cover the area where the pantograph is located, and transmitting a video image acquired in real time to the processing equipment 2; the processing device 2 is configured to execute the pantograph foreign object intrusion detection method according to any one of the first aspect and the possible designs of the first aspect, obtain a foreign object intrusion detection result, and transmit the video image and the foreign object intrusion detection result to the on-board host 3; the vehicle-mounted host 3 is used for storing the video image and the foreign object intrusion detection result, distributing and transmitting the video image and the foreign object intrusion detection result to the display 4, and sending an alarm trigger instruction to the alarm prompting device 5 when determining that a foreign object intrusion condition is met according to the foreign object intrusion detection result; the display 4 is used for displaying the video image and the foreign object intrusion detection result; and the alarm prompting device 5 is used for starting an alarm prompting action when the alarm triggering instruction is received.

As shown in fig. 6, in the specific structure of the pantograph foreign object intrusion detection system, both themonitoring camera 1 and theprocessing device 2 are disposed on the roof, wherein themonitoring camera 1 is preferably a high-definition industrial camera capable of high-quality imaging so as to transmit high-quality imaging data to theprocessing device 2; theprocessing device 2 may be implemented by the computer device provided in the third aspect of this embodiment, and a specific manner of transmitting the video image and the foreign object intrusion detection result to the on-board host 3 may be, for example: and superposing corresponding foreign object intrusion detection results and other acquisition state information on continuous image data, then obtaining a video stream by adopting an H264 compression coding mode, and finally transmitting the video stream to the vehicle-mountedhost 3 in a video stream mode. In addition, aroof power supply 61 and asupplementary lighting lamp 7 can be arranged on the roof, wherein theroof power supply 61 is used for providing electric energy support for themonitoring camera 1, theprocessing device 2, thesupplementary lighting lamp 7 and the like so that the monitoring camera, the processing device and the supplementary lighting lamp can work normally, and thesupplementary lighting lamp 7 is used for irradiating apantograph 100 of the roof so as to ensure the imaging quality of the video image and facilitate the subsequent intrusion foreign object detection processing.

On-vehicle host computer 3 thedisplay 4 with 5 equipartitions of warning suggestion device are arranged in the car, wherein, on-vehicle host computer 3 mainly used is to coming from the data completion system distribution and the storage work ofprocessing apparatus 2 can adopt conventional server equipment to realize, simultaneously it is according to whether there is the mode of foreign matter invasion for conventional mode in foreign matter invasion testing result judgement, for example after receiving the video image that marks there is the invasion foreign matter profile, judges that the foreign matter invasion condition is established. Thedisplay 4 is used for facing workers in the vehicle and visually displaying audio and video data (including but not limited to alarm prompt information generated when the condition of foreign matter invasion is determined to be established) from the vehicle-mountedhost 3 so as to provide reference for driving safety decision. The alarm prompting action of thealarm prompting device 5 can be but not limited to an audible and visual alarm action, so that the state of foreign matters on the roof is timely fed back to corresponding workers, and the harm of bow net is eliminated. In addition, an in-vehicle power source 62 may be disposed in the vehicle to provide power support for the in-vehicle host 3, thedisplay 4, thealarm prompting device 5, and the like.

For a working process, working details, and technical effects of the foregoing detection system provided in the fourth aspect of this embodiment, reference may be made to the method in the first aspect or any one of possible designs in the first aspect, which is not described herein again.

A fifth aspect of the present embodiment provides a storage medium storing instructions including instructions of any one of the first aspect or the possible designs of the pantograph foreign object intrusion detection method, that is, the storage medium stores instructions that, when executed on a computer, perform the pantograph foreign object intrusion detection method according to any one of the first aspect or the possible designs of the first aspect. The storage medium refers to a carrier for storing data, and may include, but is not limited to, a computer-readable storage medium such as a floppy disk, an optical disk, a hard disk, a flash Memory, a flash disk and/or a Memory Stick (Memory Stick), and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

For the working process, working details and technical effects of the foregoing readable storage medium provided in the fifth aspect of this embodiment, reference may be made to the method in any one of the first aspect or the first aspect, which is not described herein again.

A sixth aspect of the present embodiment provides a computer program product containing instructions which, when run on a computer, cause the computer to execute the pantograph foreign object intrusion detection method according to the first aspect or any one of the possible designs of the first aspect. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable devices.

Finally, it should be noted that the present invention is not limited to the above alternative embodiments, and that various other forms of products can be obtained by anyone in light of the present invention. The above detailed description should not be taken as limiting the scope of the invention, which is defined in the claims, and which the description is intended to be interpreted accordingly.

Claims (10)

1. A pantograph foreign matter intrusion detection method is characterized by comprising the following steps:

acquiring a video image acquired by a monitoring camera in real time, wherein the monitoring camera is mounted on the roof of the vehicle and enables the view field of a lens to cover the area where the pantograph is located;

carrying out ROI (region of interest) positioning processing on the video image, and extracting a pantograph gray image;

segmenting the foreground and the background of the pantograph gray image by using a Gaussian mixture model to obtain a foreground object binary image, wherein the gray values of foreground pixel points in the foreground object binary image are uniform non-zero values, and the gray values of background pixel points in the foreground object binary image are zero values;

carrying out moving target detection processing on three frames of pantograph gray images with continuous collection time sequence by using a three-frame difference method to obtain a moving target binary image, wherein the three frames of pantograph gray images comprise the pantograph gray images and two frames of pantograph gray images with the collection time sequences positioned in front of the pantograph gray images, the gray values of moving target pixel points in the moving target binary image are uniform non-zero values, and the gray values of background pixel points in the moving target binary image are zero values;

carrying out image logic and operation processing on the foreground object binary image and the moving target binary image to obtain a new binary image;

and carrying out contour detection processing on the new binary image, and determining a target contour which is obtained by detection and has an enclosing area exceeding a preset area threshold value in the contour as an invading foreign body contour, wherein the target contour refers to a closed contour enclosed by a plurality of adjacent edge pixel points in the new binary image, and the edge pixel points refer to pixel points which have non-zero gray values and at least one adjacent pixel point in eight adjacent pixel points around and have a gray value of zero.

2. The method for detecting intrusion of foreign object into a pantograph according to claim 1, wherein the ROI positioning process is performed on the video image to extract a grayscale image of the pantograph, and the method comprises:

sliding screenshot windows in the transverse direction and the longitudinal direction of the video image respectively by preset step lengths, and intercepting to obtain a plurality of video sub-images with the standard size of the sample;

respectively extracting corresponding HOG (histogram of oriented gradient) features of the HOG for each video sub-image in the plurality of video sub-images;

for each video subimage in the multiple video subimages, introducing the HOG feature of the corresponding direction gradient histogram into a Support Vector Machine (SVM) classification model which is trained based on the HOG feature of a positive sample and a negative sample, judging whether the corresponding video subimage contains a pantograph graph or not, and recording the position of the area of the video subimage in the video image when the video subimage is judged to contain the pantograph graph, wherein the positive sample in the positive sample and the negative sample refers to a sample image which has a sample standard size and contains the pantograph graph, and the negative sample in the positive sample and the negative sample refers to a sample image which has a sample standard size and does not contain the pantograph graph;

according to the position of the area, intercepting a pantograph image with a sample standard size from the video image;

and carrying out graying processing on the pantograph image to obtain the pantograph grayscale image.

3. The method for detecting intrusion of a foreign object into a pantograph according to claim 2, wherein the step of extracting a Histogram of Oriented Gradients (HOG) feature from each of the plurality of video sub-images comprises:

carrying out gamma correction processing on the video sub-image to obtain a new video sub-image;

respectively calculating to obtain corresponding transverse gradient components and longitudinal gradient components aiming at each pixel point in the new video subimage;

respectively calculating corresponding gradient amplitude and gradient direction angle according to corresponding transverse gradient component and longitudinal gradient component aiming at each pixel point in the new video subimage;

dividing the new video subimage into a plurality of cell units;

aiming at each cell unit in the plurality of cell units, respectively carrying out each angle subinterval of all corresponding pixel points in the q angle subintervalsPerforming statistical processing on the histogram to obtain a corresponding q-dimensional feature vector, wherein q represents a positive integer greater than 5, and the q angle subintervals are angle intervals [ -90 ]^o,90^o]Performing q equal division on all angle subintervals, and performing statistics on the histogram in a mode of accumulating the gradient amplitude of a pixel point when the gradient direction angle of the pixel point belongs to the angle subintervals;

splicing p adjacent cell units in the cell units to form a block, and connecting q-dimensional feature vectors of all the cell units in the p cell units in series to form p multiplied by q-dimensional feature vectors of the block;

scanning in the horizontal direction and the vertical direction of the video image by taking the size of the cell unit as a step length respectively to obtain a plurality of blocks and p × q-dimensional feature vectors corresponding to the blocks in the plurality of blocks;

and connecting p × q dimensional feature vectors of the blocks in series to form HOG features corresponding to the video sub-image.

4. The method for detecting intrusion of foreign object into a pantograph according to claim 1, wherein the segmentation of foreground and background is performed on the pantograph gray scale image by using a gaussian mixture model to obtain a binary image of a foreground object, comprising:

the acquisition timing sequence is positioned before the gray level image of the pantograph and is continuous

A frame pantograph gray scale image, wherein,

represents a positive integer greater than 10;

according to the above

Frame pantograph gray image, extracting gray value of pixel point at each pixel position

Wherein, in the step (A),

indicates that is between

A positive integer between (a) and (b),

is shown in

The first arranged in the gray level image of the frame pantograph from the morning to the evening according to the acquisition time sequence

The acquisition time of the frame pantograph gray image,

is shown in the first

In the frame pantograph gray image

The gray value of the pixel point at each pixel location,

indicates that is between

A positive integer between (a) and (b),

is shown in

Total number of pixel positions in the frame pantograph gray image;

to the said

Each pixel position in the frame pantograph gray image updates the corresponding gaussian mixture model according to the following steps S331 to S338:

s331. pair

And is in the range of

The weight, mean and variance of each gaussian distribution in the gaussian mixture model corresponding to each pixel position are initialized, respectively, and then step S332 is performed, wherein,

is initialized to 1, and the weight is initialized to

，

Is shown in the specification and

the total number of Gaussian distributions in the Gaussian mixture model corresponding to each pixel position;

s332, judging a first matching condition

If yes, go to step S333, otherwise go to step S334, wherein,

is represented by the second

The first pixel position corresponds to

The current mean of the gaussian distribution of the number,

represents the first

The current variance of the gaussian distribution is calculated,

indicates that is between

A positive integer between (a) and (b),

represents a positive number greater than or equal to 2.5;

s333, updating the second step according to the following formula

Weight, mean and variance of the individual gaussian distributions:

in the formula (I), the compound is shown in the specification,

represents the first

The updated weights of the gaussian distributions,

represents the first

The current weight of the gaussian distribution is given,

represents the first

The updated mean of the individual gaussian distributions,

represents the first

The updated variance of the gaussian distribution,

a preset learning update rate of the weight value is represented,

a learning update rate representing the mean value and having

，

Indicating a matrix transpose symbol, and then performing step S337;

s334, updating the first step according to the following formula

Weight of each gaussian distribution:

in the formula (I), the compound is shown in the specification,

represents the first

The updated weights of the gaussian distributions,

a preset learning update rate representing the weight value, and then step S335 is performed;

s335, judging whether the first matching condition is matched with the second matching condition

In a Gaussian mixture model corresponding to individual pixel positions

If the gaussian distributions are not satisfied, executing step S336, otherwise executing step S337;

s336. rejecting the

The Gaussian distribution with the minimum current weight in the Gaussian distributions is added with a new Gaussian distribution to obtain a new Gaussian distribution

Gaussian distribution, and then performing step S337 in which the mean value of the new gaussian distribution is initialized to

;

S337, in the pair of

Normalizing the weight values of the Gaussian distributions in the Gaussian mixture model corresponding to each pixel position to enable the sum of the new weight values of all the Gaussian distributions to be equal to 1, and then executing the step S338;

s338, judgment

Is equal to

If yes, the updating of the Gaussian mixture model is ended, otherwise, the method is right

Updating by adding 1, and then returning to execute the step S332;

for each pixel position, according to a ratio

All the Gaussian distributions in the corresponding Gaussian mixture models are sequentially arranged from small to large to obtain corresponding Gaussian distribution queues, wherein,

the current weight value representing the gaussian distribution,

representing the current variance of the gaussian distribution;

aiming at each pixel position, determining the corresponding and front-ranked pixel position in the corresponding Gaussian distribution queue

A gaussian distribution in which, among others,

taking values according to the following formula:

in the formula (I), the compound is shown in the specification,

expressed in equation in parentheses

Reach more than the preset weight threshold

Is on the positive integer variable

The value of the process is taken as the value,

indicates that is between

A positive integer between (a) and (b),

indicates that the queue number in the corresponding Gaussian distribution queue is

The current weight of the gaussian distribution;

determining a second matching condition for each pixel position

For before corresponding

Whether any Gaussian distribution in the Gaussian distributions is established or not is judged, if yes, the corresponding pixel point in the pantograph gray level image is judged to be a background pixel point, otherwise, the corresponding pixel point is judged to be a foreground pixel point, wherein,

representing the gray values of corresponding pixel points in the pantograph gray image,

is shown before corresponding

The current mean of the gaussian distribution in the gaussian distribution,

is shown before corresponding

A current variance of the Gaussian distribution in the Gaussian distribution;

and performing image binarization processing on the pantograph gray level image according to the judgment result of the pixel point type to obtain a foreground object binarization image.

5. The method for detecting intrusion of foreign matters into a pantograph according to claim 1, wherein the moving object detection processing is performed on three frames of pantograph gray-scale images with continuous acquisition time sequence by using a three-frame difference method to obtain a binary image of the moving object, and the method comprises the following steps:

respectively calculating to obtain a first difference image of a first two-frame pantograph gray image in the three-frame pantograph gray image and a second difference image of a second two-frame pantograph gray image in the three-frame pantograph gray image, wherein the gray value of each pixel point in the first difference image is the absolute value of the gray value difference of the corresponding pixel point in the first two-frame pantograph gray image, and the gray value of each pixel point in the second difference image is the absolute value of the gray value difference of the corresponding pixel point in the second two-frame pantograph gray image;

respectively carrying out image binarization processing on the first differential image and the second differential image according to the comparison result of the gray value of the pixel point and a preset threshold value to obtain corresponding differential binarization images;

and performing image logic and operation processing on the two differential binarization images obtained through the image binarization processing to obtain the moving target binarization image.

6. The pantograph foreign object intrusion detection method according to claim 1, wherein before the logical and operation processing of the images on the foreground object binarized image and the moving target binarized image, the method further comprises:

and carrying out morphological open operation processing on the foreground object binary image and/or the moving target binary image to obtain a new moving target binary image and/or a new moving target binary image for carrying out the logic and operation processing.

7. A pantograph foreign matter intrusion detection device is characterized by comprising an image acquisition module, an image positioning module, a foreground segmentation module, a motion detection module, an operation processing module and a contour detection module;

the image acquisition module is used for acquiring a video image acquired by a monitoring camera in real time, wherein the monitoring camera is arranged on the roof of the vehicle and enables the view field of the lens to cover the area where the pantograph is located;

the image positioning module is in communication connection with the image acquisition module and is used for performing ROI (region of interest) positioning processing on the video image and extracting a pantograph gray image;

the foreground segmentation module is in communication connection with the image positioning module and is used for performing foreground and background segmentation processing on the pantograph gray level image by using a Gaussian mixture model to obtain a foreground object binary image, wherein gray values of foreground pixel points in the foreground object binary image are uniform non-zero values, and gray values of background pixel points in the foreground object binary image are zero values;

the motion detection module is in communication connection with the image positioning module and is used for performing motion target detection processing on three frames of pantograph gray level images with continuous collection time sequence by using a three-frame difference method to obtain a motion target binary image, wherein the three frames of pantograph gray level images comprise the pantograph gray level images and two frames of pantograph gray level images with collection time sequences positioned in front of the pantograph gray level images, the gray level values of motion target pixel points in the motion target binary image are uniform non-zero values, and the gray level value of background pixel points in the motion target binary image is zero;

the operation processing module is respectively in communication connection with the foreground segmentation module and the motion detection module and is used for carrying out image logic and operation processing on the foreground object binary image and the motion target binary image to obtain a new binary image;

the contour detection module is in communication connection with the operation processing module and is used for performing contour detection processing on the new binary image and determining a target contour which is obtained through detection and has an enclosing area exceeding a preset area threshold value in the contour as an invading foreign body contour, wherein the target contour refers to a closed contour which is enclosed by a plurality of adjacent edge pixel points in the new binary image, and the edge pixel points refer to pixel points which have a gray value of non-zero value and at least one adjacent pixel point in eight adjacent pixel points around and have a gray value of zero value.

8. A computer device, comprising a memory, a processor and a transceiver, wherein the memory is used for storing a computer program, the transceiver is used for transmitting and receiving data, and the processor is used for reading the computer program and executing the pantograph foreign object intrusion detection method according to any one of claims 1 to 6.

9. A pantograph foreign matter intrusion detection system is characterized by comprising a monitoring camera (1), a processing device (2), an on-board host (3), a display (4) and an alarm prompting device (5), wherein the monitoring camera (1) is in communication connection with the processing device (2), the processing device (2) is in communication connection with the on-board host (3), and the on-board host (3) is in communication connection with the display (4) and the alarm prompting device (5) respectively;

the monitoring camera (1) is used for being installed on the roof of a vehicle and enabling the lens view to cover the area where the pantograph is located, and is used for transmitting the video images acquired in real time to the processing equipment (2);

the processing device (2) is used for executing the pantograph foreign object intrusion detection method according to any one of claims 1 to 6, acquiring a foreign object intrusion detection result and transmitting the video image and the foreign object intrusion detection result to the vehicle-mounted host (3);

the vehicle-mounted host (3) is used for storing the video image and the foreign object intrusion detection result, distributing and transmitting the video image and the foreign object intrusion detection result to the display (4), and sending an alarm trigger instruction to the alarm prompting device (5) when determining that a foreign object intrusion condition is met according to the foreign object intrusion detection result;

the display (4) is used for displaying the video image and the foreign object intrusion detection result;

and the alarm prompting device (5) is used for starting an alarm prompting action when the alarm triggering instruction is received.

10. A storage medium having stored thereon instructions for performing the pantograph foreign object intrusion detection method according to any one of claims 1 to 6 when the instructions are run on a computer.

Images