Disclosure of Invention
In view of the above, the embodiment of the invention provides an intelligent identification maintenance method and system for pit and groove on the road and bridge surface based on computer vision, which are used for solving the problem of how to distinguish a normal area from a pit and groove area in a road and bridge surface image, so that pit and groove segmentation is accurately carried out.
In a first aspect, an embodiment of the present invention provides a method for intelligently identifying and maintaining pits on a road and bridge surface based on computer vision, where the method includes the following steps:
Collecting road and bridge surface images with position information, and processing the road and bridge surface images to obtain gray images;
Marking any pixel point in the gray level image as a target pixel point, acquiring the abnormality degree of each target pixel point in the gray level image according to the gray level distribution characteristics of the pixel points in the gray level image, and dividing the gray level image into an initial pit area and an initial normal area according to the abnormality degree of each target pixel point in the gray level image;
Respectively establishing a filtering window with a preset size in the gray image by taking each target pixel point as a center, and acquiring a filtering median value corresponding to each target pixel point in the initial pit slot area according to gray distribution characteristics of the pixel points in the filtering window corresponding to each target pixel point in the initial pit slot area;
obtaining a smooth image corresponding to median filtering of the gray image according to the filtering median value corresponding to each target pixel point in the initial pit area and the filtering median value corresponding to each target pixel point in the initial normal area;
and acquiring pit areas in the smooth image according to the gray value difference of each pixel point in the smooth image, and maintaining the road and bridge surfaces corresponding to the pit areas according to the position information of the road and bridge surface image.
Preferably, the obtaining the degree of abnormality of each target pixel point in the gray scale image according to the gray scale distribution characteristics of the pixel points in the gray scale image includes:
establishing a first target window with a first preset size by taking the target pixel point as a center, and acquiring the complexity of the target pixel point according to the gray distribution characteristics of the pixel point in the first target window;
Establishing a second target window with a second preset size by taking the target pixel point as a center, and acquiring the local similarity of the target pixel point according to the gray level distribution characteristics of the pixel point in the second target window, wherein the second target window comprises the first target window;
and acquiring the degree of abnormality of the target pixel point according to the complexity and the local similarity of the target pixel point.
Preferably, the obtaining the complexity of the target pixel according to the gray scale distribution feature of the pixel in the first target window includes:
Obtaining a maximum gray value and a minimum gray value of pixel points in the first target window, calculating the absolute value of the difference between the maximum gray value and the minimum gray value, and recording the absolute value of the difference as a first absolute value of the difference;
Respectively obtaining the difference absolute value of the gray value of each pixel point in the first target window and the gray value of the target pixel point, obtaining a difference absolute value sequence, and obtaining the standard deviation of the difference absolute value sequence;
Forming gray values of all pixel points in the first target window into a first gray value sequence, and obtaining information entropy of the first gray value sequence;
And acquiring an addition result of the first difference absolute value, the standard deviation and the information entropy, recording the result as a first addition result, and carrying out normalization processing on the first addition result to obtain the complexity of the target pixel point in the first target window.
Preferably, the obtaining the local similarity of the target pixel point according to the gray scale distribution feature of the pixel point in the second target window includes:
Respectively acquiring pixel points spaced from the target pixel points by a preset length on a diagonal line of the second target window, marking the pixel points as neighborhood pixel points, respectively establishing a neighborhood small window with a first preset size in the second target window by taking each neighborhood pixel point as a center, respectively acquiring the complexity of the target pixel point in the second target window and the complexity of each neighborhood pixel point in the corresponding neighborhood small window;
Obtaining the absolute value of the difference between the complexity of each neighborhood pixel point in the corresponding neighborhood small window and the complexity of the target pixel point in the first target window, obtaining a sequence of absolute value differences, recording the sequence of absolute value differences as a second sequence of absolute value differences, obtaining the ratio of each absolute value difference in the second sequence of absolute value differences to the complexity of the target pixel point in the second target window, correspondingly obtaining a ratio average value, obtaining the addition result of the ratio average value and a preset value, recording the addition result as a second addition result, and normalizing the reciprocal of the second addition result to obtain the local similarity of the target pixel point.
Preferably, the obtaining the abnormality degree of the target pixel according to the complexity and the local similarity of the target pixel includes:
and obtaining an addition result of the reciprocal of the complexity of the target pixel point and the local similarity of the target pixel point, recording the addition result as a third addition result, and carrying out normalization processing on the third addition result to obtain the abnormality degree of the target pixel point.
Preferably, the dividing the gray scale image into an initial pit area and an initial normal area according to the abnormality degree of each target pixel point in the gray scale image includes:
Setting an abnormality degree threshold, acquiring all target pixel points with the abnormality degree larger than the abnormality degree threshold in the gray level image to form an initial pit area in the gray level image, and acquiring all target pixel points with the abnormality degree smaller than or equal to the abnormality degree threshold in the gray level image to form an initial normal area in the gray level image.
Preferably, the obtaining the median value of the filtering corresponding to each target pixel point in the initial pit area according to the gray distribution characteristics of the pixel points in the filtering window corresponding to each target pixel point in the initial pit area includes:
Aiming at any target pixel point in the initial pit area, marking a filter window corresponding to the any target pixel point as a first filter window, acquiring a gray value of each pixel point in the first filter window, sorting gray values of all pixel points in the first filter window, correspondingly acquiring a sorting median of the gray values in the first filter window, and marking the sorting median as a first sorting median;
Acquiring the occurrence probability of gray values of each pixel point in the first filter window, and marking the gray value with the largest occurrence probability in the first filter window as a recommended median value of the first filter window as a first recommended median value;
Setting a first difference threshold, obtaining the absolute value of the difference between the first sorting median and the first recommended median, recording the absolute value of the difference as a second absolute value of the difference, normalizing the absolute value of the second difference to obtain a first normalized value, taking the first sorting median as a filtering median corresponding to any target pixel if the first normalized value is smaller than the first difference threshold, and taking the first recommended median as the filtering median corresponding to any target pixel if the first normalized value is greater than or equal to the first difference threshold.
Preferably, the obtaining the median value of the filtering corresponding to each target pixel point in the initial normal area according to the gray distribution characteristics of the pixel points in the filtering window corresponding to each target pixel point in the initial normal area includes:
Aiming at any target pixel point in the initial normal region, marking a filter window corresponding to any target pixel point as a second filter window, acquiring a gray value of each pixel point in the second filter window, sorting gray values of all pixel points in the second filter window, correspondingly acquiring a sorting median of the gray values in the second filter window, and marking the sorting median as a second sorting median;
acquiring a filtering median value of a pixel point which belongs to an initial pit area and is closest to any pixel point in the gray level image, and marking the filtering median value as a second recommended median value;
Setting a second difference threshold, obtaining the absolute value of the difference between the second recommended median and the second sorting median, recording the absolute value as a third absolute value of the difference, normalizing the third absolute value of the difference to obtain a second normalized value, and taking the second sorting median as a filtering median corresponding to any target pixel if the second normalized value is greater than or equal to the second difference threshold;
If the second normalized value is smaller than a second difference threshold, acquiring the occurrence probability of the gray value of each pixel point in the second filter window, taking the gray value with the largest occurrence probability in the second filter window as a recommended median of the second filter window, marking the recommended median as a third recommended median, acquiring the absolute value of the difference between the second recommended median and the third recommended median as a fourth absolute value of the difference, and carrying out normalization processing on the fourth absolute value of the difference to obtain a third normalized value, wherein when the third normalized value is larger than or equal to the second difference threshold, the third recommended median is taken as the filter median corresponding to any target pixel point;
And when the third normalized value is smaller than the second difference threshold, taking the gray value with the second largest occurrence probability in the second filtering window as a new third recommended median, repeating the method for acquiring the third normalized value until the third normalized value is larger than or equal to the second difference threshold, and taking the new third recommended median as the filtering median corresponding to any target pixel point.
Preferably, the obtaining the pit area in the smooth image according to the gray value difference of each pixel point in the smooth image includes:
and acquiring a gray value of each pixel point in the smooth image, respectively setting a first gray value threshold value and a second gray value threshold value, wherein the first gray value threshold value is smaller than the second gray value threshold value, and acquiring all pixel points of which the gray values are smaller than the first gray value threshold value or larger than the second gray value threshold value in the smooth image to form a pit area in the smooth image.
In a second aspect, an embodiment of the present invention further provides a system for intelligently identifying and maintaining a pit on a road and bridge surface based on computer vision, where the system includes a memory and a processor, and the processor executes a computer program stored in the memory, so as to implement the method for intelligently identifying and maintaining a pit on a road and bridge surface based on computer vision according to the first aspect.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
The method comprises the steps of collecting road and bridge surface images with position information, processing the road and bridge surface images to obtain gray images, recording any pixel point in the gray images as a target pixel point, obtaining the abnormality degree of each target pixel point in the gray images according to the gray distribution characteristics of the pixel point in the gray images, dividing the gray images into an initial pit area and an initial normal area according to the abnormality degree of each target pixel point in the gray images, taking each target pixel point as a center, respectively establishing a filter window with a preset size in the gray images, obtaining a filter median value corresponding to each target pixel point in the initial pit area according to the gray distribution characteristics of the pixel point in a filter window corresponding to each target pixel point in the initial pit area, obtaining the filter median value corresponding to each target pixel point in the filter window corresponding to each target pixel point in the initial normal area, obtaining the filter median value corresponding to each target pixel point in the initial normal area according to the gray distribution characteristics of the pixel point in the initial normal area, smoothing the filter median value corresponding to the image in the pit area, and smoothing the image corresponding to the image of the smooth image corresponding to the surface of each target pixel point in the initial pit area, and obtaining the image. According to the method, firstly, the gray image is divided into an initial pit area and an initial normal area according to the abnormal degree of each target pixel point in the gray image, the filtering median value corresponding to each pixel point is adaptively obtained in different areas, and the median filtering processing is carried out on the gray image to obtain a corresponding smooth image, so that the salt and pepper noise generated by image segmentation can be effectively removed, spots similar to the salt and pepper noise in the good smooth image can be effectively removed, the contrast between the normal area and the pit area is more obvious, and the pit segmentation is accurately carried out on the road and bridge surface image containing the pit area.
Detailed Description
Embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, are described in detail below. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present disclosure and are not to be construed as limiting the present disclosure.
It should be noted that the terms "first," "second," and the like in the description of the present disclosure and the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with aspects of the present disclosure.
In order to illustrate the technical scheme of the invention, the following description is made by specific examples.
Referring to fig. 1, a method flowchart of a method for intelligently identifying and maintaining pits on a road and bridge surface based on computer vision according to an embodiment of the present invention is shown in fig. 1, where the method may include:
Step S101, collecting road and bridge surface images with position information, and processing the road and bridge surface images to obtain gray images.
The vehicle-mounted road surface shooting system is a road and bridge surface image acquisition system capable of shooting road and bridge surface images in real time, the road and bridge surface images are shot in real time in the driving process through a vehicle-mounted camera, satellite signals are received through a GPS receiver, real-time geographic position information of a vehicle is obtained, and the geographic position information is associated with the shot road and bridge surface images to obtain the road and bridge surface images with position information.
The road and bridge surface image with the position information is acquired by using the vehicle-mounted road and bridge surface shooting system, the road and bridge surface image is read by using OpenCV and converted into a gray image, the gray image of the road and bridge surface is obtained, as shown in fig. 2, and gray values of each pixel point in the gray image of the road and bridge surface are extracted by using Numpy, wherein OpenCV and Numpy belong to the prior art, and are not repeated here.
Step S102, marking any pixel point in the gray level image as a target pixel point, acquiring the abnormal degree of each target pixel point in the gray level image according to the gray level distribution characteristics of the pixel points in the gray level image, and dividing the gray level image into an initial pit area and an initial normal area according to the abnormal degree of each target pixel point in the gray level image.
In order to remove spots similar to salt and pepper noise which appear when the road and bridge deck image is subjected to pit segmentation, median filtering processing is needed for the gray image, but median filtering selects a gray value of a target pixel point replaced by a median point in a window, and normal areas cannot be well distinguished from pit areas, so that the pit segmentation is difficult to carry out, therefore, according to the gray distribution characteristics of the pixel points in the gray image, the gray image is required to be divided into an initial pit area and an initial normal area, and then median filtering processing is respectively carried out on the initial pit area and the initial normal area, so that the pit area and the normal area with obvious contrast are obtained.
The pit part (the pit part comprises an inner area of the pit and an edge area of the pit) is a block-shaped recess formed by the influence of the outside on the road surface, the complexity of the pit part is lower than that of the background part, namely the normal road surface part, namely the gray scale range, the gray scale change and the gray scale non-uniformity of the pit part are smaller than those of the normal road surface part, in addition, the pit part is generally a block-shaped area, namely the distribution of pixels with similar complexity in the pit part is concentrated and the local similarity is higher, so that the abnormal degree of each pixel can be obtained through the complexity and the local similarity of each pixel in the gray scale image, and the gray scale image is distinguished through the abnormal degree of each pixel in the gray scale image, so that the initial pit area and the initial normal area are obtained.
Any pixel point in the gray level image is marked as a target pixel point, and the abnormal degree of the target pixel point in the gray level image is obtained through the complexity and the local similarity of each pixel point in the gray level image, wherein the method comprises the following steps:
(1) And establishing a first target window with a first preset size by taking the target pixel point as a center, and acquiring the complexity of the target pixel point according to the gray distribution characteristics of the pixel point in the first target window.
Specifically, a maximum gray value and a minimum gray value of a pixel point in the first target window are obtained, the absolute value of the difference between the maximum gray value and the minimum gray value is calculated, and the absolute value of the difference is recorded as a first absolute value of the difference;
Respectively obtaining the difference absolute value of the gray value of each pixel point in the first target window and the gray value of the target pixel point, obtaining a difference absolute value sequence, and obtaining the standard deviation of the difference absolute value sequence;
Forming gray values of all pixel points in the first target window into a first gray value sequence, and obtaining information entropy of the first gray value sequence;
And acquiring an addition result of the first difference absolute value, the standard deviation and the information entropy, recording the result as a first addition result, and carrying out normalization processing on the first addition result to obtain the complexity of the target pixel point in the first target window.
In one embodiment, taking a target pixel point a as an example, taking the target pixel point a as a center, establishing a first target window with n×n, where n is3, where the value is not limited, and the method can be set according to a specific implementation scene, and first obtaining a maximum gray value and a minimum gray value of the pixel points in the first target window, then respectively obtaining the difference absolute value of the gray value of each pixel point in the first target window and the gray value of the target pixel point a to form a difference absolute value sequence, obtaining a standard deviation of the difference absolute value sequence, then obtaining the gray values of all the pixel points in the first target window, forming the gray values of all the pixel points in the first target window into a first gray value sequence, obtaining the information entropy of the first gray value sequence, where the information entropy is not described in detail, and finally calculating the complexity of the target pixel point a in the first target window according to the maximum gray value and the minimum gray value of the pixel points in the first target window and the standard difference of the difference absolute value sequence, and the information of the first gray value sequence.
D is the complexity of the target pixel point A in the first target window; the maximum gray value of the pixel point in the first target window is obtained; The minimum gray value of the pixel point in the first target window is obtained; The method is characterized by comprising the steps of obtaining a standard deviation of an absolute value sequence of difference values, obtaining information entropy of a first gray value sequence by H, obtaining a normalization function by norm (), and obtaining absolute value symbols by I.
The smaller the difference between the maximum gray value and the minimum gray value of the pixel points in the first target window, the smaller the range of the gray values in the first target window, the smaller the complexity of the target pixel point A in the first target window, the smaller the standard deviation of the absolute value sequence of the difference value, the smaller the data fluctuation of the absolute value sequence of the difference value, namely the smaller the fluctuation of the gray value difference value in the first target window, the smaller the complexity of the target pixel point A in the first target window, the smaller the information entropy of the first gray value sequence, the simpler the distribution of the gray values in the first target window, namely the lower the gray non-uniformity, and the smaller the complexity of the target pixel point A in the first target window.
(2) And establishing a second target window with a second preset size by taking the target pixel point as the center, and acquiring the local similarity of the target pixel point according to the gray level distribution characteristics of the pixel point in the second target window, wherein the second target window comprises the first target window.
Specifically, on a diagonal line of the second target window, respectively acquiring pixel points spaced from the target pixel points by a preset length, marking the pixel points as neighborhood pixel points, respectively establishing a neighborhood small window with a first preset size in the second target window by taking each neighborhood pixel point as a center, respectively acquiring the complexity of the target pixel point in the second target window and the complexity of each neighborhood pixel point in the corresponding neighborhood small window;
Obtaining the absolute value of the difference between the complexity of each neighborhood pixel point in the corresponding neighborhood small window and the complexity of the target pixel point in the first target window, obtaining a sequence of absolute value differences, recording the sequence of absolute value differences as a second sequence of absolute value differences, obtaining the ratio of each absolute value difference in the second sequence of absolute value differences to the complexity of the target pixel point in the second target window, correspondingly obtaining a ratio average value, obtaining the addition result of the ratio average value and a preset value, recording the addition result as a second addition result, and normalizing the reciprocal of the second addition result to obtain the local similarity of the target pixel point.
In an embodiment, taking the target pixel point a as an example, taking the target pixel point a as the center, establishing a second target window of (2n+1) × (2n+1), where the value of n is 3, that is, the size of the second target window is 7×7, where the second target window is not limited, and the second target window may be set according to a specific implementation scenario, where the second target window is separated from the target pixel point a by two diagonal pixel points of the second target windowThe method comprises the steps that pixel points (namely 1 pixel point is spaced) of pixel points are marked as neighborhood pixel points, the pixel points spaced from a target pixel point A by 1 pixel point are marked as neighborhood pixel points, 4 neighborhood pixel points are obtained, 3×3 neighborhood windows are respectively built in a second target window by taking each neighborhood pixel point as the center, as shown in fig. 3, the schematic diagram of the second target window is shown, the target pixel point A is the center pixel point of the second target window, the pixel point 1, the pixel point 2, the pixel point 3 and the pixel point 4 are neighborhood pixel points spaced from the target pixel point A by 1 pixel point, the light gray area is a3×3 neighborhood small window which is built by taking each neighborhood pixel point as the center, the complexity of each neighborhood pixel point in the first target window is respectively obtained according to the complexity obtaining method of the target pixel point A in the first target window, the complexity of the target pixel point A in the second target window is calculated according to the complexity of each neighborhood pixel point in the first target window, and the complexity of the target pixel point A in the target window is calculated according to the complexity of the neighborhood pixel point A in the first target window, and the target pixel point A in the target window is similar to the target pixel point A:
Wherein,D is the complexity of the target pixel point A in the first target window; The method comprises the steps of obtaining complexity of a (b) th neighborhood pixel point in a neighborhood small window, obtaining a sequence number of the neighborhood pixel point in a second target window, obtaining the number of the neighborhood pixel point in the second target window, obtaining the complexity of a target pixel point A in the second target window, obtaining a preset value, obtaining a normalization function by norm (), and obtaining absolute value symbols by using the absolute value symbol.
It should be noted that, the preset value c is a constant 2, which is used for preventing the formula calculation error caused by the fact that the denominator value is 0, and the smaller the difference between the complexity of each neighborhood pixel point in the neighborhood small window and the complexity of the target pixel point A in the first target window, the more similar the distribution of the neighborhood small window in the second target window and the gray value in the first target window, the larger the local similarity of the target pixel point A.
(3) And acquiring the degree of abnormality of the target pixel point according to the complexity and the local similarity of the target pixel point.
Specifically, an addition result of the reciprocal of the complexity of the target pixel point and the local similarity of the target pixel point is obtained and is recorded as a third addition result, and normalization processing is performed on the third addition result to obtain the degree of abnormality of the target pixel point.
In one embodiment, taking the target pixel point a as an example, the anomaly degree of the target pixel point a is calculated:
Wherein,D is the complexity of the target pixel point A in the first target window; is the local similarity of the target pixel point A, and norm () is a normalization function.
The complexity of the target pixel point a in the first target window is lower, the target pixel point a accords with the pixel point of the pit area, the abnormal degree of the target pixel point a is higher, the local similarity of the target pixel point a is higher, and the abnormal degree of the target pixel point a is higher.
According to the method for acquiring the abnormality degree of the target pixel point A, the abnormality degree of each target pixel point in the gray level image is acquired. After the abnormal degree of each target pixel point in the gray level image is obtained, the gray level image can be distinguished through the abnormal degree of each target pixel point in the gray level image, and an initial pit area and an initial normal area are obtained.
Specifically, an abnormality degree threshold is set, all target pixel points with the abnormality degree larger than the abnormality degree threshold in the gray level image are obtained to form an initial pit area in the gray level image, and all target pixel points with the abnormality degree smaller than or equal to the abnormality degree threshold in the gray level image are obtained to form an initial normal area in the gray level image.
In an embodiment, the threshold value of the degree of abnormality is set to be 0.75, which is not limited herein, all target pixels with the degree of abnormality greater than 0.75 in the gray scale image can be obtained according to the specific implementation scene setting, all target pixels with the degree of abnormality greater than 0.75 in the gray scale image are formed into the initial pit area in the gray scale image, all target pixels with the degree of abnormality less than or equal to 0.75 in the gray scale image are obtained, and all target pixels with the degree of abnormality less than or equal to 0.75 in the gray scale image are formed into the initial normal area in the gray scale image.
Thus, an initial pit area and an initial normal area in the gray level image are obtained.
Step S103, taking each target pixel point as a center, respectively establishing a filtering window with a preset size in the gray image, acquiring a filtering median value corresponding to each target pixel point in the initial pit slot area according to the gray distribution characteristics of the pixel points in the filtering window corresponding to each target pixel point in the initial pit slot area, and acquiring the filtering median value corresponding to each target pixel point in the initial normal area according to the gray distribution characteristics of the pixel points in the filtering window corresponding to each target pixel point in the initial normal area.
After an initial pit area and an initial normal area in the gray image are obtained, respectively performing self-adaptive median filtering treatment on the initial pit area and the initial normal area, so that a smooth image corresponding to median filtering on the gray image is obtained, the contrast between the pit area and the normal area in the gray image is enhanced, and pit segmentation can be accurately performed.
When median filtering is performed on a gray image, a filtering window needs to be established first, and 5×5 filtering windows are respectively established by taking each target pixel point in the gray image as a center, so that the method is not limited herein, and can be set according to specific implementation scenes.
When median filtering is performed on the initial pit area, because some interference spots exist in the initial pit area, but the quantity of the interference spots is small, in order to prevent the pixel points from being used as a filtering median, the sorting median of each filtering window needs to be obtained, the occurrence probability of gray values of all the pixel points in each filtering window is counted, and the filtering median corresponding to each target pixel point in the initial pit area is obtained according to the difference between the gray value with the largest occurrence probability in each filtering window and the sorting median of the corresponding filtering window.
The method for acquiring the filtering median value corresponding to each target pixel point in the initial pit area according to the difference between the gray value with the largest occurrence probability in each filtering window and the sequencing median value of the corresponding filtering window comprises the following steps:
Aiming at any target pixel point in the initial pit area, marking a filter window corresponding to the any target pixel point as a first filter window, acquiring a gray value of each pixel point in the first filter window, sorting gray values of all pixel points in the first filter window, correspondingly acquiring a sorting median of the gray values in the first filter window, and marking the sorting median as a first sorting median;
Acquiring the occurrence probability of gray values of each pixel point in the first filter window, and marking the gray value with the largest occurrence probability in the first filter window as a recommended median value of the first filter window as a first recommended median value;
Setting a first difference threshold, obtaining the absolute value of the difference between the first sorting median and the first recommended median, recording the absolute value of the difference as a second absolute value of the difference, normalizing the absolute value of the second difference to obtain a first normalized value, taking the first sorting median as a filtering median corresponding to any target pixel if the first normalized value is smaller than the first difference threshold, and taking the first recommended median as the filtering median corresponding to any target pixel if the first normalized value is greater than or equal to the first difference threshold.
In one embodiment, taking a target pixel point I in an initial pit area as an example, taking the target pixel point I as a center, establishing a 5×5 first filter window, firstly acquiring gray values of all pixel points in the first filter window, sorting the gray values of all pixel points in the first filter window, selecting a gray value median value of the pixel points in the first filter window as a sorting median value of the first filter window, marking the sorting median value of the first filter window as a first sorting median value, then acquiring the probability of occurrence of the gray value of each pixel point in the first filter window, marking the gray value with the largest sorting median value occurrence probability of the first filter window as a recommendation median value of the first filter window, marking the recommendation median value of the first filter window as a first recommendation median value, finally setting a first difference threshold value as 0.6, and calculating a filter median value corresponding to the target pixel point I according to the first sorting median value, the difference of the first recommendation median value and the first recommendation difference threshold value without limitation, according to specific implementation scene setting:
wherein Lzi is a filtering median corresponding to the target pixel point I, zi is a first sorting median; The value is a first recommended median value, g is a first difference threshold value, norm () is a normalization function, and || is an absolute value sign.
The larger the difference between the first median of sorting and the first median of recommendation is, the more likely the first median of sorting is an interference pixel point, so the first median of recommendation is used as a filter median corresponding to the target pixel point I, and the smaller the difference between the first median of sorting and the first median of recommendation is, the less likely the first median of sorting is an interference pixel point, so the first median of sorting is used as a filter median corresponding to the target pixel point I.
And acquiring the filter median value corresponding to each target pixel point in the initial pit area according to the acquisition method of the filter median value corresponding to the target pixel point I.
When the median filtering is performed on the initial normal region, in order to prevent the gray value similar to the gray value of the target pixel point in the initial pit region from being used as the filtering median, the filtering median of the target pixel point in the initial normal region is made to be different from the filtering median of the target pixel point in the initial pit region as far as possible, so that the filtering median corresponding to each target pixel point in the initial normal region needs to be adjusted by analyzing the difference value of the sorting median of each filtering window in the initial normal region and the filtering median corresponding to the target pixel point in the initial pit region closest to the sorting median. The method for acquiring the filter median value corresponding to each target pixel point in the initial normal region comprises the following steps:
(1) Aiming at any target pixel point in the initial normal region, marking a filter window corresponding to any target pixel point as a second filter window, acquiring a gray value of each pixel point in the second filter window, sorting gray values of all pixel points in the second filter window, correspondingly acquiring a sorting median of the gray values in the second filter window, and marking the sorting median as a second sorting median;
acquiring a filtering median value of a pixel point which belongs to an initial pit area and is closest to any pixel point in the gray level image, and marking the filtering median value as a second recommended median value;
Setting a second difference threshold, obtaining the absolute value of the difference between the second recommended median and the second sorting median, recording the absolute value as a third absolute value of the difference, normalizing the third absolute value of the difference to obtain a second normalized value, and taking the second sorting median as a filtering median corresponding to any target pixel if the second normalized value is greater than or equal to the second difference threshold;
If the second normalized value is smaller than a second difference threshold, acquiring the occurrence probability of the gray value of each pixel point in the second filter window, taking the gray value with the largest occurrence probability in the second filter window as a recommended median of the second filter window, marking the recommended median as a third recommended median, acquiring the absolute value of the difference between the second recommended median and the third recommended median as a fourth absolute value of the difference, carrying out normalization processing on the fourth absolute value of the difference to obtain a third normalized value, and taking the third recommended median as the filter median corresponding to any target pixel point when the third normalized value is larger than or equal to the second difference threshold.
In an embodiment, taking a target pixel point J in an initial normal area as an example, taking the target pixel point J as a center, establishing a5×5 second filter window, firstly acquiring gray values of all pixel points in the second filter window, sorting the gray values of all pixel points in the second filter window, selecting a median value of the gray values of the pixel points in the second filter window as a sorting median value of the second filter window, and marking the sorting median value of the second filter window as a second sorting median value; selecting the pixel point belonging to the initial pit area closest to the target pixel point J in the gray image, acquiring a filtering median corresponding to the pixel point belonging to the initial pit area closest to the target pixel point J in the gray image, marking the filtering median corresponding to the pixel point belonging to the initial pit area closest to the target pixel point J in the gray image as a second recommended median, acquiring the probability of occurrence of the gray value of each pixel point in a second filtering window, marking the gray value with the largest occurrence probability in the second filtering window as the recommended median of the second filtering window, marking the recommended median of the second filtering window as a third recommended median, and finally setting a second difference threshold as 0.6, wherein the filtering median corresponding to the target pixel point J can be calculated according to specific implementation scene settings according to the second sequencing median, the second recommended median, the third recommended median and the second difference threshold:
lzj is a filtering median corresponding to the target pixel point J, wherein Zj is a second sorting median; median value for the third recommendation; is the second recommended median value, f is the second difference threshold, norm () is the normalization function, and || is the absolute sign.
The larger the difference between the second recommended median and the second sorting median is, the more dissimilar the filter median corresponding to the target pixel point in the initial normal region is to the filter median corresponding to the target pixel point in the initial pit region, so the second sorting median is used as the filter median corresponding to the target pixel point J, and the smaller the difference between the second recommended median and the second sorting median is, the more similar the filter median corresponding to the target pixel point in the initial normal region is to the filter median corresponding to the target pixel point in the initial pit region, so the third recommended median with large difference from the second recommended median is used as the filter median corresponding to the target pixel point J.
(2) And when the third normalized value is smaller than the second difference threshold, taking the gray value with the second largest occurrence probability in the second filtering window as a new third recommended median, repeating the method for acquiring the third normalized value until the third normalized value is larger than or equal to the second difference threshold, and taking the new third recommended median as the filtering median corresponding to any target pixel point.
In an embodiment, when the third normalized value is smaller than the second difference threshold value 0.6, the gray value with the second largest occurrence probability in the second filter window is used as a new third recommended median value, the new third normalized value is obtained according to the third normalized value obtaining method, if the new third normalized value is greater than or equal to the second difference threshold value 0.6, the new third recommended median value (i.e., the gray value with the second largest occurrence probability in the second filter window) is used as the filtered median value corresponding to the target pixel point J, if the new third normalized value is smaller than the second difference threshold value 0.6, the gray value with the third largest occurrence probability in the second filter window is used as the new third recommended median value, the new third normalized value is continuously obtained according to the third normalized value obtaining method, and so on until the new third normalized value is greater than or equal to the second difference threshold value 0.6, and the new third recommended value corresponding to the new third normalized value is used as the filtered median value corresponding to the target pixel point J.
And acquiring the filter median value corresponding to each target pixel point in the initial normal region according to the acquisition method of the filter median value corresponding to the target pixel point J.
And obtaining a filtering median value corresponding to each target pixel point in the initial pit area and a filtering median value corresponding to each target pixel point in the initial normal area.
And step S104, obtaining a smooth image corresponding to median filtering of the gray image according to the filtering median value corresponding to each target pixel point in the initial pit area and the filtering median value corresponding to each target pixel point in the initial normal area.
According to the filter median value corresponding to each target pixel point in the initial pit area and the filter median value corresponding to each target pixel point in the initial normal area, the filter median value corresponding to each target pixel point in the gray image is obtained, the filter median value corresponding to each target pixel point in the gray image is respectively substituted for the gray value corresponding to the target pixel point in the gray image, and a smooth image with obvious pit area and normal area contrast is obtained.
And obtaining a smooth image corresponding to the gray level image after median filtering.
Step S105, according to the gray value difference of each pixel point in the smooth image, acquiring a pit area in the smooth image, and according to the position information of the road and bridge surface image, maintaining the road and bridge surface corresponding to the pit area.
In the smooth image corresponding to the median filtering of the gray level image, the comparison between the pit area and the normal area is more obvious, and the pit area in the smooth image can be obtained according to the gray level value difference of each pixel point in the smooth image.
Specifically, the gray value of each pixel point in the smooth image is obtained, a first gray value threshold value and a second gray value threshold value are respectively set, the first gray value threshold value is smaller than the second gray value threshold value, and all pixel points with gray values smaller than the first gray value threshold value or larger than the second gray value threshold value in the smooth image are obtained to form a pit area in the smooth image.
In one embodiment, the gray value of each pixel in the smooth image and the maximum gray value and the minimum gray value of the pixels in the smooth image are obtained, and the gray value in the smooth image is higher and the shallow pit is a small pit on the surface layer of the asphalt pavement, the gray value in the smooth image is lower, and the first gray value threshold is set as the first gray value thresholdSetting a second gray value threshold as the minimum gray value of the pixel points in the multiple smooth imageAnd (3) marking the maximum gray value of the pixel points in the multiple smooth image, namely marking the pixel points with the gray value smaller than the first gray value threshold value in the smooth image as the pixel points of the shallow pit area, marking the pixel points with the gray value larger than the second gray value threshold value in the smooth image as the pixel points of the deep pit area, and obtaining all the pixel points of the shallow pit area and all the pixel points of the deep pit area in the smooth image to form the pit area in the smooth image.
After the pit area in the smooth image is obtained, the position information of the road and bridge surface image is matched with the actual road map because the road and bridge surface image is provided with the position information, the actual road and bridge section where the pit area is located is obtained, and the pit in the actual road and bridge section is maintained according to the actual road and bridge section where the pit area is located.
The vehicle-mounted road surface shooting system can acquire road and bridge surface images in real time, analyze each road and bridge surface image according to the acquisition method of the pit area, mark the road and bridge surface image containing the pit area, acquire the actual road and bridge section of each road and bridge surface image containing the pit area, and maintain and repair each pit according to the actual road and bridge section of each road and bridge surface image containing the pit area, thereby avoiding the continuous expansion of the pit and reducing the influence of the pit on the safety of vehicles and pedestrians.
In summary, the embodiment of the invention collects road and bridge surface images with position information, processes the road and bridge surface images to obtain gray images, marks any pixel point in the gray images as a target pixel point, obtains the abnormality degree of each target pixel point in the gray images according to the gray distribution characteristics of the pixel points in the gray images, divides the gray images into an initial pit area and an initial normal area according to the abnormality degree of each target pixel point in the gray images, respectively establishes a filter window with a preset size in the gray images by taking each target pixel point as a center, obtains a filter median value corresponding to each target pixel point in the initial pit area according to the gray distribution characteristics of the pixel points in the filter window corresponding to each target pixel point in the initial pit area, obtains the filter median value corresponding to each target pixel point in the initial normal area according to the gray distribution characteristics of the pixel points in the filter window corresponding to each target pixel point in the initial normal area, smoothes the image corresponding to the median value in the image according to the initial pit area, and smoothes the image corresponding to the median value of each target pixel point in the initial pit area, and smoothes the image corresponding to the peak value in the pit area. According to the embodiment of the invention, firstly, the gray image is divided into an initial pit area and an initial normal area according to the abnormality degree of each target pixel point in the gray image, the filtering median value corresponding to each pixel point is adaptively obtained in different areas, and the median filtering processing is carried out on the gray image to obtain a corresponding smooth image, so that the salt and pepper noise generated by image segmentation can be effectively removed, spots similar to the salt and pepper noise in the good smooth image can be effectively removed, the contrast between the normal area and the pit area is more obvious, and the pit segmentation is accurately carried out on the road and bridge surface image containing the pit area.
Based on the same inventive concept as the method, the embodiment of the invention also provides a road and bridge surface pit intelligent identification maintenance system based on computer vision, which comprises a memory and a processor, wherein the processor executes a computer program stored in the memory so as to realize the road and bridge surface pit intelligent identification maintenance method based on computer vision.
The foregoing embodiments are merely for illustrating the technical solution of the present invention, but not for limiting the same, and although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solution described in the foregoing embodiments may be modified or substituted for some of the technical features thereof, and that these modifications or substitutions should not depart from the spirit and scope of the technical solution of the embodiments of the present invention and should be included in the protection scope of the present invention.