技术领域technical field
本发明涉及图像处理领域，尤其涉及一种去除前景检测结果阴影的方法及装置。The invention relates to the field of image processing, in particular to a method and device for removing the shadow of a foreground detection result.
背景技术Background technique
由于阴影具有与运动物体相同的运动特征，通常被误检测为前景，视频图像中存在的阴影是影响运动目标检测效果的关键因素之一，对阴影进行检测和消除已成为运动检测中的重要研究内容。如果阴影和运动物体融合，将影响目标的几何特征；如果阴影与运动目标相分离，则容易被误检测为新的目标。这些误检测对高层的运动目标分类、跟踪以及行为分析等都造成很大的影响，因此在运动目标检测过程中对阴影进行消除具有重要意义。目前，从前景目标中消除阴影还没有简单有效的办法，是图像领域难解问题之一。Since the shadow has the same motion characteristics as the moving object, it is usually misdetected as the foreground. The shadow in the video image is one of the key factors affecting the detection effect of the moving object. The detection and elimination of the shadow has become an important research in motion detection. content. If the shadow merges with the moving object, it will affect the geometric characteristics of the target; if the shadow is separated from the moving object, it is easy to be misdetected as a new target. These false detections have a great impact on the classification, tracking and behavior analysis of high-level moving objects, so it is of great significance to eliminate shadows in the process of moving object detection. At present, there is no simple and effective way to remove shadows from foreground objects, which is one of the difficult problems in the image field.
目前大多数的阴影检测和消除方法都是基于阴影本身的特征来开发的，例如阴影的灰度特征、色度特征、纹理统计特征等等。现有技术中，进行阴影去除的方法包括：将图像进行灰度化处理，再进行前景检测处理，以剔除背景，对二值化的前景检测图分别进行水平和垂直投影，分别得到水平投影直方图及垂直投影直方图，并设定门限值，将小于某一门限值的部分判定为阴影并去除。Most of the current shadow detection and elimination methods are developed based on the characteristics of the shadow itself, such as grayscale features, chromaticity features, texture statistical features, etc. of shadows. In the prior art, the method for removing shadows includes: performing grayscale processing on the image, and then performing foreground detection processing to eliminate the background, respectively performing horizontal and vertical projection on the binarized foreground detection map to obtain horizontal projection histograms respectively Figure and vertical projection histogram, and set the threshold value, determine the part smaller than a certain threshold value as shadow and remove it.
另外也存在基于像素点进行阴影去除的方法，即根据阴影的亮度等特点设置阈值，当像素点超过对应的阈值时，将像素点判定为阴影进行去除。In addition, there is also a method of shadow removal based on pixels, that is, thresholds are set according to characteristics such as shadow brightness, and when a pixel exceeds the corresponding threshold, the pixel is judged as a shadow and removed.
由于各场景不同，门限值或阈值的取值范围也就不同，针对不同的场景，不存在普遍适用的门限值或阈值，从而导致容易造成阴影的误判。因此无论基于像素点的方法还是简单的投影方法，虽然在某些阈值下能有效检测出阴影的部分，但同时容易将车窗、行人或机动车、非机动车等与阴影特征相似的局部区域误检测为阴影，一并去除，这样容易造成前景空洞、分割错误等问题。Since each scene is different, the value range of the threshold value or the threshold value is also different. For different scenes, there is no generally applicable threshold value or threshold value, which may easily cause misjudgment of shadows. Therefore, regardless of the pixel-based method or the simple projection method, although the shadow part can be effectively detected under certain thresholds, it is easy to identify local areas similar to shadow features such as car windows, pedestrians, motor vehicles, and non-motor vehicles. If it is falsely detected as a shadow, it will be removed together, which will easily cause problems such as foreground holes and segmentation errors.
因此，现有技术存在将非阴影区域误检为阴影区域的问题。Therefore, the prior art has the problem of falsely detecting non-shaded areas as shadowed areas.
发明内容Contents of the invention
为解决现有技术阴影区域误检率高的问题，本发明提供了一种方法，减少阴影区域被误检的概率。In order to solve the problem of high false detection rate of shadow areas in the prior art, the present invention provides a method to reduce the probability of false detection of shadow areas.
一种去除前景检测结果阴影的方法，包括：由输入的原始图像得到灰度图及前景检测图，并获取图像前景中的阴影候选区，还包括：A method for removing the shadow of a foreground detection result, comprising: obtaining a grayscale image and a foreground detection image from an input original image, and obtaining a shadow candidate area in the foreground of the image, and further comprising:
将所述灰度图去除背景得到前景灰度图，根据所述前景灰度图，在水平方向上计算各像素行的纹理统计特征值，在垂直方向上计算各像素列的纹理统计特征值；Remove the background from the grayscale image to obtain a foreground grayscale image, calculate the texture statistical feature value of each pixel row in the horizontal direction, and calculate the texture statistical feature value of each pixel column in the vertical direction according to the foreground grayscale image;
利用像素行的纹理统计特征值找到水平方向上的分割位置，且利用像素列的纹理统计特征值找到垂直方向上的分割位置，其中分割位置为使运动目标与阴影差异最大的像素位置；Using the texture statistical feature value of the pixel row to find the segmentation position in the horizontal direction, and using the texture statistical feature value of the pixel column to find the segmentation position in the vertical direction, wherein the segmentation position is the pixel position that makes the difference between the moving object and the shadow the largest;
分别计算水平及垂直方向上的分割位置两侧的阴影候选区占所对应前景部分的比例，并从分割位置的两侧中选择比例较大的一侧作为阴影去除。Calculate the proportion of the shadow candidate area on both sides of the segmentation position in the horizontal and vertical directions to the corresponding foreground part, and select the side with a larger proportion from the two sides of the segmentation position as the shadow removal.
从基于纹理统计特征进行的投影中找到使运动目标与阴影差异最大的分割点，将分割结果结合阴影候选区域，进一步确定物体阴影的检测范围，从而避免一些类似阴影的区域被当作阴影而被去除，提高了阴影检测的准确率。另外这种方法本身已对运动物体和阴影部分进行了准确分割，因此在选择候选阴影区域时，针对不同场景不需要设置严格的门限值，可以对大部分场景设置普遍适用的统一门限值，从而提高了方法的普遍适用性。在水平方向上所述像素位置为像素行位置，在垂直方向上所述像素位置为像素列的位置，在灰度图中包含有前景纹理统计特征，但同时也包含背景的纹理统计特征，而背景的纹理统计特征在进行投影时为无效信息，会对后面的分割过程造成干扰，通过将灰度图与前景检测图相乘，将灰度图去除背景，从所得到的前景灰度图中直接可获取前景的纹理统计特征，避免背景的干扰，从而提高分割准确率。将所述前景检测图与所述灰度图相乘，即可将灰度图中的背景去除。From the projection based on texture statistical features, find the segmentation point that makes the difference between the moving object and the shadow the largest, combine the segmentation results with the shadow candidate area, and further determine the detection range of the object shadow, so as to avoid some shadow-like areas being regarded as shadows. Removed to improve the accuracy of shadow detection. In addition, this method itself has accurately segmented moving objects and shadow parts, so when selecting candidate shadow areas, it is not necessary to set strict threshold values for different scenes, and a uniform threshold value that is generally applicable to most scenes can be set , thus improving the general applicability of the method. The pixel position in the horizontal direction is the position of the pixel row, and the pixel position in the vertical direction is the position of the pixel column. The grayscale image contains the statistical characteristics of the foreground texture, but also contains the statistical characteristics of the texture of the background. The texture statistical features of the background are invalid information during projection, which will interfere with the subsequent segmentation process. By multiplying the grayscale image with the foreground detection image, the grayscale image is removed from the background, and the obtained foreground grayscale image is The texture statistical features of the foreground can be obtained directly, and the interference of the background can be avoided, thereby improving the segmentation accuracy. The background in the grayscale image can be removed by multiplying the foreground detection image with the grayscale image.
其中获取阴影候选区的图像可能由前景检测图得到也可能由原始图像得到，例如通过将前景检测图进行水平和垂直方向上投影的方式得到阴影候选区，则获取的阴影候选区在前景检测图中；利用色度、亮度结合饱和度的方式得到阴影候选区时，所获取的阴影候选区在原始图像中，或者，根据后续要进行的处理，阴影候选区也可能在灰度图中。The image of the shadow candidate area obtained may be obtained from the foreground detection map or the original image. For example, the shadow candidate area is obtained by projecting the foreground detection map in the horizontal and vertical directions, and the obtained shadow candidate area is in the foreground detection map. Medium; when the shadow candidate area is obtained by combining chroma, brightness and saturation, the acquired shadow candidate area is in the original image, or, according to the subsequent processing, the shadow candidate area may also be in the grayscale image.
由于划分的位置并不一定是中心位置，因此，如果仅仅是判断各部分被判断为阴影的像素行或像素列出现频次，则可能造成被划分出来面积较大的部分被判断为阴影的像素行或像素列或是像素点比划分出来面积较小的部分出现频次高，从而导致总是将划分出的面积较大的部分被作为阴影区剔除。为了防止这种现象出现，通过将阴影候选区在所划分部分中的前景部分所占比例来判断阴影区出现的可能性大小。不同的计算方法得到的阴影候选区不同，例如通过投影方法得到的阴影候选区由像素行或像素列组成，由色度、亮度结合饱和度得到的阴影候选区由像素点组成。如果阴影候选区由像素行或像素列组成，则计算该阴影候选区在所划分出的部分中对应的被判定为前景的像素行或像素列的比例；如果阴影候选区由像素点组成，则计算被判定为阴影候选区的像素点个数在所划分出的部分中占前景像素点个数的比例，将计算比例较高的整个部分作为阴影区剔除。Since the divided position is not necessarily the central position, if only the frequency of occurrence of the pixel row or pixel column judged to be shadowed in each part is judged, it may cause the divided part with a larger area to be judged as the shadowed pixel row Either the pixel row or the pixel point appears more frequently than the divided part with a smaller area, so that the divided part with a larger area is always removed as a shadow area. In order to prevent this phenomenon from appearing, the possibility of the shadow area is judged by the proportion of the shadow candidate area in the foreground part of the divided part. The shadow candidate areas obtained by different calculation methods are different. For example, the shadow candidate area obtained by the projection method is composed of pixel rows or pixel columns, and the shadow candidate area obtained by combining chroma, brightness and saturation is composed of pixel points. If the shadow candidate area is composed of pixel rows or pixel columns, then calculate the ratio of the shadow candidate area in the divided part that is determined to be the foreground pixel row or pixel column; if the shadow candidate area is composed of pixel points, then Calculate the ratio of the number of pixels determined as shadow candidate areas to the number of foreground pixels in the divided part, and remove the entire part with a higher calculation ratio as a shadow area.
水平方向上，每个像素位置表示一行像素行，垂直纹理方向上，每个像素位置表示一列像素列。In the horizontal direction, each pixel position represents a row of pixels, and in the vertical texture direction, each pixel position represents a column of pixel columns.
进一步而言，所计算的纹理统计特征值包括以下至少一个：梯度、方差、Sobel算子、熵、拉普拉斯算子以及LBP特征值。Further, the calculated texture statistical feature value includes at least one of the following: gradient, variance, Sobel operator, entropy, Laplacian operator, and LBP feature value.
采用其中一种时，利用所选择的纹理统计特征值计算方法分别计算在水平方向上各像素行的纹理统计特征值和在垂直方向上各像素列的纹理统计特征值。当采用多种时，利用所选择的纹理统计特征值计算方法分别计算在水平方向上各像素行的纹理统计特征值和垂直方向上各像素列的纹理统计特征值。When one of them is adopted, the texture statistical feature value of each pixel row in the horizontal direction and the texture statistical feature value of each pixel column in the vertical direction are respectively calculated by using the selected texture statistical feature value calculation method. When multiple methods are used, the texture statistical feature value of each pixel row in the horizontal direction and the texture statistical feature value of each pixel column in the vertical direction are respectively calculated using the selected texture statistical feature value calculation method.
进一步而言，所述纹理统计特征值为方差，对于具有p行q列的前景灰度图，水平和垂直方向上的纹理统计特征值获取方法如下：Further, the statistical feature value of the texture is the variance. For a foreground grayscale image with p rows and q columns, the method for obtaining the statistical feature value of the texture in the horizontal and vertical directions is as follows:
水平方向上第i行像素行的方差HorizontalVariance(i)为第i行相邻两侧r行上所有不为0的像素点的灰度值方差统计结果，其中r为指定值：The variance HorizontalVariance(i) of the i-th row of pixels in the horizontal direction is the statistical result of the variance of the gray value of all pixels that are not 0 on the r rows adjacent to the i-th row, where r is the specified value:
其中，FG′(i，j)表示在所述前景灰度图中第i行第j列像素点的灰度值，FG′(i，j)中j的取值为1到q且i的取值为i-r到i+r，为所述第i-r到i+r共2*r+1行像素的灰度值均值，m为2r+1行像素的总个数；Among them, FG'(i, j) represents the gray value of the pixel point in the i-th row and j-th column in the foreground grayscale image, and the value of j in FG'(i, j) is 1 to q and i The value is from ir to i+r, It is the mean value of the gray value of the 2*r+1 rows of pixels from the irth to i+r, and m is the total number of pixels in the 2r+1 rows;
垂直方向上，第j列像素列的方差VerticalVariance(j)为第j列相邻两侧c列上所有不为0的像素点的方差统计结果：In the vertical direction, the variance VerticalVariance(j) of the jth column pixel column is the statistical variance result of all the non-zero pixel points on the c columns adjacent to the jth column:
其中，FG′(i，j)表示在所述前景灰度图中第i行第j列像素点的灰度值，FG′(i，j)中j的取值为j-c到j+c，且i的取值为1到p，为所述第j-c到j+c共2*c+1列的像素灰度值均值，m为2c+1列像素的总个数。Wherein, FG'(i, j) represents the grayscale value of the i-th row and j-th column pixel point in the foreground grayscale image, and the value of j in FG'(i, j) is jc to j+c, And the value of i is 1 to p, is the mean value of the pixel gray value of the 2*c+1 column from jc to j+c, and m is the total number of pixels in the 2c+1 column.
图像中单个像素行或像素列的方差可能存在其中某一像素行或像素列的方差由于噪声或其他原因而出现异常，不满足正常的统计特征，在后续的分割位置查找中可能误将该像素位置作为分割位置，因此为避免出错，考虑邻近的像素行或像素列的信息，类似于滤波的作用，从而使得计算得到的纹理统计特征方差具有良好的统计特性。类似的采用统计方式得到的熵作为纹理统计特征值时，对应的像素行或像素列也需要将邻近的对应像素行或像素列信息加以考虑。The variance of a single pixel row or pixel column in the image may be abnormal due to noise or other reasons, and the variance of a certain pixel row or pixel column does not meet the normal statistical characteristics. In the subsequent segmentation position search, the pixel may be mistaken The position is used as the segmentation position, so in order to avoid errors, the information of adjacent pixel rows or pixel columns is considered, which is similar to the role of filtering, so that the calculated variance of texture statistical features has good statistical properties. Similarly, when the entropy obtained in a statistical manner is used as the statistical feature value of the texture, the corresponding pixel row or pixel column also needs to take into account the information of the adjacent corresponding pixel row or pixel column.
进一步而言，所述纹理统计特征值为Sobel算子，对所述前景灰度图中第i行第j列的像素点，Further, the statistical feature value of the texture is a Sobel operator, and for the pixel in the i-th row and j-th column in the foreground grayscale image,
水平Sobel算子计算结果为：The calculation result of the horizontal Sobel operator is:
S(i，j)＝FG′(i-1，j-1)+2·FG′(i-1，j)+FG′(i-1，j+1)S(i,j)=FG'(i-1,j-1)+2·FG'(i-1,j)+FG'(i-1,j+1)
-FG′(i+1，j-1)-2·FG′(i+1，j)-FG′(i+1，j+1)-FG'(i+1, j-1)-2 · FG'(i+1, j)-FG'(i+1, j+1)
垂直Sobel算子计算结果为：S(i，j)＝FG′(i-1，j+1)+2·FG′(i，j+1)+FG′(i+1，j+1)-FG′(i-1，j-1)-2·FG′(，，j-1)-FG′(i+1，j-1)The calculation result of the vertical Sobel operator is: S(i, j) = FG'(i-1, j+1)+2·FG'(i, j+1)+FG'(i+1, j+1) -FG'(i-1, j-1)-2 · FG'(,, j-1)-FG'(i+1, j-1)
其中FG′为像素点的灰度值，在水平方向上各像素行的纹理统计特征值为该行像素点中Sobel算子的最大值；Where FG' is the gray value of the pixel, and the texture statistical feature value of each pixel row in the horizontal direction is the maximum value of the Sobel operator in the row of pixels;
在垂直方向上各像素列的纹理统计特征值为该列像素点中Sobel算子的最大值。The texture statistical feature value of each pixel column in the vertical direction is the maximum value of the Sobel operator in the pixel points of the column.
Sobel算子作为特征值时，已将邻近像素的信息加以考虑，在采用其他的局部纹理统计特征值如梯度、拉普拉斯算子、LBP特征值进行投影时，像素行或像素列所对应的投影特征值也为该像素行或像素列所计算得到像素点纹理统计特征值中的最大值。When the Sobel operator is used as the eigenvalue, the information of adjacent pixels has been considered. When using other local texture statistical eigenvalues such as gradient, Laplacian operator, and LBP eigenvalues for projection, the pixel row or pixel column corresponds to The projection eigenvalue of is also the maximum value among the statistical eigenvalues of the pixel point texture calculated for the pixel row or pixel column.
进一步而言，在水平和垂直方向上找到分割位置的方法包括，对于单个方向上的第n个像素位置，分别计算：Further, the method for finding the segmentation position in the horizontal and vertical directions includes, for the nth pixel position in a single direction, respectively calculating:
其中，k为像素行或像素列的总长度，H(i)表示第i个像素位置的纹理统计特征值，为第n个像素位置之前所有像素位置的纹理统计特征值均值，为第n个像素位置之后所有像素位置的纹理统计特征值均值，在水平方向上第n个像素位置为第n个像素行，在垂直方向上第n个像素位置为第n个像素列；Among them, k is the total length of the pixel row or pixel column, H(i) represents the texture statistical feature value of the i-th pixel position, is the mean value of texture statistical feature values of all pixel positions before the nth pixel position, is the mean value of the texture statistical feature value of all pixel positions after the nth pixel position, the nth pixel position in the horizontal direction is the nth pixel row, and the nth pixel position in the vertical direction is the nth pixel column;
获取σ1与σ2之和达到最小值所在的像素位置，作为运动目标与阴影差异最大的分割位置。Obtain the pixel position where the sum of σ1 and σ2 reaches the minimum value, and use it as the segmentation position where the difference between the moving object and the shadow is the largest.
σ1表示在第n个像素位置之前各像素位置的纹理统计特征值与均值之间的差值之和，σ2表示在第n个像素位置之后各像素位置的纹理统计特征值与均值之间的差值之和，当σ1与σ2之和达到最小，则意味着在当前的n取值处左侧及右侧的相邻像素位置之间纹理统计特征值的差异均达到了最小，同时表明该像素位置运动目标与阴影差异最大，从而将运动目标与阴影分开。σ1 represents the sum of the difference between the texture statistical feature value and the average value of each pixel position before the nth pixel position, andσ2 represents the difference between the texture statistical feature value and the mean value of each pixel position after the nth pixel position When the sum of σ1 and σ2 reaches the minimum, it means that the difference of texture statistical feature values between the adjacent pixel positions on the left and right of the current n value has reached the minimum, At the same time, it indicates that the pixel position has the largest difference between the moving object and the shadow, so that the moving object and the shadow are separated.
进一步而言，获取图像前景中的阴影候选区的方法包括以下一者：Further, the method for obtaining the shadow candidate area in the foreground of the image includes one of the following:
对图像中的前景分别进行水平和垂直投影并限定门限值，得到小于所述门限值的像素位置在图像中对应的区域作为所述阴影候选区；Horizontally and vertically project the foreground in the image respectively and define a threshold value, and obtain an area corresponding to a pixel position smaller than the threshold value in the image as the shadow candidate area;
分别预设亮度、色度以及饱和度的阈值，并将图像中将亮度、色度以及饱和度同时超出阈值的像素点作为所述阴影候选区。The thresholds of brightness, chroma and saturation are respectively preset, and the pixels in the image whose brightness, chroma and saturation exceed the threshold at the same time are used as the shadow candidate areas.
采用对前景检测图进行水平和垂直投影并限定门限值的方法，在水平投影得到的水平投影直方图中，所得到的阴影候选区为像素灰度值小于门限值的各像素行，在垂直投影得到的水平投影直方图中，所得到的阴影候选区为所得到的阴影候选区为像素灰度值小于门限值的各像素列。Using the method of horizontally and vertically projecting the foreground detection map and defining the threshold value, in the horizontal projection histogram obtained by horizontal projection, the obtained shadow candidate area is each pixel row whose pixel gray value is less than the threshold value, in In the horizontal projection histogram obtained by the vertical projection, the obtained shadow candidate regions are the pixel columns whose gray value of the pixel is smaller than the threshold value.
采用预设亮度、色度以及饱和度的阈值方法所得到的各像素点组成所述的阴影候选区。The pixel points obtained by adopting the threshold method of preset brightness, chroma and saturation constitute the shadow candidate area.
进一步而言，在所计算的纹理统计特征值有多个的情况下，对应地在水平和垂直方向上得到的分割位置均有多个，对于单个方向，在分别计算各分割位置两侧阴影候选区与前景像素的比例之后，先选择两侧比例相差较大的分割位置作为最终分割位置，再从所选最终分割位置的两侧中选择比例较大的一侧作为阴影去除。Furthermore, when there are multiple texture statistical feature values to be calculated, there are correspondingly multiple segmentation positions obtained in the horizontal and vertical directions. For a single direction, the shadow candidates on both sides of each segmentation position are calculated respectively. After the ratio of the area to the foreground pixel, first select the segmentation position with a larger ratio difference on both sides as the final segmentation position, and then select the side with a larger ratio from the two sides of the selected final segmentation position as the shadow removal.
结合多个纹理统计特征值综合考虑，可以提高阴影区判定的准确性，在利用分割位置进行划分时，不同分割位置划分出的两个部分中，阴影候选区所占前景的比例不同，而比例相差较大表明阴影与非阴影之间的区分度高，因此将两部分比例相差较大所对应的分割位置作为最终的分割位置可以更准确地剔除阴影。Combined with the comprehensive consideration of multiple texture statistical feature values, the accuracy of shadow area determination can be improved. When using the segmentation position for division, in the two parts divided by different segmentation positions, the proportion of the shadow candidate area to the foreground is different, and the proportion A large difference indicates a high degree of discrimination between shadows and non-shadows. Therefore, taking the segmentation position corresponding to the large difference between the two parts as the final segmentation position can remove shadows more accurately.
为实施本发明方法，本发明还提供了对应的装置，以降低图像中的阴影误检率。In order to implement the method of the present invention, the present invention also provides a corresponding device to reduce the shadow false detection rate in the image.
一种去除前景检测结果阴影的装置，包括：预处理模块，由输入的原始图像得到灰度图及前景检测图，阴影候选区获取模块获取图像前景中的阴影候选区，还包括：A device for removing the shadow of a foreground detection result, including: a preprocessing module, which obtains a grayscale image and a foreground detection image from an input original image, and a shadow candidate area acquisition module acquires a shadow candidate area in the image foreground, and also includes:
纹理统计特征值计算模块，将所述灰度图去除背景得到前景灰度图，从所述前景灰度图计算水平方向上以及垂直方向上的纹理统计特征值；The texture statistical feature value calculation module removes the background from the grayscale image to obtain a foreground grayscale image, and calculates the texture statistical feature values in the horizontal direction and the vertical direction from the foreground grayscale image;
分割位置查找模块，利用水平方向上的纹理统计特征值找到水平方向上的分割位置且利用垂直方向上的纹理统计特征值找到垂直方向上的分割位置，其中各分割位置在对应方向上使运动目标与阴影差异最大；The segmentation position search module uses the texture statistical feature value in the horizontal direction to find the segmentation position in the horizontal direction and uses the texture statistical feature value in the vertical direction to find the segmentation position in the vertical direction, wherein each segmentation position makes the moving target in the corresponding direction the largest difference from shadow;
阴影去除模块，分别计算水平及垂直方向上的分割位置两侧的阴影候选区占所对应前景部分的比例，并从分割位置的两侧中选择比例较大的一侧作为阴影去除。The shadow removal module calculates the ratio of the shadow candidate areas on both sides of the segmentation position in the horizontal and vertical directions to the corresponding foreground part, and selects the side with a larger ratio from the two sides of the segmentation position as the shadow removal.
从基于纹理统计特征进行的投影中找到使运动目标与阴影差异最大的分割点，将分割结果结合阴影候选区域，进一步确定物体阴影的检测范围，从而避免一些类似阴影的区域被当作阴影而被去除，提高了阴影检测的准确率。另外这种方法本身已对运动物体和阴影部分进行了准确分割，因此在选择候选阴影区域时，针对不同场景不需要设置严格的门限值，可以对大部分场景设置普遍适用的统一门限值，从而提高了方法的普遍适用性。在水平方向上所述像素位置为像素行位置，在垂直方向上所述像素位置为像素列的位置，在灰度图中包含有前景纹理统计特征，但同时也包含背景的纹理统计特征，而背景的纹理统计特征在进行投影时为无效信息，会对后面的分割过程造成干扰，通过将灰度图与前景检测图相乘，将灰度图去除背景，从所得到的前景灰度图中直接可获取前景的纹理统计特征，避免背景的干扰，从而提高分割准确率。将所述前景检测图与所述灰度图相乘，即可将灰度图中的背景去除。From the projection based on texture statistical features, find the segmentation point that makes the difference between the moving object and the shadow the largest, combine the segmentation results with the shadow candidate area, and further determine the detection range of the object shadow, so as to avoid some shadow-like areas being regarded as shadows. Removed to improve the accuracy of shadow detection. In addition, this method itself has accurately segmented moving objects and shadow parts, so when selecting candidate shadow areas, it is not necessary to set strict threshold values for different scenes, and a uniform threshold value that is generally applicable to most scenes can be set , thus improving the general applicability of the method. The pixel position in the horizontal direction is the position of the pixel row, and the pixel position in the vertical direction is the position of the pixel column. The grayscale image contains the statistical characteristics of the foreground texture, but also contains the statistical characteristics of the texture of the background. The texture statistical features of the background are invalid information during projection, which will interfere with the subsequent segmentation process. By multiplying the grayscale image with the foreground detection image, the grayscale image is removed from the background, and the obtained foreground grayscale image is The texture statistical features of the foreground can be obtained directly, and the interference of the background can be avoided, thereby improving the segmentation accuracy. The background in the grayscale image can be removed by multiplying the foreground detection image with the grayscale image.
其中获取阴影候选区的图像可能是前景检测图也可能是原始图像，例如通过将前景检测图进行水平和垂直方向上投影的方式得到阴影候选区，则获取的阴影候选区在前景检测图中；利用色度、亮度结合饱和度的方式得到阴影候选区时，所获取的阴影候选区在原始图像中，或者，根据后续要进行的处理，阴影候选区也可能在灰度图中。Wherein the image for obtaining the shadow candidate area may be a foreground detection map or an original image, for example, the shadow candidate area is obtained by projecting the foreground detection map in the horizontal and vertical directions, then the obtained shadow candidate area is in the foreground detection map; When the shadow candidate area is obtained by combining chroma, brightness and saturation, the acquired shadow candidate area is in the original image, or, according to subsequent processing, the shadow candidate area may also be in the grayscale image.
由于划分的位置并不一定是中心位置，因此，如果仅仅是判断各部分被判断为阴影的像素行或像素列出现频次，则可能造成被划分出来面积较大的部分被判断为阴影的像素行或像素列或是像素点比划分出来面积较小的部分出现频次高，从而导致总是将划分出的面积较大的部分被作为阴影区剔除。为了防止这种现象出现，通过将阴影候选区在所划分部分中的前景部分所占比例来判断阴影区出现的可能性大小。不同的计算方法得到的阴影候选区不同，例如通过投影方法得到的阴影候选区由像素行或像素列组成，由色度、亮度结合饱和度得到的阴影候选区由像素点组成。如果阴影候选区由像素行或像素列组成，则计算该阴影候选区在所划分出的部分中对应的被判定为前景的像素行或像素列的比例；如果阴影候选区由像素点组成，则计算被判定为阴影候选区的像素点个数在所划分出的部分中占前景像素点个数的比例，将计算比例较高的整个部分作为阴影区剔除。Since the divided position is not necessarily the central position, if only the frequency of occurrence of the pixel row or pixel column judged to be shadowed in each part is judged, it may cause the divided part with a larger area to be judged as the shadowed pixel row Either the pixel row or the pixel point appears more frequently than the divided part with a smaller area, so that the divided part with a larger area is always removed as a shadow area. In order to prevent this phenomenon from appearing, the possibility of the shadow area is judged by the proportion of the shadow candidate area in the foreground part of the divided part. The shadow candidate areas obtained by different calculation methods are different. For example, the shadow candidate area obtained by the projection method is composed of pixel rows or pixel columns, and the shadow candidate area obtained by combining chroma, brightness and saturation is composed of pixel points. If the shadow candidate area is composed of pixel rows or pixel columns, then calculate the ratio of the shadow candidate area in the divided part that is determined to be the foreground pixel row or pixel column; if the shadow candidate area is composed of pixel points, then Calculate the ratio of the number of pixels determined as shadow candidate areas to the number of foreground pixels in the divided part, and remove the entire part with a higher calculation ratio as a shadow area.
水平方向上，每个像素位置表示一行像素行，垂直纹理方向上，每个像素位置表示一列像素列。In the horizontal direction, each pixel position represents a row of pixels, and in the vertical texture direction, each pixel position represents a column of pixel columns.
进一步而言，所述纹理统计特征值计算模块所计算的纹理统计特征值包括以下至少一个：梯度、方差、Sobel算子、熵、拉普拉斯算子以及LBP特征值。Further, the texture statistical feature value calculated by the texture statistical feature value calculation module includes at least one of the following: gradient, variance, Sobel operator, entropy, Laplacian operator and LBP feature value.
采用其中一种时，利用所选择的纹理统计特征值计算方法分别计算在水平方向上各像素行的纹理统计特征值和在垂直方向上各像素列的纹理统计特征值。当采用多种时，利用所选择的纹理统计特征值计算方法分别计算在水平方向上各像素行的纹理统计特征值和垂直方向上各像素列的纹理统计特征值。When one of them is adopted, the texture statistical feature value of each pixel row in the horizontal direction and the texture statistical feature value of each pixel column in the vertical direction are respectively calculated by using the selected texture statistical feature value calculation method. When multiple methods are used, the texture statistical feature value of each pixel row in the horizontal direction and the texture statistical feature value of each pixel column in the vertical direction are respectively calculated using the selected texture statistical feature value calculation method.
进一步而言，所述分割位置查找模块在水平和垂直方向上找到分割位置的方法包括，对于单个方向上的第n个像素位置，分别计算：Further, the method for finding the segmentation position in the horizontal and vertical directions by the segmentation position search module includes, for the nth pixel position in a single direction, calculating respectively:
其中，k为像素行或像素列的总长度，H(i)表示第i个像素位置的纹理统计特征值，为第n个像素位置之前所有像素位置的纹理统计特征值均值，为第n个像素位置之后所有像素位置的纹理统计特征值均值，在水平方向上第n个像素位置为第n个像素行，在垂直方向上第n个像素位置为第n个像素列；Among them, k is the total length of the pixel row or pixel column, H(i) represents the texture statistical feature value of the i-th pixel position, is the mean value of texture statistical feature values of all pixel positions before the nth pixel position, is the mean value of the texture statistical feature value of all pixel positions after the nth pixel position, the nth pixel position in the horizontal direction is the nth pixel row, and the nth pixel position in the vertical direction is the nth pixel column;
获取σ1与σ2之和达到最小值所在的像素位置，作为运动目标与阴影差异最大的分割位置。Obtain the pixel position where the sum of σ1 and σ2 reaches the minimum value, and use it as the segmentation position where the difference between the moving object and the shadow is the largest.
σ1表示在第n个像素位置之前各像素位置的纹理统计特征值与均值之间的差值之和，σ2表示在第n个像素位置之后各像素位置的纹理统计特征值与均值之间的差值之和，当σ1与σ2之和达到最小，则意味着在当前的n取值处左侧及右侧的相邻像素位置之间纹理统计特征值的差异均达到了最小，同时表明该像素位置运动目标与阴影差异最大，从而将运动目标与阴影分开。σ1 represents the sum of the difference between the texture statistical feature value and the average value of each pixel position before the nth pixel position, andσ2 represents the difference between the texture statistical feature value and the mean value of each pixel position after the nth pixel position When the sum of σ1 and σ2 reaches the minimum, it means that the difference of texture statistical feature values between the adjacent pixel positions on the left and right of the current n value has reached the minimum, At the same time, it indicates that the pixel position has the largest difference between the moving object and the shadow, so that the moving object and the shadow are separated.
进一步而言，所述纹理统计特征值计算模块所计算的纹理统计特征值有多个的情况下，对应地分割位置查找模块在水平和垂直方向上得到的分割位置均有多个，对于单个方向，所述阴影去除模块在分别计算各分割位置两侧阴影候选区与前景像素的比例之后，先选择两侧比例相差较大的分割位置作为最终分割位置，再从所选最终分割位置的两侧中选择比例较大的一侧作为阴影去除。Further, when there are multiple texture statistical feature values calculated by the texture statistical feature value calculation module, there are multiple segmentation positions obtained by the corresponding segmentation position search module in the horizontal and vertical directions. For a single direction , after the shadow removal module calculates the proportions of the shadow candidate area and the foreground pixel on both sides of each segmentation position, first selects the segmentation position with a large difference in the proportions of the two sides as the final segmentation position, and then selects from both sides of the selected final segmentation position Select the side with a larger ratio as the shadow removal.
结合多个纹理统计特征值综合考虑，可以提高阴影区判定的准确性，在利用分割位置进行划分时，不同分割位置划分出的两个部分中，阴影候选区所占前景的比例不同，而比例相差较大表明阴影与非阴影之间的区分度高，因此将两部分比例相差较大所对应的分割位置作为最终的分割位置可以更准确地剔除阴影。Combined with the comprehensive consideration of multiple texture statistical feature values, the accuracy of shadow area determination can be improved. When using the segmentation position for division, in the two parts divided by different segmentation positions, the proportion of the shadow candidate area to the foreground is different, and the proportion A large difference indicates a high degree of discrimination between shadows and non-shadows. Therefore, taking the segmentation position corresponding to the large difference between the two parts as the final segmentation position can remove shadows more accurately.
本发明的方法及装置突出效果在于，能够有效地降低前景检测结果中阴影的误检率，避免将非阴影区域误检为阴影区域，提高阴影识别准确率，从而有助于后续处理效率的提高；不需要针对具体场景设定具体的门限值，从而提高了方法适用的普遍性；结合多个纹理统计特征值综合考虑，可以提高阴影区判定的准确性。The outstanding effect of the method and device of the present invention is that it can effectively reduce the false detection rate of shadows in the foreground detection results, avoid misdetection of non-shadowed areas as shadowed areas, improve the accuracy of shadow recognition, and thus contribute to the improvement of subsequent processing efficiency ; There is no need to set a specific threshold value for a specific scene, thereby improving the universality of the method; combined with the comprehensive consideration of multiple texture statistical feature values, the accuracy of shadow area determination can be improved.
附图说明Description of drawings
图1为本发明一个实施例的方法流程图；Fig. 1 is the method flowchart of an embodiment of the present invention;
图2为本发明当前实施例的垂直方向纹理投影直方图；FIG. 2 is a histogram of vertical texture projection in the current embodiment of the present invention;
图3为本发明当前实施例查找分割位置所得到的查找结果示意图。FIG. 3 is a schematic diagram of a search result obtained by searching for a segmentation position according to the current embodiment of the present invention.
具体实施方式detailed description
现结合实施例及说明书附图对本发明方案进行详细的解释说明。The scheme of the present invention is now explained in detail in conjunction with the embodiments and the accompanying drawings.
本发明根据实际使用中的阴影情况进行分析，用一种简单有效的方式，将纹理信息与几何信息相结合，去除阴影，正确分割，提高后期跟踪与识别的准确率。本发明一个实施例去除前景检测结果阴影的装置包括：预处理模块、纹理统计特征值计算模块、分割位置查找模块、阴影候选区获取模块以及阴影去除模块。The present invention analyzes the shadow situation in actual use, combines texture information and geometric information in a simple and effective way, removes shadows, correctly divides, and improves the accuracy of later tracking and recognition. In one embodiment of the present invention, the device for removing the shadow of the foreground detection result includes: a preprocessing module, a texture statistical feature value calculation module, a segmentation position search module, a shadow candidate area acquisition module and a shadow removal module.
当前实施例利用去除前景检测结果阴影的装置对前景检测结果进行阴影去除的方法如图1所示，包括如下步骤：In the present embodiment, the method for removing the shadow of the foreground detection result by using the device for removing the shadow of the foreground detection result is shown in Figure 1, including the following steps:
A，预处理模块根据将输入的原始图像得到灰度图及前景检测图。A. The preprocessing module obtains a grayscale image and a foreground detection image from the input original image.
在当前实施例中，预处理模块对输入的原始图像进行灰度化处理，得到灰度图；并对灰度图进行前景检测得到二值化的前景检测图。In the current embodiment, the preprocessing module grayscales the input original image to obtain a grayscale image; and performs foreground detection on the grayscale image to obtain a binarized foreground detection image.
B，纹理统计特征值计算模块将灰度图与前景检测图进行点乘得到前景灰度图，并根据前景灰度图在水平方向上计算各像素行的纹理统计特征值，在垂直方向上计算各像素列的纹理统计特征值。B. The texture statistical eigenvalue calculation module performs point multiplication of the grayscale image and the foreground detection image to obtain the foreground grayscale image, and calculates the texture statistical eigenvalues of each pixel row in the horizontal direction according to the foreground grayscale image, and calculates in the vertical direction Texture statistics feature values for each pixel column.
后续进行纹理统计特征的计算需要灰度信息，而前景检测图为二值化图像，不存在灰度信息，而如果直接采用灰度图，由于灰度图中包含有前景纹理统计特征，但同时也包含背景的纹理统计特征，而背景的纹理统计特征在进行投影时为无效信息，会对后面的分割过程造成干扰。首先以粗略分割得到的前景检测图为掩码图像，将用相同的坐标分割出来的灰度图对应区域提取出来，即将前景检测图与灰度图进行点乘，得到前景灰度图。前景灰度图为去除了背景的灰度图。The subsequent calculation of texture statistical features requires grayscale information, and the foreground detection image is a binary image without grayscale information. If the grayscale image is directly used, since the grayscale image contains foreground texture statistical features, but at the same time It also contains the texture statistical features of the background, and the texture statistical features of the background are invalid information during projection, which will interfere with the subsequent segmentation process. First, the foreground detection image obtained by rough segmentation is used as a mask image, and the corresponding area of the grayscale image segmented with the same coordinates is extracted, that is, the foreground detection image and the grayscale image are dot-multiplied to obtain the foreground grayscale image. The foreground grayscale image is the grayscale image with the background removed.
这里图像的纹理可以用多种不同的方式进行描述，例如梯度、方差、Sobel、熵等，均可表征图像细节的丰富程度。多种纹理统计特征描述方式可选择其中一种使用，也可使用多种然后综合判断，需要对性能和效果进行权衡。当前实施例中采用各像素行及各像素列在指定范围内的方差，每行像素行的方差用当前像素行与两侧相邻c行的方差表示，每列像素列的方差用两侧相邻r列的方差表示，当前实施例中，c以及r的取值均为1，即对于第i行像素，其方差为i-1、i、i+1三行的灰度值方差，即对于第j列像素，其方差为j-1、j、j+1三列的灰度值方差。The texture of the image here can be described in many different ways, such as gradient, variance, Sobel, entropy, etc., all of which can represent the richness of image details. A variety of texture statistical feature description methods can be selected for use, or multiple methods can be used and then comprehensively judged, and performance and effect need to be weighed. In the current embodiment, the variance of each pixel row and each pixel column within a specified range is used. The variance of each pixel row is represented by the variance of the current pixel row and the adjacent c rows on both sides. The variance of the adjacent r columns indicates that in the current embodiment, the values of c and r are both 1, that is, for the i-th row of pixels, the variance is the variance of the gray value of the three rows of i-1, i, and i+1, namely For the jth column of pixels, its variance is the variance of the gray value of the three columns j-1, j, j+1.
水平方向上第i行像素行的方差HorizontalVariance(i)为i-1、i、i+1三行上所有不为0的像素点的灰度值方差统计结果：The variance HorizontalVariance(i) of the i-th row of pixels in the horizontal direction is the statistical result of the variance of the gray value of all pixels that are not 0 on the three rows i-1, i, and i+1:
其中，FG′(i，j)表示在前景灰度图中第i行第j列像素点的灰度值，FG′(i，j)中j的取值为1到q且i的取值为i-1，i以及i+1，为所述第i-1、i、i+1三行像素的灰度值均值，m为i-1到i+1这3行的像素点个数总和；Among them, FG'(i, j) represents the gray value of the pixel in the i-th row and j-th column in the foreground grayscale image, and the value of j in FG'(i, j) is 1 to q and the value of i for i-1, i and i+1, is the mean gray value of the pixels in the i-1th, i, and i+1th three rows, and m is the sum of the number of pixels in the three rows from i-1 to i+1;
垂直方向上，第j列像素列的方差为第j-1、j、j+1三列上所有不为0的像素点的方差统计结果：In the vertical direction, the variance of the pixel column in the jth column is the variance statistical result of all the pixels that are not 0 on the j-1, j, and j+1 columns:
其中，FG′(i，j)表示在前景灰度图中第i行第j列像素点的灰度值，FG′(i，j)中j的取值为j-1到j+1，且i的取值为1到p，为第j-1、j、j+1三列的像素灰度值均值，m为j-1到j+1这3列的像素点个数总和。Among them, FG'(i, j) represents the gray value of the pixel point in the i-th row and j-th column in the foreground grayscale image, and the value of j in FG'(i, j) is j-1 to j+1, And the value of i is 1 to p, is the mean value of the pixel gray value of the j-1, j, j+1 three columns, and m is the sum of the number of pixels in the three columns j-1 to j+1.
纹理统计特征值也采用其他方式计算得到，例如采用Sobel算子作为纹理统计特征值，对第i行第j列的像素点，水平Sobel算子计算结果为：The statistical feature value of texture can also be calculated by other methods. For example, the Sobel operator is used as the statistical feature value of texture. For the pixel point in the i-th row and the j-th column, the calculation result of the horizontal Sobel operator is:
S(i，j)＝FG′(i-1，j-1)+2·FG′(i-1，j)+FG′(i-1，j+1)S(i,j)=FG'(i-1,j-1)+2·FG'(i-1,j)+FG'(i-1,j+1)
-FG′(i+1，j-1)-2·FG′(i+1，j)-FG′(i+1，j+1)-FG'(i+1, j-1)-2 · FG'(i+1, j)-FG'(i+1, j+1)
对第i行第j列的像素点，垂直Sobel算子计算结果为：For the pixel point in row i and column j, the calculation result of the vertical Sobel operator is:
S(i，j)＝FG′(i-1，j+1)+2·FG′(i，j+1)+FG′(i+1，j+1)S(i, j) = FG'(i-1, j+1)+2·FG'(i, j+1)+FG'(i+1, j+1)
-FG′(i-1，j-1)-2·FG′(i，j-1)-FG′(i+1，j-1)-FG'(i-1, j-1)-2 · FG'(i, j-1)-FG'(i+1, j-1)
其中FG’表示像素点的灰度值，括号中参数分别表示像素点的行列位置。Among them, FG' represents the gray value of the pixel, and the parameters in the brackets represent the row and column positions of the pixel.
当前实施例中，水平方向像素行的纹理统计特征值用水平纹理投影直方图表示，垂直方向像素列的纹理统计特征值用垂直纹理投影直方图表示。In the current embodiment, the texture statistical feature values of the horizontal pixel rows are represented by a horizontal texture projection histogram, and the texture statistical feature values of the vertical pixel columns are represented by a vertical texture projection histogram.
对于方差而言，水平纹理投影直方图HorizontalProject(i)为：For variance, the horizontal texture projection histogram HorizontalProject(i) is:
HorizontalProject(i)＝HorizontalVariance(i)；HorizontalProject(i) = HorizontalVariance(i);
即，水平纹理投影直方图的横坐标为各像素行，纵坐标为各像素行的方差，此处各像素行的方差即为计算得到的HorizontalVariance(i)。That is, the abscissa of the horizontal texture projection histogram is each pixel row, and the ordinate is the variance of each pixel row, where the variance of each pixel row is the calculated HorizontalVariance(i).
垂直纹理投影直方图VerticalProject(j)为：The vertical texture projection histogram VerticalProject(j) is:
VerticalProject(j)＝VerticalVariance(j)。VerticalProject(j) = VerticalVariance(j).
同理，垂直纹理投影直方图的横坐标为各像素列，纵坐标为各像素列的方差，此处各像素列的方差即为计算得到的VerticalVariance(j)。Similarly, the abscissa of the vertical texture projection histogram is each pixel column, and the ordinate is the variance of each pixel column, where the variance of each pixel column is the calculated VerticalVariance(j).
采用方差作为纹理统计特征值，在垂直方向上投影得到的垂直纹理投影直方图如图2所示。Using the variance as the texture statistical feature value, the vertical texture projection histogram obtained by projecting in the vertical direction is shown in Figure 2.
如果采用Sobel算子，则在水平纹理投影直方图中，对于任意像素行，像素行的纹理统计特征值为该行像素点中Sobel算子的最大值；在垂直纹理投影直方图中，对于任意像素列，像素列的纹理统计特征值为该列像素点中Sobel算子的最大值。其他局部纹理统计特征值例如拉普拉斯算子、梯度、LBP特征值在进行纹理统计特征值的计算时，某一像素行或像素列的纹理统计特征值中均为像素行或像素列中像素点的最大纹理统计特征值。If the Sobel operator is used, in the horizontal texture projection histogram, for any pixel row, the texture statistical feature value of the pixel row is the maximum value of the Sobel operator in the pixel points of the row; in the vertical texture projection histogram, for any pixel row Pixel column, the texture statistical feature value of the pixel column is the maximum value of the Sobel operator in the pixel points of the column. Other local texture statistical feature values such as Laplacian operator, gradient, and LBP feature value are calculated in the texture statistical feature value, and the texture statistical feature value of a certain pixel row or pixel column is in the pixel row or pixel column The maximum texture statistical feature value of a pixel.
C，分割位置查找模块分别从水平纹理投影图及垂直纹理投影图中找到使运动目标与阴影部分在投影上差异最大的分割位置。C. The segmentation position search module finds the segmentation position that makes the largest difference in projection between the moving object and the shadow part from the horizontal texture projection map and the vertical texture projection map.
由于步骤B已经用纹理统计特征很好地描述了图像信息，得到了纹理投影直方图，只需要通过纹理投影直方图的信息来找到差异最大的分割位置。对纹理投影直方图的第n个像素位置，计算：Since the image information has been well described by the texture statistical features in step B, and the texture projection histogram is obtained, it is only necessary to find the segmentation position with the largest difference through the information of the texture projection histogram. For the nth pixel position of the texture projection histogram, compute:
其中k为纹理投影直方图的总长度，即水平纹理投影直方图中总长度为像素行的总行数，垂直纹理投影直方图中总长度为像素列的总列数，为第n个像素位置之前所有像素位置的纹理统计特征值均值，为第n个像素位置之后所有像素位置的纹理统计特征值均值。当σ1与σ2之和达到最小值时，表示在该像素位置运动目标与阴影部分被最大程度地区分开来。Where k is the total length of the texture projection histogram, that is, the total length of the horizontal texture projection histogram is the total number of rows of pixel rows, and the total length of the vertical texture projection histogram is the total number of columns of pixel columns, is the mean value of texture statistical feature values of all pixel positions before the nth pixel position, is the mean value of texture statistical feature values of all pixel positions after the nth pixel position. When the sum of σ1 and σ2 reaches the minimum value, it means that the moving target and the shadow part are distinguished to the greatest extent at this pixel position.
最小值所对应的坐标位置即为所需要的分割位置，当前实施例中采用大津法(Ostu)查找运动目标与阴影部分在投影上差异最大的分割位置。利用大津法对图2所示垂直纹理投影直方图处理得到的结果如图3所示，其中图3横坐标为像素列，纵坐标为像素列所对应的σ1与σ2之和。在垂直纹理投影直方图中查找到的分割位置为j＝38，也可以用其他已知的分类方法例如K-means聚类算法找到该最小值的位置。The coordinate position corresponding to the minimum value is the required segmentation position. In the current embodiment, the Otsu method (Ostu) is used to find the segmentation position with the largest difference in projection between the moving object and the shadow part. The results obtained by processing the vertical texture projection histogram shown in Figure 2 using the Otsu method are shown in Figure 3, where the abscissa in Figure 3 is the pixel column, and the ordinate is the sum of σ1 and σ2 corresponding to the pixel column. The segmentation position found in the vertical texture projection histogram is j=38, and other known classification methods such as K-means clustering algorithm can also be used to find the position of the minimum value.
如果计算的纹理统计特征值有多个，例如既有方差也有Sobel算子，则得到的各投影方向上的纹理投影直方图也有对应地有多个，分别是方差的纹理投影直方图以及Sobel算子的纹理投影直方图。相对应地，每个纹理投影直方图中均会得到一个对应的分割位置。例如，利用方差在垂直纹理投影所得到的分割位置为j＝38，利用Sobel算子在垂直纹理投影所得到的分割位置则为j＝42，则在后续的分割过程中需要进一步进行处理。If there are multiple texture statistical eigenvalues, for example, there are both variance and Sobel operators, then there are correspondingly multiple texture projection histograms in each projection direction, which are texture projection histograms of variance and Sobel operators. The texture projection histogram of the sub. Correspondingly, a corresponding segmentation position will be obtained in each texture projection histogram. For example, the segmentation position obtained by using the variance in the vertical texture projection is j=38, and the segmentation position obtained by using the Sobel operator in the vertical texture projection is j=42, and further processing is required in the subsequent segmentation process.
D，阴影候选区获取模块获取图像前景中的阴影候选区。D, the shadow candidate area acquisition module acquires the shadow candidate area in the foreground of the image.
步骤D采用现有的阴影判断方法获取阴影候选区域，步骤D在步骤A完成之后即可进行，可以与步骤B或C同时进行，也可以放在步骤B或C之前或之后进行，当前实施例中，步骤D在完成步骤C后进行。Step D adopts the existing shadow judgment method to obtain shadow candidate areas. Step D can be carried out after step A is completed. It can be carried out simultaneously with step B or C, or it can be carried out before or after step B or C. The current embodiment In, step D is carried out after completing step C.
现有阴影候选区的获取方法包括：对前景检测图进行水平及垂直投影，或者将亮度与色度信息结合。采用亮度结合色度信息获取灰度图中阴影候选区的获取方式为，将同时满足以下三个不等式的像素点作为阴影候选区：Existing methods for obtaining shadow candidate regions include: horizontally and vertically projecting the foreground detection map, or combining luminance and chrominance information. The way to obtain shadow candidate areas in the grayscale image by combining brightness and chrominance information is to use pixels that satisfy the following three inequalities at the same time as shadow candidate areas:
α≤L(x，y)/LB≤β；α≤L(x,y)/LB ≤β;
|H(x，y)-HB|≤τH；|H(x,y)-HB |≤τH ;
S(x，y)-SB≤τs。S(x,y)-SB ≤τs .
对于坐标为(x,y)的像素点，其中L(x,y)表示该像素点的亮度。H(x,y)表示该像素点的色度、S(x,y)表示该像素点的饱和度，LB、HB及SB分别表示灰度图中的背景亮度、背景色度及背景饱和度，在得到前景检测图之后，LB、HB及SB这几个值可以通过对背景的相应计算处理得到。其中α、β及τH均为大于0且小于1的值，τs为大于-1且小于0的值，各值均为预设值，由于色度、亮度以及饱和度信息需要用原始图像(具有RGB信息)结合背景所得到的相关值进行处理，因此阴影候选区在原始图像的前景中获取。For a pixel point whose coordinates are (x, y), where L(x, y) represents the brightness of the pixel point. H(x,y) represents the chroma of the pixel, S(x,y) represents the saturation of the pixel, LB , HB and SB represent the background brightness, background chroma and For background saturation, after obtaining the foreground detection map, the values of LB , HB and SB can be obtained through corresponding calculation and processing of the background. Among them, α, β, and τH are all values greater than 0 and less than 1, τs is a value greater than -1 and less than 0, and each value is a preset value, because the information of chroma, brightness and saturation needs to use the original image (with RGB information) are processed in combination with the correlation values obtained from the background, so shadow candidates are obtained in the foreground of the original image.
本发明当前实施例利用前景检测图投影的方法，是基于行和列来处理的分割位置两侧满足条件的行/列数与分割位置两侧总的行列数。对前景检测图分别进行水平和垂直投影，分别得到水平投影直方图及垂直投影直方图并设定门限值，其中水平投影直方图中横坐标对应灰度图中的像素行，垂直投影直方图中的横坐标对应灰度图中的像素列，各投影直方图中的纵坐标对应像素行或像素列的灰度值，将小于门限值的像素行及像素列判定为阴影候选区，因此得到的图像前景中的阴影候选区是从前景检测图中的前景中获取的。The current embodiment of the present invention utilizes the projection method of the foreground detection map, which processes the number of rows/columns satisfying the conditions on both sides of the segmentation position and the total number of rows and columns on both sides of the segmentation position based on rows and columns. Horizontally and vertically project the foreground detection map to obtain the horizontal projection histogram and vertical projection histogram respectively and set the threshold value, where the abscissa in the horizontal projection histogram corresponds to the pixel row in the grayscale image, and the vertical projection histogram The abscissa in corresponds to the pixel column in the grayscale image, and the ordinate in each projection histogram corresponds to the gray value of the pixel row or pixel column, and the pixel row and pixel column that are less than the threshold value are determined as shadow candidate areas, so The shadow proposals in the foreground of the resulting image are obtained from the foreground in the foreground detection map.
E，阴影去除模块分别计算水平及垂直方向上的分割位置两侧的阴影候选区与前景部分的比例，并从分割位置的两侧中选择比例较大的一侧作为阴影去除。E, the shadow removal module calculates the ratio of the shadow candidate area and the foreground part on both sides of the segmentation position in the horizontal and vertical directions, and selects the side with a larger ratio from the two sides of the segmentation position as the shadow removal.
分割位置在图像中对应像素列或像素行，在当前步骤中，首先利用分割位置将图像进行划分，对应地，将阴影候选区及前景部分被分割位置划分。以垂直纹理投影直方图为例，当前实施例利用方差纹理统计特征值得到的垂直方向上分割位置为j＝38列处，则该分割位置将图像在对应的垂直方向上划分为两个部分。在该方向上，获取各部分阴影候选区在对应部分的前景中所占比例，并将所对应方向中比例较大的部分作为阴影部分去除。当前实施例中第38列左侧的部分小于门限值的像素列(被判定为阴影候选区的像素列)占前景像素列的38％，右侧为24％，则表明左侧部分为阴影部分的可能性较大，从而将第38列左侧部分作为阴影区全部进行剔除。The segmentation position corresponds to the pixel column or pixel row in the image. In the current step, the image is first divided by the segmentation position, and correspondingly, the shadow candidate area and the foreground part are divided by the segmentation position. Taking the vertical texture projection histogram as an example, the segmentation position in the vertical direction obtained by using variance texture eigenvalues in the current embodiment is j=38 columns, and the segmentation position divides the image into two parts in the corresponding vertical direction. In this direction, the proportion of each part of the shadow candidate area in the corresponding part of the foreground is obtained, and the part with a larger proportion in the corresponding direction is removed as a shadow part. In the current embodiment, the pixel columns on the left side of the 38th column that are smaller than the threshold value (the pixel columns determined to be shadow candidate regions) account for 38% of the foreground pixel columns, and the right side is 24%, indicating that the left part is a shadow Part of it is more likely, so the left part of the 38th column is taken as the shadow area and all are eliminated.
如果所计算的纹理统计特征值有多个，例如有方差和Sobel算子，则各个方向上的分割位置也有多个，此时需要通过比较来确立最终用于划分的分割位置，例如Sobel算子在垂直纹理投影直方图中得到的分割位置为第42列，则对其左右两侧的部分同样进行阴影候选区占前景部分比例的计算，例如，得到左侧所占比例为43％，右侧所占比例为20％，在这种情况下，可以发现，在垂直方向上方差所对应的分割位置划分出的两部分阴影候选区比例差异要小于Sobel算子所对应的分割位置划分出的两部分阴影候选区比例差异，从而将Sobel算子所对应的分割位置作为最终用于划分的分割位置，并且将阴影候选区在前景中所占比例较大的部分，即左侧部分作为阴影部分剔除。在水平方向上可以通过同样的方式确定分割位置及阴影。If there are multiple texture statistical feature values calculated, such as variance and Sobel operator, there are also multiple segmentation positions in each direction. At this time, it is necessary to establish the final segmentation position for division through comparison, such as Sobel operator The segmentation position obtained in the vertical texture projection histogram is the 42nd column, and the proportion of the shadow candidate area to the foreground part is also calculated for the parts on the left and right sides. For example, the proportion of the left side is 43%, and the proportion of the right side is 43%. The proportion is 20%. In this case, it can be found that the difference in the ratio of the two parts of the shadow candidate area divided by the segmentation position corresponding to the difference in the vertical direction is smaller than that of the two parts divided by the segmentation position corresponding to the Sobel operator. The proportion difference of the partial shadow candidate area, so that the segmentation position corresponding to the Sobel operator is used as the final segmentation position for division, and the part of the shadow candidate area with a large proportion in the foreground, that is, the left part is removed as the shadow part . In the horizontal direction, the division position and shadow can be determined in the same way.
如果是基于像素点的亮度、色度以及饱和度信息来判定阴影候选区的，同理，例如得到的垂直分割位置左侧，假设原本有XL个像素点为前景像素点，而被判定为阴影候选区的有YL个像素点，则左侧的阴影占前景像素点PL＝YL/XL；同样，分割位置右侧的前景像素点个数记为XR，被判定为阴影候选区的像素点个数记为YR，右侧的阴影候选区占前景像素点比例为PR＝YR/XR。比较PL和PR的大小，概率较大的一侧判定为阴影，进行去除；所占比例较小的一侧判定为运动物体，予以保留。如果单个投影方向上分割位置有多个，同样也通过先比较分割位置所划分的两部分阴影所占比例差异进行比较来确定分割位置，两部分差异较大的分割位置作为最终的分割位置，再进行阴影的剔除。If the shadow candidate area is determined based on the brightness, chroma, and saturation information of the pixels, in the same way, for example, on the left side of the obtained vertical segmentation position, assuming that there are originally XL pixels that are foreground pixels, they are determined as There are YL pixels in the shadow candidate area, then the shadow on the left occupies the foreground pixel PL =YL /XL ; similarly, the number of foreground pixels on the right side of the segmentation position is recorded as XR , and it is judged as a shadow The number of pixels in the candidate area is denoted as YR , and the ratio of the shadow candidate area on the right to the foreground pixels is PR =YR /XR . Comparing the size ofPL andPR , the side with higher probability is judged as shadow and removed; the side with smaller proportion is judged as moving object and is retained. If there are multiple division positions in a single projection direction, the division position is also determined by comparing the proportion difference of the two parts of the shadow divided by the division position first, and the division position with a large difference between the two parts is taken as the final division position, and then Remove shadows.
本发明的方法及装置突出效果在于，能够有效地降低前景检测结果中阴影的误检率，避免将非阴影区域误检为阴影区域，提高阴影识别准确率，从而有助于后续处理效率的提高；不需要针对具体场景设定具体的门限值，从而提高了方法适用的普遍性；结合多个纹理统计特征值综合考虑，可以提高阴影区判定的准确性。The outstanding effect of the method and device of the present invention is that it can effectively reduce the false detection rate of shadows in the foreground detection results, avoid misdetection of non-shadowed areas as shadowed areas, improve the accuracy of shadow recognition, and thus contribute to the improvement of subsequent processing efficiency ; There is no need to set a specific threshold value for a specific scene, thereby improving the universality of the method; combined with the comprehensive consideration of multiple texture statistical feature values, the accuracy of shadow area determination can be improved.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510679083.6ACN105261021B (en) | 2015-10-19 | 2015-10-19 | Method and device for removing shadow from foreground detection result |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510679083.6ACN105261021B (en) | 2015-10-19 | 2015-10-19 | Method and device for removing shadow from foreground detection result |
| Publication Number | Publication Date |
|---|---|
| CN105261021Atrue CN105261021A (en) | 2016-01-20 |
| CN105261021B CN105261021B (en) | 2019-03-08 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510679083.6AActiveCN105261021B (en) | 2015-10-19 | 2015-10-19 | Method and device for removing shadow from foreground detection result |
| Country | Link |
|---|---|
| CN (1) | CN105261021B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127744A (en)* | 2016-06-17 | 2016-11-16 | 广州市幸福网络技术有限公司 | Display foreground and background border Salience estimation and system |
| CN106339995A (en)* | 2016-08-30 | 2017-01-18 | 电子科技大学 | Space-time multiple feature based vehicle shadow eliminating method |
| CN107168049A (en)* | 2017-05-19 | 2017-09-15 | 浙江工业大学 | Method for acquiring photovoltaic array output characteristic curve in real time |
| CN110659687A (en)* | 2019-09-24 | 2020-01-07 | 华南农业大学 | Yellow trapping plate-based phyllotreta striolata detection method, medium and equipment |
| CN111161299A (en)* | 2018-11-08 | 2020-05-15 | 深圳富泰宏精密工业有限公司 | Image segmentation method, computer program, storage medium, and electronic device |
| CN113192101A (en)* | 2021-05-06 | 2021-07-30 | 影石创新科技股份有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN113487502A (en)* | 2021-06-30 | 2021-10-08 | 中南大学 | Shadow removing method for hollow image |
| CN113870237A (en)* | 2021-10-09 | 2021-12-31 | 西北工业大学 | Composite material image shadow detection method based on horizontal diffusion |
| WO2023284313A1 (en)* | 2021-07-16 | 2023-01-19 | 稿定（厦门）科技有限公司 | Automatic slicing method and device for psd picture |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6999620B1 (en)* | 2001-12-10 | 2006-02-14 | Hewlett-Packard Development Company, L.P. | Segmenting video input using high-level feedback |
| CN101098462A (en)* | 2007-07-12 | 2008-01-02 | 上海交通大学 | Video Moving Object Detection Method Combining Chroma Deviation and Luminance Deviation |
| CN101324927A (en)* | 2008-07-18 | 2008-12-17 | 北京中星微电子有限公司 | Method and apparatus for detecting shadows |
| CN102184553A (en)* | 2011-05-24 | 2011-09-14 | 杭州华三通信技术有限公司 | Moving shadow detecting method and device |
| CN102298781A (en)* | 2011-08-16 | 2011-12-28 | 长沙中意电子科技有限公司 | Motion shadow detection method based on color and gradient characteristics |
| CN103810722A (en)* | 2014-02-27 | 2014-05-21 | 云南大学 | Moving target detection method combining improved LBP (Local Binary Pattern) texture and chrominance information |
| CN104537695A (en)* | 2015-01-23 | 2015-04-22 | 贵州现代物流工程技术研究有限责任公司 | Anti-shadow and anti-covering method for detecting and tracing multiple moving targets |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6999620B1 (en)* | 2001-12-10 | 2006-02-14 | Hewlett-Packard Development Company, L.P. | Segmenting video input using high-level feedback |
| CN101098462A (en)* | 2007-07-12 | 2008-01-02 | 上海交通大学 | Video Moving Object Detection Method Combining Chroma Deviation and Luminance Deviation |
| CN101324927A (en)* | 2008-07-18 | 2008-12-17 | 北京中星微电子有限公司 | Method and apparatus for detecting shadows |
| CN102184553A (en)* | 2011-05-24 | 2011-09-14 | 杭州华三通信技术有限公司 | Moving shadow detecting method and device |
| CN102298781A (en)* | 2011-08-16 | 2011-12-28 | 长沙中意电子科技有限公司 | Motion shadow detection method based on color and gradient characteristics |
| CN103810722A (en)* | 2014-02-27 | 2014-05-21 | 云南大学 | Moving target detection method combining improved LBP (Local Binary Pattern) texture and chrominance information |
| CN104537695A (en)* | 2015-01-23 | 2015-04-22 | 贵州现代物流工程技术研究有限责任公司 | Anti-shadow and anti-covering method for detecting and tracing multiple moving targets |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127744A (en)* | 2016-06-17 | 2016-11-16 | 广州市幸福网络技术有限公司 | Display foreground and background border Salience estimation and system |
| CN106127744B (en)* | 2016-06-17 | 2019-05-14 | 广州市幸福网络技术有限公司 | Display foreground and background border Salience estimation and system |
| CN106339995A (en)* | 2016-08-30 | 2017-01-18 | 电子科技大学 | Space-time multiple feature based vehicle shadow eliminating method |
| CN107168049A (en)* | 2017-05-19 | 2017-09-15 | 浙江工业大学 | Method for acquiring photovoltaic array output characteristic curve in real time |
| CN107168049B (en)* | 2017-05-19 | 2019-11-29 | 浙江工业大学 | Method for acquiring photovoltaic array output characteristic curve in real time |
| CN111161299A (en)* | 2018-11-08 | 2020-05-15 | 深圳富泰宏精密工业有限公司 | Image segmentation method, computer program, storage medium, and electronic device |
| CN111161299B (en)* | 2018-11-08 | 2023-06-30 | 深圳富泰宏精密工业有限公司 | Image segmentation method, storage medium and electronic device |
| CN110659687A (en)* | 2019-09-24 | 2020-01-07 | 华南农业大学 | Yellow trapping plate-based phyllotreta striolata detection method, medium and equipment |
| CN113192101A (en)* | 2021-05-06 | 2021-07-30 | 影石创新科技股份有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN113192101B (en)* | 2021-05-06 | 2024-03-29 | 影石创新科技股份有限公司 | Image processing method, device, computer equipment and storage medium |
| CN113487502A (en)* | 2021-06-30 | 2021-10-08 | 中南大学 | Shadow removing method for hollow image |
| CN113487502B (en)* | 2021-06-30 | 2022-05-03 | 中南大学 | A shadow removal method for pothole images |
| WO2023284313A1 (en)* | 2021-07-16 | 2023-01-19 | 稿定（厦门）科技有限公司 | Automatic slicing method and device for psd picture |
| CN113870237A (en)* | 2021-10-09 | 2021-12-31 | 西北工业大学 | Composite material image shadow detection method based on horizontal diffusion |
| CN113870237B (en)* | 2021-10-09 | 2024-03-08 | 西北工业大学 | Composite material image shadow detection method based on horizontal diffusion |
| Publication number | Publication date |
|---|---|
| CN105261021B (en) | 2019-03-08 |
| Publication | Publication Date | Title |
|---|---|---|
| CN105261021A (en) | Method and apparatus of removing foreground detection result shadows | |
| CN110929560B (en) | Video semi-automatic target labeling method integrating target detection and tracking | |
| US10255691B2 (en) | Method and system of detecting and recognizing a vehicle logo based on selective search | |
| CN101770583B (en) | Template matching method based on global features of scene | |
| CN104732211B (en) | A kind of method for traffic sign detection based on adaptive threshold | |
| US20200250840A1 (en) | Shadow detection method and system for surveillance video image, and shadow removing method | |
| CN103903018A (en) | Method and system for positioning license plate in complex scene | |
| CN105761266A (en) | Method of extracting rectangular building from remote sensing image | |
| CN104966285B (en) | A kind of detection method of salient region | |
| CN103810716B (en) | Move and the image partition method of Renyi entropy based on gray scale | |
| CN103119625B (en) | Method and device for video character segmentation | |
| CN104599511B (en) | Traffic flow detection method based on background modeling | |
| CN102768731A (en) | System and method for automatic positioning and recognition of targets based on high-definition video images | |
| CN114299002A (en) | An intelligent detection system and method for abnormal behavior of pavement throwing | |
| CN106815587B (en) | Image processing method and device | |
| CN104715252A (en) | License plate character segmentation method with combination of dynamic template and pixel points | |
| CN103955949A (en) | Moving target detection method based on Mean-shift algorithm | |
| CN104657980A (en) | Improved multi-channel image partitioning algorithm based on Meanshift | |
| EP3584763A1 (en) | Vehicle-mounted environment recognition device | |
| CN111753749A (en) | A Lane Line Detection Method Based on Feature Matching | |
| CN101114337A (en) | A method for identifying and locating buildings on the ground | |
| CN108010044A (en) | A kind of method of video boundaries detection | |
| CN102004924A (en) | Human head detection system and method | |
| CN103106409A (en) | Composite character extraction method aiming at head shoulder detection | |
| CN104463134A (en) | License plate detection method and system |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |