CN104463853A

Movatterモバイル変換

Info

Publication number: CN104463853A
Application number: CN201410675195.XA
Authority: CN
Inventors: 刘颜丽; 陈卓
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2014-11-22
Filing date: 2014-11-22
Publication date: 2015-03-25

Abstract

Translated fromChinese

本发明公开了一种基于图像分割的阴影检测与去除算法，其涉及图像处理技术领域，主要解决如何判断一个区域是否为阴影或是一条边缘是否为阴影以及相应阴影的如何去除的问题。首先利用纹理和亮度特征，结合局部信息和全局信息估计每个像素点是阴影边缘的概率；使用分水岭算法利用轮廓信息分割图像；利用基于边缘的区域融合算法将图像中阴影区域和非阴影区域分割开来，同时分别把阴影区域和非阴影区域分割成若干个子区域；然后分别训练一个分类器SVM，对阴影进行识别；随后使用图割算法求解检测阴影的能量方程，得到最终的阴影检测结果；最后根据阴影检测的结果，使用抠图算法计算阴影标签，利用标签点亮阴影区域，恢复阴影区域的光照。

The invention discloses a shadow detection and removal algorithm based on image segmentation, which relates to the technical field of image processing and mainly solves the problems of how to judge whether a region is a shadow or whether an edge is a shadow and how to remove the corresponding shadow. First, use texture and brightness features, combine local information and global information to estimate the probability that each pixel is a shadow edge; use the watershed algorithm to utilize contour information Segment the image; use the edge-based area fusion algorithm to separate the shadow area and non-shadow area in the image, and divide the shadow area and non-shadow area into several sub-areas; then train a classifier SVM to identify the shadow ; Then use the graph cut algorithm to solve the energy equation for shadow detection to get the final shadow detection result; finally, according to the shadow detection result, use the matting algorithm to calculate the shadow label, use the label to lighten the shadow area, and restore the illumination of the shadow area.

Description

Translated fromChinese

一种基于图像分割的阴影检测与去除算法A Shadow Detection and Removal Algorithm Based on Image Segmentation

技术领域technical field

本发明具体涉及一种基于图像分割的阴影检测与去除算法，用于图像阴影检测和半影区域去除，属于图像处理技术领域。The invention specifically relates to a shadow detection and removal algorithm based on image segmentation, which is used for image shadow detection and penumbra area removal, and belongs to the technical field of image processing.

背景技术Background technique

阴影检测一直是图像处理领域的研究热点之一。由于阴影的存在会增加物体识别、视频分割等算法的难度，将阴影检测出来并将其去除可显著提高图像处理领域许多算法的性能。Shadow detection has always been one of the research hotspots in the field of image processing. Since the existence of shadows will increase the difficulty of algorithms such as object recognition and video segmentation, detecting and removing shadows can significantly improve the performance of many algorithms in the field of image processing.

很多阴影检测的方法是基于光照模型，或者颜色模型提出的。如在颜色空间HSI下的阴影检测方法，该方法利用HSI空间中H和I的比值来检测阴影。但该方法更适合航拍图或者阴影比较明显的图像，针对复杂场景下的阴影检测效果则差强人意。但是，由于不能很好地判断一个像素点属于阴影还是属于颜色比较暗的非阴影，自动检测阴影的方法仍然是阴影检测的一大挑战。Many shadow detection methods are proposed based on illumination models or color models. Such as the shadow detection method under the color space HSI, this method uses the ratio of H and I in the HSI space to detect shadows. However, this method is more suitable for aerial images or images with obvious shadows, and the effect of shadow detection in complex scenes is not satisfactory. However, since it is impossible to judge whether a pixel belongs to a shadow or a non-shadow with a darker color, the method of automatically detecting shadows is still a big challenge for shadow detection.

迄今为止，图像处理领域已经存在大量的阴影检测算法。根据检测手段的不同，可将现有的阴影检测算法分为基于边缘的阴影检测和基于学习的阴影检测。So far, there have been a large number of shadow detection algorithms in the field of image processing. According to different detection means, existing shadow detection algorithms can be divided into edge-based shadow detection and learning-based shadow detection.

基于边缘的阴影检测算法首先需要场景彩色图和光照不变的灰度图，灰度图由校准的摄像机获得，通过比较灰度图的边缘和原图像的边缘来检测阴影(G.D.Finlayson,2006)。该方法对高质量图像有着极佳的处理效果，而对普通图像效果一般。基于学习的阴影检测算法则考虑到阴影边缘的复杂性，转而从经验的角度将数据驱动的方法引入到阴影检测，如基于光照强度、梯度等信息利用条件随机场(Conditional random field,CRF)，判断一个区域是否为阴影；或是利用CRF来判断一条边缘是否为阴影。虽然这种利用条件随机场CRF的方法可以在一定条件下很好地检测阴影，但它训练过程冗长，并对训练集的依赖度较高。时至今日，在光照、物体反射率和阴影几何形状等因素的影响下，阴影检测仍然是一个非常具有挑战性的问题。The edge-based shadow detection algorithm first needs the scene color image and the grayscale image with constant illumination. The grayscale image is obtained by a calibrated camera, and the shadow is detected by comparing the edge of the grayscale image with the edge of the original image (G.D.Finlayson, 2006) . This method has an excellent processing effect on high-quality images, but not so much on ordinary images. The learning-based shadow detection algorithm takes into account the complexity of shadow edges, and instead introduces data-driven methods into shadow detection from an empirical point of view, such as using Conditional Random Field (CRF) based on information such as light intensity and gradient. , to judge whether an area is a shadow; or use CRF to judge whether an edge is a shadow. Although this method using conditional random field (CRF) can detect shadows well under certain conditions, it has a lengthy training process and a high degree of dependence on the training set. Today, shadow detection is still a very challenging problem under the influence of factors such as lighting, object reflectivity, and shadow geometry.

发明内容Contents of the invention

针对上述现有技术，本发明的目的在于如何提供一种基于图像分割的阴影检测与去除算法，其可以有效地分割阴影和非阴影区域，还可以更好地检测和去除场景中的自阴影和投射阴影。For the prior art above, the purpose of the present invention is how to provide a shadow detection and removal algorithm based on image segmentation, which can effectively segment shadow and non-shadow areas, and can also better detect and remove self-shadows and shadows in the scene. Cast shadows.

为了解决上述技术问题，本发明采用如下技术方案：In order to solve the above technical problems, the present invention adopts the following technical solutions:

一种基于图像分割的阴影检测与去除算法，其特征在于，包括如下步骤：A kind of shadow detection and removal algorithm based on image segmentation, it is characterized in that, comprises the steps:

S100：利用纹理和亮度特征，结合局部信息和全局信息估计每个像素点是阴影边缘的概率；S100: Using texture and brightness features, combining local information and global information, to estimate the probability that each pixel is a shadow edge;

S200：使用分水岭算法利用轮廓信息gPb分割图像； S200: Use the watershed algorithm to segment the image using the contour information gPb;

S300：利用基于边缘的区域融合算法将图像中阴影区域和非阴影区域分割开来，同时分别把阴影区域和非阴影区域分割成若干个子区域；然后，利用单个区域信息和匹配区域的信息，分别训练一个分类器SVM，对阴影进行识别；随后使用图割算法求解检测阴影的能量方程，得到最终的阴影检测结果。S300: Use an edge-based area fusion algorithm to separate the shadow area and the non-shadow area in the image, and at the same time divide the shadow area and the non-shade area into several sub-areas; then, use the information of a single area and the information of the matching area, respectively Train a classifier SVM to identify shadows; then use the graph cut algorithm to solve the energy equation for detecting shadows to obtain the final shadow detection results.

S400，根据阴影检测的结果，使用抠图算法计算阴影标签，利用得到的标签点亮阴影区域，恢复阴影区域的光照，使其与周围非阴影的区域光照相同。S400. According to the result of the shadow detection, use a matting algorithm to calculate a shadow label, use the obtained label to light up the shadow area, and restore the illumination of the shadow area to make it the same as that of the surrounding non-shaded area.

所述步骤S100主要以下几步组成：The step S100 mainly consists of the following steps:

S101：通过计算方向梯度信息G(x,y,θ)构建阴影边缘检测器Pb，阴影边缘检测器分别计算亮度和纹理基元两个通道的梯度信息G(x,y,θ)，阴影边缘检测器的构建方法是在图像中一点(x,y)为中心，以r为半径画一圆，该圆被方向为θ的直径分割为两个半圆；S101: Construct the shadow edge detector Pb by calculating the direction gradient information G(x, y, θ). The construction method of the detector is to draw a circle with a radius of r at a point (x, y) in the image, and the circle is divided into two semicircles by a diameter whose direction is θ;

S102：通过计算这两个半圆的柱状图之间的χ²距离得到方向梯度G：S102: Obtain the directional gradient G by calculating the^χ distance between the histograms of the two semicircles:

${χ χ}^{22} ((g g,, h h)) = = \frac{11}{22} \underset{i i}{Σ Σ} \frac{{((g g ((i i)) - - h h ((i i))))}^{22}}{g g ((i i)) + + h h ((i i))} - - - - - - ((11))$

其中，g和h代表两个半圆，i代表图像值域上的值；Among them, g and h represent two semicircles, and i represents the value on the image range;

S103：将计算得到的检测器Pb结合起来，得到不同尺度、不同通道组合起来的局部信息(local cues)mPb：S103: Combine the calculated detectors Pb to obtain local information (local cues) mPb of different scales and different channels:

$mPb mPb ((x x,, yθ yθ)) = = \underset{s the s}{Σ Σ} \underset{i i}{Σ Σ} {α α}_{i i,, s the s} {G G}_{i i,, σ σ ((i i,, s the s))} ((x x,, y the y,, θ θ)) - - - - - - ((22))$

其中，s代表圆的半径，i代表特征通道(亮度、纹理基元)；G_i,σ(i,s)(x,y,θ)比较了以(x,y)为中心、σ(i,s)为半径大小、θ为半径方向的两个半圆之间的差异；α_i,s则是各个梯度信息所代表的权重；Among them, s represents the radius of the circle, i represents the feature channel (brightness, texture primitive); G_i,σ(i,s) (x,y,θ) compares the center of (x,y), ,s) is the radius size, θ is the difference between two semicircles in the radius direction; α_i,s is the weight represented by each gradient information;

在每个点上，选取梯度信息G的最大值作为该点的mPb值，该值表示最终的局部信息：At each point, the maximum value of the gradient information G is selected as the mPb value of the point, which represents the final local information:

$mPb mPb ((x x,, y the y,, θ θ)) = = \underset{θ θ}{max max} {{mPb mPb ((x x,, y the y,, θ θ))}} - - - - - - ((33))$

步骤S104：以mPb构建一个稀疏矩阵，通过计算该矩阵的特征值和特征向量，得到所需的全局信息；稀疏矩阵是在一个半径r＝5像素的区域内，将每个像素链接起来：Step S104: Construct a sparse matrix with mPb, and obtain the required global information by calculating the eigenvalues and eigenvectors of the matrix; the sparse matrix is in an area with a radius of r=5 pixels, linking each pixel:

${W W}_{ij ij} = = exp exp ((- - \underset{p p &Element; &Element; \overset{&OverBar; &OverBar;}{ij ij}}{max max} \frac{{{mPb mPb ((p p))}}}{ρ ρ})) - - - - - - ((44))$

其中，代表的是i和j之间的联系，ρ＝0.1，接着定义D_ii＝∑_jW_ij,通过in, represents the connection between i and j, ρ=0.1, then define D_ii =∑_j W_ij , through

(D-W)v＝λDv (5)(D-W)v=λDv (5)

计算得到特征向量{v₀,v₁,…,v_n}，和特征值0＝λ₀≤λ₁≤…≤λ_n。Eigenvectors {v₀ , v₁ ,...,v_n } and eigenvalues 0=λ₀ ≤λ₁ ≤...≤λ_n are obtained through calculation.

步骤S105：将步骤S104中的每一个特征向量看做一副图像，通过计算不同方向下的高斯滤波，得到方向信息然后，将不同特征向量下的方向信息线性叠加起来得到全局信息sPb：Step S105: Treat each feature vector in step S104 as an image, and obtain direction information by calculating Gaussian filtering in different directions Then, the direction information under different eigenvectors is linearly superimposed to obtain the global information sPb:

$sPb sPb ((x x,, y the y,, θ θ)) = = {Σ Σ}_{k k = = 11}^{n no} \frac{11}{\sqrt{{λ λ}_{k k}}} \cdot &Center Dot; {&dtri; &dtri;}_{θ θ} {v v}_{k k} ((x x,, y the y)) - - - - - - ((66))$

步骤S106：将局部信息mPb和全局信息sPb有机地结合起来分析图像轮廓信息gPb：Step S106: Organically combine local information mPb and global information sPb to analyze image contour information gPb:

$gPb PPML ((x x,, y the y,, θ θ)) = = \underset{s the s}{Σ Σ} \underset{i i}{Σ Σ} {{{β β}_{i i,, s the s} {G G}_{i i,, σ σ} ((x x,, y the y,, θ θ)) + + γ γ \cdot \cdot sPb sPb ((x x,, y the y,, θ θ))}} - - - - - - ((77))$

其中，β_i,s和γ分别表示mPb和sPb的系数。Among them, βi_{, s} and γ represent the coefficients of mPb and sPb, respectively.

所述步骤S200主要以下几步组成：The step S200 mainly consists of the following steps:

步骤S201：估计出图像中任意一点(x,y)在方向θ上是轮廓的概率，求出该点轮廓检测的最大值：Step S201: Estimate the probability that any point (x, y) in the image is a contour in the direction θ, and find the maximum value of contour detection at this point:

$E E. ((x x,, y the y)) = = \underset{θ θ}{max max} E E. ((x x,, y the y,, θ θ)) - - - - - - ((88))$

步骤S202：利用数学形态学，以区域中E(x,y)的最小值为“集水盆地”计算每个区域，每个“集水盆地”对应一个区域，记为P₀；两个集水盆地交汇处是“分水岭”，记为K₀；Step S202: Using mathematical morphology, calculate each area with the minimum value of E(x, y) in the area as the "catchment basin", and each "catchment basin" corresponds to an area, which is recorded as P₀ ; two sets The intersection of water basins is the "watershed", denoted as K₀ ;

步骤S203：分水岭算法会产生过度分割问题，即将本不应该是边的地方标记为分水岭，利用区域合并算法解决过度分割问题；Step S203: the watershed algorithm will cause over-segmentation problem, that is, mark the place that should not be an edge as a watershed, and use the region merging algorithm to solve the over-segmentation problem;

所述区域合并算法如下：定义一个无向图G＝(P₀,K₀,W(K₀),E(P₀))，其中W(K₀)表示每条分水岭的权值，由分水岭上点的总能量除以分水岭上点的个数而得，E(P₀)表示每个集水盆地的能量值，每个盆地的初始能量均为零，W(K₀)描述了相邻两个区域之间的相异性；将分水岭按照其权值，由小到大存入队列。The region merging algorithm is as follows: define an undirected graph G=(P₀ , K₀ , W(K₀ ), E(P₀ )), where W(K₀ ) represents the weight of each watershed, and the watershed The total energy of the upper point is divided by the number of points on the watershed, E(P₀ ) represents the energy value of each catchment basin, and the initial energy of each basin is zero, W(K₀ ) describes the adjacent The dissimilarity between the two areas; the watershed is stored in the queue according to its weight, from small to large.

区域合并算法包括以下步骤为：The region merging algorithm includes the following steps:

一、找到权重最小的边C^*＝argminW(C)1. Find the edge with the smallest weight C^* ＝argminW(C)

假定R₁和R₂由边C^*分割，且R＝R₁∪R₂，如果min{E(R₁),E(R₂)}≠0，判断是否合并，合并条件为：则W(K₀)≤τ·min{E(R₁),E(R₂)} (9) Suppose R₁ and R₂ are separated by side C^* , and R=R₁ ∪R₂ , if min{E(R₁ ),E(R₂ )}≠0, judge whether to merge, the merge condition is: then W( K₀ )≤τ·min{E(R₁ ),E(R₂ )} (9)

或min{E(R₁),E(R₂)}＝0 (10)，or min{E(R₁ ),E(R₂ )}=0 (10),

其中τ为一个常数；where τ is a constant;

二、若合并，则更新E(R)、P₀和K₀，E(R)、P₀和K₀的更新方法是：2. If combined, then update E(R), P₀ and K₀ , the update method of E(R), P₀ and K₀ is:

E(R)＝max{E(R₁),E(R₂),W(C^*)} (11) E(R)=max{E(R₁ ),E(R₂ ),W(C^* )} (11)

P₀←P₀\{R₁,R₂}∪R (12) P₀ ←P₀ \{R₁ ,R₂ }∪R (12)

K₀←K₀\{C^*} (13) K₀ ←K₀ \{C^* } (13)

更地一步地，通过调整τ调整合并条件，进面控制最终区域的大小，τ越大最终合并的区域面积越大。Furthermore, adjust the merging conditions by adjusting τ, and control the size of the final region. The larger τ is, the larger the area of the final merging region will be.

所述步骤S300中通过解下述能量方程得到最终的阴影检测结果，能量方程是由图割算法解得：In the step S300, the final shadow detection result is obtained by solving the following energy equation, and the energy equation is solved by the graph cut algorithm:

$\begin{matrix} \overset{^^}{y the y} = = arg arg \underset{y the y}{min min} \underset{k k}{Σ Σ} {cos cos t t}_{k k}^{unary unary} (({y the y}_{k k})) \\ + + {α α}_{22} \underset{{{i i,, j j}} &Element; &Element; {E E.}_{same the same}}{Σ Σ} {c c}_{ij ij}^{same the same} 11 (({y the y}_{i i} &NotEqual; &NotEqual; {y the y}_{j j})) \end{matrix} - - - - - - ((1515))$

同时，at the same time,

$\begin{matrix} {cos cos t t}_{k k}^{unary unary} (({y the y}_{k k})) = = - - {c c}_{k k}^{shadow shadow} {y the y}_{k k} \\ - - {α α}_{11} \underset{{{i i = = k k,, j j}} &Element; &Element; {E E.}_{diff diff}}{Σ Σ} {c c}_{ij ij}^{diff diff} {y the y}_{k k} \\ + + {α α}_{11} \underset{{{i i,, j j = = k k}} &Element; &Element; {E E.}_{diff diff}}{Σ Σ} {c c}_{ij ij}^{diff diff} {y the y}_{k k} \end{matrix} - - - - - - ((1616))$

其中，表示区域匹配分类器对于两区域光照相同的估计，表示区域匹配分类器对于两区域光照不同的估计，是单区域分类器对单区域区域是否为阴影的估计，{i,j}∈E_same表示相同光照的两个区域，{i,j}∈E_diff表示不同光照的两个区域；y＝{-1,1}ⁿ，当为1时表示该区域是阴影区域。in, Indicates that the area matching classifier estimates that the illumination of the two areas is the same, Indicates the estimation of the area matching classifier for the different illumination of the two areas, is the single-region classifier’s estimate of whether the single-region region is a shadow, {i,j}∈E_same represents two regions with the same illumination, {i,j}∈E_diff represents two regions with different illumination; y={ -1,1}ⁿ , when it is 1, it means that the area is a shadow area.

对所述步骤S400更进一步地描述为：使用抠图算法计算阴影标签，该算法认为一个图像I_i可由前景F_i和背景B_i混合而成，其公式如下：The step S400 is further described as: use the matting algorithm to calculate the shadow label, and the algorithm believes that an image I_i can be formed by mixing the foreground F_i and the background B_i , and the formula is as follows:

I_i＝k_iF_i+(1-k_i)B_i (18) I_i =k_i F_i +(1-k_i )B_i (18)

＝k_i(L_dR_i+L_eR_i)+(1-k_i)L_eR_i (19)=k_i (L_d R_i +L_e R_i )+(1-k_i )L_e R_i (19)

其中，L_d是直射光、L_e是环境光、k_i是阴影标签、R_i是点i的反射系数。 where L_d is the direct light, L_e is the ambient light, ki_is the shadow label, and R_i is the reflection coefficient of point i.

将前景标记为非阴影，背景标记为阴影，通过计算下述能量方程的最小值得到k_i的大小，k_i表示点i的标签，由(20)得到，k是k_i组成的向量，得到k就得到了k_i。Mark the foreground as non-shaded and the background as shadowed, and obtain the size of_ki by calculating the minimum value of the following energy equation,_ki represents the label of point i, obtained from (20), k is a vector composed of_ki , and k gets k_i .

$E E. ((k k)) = = {k k}^{T T} Lk Lk + + λ λ {((k k - - \overset{^^}{k k}))}^{T T} D D. ((k k - - \overset{^^}{k k})) - - - - - - ((2020))$

k^T是k的转置，k是k_i组成的向量，λ是一个很大的数值，具体由实践而定。是由步骤S300的能量方程得到的标签y_k组成的向量，里每个元素的值就是公式里得到的y_k，这里每一个元素值不是0就是1。请注意：该公式就是为了计算k，计算得到的k是一个向量，向量里的每个元素表示每个像素的标签；但是每个元素的值的取值范围变为[0、1]，也就是说一部分像素的标签由1或0变为了一个0到1区间范围的值；标签仍为0或1的就表示阴影区域和非阴影区域，标签值在0和1之间的表示半影区域。k^T is the transpose of k, k is the vector composed of k_i , and λ is a large value, which depends on practice. is a vector consisting of labels y_k obtained from the energy equation in step S300, The value of each element in is the y_k obtained in the formula, where the value of each element is either 0 or 1. Please note: this formula is to calculate k, the calculated k is a vector, each element in the vector represents the label of each pixel; but the value range of each element becomes [0, 1], also That is to say, the label of some pixels has changed from 1 or 0 to a value ranging from 0 to 1; if the label is still 0 or 1, it represents the shadow area and non-shadow area, and if the label value is between 0 and 1, it represents the penumbra area .

其中，L是抠图的拉普拉斯矩阵，D(i,i)是一个对角阵，D(i,i)＝1表示像素i是阴影区域的边缘，D(i,i)＝0则表示其它所有点；Among them, L is the Laplacian matrix of matting, D(i,i) is a diagonal matrix, D(i,i)=1 means that pixel i is the edge of the shadow area, D(i,i)=0 then represent all other points;

利用得到的标签点亮阴影区域，恢复阴影区域的光照：Use the obtained label to lighten the shadow area and restore the lighting of the shadow area:

根据阴影模型，如果一个像素被点亮，则According to the shadow model, if a pixel is lit, then

$\begin{matrix} {I I}_{i i}^{shadow shadow - - free free} = = (({L L}_{d d} + + {L L}_{e e})) {R R}_{i i} & ((21 twenty one)) \\ = = (({k k}_{i i} \cdot &Center Dot; {L L}_{d d} + + {L L}_{e e})) {R R}_{i i} \frac{(({L L}_{d d} + + {L L}_{e e}))}{(({k k}_{i i} \cdot &Center Dot; {L L}_{d d} + + {L L}_{e e}))} & ((22 twenty two)) \\ = = \frac{r r + + 11}{{k k}_{i i} r r + + 11} {I I}_{i i} & ((23 twenty three)) \end{matrix}$

其中，r＝L_d/L_e是直射光L_d和环境光L_e的比值，I_i表示像素i原本的值，所以，计算得到r就可将阴影去除；Among them, r=L_d /L_e is the ratio of the direct light L_d to the ambient light L_e , and I_i represents the original value of the pixel i, so the shadow can be removed by calculating r;

已知A known

I_i＝(k_i·L_d+L_e)R_i (24) I_i =(k_i ·L_d +L_e )R_i (24)

I_j＝(k_j·L_d+L_e)R_j (25) I_j ＝(k_j ·L_d +L_e )R_j (25)

如果两个像素点的反射系数相同，即R_i＝R_j，则If the reflection coefficients of two pixels are the same, that is, R_i =R_j , then

$r r = = \frac{{I I}_{j j} - - {I I}_{i i}}{{k k}_{j j} {I I}_{i i} - - {k k}_{i i} {I I}_{j j}}$

公式(24)和(25)的目的是计算r，计算该值的思想就是，找到材质相同(即反射系数相同，R_i＝R_j)、光照不同同(L_d和L_e一样，但阴影标签k不同)的两个点i,j，利用两者之间的数学关系就可以得到r。I_j就表示这样一个点j的像素值。_The purpose of formulas (24) and (25₎ is to calculate r_._The idea of calculating this value is to find For two points i, j with different labels k), r can be obtained by using the mathematical relationship between the two. I_j represents the pixel value of such a point j.

与现有技术相比，本发明具有以下有益效果：Compared with the prior art, the present invention has the following beneficial effects:

一、利用相同的阴影特征将分割图像和检测阴影有机地结合起来，很好地分割了图像中的阴影和非阴影区域，使图像分割更加准确，阴影区域检测更准确、效果更好；1. Using the same shadow feature to organically combine image segmentation and shadow detection, the shadow and non-shadow areas in the image are well segmented, making image segmentation more accurate, and shadow area detection more accurate and better;

二、较完整地检测到了阴影的轮廓，本发明提供的检测算法对复杂背景的抗干扰能力较好，去除的图像不仅在阴影区域内部保存了较好的纹理特征，而且在阴影边缘处较为光滑，表明本文算法在半影区域很好地实现了光照过渡效果。2. The outline of the shadow is detected relatively completely. The detection algorithm provided by the present invention has better anti-interference ability to complex backgrounds. The removed image not only preserves better texture features inside the shadow area, but also is relatively smooth at the edge of the shadow , indicating that the algorithm in this paper can well realize the light transition effect in the penumbra area.

三、根据轮廓信息，提出了一种区域合并的算法，并且可以通过控制区域合并的参数，方便地控制合并区域的大小，这些优点共同提升了阴影去除的效果。3. According to the contour information, a region merging algorithm is proposed, and the size of the merging region can be conveniently controlled by controlling the parameters of the region merging. These advantages jointly improve the effect of shadow removal.

附图说明Description of drawings

图1为本发明的算法流程图；Fig. 1 is the algorithm flowchart of the present invention;

(a)为原图像；(b)为分割后的图像；(c)为单区域识别器检测的阴影和非阴影区域，白色表示阴影区域；(d)为匹配区域识别器检测的阴影和非阴影区域；(e)为经过抠图算法计算得到的阴影标签k_i的灰度图；(f)为阴影去除后的图像。(a) is the original image; (b) is the segmented image; (c) is the shadow and non-shadow area detected by the single-region recognizer, and white represents the shadow region; (d) is the shadow and non-shadow detected by the matching region recognizer The shadow area; (e) is the grayscale image of the shadow label k_i calculated by the matting algorithm; (f) is the image after shadow removal.

图2为区域合并算法流程图；Fig. 2 is a flow chart of the region merging algorithm;

图3为区域匹配图，图中红线表示相同光照的匹配区域，蓝线表示不同光照的匹配区域。Figure 3 is an area matching diagram. The red line in the figure indicates the matching area with the same illumination, and the blue line indicates the matching area with different illumination.

具体实施方式Detailed ways

下面将结合附图及具体实施方式对本发明作进一步的描述。The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

本发明根据阴影的特征，融合阴影边缘检测和阴影区域检测，分割阴影和非阴影区域。算法分为两个步骤：阴影边缘检测和阴影分割。According to the characteristics of the shadow, the invention combines shadow edge detection and shadow area detection to segment shadow and non-shadow areas. The algorithm is divided into two steps: shadow edge detection and shadow segmentation.

实施例阴影边缘检测Example shadow edge detection

首先，通过计算方向梯度信息G(x,y,θ)构建阴影边缘检测器Pb。构建的方法是在图像中一点(x,y)为中心，以r为半径画一圆，该圆被方向为θ的直径分割为两个半圆。最后通过计算这两个半圆的柱状图之间的χ²距离得到方向梯度G：First, the shadow edge detector Pb is constructed by computing the directional gradient information G(x, y, θ). The method of construction is to draw a circle with a radius of r at a point (x, y) in the image, and the circle is divided into two semicircles by a diameter whose direction is θ. Finally, the directional gradient G is obtained by calculating the^χ2 distance between the histograms of these two semicircles:

其中，g和h代表两个半圆，i代表图像值域上的值。Among them, g and h represent two semicircles, and i represents the value on the image range.

检测器分别计算亮度和纹理基元两个通道的梯度信息G(x,y,θ)。纹理基元的计算方式是选择16个方向的奇偶高斯滤波器进行滤波，经过k平均算法聚类的纹理基元分为32组。The detector calculates the gradient information G(x, y, θ) of two channels of brightness and texture primitives respectively. The calculation method of the texture primitive is to select the odd-even Gaussian filter in 16 directions for filtering, and the texture primitives clustered by the k-means algorithm are divided into 32 groups.

其次，将计算得到的检测器Pb结合起来，得到不同尺度、不同通道组合起来的局部信息(local cues)mPb：Secondly, the calculated detectors Pb are combined to obtain local information (local cues) mPb of different scales and different channels:

$mPb mPb ((x x,, y the y,, θ θ)) = = \underset{s the s}{Σ Σ} \underset{i i}{Σ Σ} {α α}_{i i,, s the s} {G G}_{i i,, σ σ ((i i,, s the s))} ((x x,, y the y,, θ θ)) - - - - - - ((22))$

其中，s代表圆的半径，i代表特征通道(亮度，纹理基元)；G_i,σ(i,s)(x,y,θ)比较了以(x,y)为中心、σ(i,s)为半径大小、θ为半径方向的两个半圆之间的差异，α_i,s则是各个梯度信息所代表的权重。Among them, s represents the radius of the circle,_i represents the feature channel (luminance, texture primitive); ,s) is the size of the radius, θ is the difference between the two semicircles in the radial direction, and α_i,s is the weight represented by each gradient information.

在每个点上，选取梯度信息G的最大值作为该点的mPb值，该值表示的最终的局部信息：At each point, the maximum value of the gradient information G is selected as the mPb value of the point, which represents the final local information:

接着，为了更好地提取轮廓信息，使用标准归一化图割的方法获取全局信息(global cues)。以mPb构建一个稀疏矩阵，通过计算该矩阵的特征值和特征向量，得到所需的全局信息。稀疏矩阵是在一个半径r＝5像素的区域内，将每个像素链接起来：Next, in order to better extract contour information, the standard normalized graph cut method is used to obtain global information (global cues). Construct a sparse matrix with mPb, and obtain the required global information by calculating the eigenvalues and eigenvectors of the matrix. The sparse matrix is to link each pixel in an area with a radius of r=5 pixels:

(D-W)v＝λDv (5)(D-W)v=λDv (5)

虽然归一化图割不能很好地分割图像，但却可以很好地反映图像的轮廓。所以，将每一个特征向量看做一副图像，通过计算不同方向下的高斯滤波，得到方向信息然后，将不同特征向量下的方向信息线性叠加起来：Although the normalized graph cut cannot segment the image very well, it can reflect the contour of the image well. Therefore, each eigenvector is regarded as an image, and the direction information is obtained by calculating the Gaussian filter in different directions Then, the direction information under different eigenvectors is linearly superimposed:

mPb和sPb分别代表了不同的边缘信息。mPb涵盖了所有边缘的信息，是局部信息；而sPb则表示了最为突出的边的信息，是全局信息。正因为此，将两者有机地结合起来可以有效地分析图像轮廓信息，本发明将其定义为gPb：mPb and sPb represent different edge information respectively. mPb covers all edge information, which is local information; while sPb represents the most prominent edge information, which is global information. Just because of this, the image profile information can be effectively analyzed by combining the two organically, which is defined as gPb by the present invention:

$gPb PPML ((x x,, y the y,, θ θ)) = = \underset{s the s}{Σ Σ} \underset{i i}{Σ Σ} {{{β β}_{i i,, s the s} {G G}_{i i,, σ σ} ((x x,, y the y,, θ θ)) + + γ γ \cdot &Center Dot; sPb sPb ((x x,, y the y,, θ θ))}} - - - - - - ((77))$

式中，β_i,s和γ分别表示mPb和sPb的系数。In the formula, β_{i, s} and γ represent the coefficients of mPb and sPb, respectively.

实施例图像分割Example image segmentation

gPb能够有效的表示轮廓，但是这些轮廓也不是完全封闭的，因此不能用作分割图像。为了更好地分割图像，本文使用分水岭算法利用轮廓信息gPb来进行区域分割。gPb can effectively represent contours, but these contours are not completely closed, so they cannot be used to segment images. In order to better segment the image, this paper uses the watershed algorithm to use the contour information gPb to segment the region.

为了更好的描述分水岭算法，首先考虑任意一个轮廓检测器E(x,y,θ)。这个检测器可以估计出图像中任意一点(x,y)在方向θ上是轮廓的概率，并且这个值越大，表示该点是轮廓的概率越高。对于每个点求轮廓检测的最大值：To better describe the watershed algorithm, first consider any contour detector E(x,y,θ). This detector can estimate the probability that any point (x, y) in the image is a contour in the direction θ, and the larger the value, the higher the probability that the point is a contour. Find the maximum contour detection for each point:

接着，利用数学形态学，以区域中E(x,y)的最小值为“集水盆地”计算每个区域。每个“集水盆地”对应一个区域，记为P₀；两个集水盆地交汇处是“分水岭”，记为K₀。然而，分水岭算法会产生过度分割问题，将本不应该是边的地方标记为分水岭。为了解决这一问题，利用一种新的区域合并算法。该算法的前提是：分水岭算法一定会产生过分问题，即初始的每个集水盆地都需要合并。Next, using mathematical morphology, each region is calculated with the minimum value of E(x,y) in the region as the "catchment basin". Each "catchment basin" corresponds to an area, denoted as P₀ ; the intersection of two catchment basins is a "watershed", denoted as K₀ . However, the watershed algorithm creates an over-segmentation problem, marking places that should not be edges as watershed. To solve this problem, a new region merging algorithm is utilized. The premise of this algorithm is: the watershed algorithm will inevitably produce excessive problems, that is, each initial catchment basin needs to be merged.

区域合并算法描述如下：定义一个无向图G＝(P₀,K₀,W(K₀),E(P₀))，其中W(K₀)表示每条分水岭的权值，由分水岭上点的总能量处以分水岭上点的个数而得；E(P₀)表示每个集水盆地的能量值，每个盆地的初始能量均为零。同时需要注意的是在图中，每条都恰好分割两个区域，W(K₀)描述了相邻两个区域之间的相异性。将分水岭按照其权值，由小到大存入队列，算法的流程图如图2所示。The region merging algorithm is described as follows: Define an undirected graph G=(P₀ ,K₀ ,W(K₀ ),E(P₀ )), where W(K₀ ) represents the weight of each watershed, which is determined by The total energy of a point is obtained by the number of points on the watershed; E(P₀ ) represents the energy value of each catchment basin, and the initial energy of each basin is zero. At the same time, it should be noted that in the figure, each bar divides exactly two regions, and W(K₀ ) describes the dissimilarity between two adjacent regions. The watershed is stored in the queue from small to large according to its weight, and the flow chart of the algorithm is shown in Figure 2.

假定R₁和R₂由边C^*分割，且R＝R₁∪R₂。合并的条件是：如果min{E(R₁),E(R₂)}≠0，则Assume R₁ and R₂ are separated by edge C^* , and R=R₁ ∪_{R 2} . The condition for merging is: if min{E(R₁ ),E(R₂ )}≠0, then

W(K₀)≤τ·min{E(R₁),E(R₂)} (9) W(K₀ )≤τ·min{E(R₁ ),E(R₂ )} (9)

或min{E(R₁),E(R₂)}＝0 (10) or min{E(R₁ ),E(R₂ )}=0 (10)

其中，τ表示一个常数。通过调整τ就可以调整合并条件，从而控制最终区域的大小，τ越大最终合并的区域面积越大。Among them, τ represents a constant. By adjusting τ, the merging condition can be adjusted, thereby controlling the size of the final region. The larger τ is, the larger the area of the final merging region will be.

E(R)、P₀和K₀的更新方法是：The update method of E(R), P₀ and K₀ is:

P₀←P₀\{R₁,R₂}∪R (12) P₀ ←P₀ \{R₁ ,R₂ }∪R (12)

K₀←K₀\{C^*} (13) K₀ ←K₀ \{C^* } (13)

实施例阴影检测Example shadow detection

单区域识别：训练一个SVM分类器来判断单个区域是阴影的概率；训练集已手工标记出了阴影部分，分类器利用阴影的亮度和纹理基元来进行分类，输出为表示区域为阴影的概率。Single region recognition: train an SVM classifier to determine the probability of a single region being a shadow; the training set has manually marked the shadow part, the classifier uses the brightness of the shadow and texture primitives to classify, and the output is the probability that the region is a shadow .

匹配区域识别：检测一个区域是否为阴影区域，应该与其纹理相似的区域进行比较。如果和的亮度相似，则二者处于同一光照强度下；如果和的亮度相异，则认定亮度较暗的区域为阴影。Matching Region Recognition: Detects whether a region is a shaded region and should be compared to regions with similar textures. If the brightness of the sum is similar, the two are under the same light intensity; if the brightness of the sum is different, the area with a darker brightness is considered a shadow.

本发明利用分类器训练四个特征来判断阴影区域：The present invention uses a classifier to train four features to judge shadow areas:

①亮度和纹理基元的χ²距离；① The^χ2 distance between brightness and texture primitives;

②平均RGB比②Average RGB ratio

当对比的匹配区域是相同材质时，非阴影区域的三通道值更高。计算公式如下：When the compared matching area is the same material, the three-channel value is higher in the non-shaded area. Calculated as follows:

${ρ ρ}_{R R} = = \frac{{R R}_{avg avg 11}}{{R R}_{avg avg 22}} {ρ ρ}_{G G} = = \frac{{G G}_{avg avg 11}}{{G G}_{avg avg 22}} {ρ ρ}_{B B} = = \frac{{B B}_{avg avg 11}}{{B B}_{avg avg 22}} - - - - - - ((1414))$

其中，R_avg1表示第一个区域R通道的平均值。Among them, R_avg1 represents the average value of the R channel in the first region.

③色彩对齐度 ③Color Alignment

相同材质的阴影/非阴影对在RGB空间上的色彩保持对齐。该参数通过计算ρ_R/ρ_G和ρ_G/ρ_B得到。Shaded/non-shaded pairs of the same material maintain color alignment in RGB space. This parameter is obtained by calculating ρ_R /ρ_G and ρ_G /ρ_B.

④区域归一化距离④ Regional normalized distance

由于匹配区域的材质是否相同不一定和是否相邻有关，所以本将该参数也列为训练特征之一。该值是由匹配区域之间的欧式距离计算而得的。Since the material of the matching area is not necessarily related to whether it is adjacent, this parameter is also listed as one of the training features. This value is calculated from the Euclidean distance between matching regions.

构建如下能量方程，使用图割算法得到最终的阴影检测结果：Construct the following energy equation, and use the graph cut algorithm to obtain the final shadow detection result:

$\overset{^^}{y the y} = = arg arg \underset{y the y}{min min} \underset{k k}{Σ Σ} {cos cos t t}_{k k}^{unary unary} (({y the y}_{k k})) + + {α α}_{22} \underset{{{i i,, j j}} &Element; &Element; {E E.}_{same the same}}{Σ Σ} {c c}_{ij ij}^{same the same} 11 (({y the y}_{i i} &NotEqual; &NotEqual; {y the y}_{j j})) - - - - - - ((1515))$

同时，at the same time,

其中，表示区域匹配分类器对于两区域光照相同的估计，表示区域匹配分类器对于两区域光照不同的估计，是单区域分类器对单区域区域是否为阴影的估计；{i,j}∈E_same表示相同光照的两个区域，{i,j}∈E_di^f表示不同光照的两个区域；y＝{-1,1}ⁿ，当为1时表示该区域是阴影区域。in, Indicates that the area matching classifier estimates that the illumination of the two areas is the same, Indicates the estimation of the area matching classifier for the different illumination of the two areas, is the single-region classifier’s estimate of whether a single-region region is shaded; {i,j}∈E_same represents two regions with the same illumination, and {i,j}∈E_di^f represents two regions with different illumination; y= {-1,1}ⁿ , when it is 1, it means that the area is a shaded area.

实施例阴影去除Example shadow removal

为有效地进行阴影移除，必须建立一个适当的阴影模型。阴影模型的光照由直射光和环境光共同决定：For effective shadow removal, an appropriate shadow model must be established. The lighting of the shadow model is determined by the direct light and the ambient light:

I_i＝(k_i·L_d+L_e)R_i (17) I_i =(k_i ·L_d +L_e )R_i (17)

对于像素i，k_i表示阴影标签。当k_i＝0时，表示该像素点在阴影区域内；当k_i＝1时，表示该像素点在非阴影区域内；当0<k_i<1，表示该像素处在半影区域内。For pixel i,_ki denotes the shadow label. When k_i =0, it means the pixel is in the shaded area; when k_i =1, it means the pixel is in the non-shaded area; when 0<k_i <1, it means the pixel is in the penumbra area .

尽管阴影检测已经为每个像素赋予了一个标签(0或1)，然而实际场景中阴影边缘是由非阴影到阴影逐步过渡的。为了更好地对阴影边缘进行去除，本文使用抠图算法计算半影区域：Although shadow detection has assigned a label (0 or 1) to each pixel, in actual scenes shadow edges gradually transition from non-shadow to shadow. In order to better remove the shadow edge, this paper uses the matting algorithm to calculate the penumbra area:

I_i＝k_iF_i+(1-k_i)B_i (18) I_i =k_i F_i +(1-k_i )B_i (18)

将前景标记为非阴影，背景标记为阴影。通过计算下述能量方程的最小值得到k_i的大小：Mark the foreground as non-shaded and the background as shaded. The size of_ki is obtained by calculating the minimum value of the following energy equation:

其中，L是抠图的拉普拉斯矩阵，D(i,i)是一个对角阵。实验中，D(i,i)＝1表示像素i是阴影区域的边缘，D(i,i)＝0则表示其它所有点。Among them, L is the Laplacian matrix of matting, and D(i,i) is a diagonal matrix. In the experiment, D(i,i)=1 indicates that pixel i is the edge of the shaded area, and D(i,i)=0 indicates all other points.

其中，r＝L_d/L_e是直射光L_d和环境光L_e的比值，I_i表示像素i原本的值。所以，计算得到r就可以将阴影去除。Wherein, r=L_d /L_e is the ratio of the direct light L_d to the ambient light L_e , and I_i represents the original value of the pixel i. Therefore, the shadow can be removed by calculating r.

已知A known

I_i＝(k_i·L_d+L_e)R_i (24) I_i =(k_i ·L_d +L_e )R_i (24)

I_j＝(k_j·L_d+L_e)R_j (25) I_j ＝(k_j ·L_d +L_e )R_j (25)

在相邻的阴影和非阴影区域两边选取最相近的像素点，然后根据式(24)计算得到r值。Select the closest pixel points on both sides of the adjacent shaded and non-shaded areas, and then calculate the r value according to formula (24).

以上内容仅为结合具体方案对本发明进行的一些详细说明，不能认定发明的具体实施只限于这些说明。对本发明所属技术领域的普通技术人员来说，在不脱离本发明的构思前提下，还可以做出简单的推演及替换，都应当视为在本发明的保护范围内。The above content is only some detailed descriptions of the present invention in combination with specific solutions, and it cannot be assumed that the specific implementation of the invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, they can also make simple deduction and replacement, which should be regarded as within the protection scope of the present invention.