技术领域technical field
本发明涉及图像处理技术领域，更具体地讲，涉及一种修复在拍摄时受到雨滴干扰的视频图像的方法。The present invention relates to the technical field of image processing, and more specifically, relates to a method for repairing video images disturbed by raindrops during shooting.
背景技术Background technique
雨对图像成像有很大的影响，会造成图像成像模糊、信息被覆盖，使得视频图像的清晰度下降、视频图像的数字化处理性能下降。因此，对拍摄时受到雨滴干扰的视频图像进行修复处理有利于图像的进一步处理，包括提高基于图像的目标检测、识别、追踪、分割和监控等技术的性能。视频图像去雨技术在现代军事、交通以及安全监控等领域有着广泛的应用前景。Rain has a great impact on image formation, which will cause blurring of image formation and coverage of information, resulting in a decrease in the definition of video images and a decrease in the performance of digital processing of video images. Therefore, repairing the video image disturbed by raindrops during shooting is beneficial to the further processing of the image, including improving the performance of image-based target detection, recognition, tracking, segmentation and monitoring technologies. Video image deraining technology has broad application prospects in the fields of modern military, transportation and security monitoring.
现有的用于视频图像的去雨算法，应用在静态场景的视频图像上较为成熟，但应用在动态场景的视频图像上无法达到理想的检测效果。The existing rain removal algorithms for video images are relatively mature when applied to video images of static scenes, but cannot achieve ideal detection results when applied to video images of dynamic scenes.
此外，实时对拍摄时受到雨滴干扰的视频图像进行修复处理在自动导航系统、安全监控系统等场合中有很大的需求。这些应用场合中往往需要及时得到处理结果，反馈给用户，视频处理的滞后有可能导致用户做出错误的判断。而现有的用于视频图像的去雨算法由于其计算的复杂度导致效率较低，实时性还有待提高。In addition, there is a great demand for real-time restoration of video images disturbed by raindrops during shooting in automatic navigation systems, security monitoring systems, and other occasions. In these applications, it is often necessary to obtain processing results in a timely manner and feed them back to users. The lag of video processing may cause users to make wrong judgments. However, the existing rain removal algorithms for video images are less efficient due to their computational complexity, and their real-time performance needs to be improved.
发明内容Contents of the invention
本发明的示例性实施例在于提供一种修复在拍摄时受到雨滴干扰的视频图像的方法，以克服现有技术中修复精度和修复效率不理想的问题。An exemplary embodiment of the present invention provides a method for repairing a video image disturbed by raindrops during shooting, so as to overcome the problems of unsatisfactory repair accuracy and repair efficiency in the prior art.
本发明提供一种修复在拍摄时受到雨滴干扰的视频图像的方法，其特征在于，包括：(A)接收基于RGB色彩空间的视频的一帧图像；(B)将接收到的一帧图像转换成基于HSL色彩空间的图像；(C)基于色调分量和饱和度分量检测转换后的图像中的纯雨滴部分以及雨滴和运动物体重叠部分；(D)基于亮度分量分别对纯雨滴部分以及雨滴和运动物体重叠部分进行雨滴去除；(E)将去除雨滴后的图像转换成基于RGB色彩空间的图像并输出。The present invention provides a method for repairing a video image disturbed by raindrops when shooting, which is characterized in that it includes: (A) receiving a frame of image based on RGB color space video; (B) converting the received frame of image into an image based on the HSL color space; (C) detect the pure raindrop part and the overlapping part of the raindrop and the moving object in the converted image based on the hue component and the saturation component; (D) respectively detect the pure raindrop part and the raindrop and moving object based on the brightness component Remove the raindrops on the overlapping part of the moving object; (E) convert the image after removing the raindrops into an image based on the RGB color space and output it.
可选地，步骤(C)包括：(C1)基于色调分量和饱和度分量检测所述转换后的图像中的运动物体部分；(C2)根据所述一帧图像和与所述一帧图像相邻的一帧图像的灰度差、雨滴的光学特性和色彩特性检测所述转换后的图像中的受雨滴污染部分；(C3)将所述运动物体部分和所述受雨滴污染部分的重叠部分确定为雨滴和运动物体重叠部分，并将所述受雨滴污染部分中除与所述运动物体部分重叠之外的部分确定为纯雨滴部分。Optionally, the step (C) includes: (C1) detecting a moving object part in the converted image based on a hue component and a saturation component; (C2) The grayscale difference of an adjacent frame image, the optical characteristics and color characteristics of raindrops detect the part polluted by raindrops in the image after the conversion; (C3) the overlapping part of the part of the moving object and the part polluted by raindrops It is determined as the overlapping part of the raindrop and the moving object, and the part of the part polluted by raindrops except the part overlapping with the moving object is determined as the pure raindrop part.
可选地，步骤(C1)包括：(C11)基于色调分量和饱和度分量检测所述转换后的图像中的运动物体的边缘；(C12)按照色彩特征将所述转换后的图像中的像素点进行聚类，以将所述转换后的图像分割成多个块；(C13)将检测到的运动物体的边缘所属的块内部的像素点标记为属于运动物体部分的像素点。Optionally, the step (C1) includes: (C11) detecting the edge of the moving object in the converted image based on the hue component and the saturation component; (C12) converting the pixel in the converted image according to the color feature Points are clustered to divide the converted image into multiple blocks; (C13) Mark the pixels inside the block to which the edge of the detected moving object belongs as the pixels belonging to the moving object part.
可选地，步骤(C11)包括：当所述转换后的图像中的像素点的度量函数值大于预定阈值时，确定该像素点位于运动物体的边缘上，其中，度量函数F(x,y)为：Optionally, step (C11) includes: when the metric function value of a pixel in the converted image is greater than a predetermined threshold, determine that the pixel is located on the edge of the moving object, wherein the metric function F(x, y )for:
其中，Hr(x,y)指示像素点(x,y)的色调值，Hb(x,y)指示像素点(x,y)的背景色调值，Sr(x,y)指示像素点(x,y)的饱和度值，Sb(x,y)指示像素点(x,y)的背景饱和度值，指示像素点(x,y)的灰度值，指示像素点(x,y)的背景灰度值。Among them, Hr (x, y) indicates the hue value of the pixel point (x, y), Hb (x, y) indicates the background hue value of the pixel point (x, y), and Sr (x, y) indicates the pixel The saturation value of the point (x, y), Sb (x, y) indicates the background saturation value of the pixel point (x, y), Indicates the gray value of the pixel point (x, y), Indicates the background gray value of the pixel (x, y).
可选地，步骤(D)包括：将纯雨滴部分中的每个像素点的亮度值替换为：所述一帧图像的前M帧图像至后M帧图像中的对应像素点的亮度值的加权平均值，其中，M为大于0的整数；将雨滴和运动物体重叠部分中的每个像素点的亮度值替换为：所述一帧图像的前一帧图像至后一帧图像中的各帧图像中的对应像素点及对应像素点的相邻像素点的亮度值的加权平均值的平均值。Optionally, step (D) includes: replacing the brightness value of each pixel point in the pure raindrop part with: the brightness value of the corresponding pixel point in the first M frame images to the last M frame images of the one frame image Weighted average, wherein, M is an integer greater than 0; the brightness value of each pixel in the overlapping part of the raindrop and the moving object is replaced by: each of the previous frame image to the next frame image of the one frame image The average value of the weighted average of the brightness values of the corresponding pixel point and the adjacent pixel points of the corresponding pixel point in the frame image.
可选地，M＝3，其中，纯雨滴部分中的像素点(x,y)的亮度值通过下式计算得到替换值：Optionally, M=3, wherein, the brightness value of the pixel point (x, y) in the pure raindrop part is calculated by the following formula to obtain the replacement value:
其中，N为所述一帧图像的帧序号，L(x,y,t)指示第t帧图像的像素点(x,y)的亮度值，Fb(t)为加权系数矩阵，Fb(t)[1,2,4,0,4,2,1]。Wherein, N is the frame sequence number of the one frame image, L(x, y, t) indicates the brightness value of the pixel point (x, y) of the tth frame image, Fb (t) is a matrix of weighting coefficients, Fb (t)[1,2,4,0,4,2,1].
可选地，雨滴和运动物体重叠部分中的像素点(x,y)的亮度值通过下式计算得到替换值：Optionally, the brightness value of the pixel point (x, y) in the overlapping part of the raindrop and the moving object is calculated by the following formula to obtain the replacement value:
其中，N为所述一帧图像的帧序号，L(x,y,t)指示第t帧图像的像素点(x,y)的亮度值，V指示所述一帧图像的前一帧图像至后一帧图像中的与像素点(x,y)对应的像素点及对应的像素点的相邻像素点所构成的域，Fm(x,y,t)为加权系数矩阵，Wherein, N is the frame number of the one-frame image, L(x, y, t) indicates the brightness value of the pixel point (x, y) of the t-th frame image, and V indicates the previous frame image of the one-frame image To the domain formed by the pixel point corresponding to the pixel point (x, y) and the adjacent pixel point of the corresponding pixel point in the next frame image, Fm (x, y, t) is a matrix of weighting coefficients,
根据本发明示例性实施例的修复在拍摄时受到雨滴干扰的视频图像的方法，基于视频图像的HSL色彩空间和对运动目标的识别实现在高鲁棒性的前提下提高修复视频图像的精度和效率。According to the method for repairing video images disturbed by raindrops during shooting according to an exemplary embodiment of the present invention, based on the HSL color space of video images and the recognition of moving objects, the accuracy and accuracy of repairing video images can be improved under the premise of high robustness. efficiency.
将在接下来的描述中部分阐述本发明总体构思另外的方面和/或优点，还有一部分通过描述将是清楚的，或者可以经过本发明总体构思的实施而得知。Additional aspects and/or advantages of the present general inventive concept will be partially set forth in the following description, and some will be clear from the description, or can be learned through practice of the present general inventive concept.
附图说明Description of drawings
图1示出根据本发明示例性实施例的修复在拍摄时受到雨滴干扰的视频图像的方法的流程图；FIG. 1 shows a flow chart of a method for repairing a video image disturbed by raindrops during shooting according to an exemplary embodiment of the present invention;
图2示出根据本发明示例性实施例的检测运动物体部分的方法的流程图。Fig. 2 shows a flowchart of a method for detecting moving object parts according to an exemplary embodiment of the present invention.
具体实施方式Detailed ways
现将详细参照本发明的实施例，所述实施例的示例在附图中示出，其中，相同的标号始终指的是相同的部件。以下将通过参照附图来说明所述实施例，以便解释本发明。Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like numerals refer to like parts throughout. The embodiments are described below in order to explain the present invention by referring to the figures.
图1示出根据本发明示例性实施例的修复在拍摄时受到雨滴干扰的视频图像的方法的流程图。Fig. 1 shows a flowchart of a method for repairing a video image disturbed by raindrops during shooting according to an exemplary embodiment of the present invention.
参照图1，在步骤S10，接收基于RGB色彩空间的视频的一帧图像。Referring to FIG. 1 , in step S10 , a frame of video based on RGB color space is received.
在步骤S20，将接收到的一帧图像转换成基于HSL色彩空间的图像。In step S20, the received frame of image is converted into an image based on HSL color space.
具体说来，将接收到的基于RGB色彩空间的红色(R)分量、绿色(G)分量和蓝色(B)分量表示的RGB图像转换成基于HSL色彩空间中的色调(H)分量、饱和度(S)分量和亮度(L)分量表示的图像，以在HSL色彩空间下进行雨滴的检测和去除。Specifically, the received RGB image represented by the red (R) component, green (G) component and blue (B) component based on the RGB color space is converted into a hue (H) component based on the HSL color space, saturation The image represented by the degree (S) component and the brightness (L) component is used to detect and remove raindrops in the HSL color space.
可通过下式将基于RGB色彩空间的图像转换成基于HSL色彩空间的图像：An image based on the RGB color space can be converted into an image based on the HSL color space by the following formula:
其中，max指示R、G、B分量中的最大值，min指示R、G、B分量中的最小值。Wherein, max indicates the maximum value among the R, G, and B components, and min indicates the minimum value among the R, G, and B components.
由于去除雨滴与雨滴的亮度特性有关，与雨滴的色调特性和饱和度特性无关，因此，将RGB图像转换成基于HSL色彩空间的图像，就可仅针对雨滴的亮度分量进行雨滴去除，并且可基于色调分量和饱和度分量进行运动目标的检测，从而大大降低算法的复杂度，提高计算效率。Since the removal of raindrops is related to the brightness characteristics of raindrops, and has nothing to do with the hue and saturation characteristics of raindrops, therefore, converting the RGB image into an image based on the HSL color space can remove raindrops only for the brightness component of raindrops, and can be based on Hue component and saturation component are used to detect moving objects, which greatly reduces the complexity of the algorithm and improves the calculation efficiency.
在步骤S30，基于色调分量和饱和度分量检测转换后的图像中的纯雨滴部分以及雨滴和运动物体重叠部分。In step S30, the pure raindrop part and the overlapping part of the raindrop and the moving object in the converted image are detected based on the hue component and the saturation component.
在一个示例中，可首先基于色调分量和饱和度分量检测转换后的图像中的运动物体部分。这里，可使用各种适合的方法基于色调分量和饱和度分量检测转换后的图像中的运动物体部分。优选地，可通过图2所示的检测运动物体部分的方法来实现。In one example, moving object parts in the converted image may first be detected based on hue and saturation components. Here, various suitable methods may be used to detect the moving object part in the converted image based on the hue component and the saturation component. Preferably, it can be realized by the method for detecting moving object parts shown in FIG. 2 .
然后，根据所述一帧图像和与所述一帧图像相邻的一帧图像的灰度差、雨滴的光学特性和色彩特性检测所述转换后的图像中的受雨滴污染部分。Then, detecting the part polluted by raindrops in the converted image according to the gray level difference between the one frame image and the one frame image adjacent to the one frame image, the optical characteristic and the color characteristic of the raindrop.
在一个示例中，可首先求得各像素点在连续两帧视频图像中的灰度差，当一像素点的灰度差大于差值阈值时，则确定该像素点是候选的受雨滴污染的像素点。这里，差值阈值大小的选取要使得所有被雨滴污染的像素点的灰度值的变化都能够被检测出来。例如，差值阈值大小可取为3/255。In an example, the gray level difference of each pixel point in two consecutive frames of video images can be obtained first, and when the gray level difference of a pixel point is greater than the difference threshold, it is determined that the pixel point is a candidate polluted by raindrops. pixel. Here, the size of the difference threshold is selected so that the changes in the gray values of all pixels polluted by raindrops can be detected. For example, the magnitude of the difference threshold may be 3/255.
然后，基于雨滴的光学和色彩特性对候选的受雨滴污染的像素点进行进一步的筛选，得到受雨滴污染的像素点，从而确定受雨滴污染部分。这些特性包括：强度波动范围α，最大连通区域面积β以及RGB色彩分量的变化值等。例如，将α的值在3/255-30/255之间，β的值在30-50个像素点之间，ΔR、ΔG和ΔB近似相等的像素点确定为受雨滴污染的像素点，否则确定为非受雨滴污染的像素点。这里，α和β的取值可根据雨滴的大小，视频帧的大小以及拍摄焦距等进行设置。Then, based on the optical and color characteristics of the raindrops, the candidate pixels polluted by raindrops are further screened to obtain the pixels polluted by raindrops, so as to determine the part polluted by raindrops. These characteristics include: the intensity fluctuation range α, the area of the largest connected region β and the change value of RGB color components, etc. For example, the value of α is between 3/255-30/255, the value of β is between 30-50 pixels, and the pixels whose ΔR, ΔG and ΔB are approximately equal are determined as the pixels polluted by raindrops, otherwise Pixels determined not to be polluted by raindrops. Here, the values of α and β can be set according to the size of the raindrop, the size of the video frame, and the shooting focal length.
将所述运动物体部分和所述受雨滴污染部分的重叠部分确定为雨滴和运动物体重叠部分，所述运动物体部分中除与所述受雨滴污染部分重叠之外的部分确定为纯运动物体部分，并将所述受雨滴污染部分中除与所述运动物体部分重叠之外的部分确定为纯雨滴部分，从而检测出纯雨滴部分、雨滴和运动物体重叠部分、纯运动物体部分。Determining the overlapping portion of the moving object portion and the raindrop-contaminated portion as the overlapping portion of the raindrop and the moving object, and determining the portion of the moving object portion other than the overlap with the raindrop-contaminated portion as a pure moving object portion , and determine the part of the part polluted by raindrops other than the part overlapping with the moving object as a pure raindrop part, so as to detect the pure raindrop part, the overlapping part of the raindrop and the moving object, and the pure moving object part.
换言之，结合运动物体部分和受雨滴污染部分得到他们的交集作为雨滴和运动物体重叠部分，运动物体部分除该交集之外的部分为纯运动物体部分，受雨滴污染部分除该交集之外的部分为纯雨滴部分。In other words, combine the part of the moving object and the part polluted by raindrops to get their intersection as the overlapping part of the raindrop and the moving object, the part of the moving object except the intersection is the pure moving part, and the part polluted by raindrops except the intersection is the pure raindrop part.
应该理解，也可使用其他适合的方法基于色调分量和饱和度分量检测转换后的图像中的纯雨滴部分以及雨滴和运动物体重叠部分。It should be understood that other suitable methods may also be used to detect the pure raindrop portion and the overlapping portion of the raindrop and the moving object in the converted image based on the hue component and the saturation component.
在步骤S40，基于亮度分量分别对纯雨滴部分以及雨滴和运动物体重叠部分进行雨滴去除。In step S40, raindrop removal is performed on the pure raindrop part and the overlapping part of the raindrop and the moving object based on the luminance component.
由于纯运动物体部分没有受到雨滴的污染，亮度值保持不变，因此，只需基于亮度分量分别对纯雨滴部分以及雨滴和运动物体重叠部分进行雨滴去除即可。Since the pure moving object part is not polluted by raindrops, the brightness value remains unchanged. Therefore, it is only necessary to remove the raindrops on the pure raindrop part and the overlapping part of raindrops and moving objects based on the brightness component.
在一个示例中，可将纯雨滴部分中的每个像素点的亮度值替换为：所述一帧图像的前M帧图像至后M帧图像中的对应像素点的亮度值的加权平均值，其中，M为大于0的整数。即，可通过时域空间中的前后M帧中的该像素点的亮度值的加权平均值来代替纯雨滴部分中的该像素点的亮度值。In an example, the brightness value of each pixel point in the pure raindrop part may be replaced by: the weighted average of the brightness values of the corresponding pixel points in the first M frame images to the last M frame images of the one frame image, Wherein, M is an integer greater than 0. That is, the luminance value of the pixel point in the pure raindrop part may be replaced by a weighted average of the luminance values of the pixel point in the preceding and subsequent M frames in the time domain space.
将雨滴和运动物体重叠部分中的每个像素点的亮度值替换为：所述一帧图像的前一帧图像至后一帧图像中的各帧图像中的对应像素点及对应像素点的相邻像素点的亮度值的加权平均值的平均值。对于雨滴与运动物体重叠部分中的像素点的亮度值，由于帧之间的时域相关性并不大，反而空间相关性更大，所以时域上可仅仅选取前后各一帧取平均，而分别在每一帧中进行该像素点及该像素点的相邻像素点的加权平均，从而综合时空相关性进行雨滴的去除。Replace the brightness value of each pixel point in the overlapping part of the raindrop and the moving object with: the corresponding pixel point and the corresponding pixel point in each frame image from the previous frame image to the next frame image of the one frame image The average value of the weighted average of the brightness values of adjacent pixels. For the brightness value of the pixels in the overlapping part of the raindrop and the moving object, since the time domain correlation between the frames is not large, but the spatial correlation is greater, so in the time domain, only one frame before and after each can be selected to take the average, and The weighted average of the pixel and the adjacent pixels of the pixel is carried out in each frame, so as to remove the raindrops by integrating the spatial-temporal correlation.
例如，M＝3，其中，纯雨滴部分中的像素点(x,y)的亮度值可通过下式计算得到替换值：For example, M=3, wherein, the brightness value of the pixel point (x, y) in the pure raindrop part can be calculated by the following formula to obtain the replacement value:
其中，N为所述一帧图像的帧序号，L(x,y,t)指示第t帧图像的像素点(x,y)的亮度值，Fb(t)为加权系数矩阵，Fb(t)[1,2,4,0,4,2,1]。Wherein, N is the frame sequence number of the one frame image, L(x, y, t) indicates the brightness value of the pixel point (x, y) of the tth frame image, Fb (t) is a matrix of weighting coefficients, Fb (t)[1,2,4,0,4,2,1].
雨滴和运动物体重叠部分中的像素点(x,y)的亮度值可通过下式计算得到替换值：The brightness value of the pixel point (x, y) in the overlapping part of the raindrop and the moving object can be calculated by the following formula to obtain the replacement value:
在步骤S50，将去除雨滴后的图像转换成基于RGB色彩空间的图像并输出。即，将完成去雨处理的HSL图像再转换成RGB图像以输出。In step S50, the image after the raindrop removal is converted into an image based on RGB color space and output. That is, the HSL image after the deraining process is converted into an RGB image for output.
图2示出根据本发明示例性实施例的检测运动物体部分的方法的流程图。可在执行步骤S30时执行。Fig. 2 shows a flowchart of a method for detecting moving object parts according to an exemplary embodiment of the present invention. It can be executed when step S30 is executed.
如图2所示，在步骤S301，基于色调分量和饱和度分量检测所述转换后的图像中的运动物体的边缘。As shown in FIG. 2, in step S301, the edge of the moving object in the converted image is detected based on the hue component and the saturation component.
由于雨滴下落速度较快，在正常曝光速度下，图像上基本观测不到球形的雨滴，而是雨滴由于快速运动所形成的雨线。在自然环境下，式(6)可描述雨滴的物理成像过程，并可定量地描述雨滴下落时产生的模糊：Due to the fast falling speed of raindrops, under normal exposure speed, spherical raindrops are basically not observed on the image, but rainlines formed by the rapid movement of raindrops. In the natural environment, formula (6) can describe the physical imaging process of raindrops and quantitatively describe the blurring of raindrops when they fall:
Ir(x,y)αIE(x,y)+(1-α)Ib(x,y) (6)Ir (x,y)αIE (x,y)+(1-α)Ib (x,y) (6)
其中，Ir(x,y)指示像素点(x,y)的灰度值，IE(x,y)指示在曝光时间T内，假设雨滴一直覆盖在像素点(x,y)所形成的等效理想灰度值，Ib(x,y)指示像素点(x,y)的背景灰度值，即，没有被雨滴污染时的灰度值，ατ/T，表示雨滴下落经过像素点(x,y)所需的时间与曝光时间的比值。Among them, Ir (x, y) indicates the gray value of the pixel point (x, y), and IE (x, y) indicates that within the exposure time T, it is assumed that raindrops always cover the pixel point (x, y) The equivalent ideal gray value of , Ib (x, y) indicates the background gray value of the pixel point (x, y), that is, the gray value when it is not polluted by raindrops, ατ/T, which means that the raindrops fall through the pixel The ratio of the time required for a point (x,y) to the exposure time.
自然光线是由不同频率的光混合而成的，因此雨滴成像的光学模型在基于任一颜色分量的通道内仍然成立。即，式(6)中各个变量用其在R、G、B三个通道的分量表示后仍然成立。将三个分量用矢量表示为：Natural light is a mixture of light of different frequencies, so the optical model of raindrop imaging still holds within a channel based on either color component. That is, each variable in formula (6) still holds true after being represented by its components in the three channels of R, G, and B. Represent the three components as vectors:
其中，Rr(x,y)指示像素点(x,y)的R分量值，Gr(x,y)指示像素点(x,y)的G分量值，Br(x,y)指示像素点(x,y)的B分量值，Rb(x,y)指示像素点(x,y)的背景R分量值(即，没有被雨滴污染时的R分量值)，Gb(x,y)指示像素点(x,y)的背景G分量值(即，没有被雨滴污染时的G分量值)，Bb(x,y)指示像素点(x,y)的背景B分量值(即，没有被雨滴污染时的B分量值)。Among them, Rr (x, y) indicates the R component value of the pixel point (x, y), Gr (x, y) indicates the G component value of the pixel point (x, y), and Br (x, y) indicates The B component value of the pixel point (x, y), Rb (x, y) indicates the background R component value of the pixel point (x, y) (that is, the R component value when not polluted by raindrops), Gb (x ,y) indicates the background G component value of the pixel point (x, y) (that is, the G component value when not polluted by raindrops), Bb (x, y) indicates the background B component value of the pixel point (x, y) (ie, the B component value when not polluted by raindrops).
由于雨滴下落速度较快，所以ατ/T趋近于零，α/(1-α)趋近于零，由式(1)、式(2)、式(3)以及式(8)联合可知，受雨滴污染的像素点中，Hr-Hb和Sr-Sb趋近零，即受雨滴污染的像素点的色调值和饱和度值与该像素点没有受雨滴污染时的色调值(即，背景色调值)和饱和度值(即，背景饱和度值)相比变化较小。而对于运动物体的边缘上的像素点来说，色调值和饱和度值会发生比较明显的变化。Due to the fast falling speed of raindrops, ατ/T tends to zero, and α/(1-α) tends to zero, which can be known from the combination of formula (1), formula (2), formula (3) and formula (8) , in the pixels polluted by raindrops, Hr -Hb and Sr -Sb tend to zero, that is, the hue value and saturation value of the pixel polluted by raindrops are the same as the hue value when the pixel is not polluted by raindrops (ie, the background hue value) and the saturation value (ie, the background saturation value) vary less. For pixels on the edge of a moving object, the hue value and saturation value will change significantly.
由于受雨滴污染的像素点的色调值与背景色调值相比变化较小，而运动物体的边缘上的像素点的色调值变化较大。但由于受不同视频质量以及远景雨雾形成的模糊效应的影响，无法获得准确的受雨滴污染的像素点的色调值和背景色调值，因此单一使用色调值无法准确地检测出运动物体的边缘，需要结合饱和度值以及灰度值来构造度量函数。Compared with the background tone value, the hue value of the pixel polluted by raindrops changes slightly, while the hue value of the pixel point on the edge of the moving object changes greatly. However, due to the influence of different video quality and the blur effect caused by rain and fog in the distant view, it is impossible to obtain the accurate hue value of the pixels polluted by raindrops and the background hue value, so the single use of hue value cannot accurately detect the edge of the moving object. Combining saturation values and grayscale values to construct a metric function.
因此，可构造度量函数F(x,y)为：Therefore, the metric function F(x,y) can be constructed as:
其中，Hr(x,y)指示像素点(x,y)的色调值，Hb(x,y)指示像素点(x,y)的背景色调值，Sr(x,y)指示像素点(x,y)的饱和度值，Sb(x,y)指示像素点(x,y)的背景饱和度值，指示像素点(x,y)的灰度值，指示像素点(x,y)的背景灰度值。Among them, Hr (x, y) indicates the hue value of the pixel point (x, y), Hb (x, y) indicates the background hue value of the pixel point (x, y), and Sr (x, y) indicates the pixel The saturation value of the point (x, y), Sb (x, y) indicates the background saturation value of the pixel point (x, y), Indicates the gray value of the pixel point (x, y), Indicates the background gray value of the pixel (x, y).
应该理解，像素点(x,y)的背景色调值可通过该像素点没有被雨滴污染的相邻帧中该像素点的色调值获得，像素点的背景饱和度值和背景灰度值也可通过相应的方式获得。It should be understood that the background hue value of the pixel point (x, y) can be obtained from the hue value of the pixel point in the adjacent frame where the pixel point is not polluted by raindrops, and the background saturation value and background gray value of the pixel point can also be obtained obtained in a corresponding manner.
由于雨滴的色彩特性，受雨滴污染的像素点的度量函数值趋近于零，而运动物体的边缘上的像素点由于色调值和饱和度值都会发生比较明显变化，因此，度量函数值是一个比较大的值。因此，可设置一个阈值来筛选出运动物体的边缘上的像素点。即当所述转换后的图像中的像素点的度量函数值大于预定阈值时，确定该像素点位于运动物体的边缘上。Due to the color characteristics of raindrops, the metric function value of the pixels polluted by raindrops tends to zero, while the pixels on the edge of the moving object will change significantly due to the hue value and saturation value. Therefore, the metric function value is a relatively large value. Therefore, a threshold can be set to filter out the pixels on the edge of the moving object. That is, when the metric function value of a pixel in the converted image is greater than a predetermined threshold, it is determined that the pixel is located on the edge of the moving object.
在步骤S302，按照色彩特征将所述转换后的图像中的像素点进行聚类，以将所述转换后的图像分割成多个块。In step S302, the pixels in the converted image are clustered according to color features, so as to divide the converted image into multiple blocks.
在步骤S303，将检测到的运动物体的边缘所属的块内部的像素点标记为属于运动物体部分的像素点。即，采用彩色聚类图像分割方法来确定运动物体部分。In step S303, the pixels inside the block to which the detected edge of the moving object belongs are marked as the pixels belonging to the moving object part. That is, the color clustering image segmentation method is used to determine the part of the moving object.
此外，根据本发明的示例性实施例的上述方法可以被实现为计算机程序，从而当运行该程序时，实现上述方法。Furthermore, the above-described methods according to exemplary embodiments of the present invention can be implemented as a computer program so that when the program is executed, the above-described methods are implemented.
根据本发明示例性实施例的修复在拍摄时受到雨滴干扰的视频图像的方法，基于视频图像的HSL色彩空间和对运动目标的识别实现在高鲁棒性的前提下提高修复视频图像的精度和效率。According to the method for repairing video images disturbed by raindrops during shooting according to an exemplary embodiment of the present invention, based on the HSL color space of video images and the recognition of moving objects, the accuracy and accuracy of repairing video images can be improved under the premise of high robustness. efficiency.
虽然已表示和描述了本发明的一些示例性实施例，但本领域技术人员应该理解，在不脱离由权利要求及其等同物限定其范围的本发明的原理和精神的情况下，可以对这些实施例进行修改。While a few exemplary embodiments of the present invention have been shown and described, it should be understood by those skilled in the art that such modifications may be made without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents. Examples are modified.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410856621.XACN104463812B (en) | 2014-12-31 | 2014-12-31 | The method for repairing the video image by raindrop interference when shooting |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410856621.XACN104463812B (en) | 2014-12-31 | 2014-12-31 | The method for repairing the video image by raindrop interference when shooting |
| Publication Number | Publication Date |
|---|---|
| CN104463812A CN104463812A (en) | 2015-03-25 |
| CN104463812Btrue CN104463812B (en) | 2018-08-24 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410856621.XAActiveCN104463812B (en) | 2014-12-31 | 2014-12-31 | The method for repairing the video image by raindrop interference when shooting |
| Country | Link |
|---|---|
| CN (1) | CN104463812B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105046670A (en)* | 2015-08-28 | 2015-11-11 | 中国科学院深圳先进技术研究院 | Image rain removal method and system |
| CN106056545A (en)* | 2016-05-24 | 2016-10-26 | 中国科学院深圳先进技术研究院 | Image rain removing method and image rain removing system |
| US10922846B2 (en)* | 2017-08-29 | 2021-02-16 | Guangdong Virtual Reality Technology Co., Ltd. | Method, device and system for identifying light spot |
| CN113362274B (en)* | 2021-02-19 | 2023-09-22 | 西北工业大学 | Rainfall monitoring and calculating method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2641230B1 (en)* | 2010-11-15 | 2017-09-06 | Indian Institute Of Technology, Kharagpur | Method and apparatus for detection and removal of rain from videos using temporal and spatiotemporal properties |
| CN102254149B (en)* | 2011-06-13 | 2013-05-08 | 南京航空航天大学 | Method for detecting and identifying raindrops in video image |
| Publication number | Publication date |
|---|---|
| CN104463812A (en) | 2015-03-25 |
| Publication | Publication Date | Title |
|---|---|---|
| JP7077395B2 (en) | Multiplexed high dynamic range image | |
| US11700457B2 (en) | Flicker mitigation via image signal processing | |
| EP2641230B1 (en) | Method and apparatus for detection and removal of rain from videos using temporal and spatiotemporal properties | |
| US8582915B2 (en) | Image enhancement for challenging lighting conditions | |
| US9262811B2 (en) | System and method for spatio temporal video image enhancement | |
| CN108174057B (en) | Method and device for rapidly reducing noise of picture by utilizing video image inter-frame difference | |
| CN107292830B (en) | Low-illumination image enhancement and evaluation method | |
| CN104616290A (en) | Target detection algorithm in combination of statistical matrix model and adaptive threshold | |
| CN104144298A (en) | A Wide Dynamic Image Synthesis Method | |
| CN104408707A (en) | Rapid digital imaging fuzzy identification and restored image quality assessment method | |
| US20230127009A1 (en) | Joint objects image signal processing in temporal domain | |
| CN104463812B (en) | The method for repairing the video image by raindrop interference when shooting | |
| CN108053382A (en) | A kind of visual characteristic defogging is surely as detection system | |
| CN105046670A (en) | Image rain removal method and system | |
| CN112233032B (en) | A method for removing ghosts in high dynamic range images | |
| CN101299239B (en) | Method and device for acquiring character area image and character recognition system | |
| CN104915940A (en) | Alignment-based image denoising method and system | |
| CN107395991A (en) | Image combining method, device, computer-readable recording medium and computer equipment | |
| CN113923431A (en) | Real-time keying method, device, device and storage medium based on high-definition video | |
| CN103020924A (en) | Low-illumination monitored image enhancement method based on similar scenes | |
| CN104021527A (en) | Rain and snow removal method in image | |
| CN113222870A (en) | Image processing method, device and equipment | |
| US20110085026A1 (en) | Detection method and detection system of moving object | |
| KR102136716B1 (en) | Apparatus for Improving Image Quality and Computer-Readable Recording Medium with Program Therefor | |
| CN104484865A (en) | Method for removing raindrops in video image |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |