Method and device for rapidly reducing noise of picture by utilizing video image inter-frame differenceTechnical Field
The invention belongs to the technical field of video image processing, and particularly relates to a method and a device for quickly reducing noise of a picture by utilizing video image inter-frame difference.
Background
In reality, digital images are often affected by interference of imaging equipment and external environment noise during digitization and transmission, and are called noisy images or noisy images. The process of reducing noise in a digital image is called image denoising, sometimes referred to as image denoising. Noise is an important cause of image disturbance. An image may have various noises in practical application, and these noises may be generated in transmission, quantization, etc. The noise can be mainly divided into three forms, namely additive noise, multiplicative noise, quantization noise and the like according to the relation between the noise and the signal.
The traditional noise processing method comprises an average filter, an adaptive wiener filter, a median filter, a morphological noise filter, a wavelet de-noising and other methods. The traditional denoising algorithm only utilizes a still picture, and has real-time requirements when video denoising is performed, however, the traditional algorithm has high processing speed and can sacrifice certain effect, and the traditional algorithm has good effect and can sacrifice certain speed, so that the speed and the processing effect cannot be compatible.
Disclosure of Invention
Aiming at the defects or improvement requirements in the prior art, the invention provides a method and a device for rapidly reducing noise of a picture by utilizing video image interframe difference, and solves the problem that the speed and the effect of the traditional noise reduction algorithm in the prior art cannot be compatible.
In order to achieve the above object, according to an aspect of the present invention, there is provided a method for fast noise reduction of a picture using a difference between video image frames, comprising:
determining a first frame key frame of a video stream to be denoised, detecting a non-key frame of the video stream by taking the first frame key frame as a starting point until a next frame key frame is encountered, detecting and replacing the non-key frame of the video stream by taking the next frame key frame as the starting point, and repeating the processing process until all the non-key frames are detected; wherein detecting and replacing non-key frames of the video stream comprises:
judging whether each pixel point in the non-key frame is the maximum brightness value or the minimum brightness value of the pixel point and the adjacent pixel points around the pixel point; if not, determining the pixel point as a non-noise point;
if the brightness value is the maximum brightness value, calculating the difference value between the brightness value of the pixel point and the next largest brightness value of the pixel point and the adjacent pixel points around the pixel point; if the difference value is smaller than a preset brightness threshold value, determining the pixel point as a non-noise point, and if not, calculating the difference value between the brightness value of the pixel point and the maximum brightness values of the pixel point at the same position in the previous frame and the adjacent pixel points around the pixel point; if the difference value is smaller than a preset brightness threshold value, determining the pixel point as a non-noise point, and if not, determining the pixel point as a noise point;
if the brightness value is the minimum brightness value, calculating the difference value between the brightness value of the pixel point and the next minimum brightness value of the pixel point and the adjacent pixel points around the pixel point; if the difference value is smaller than a preset brightness threshold value, determining the pixel point as a non-noise point, and if not, calculating the difference value between the brightness value of the pixel point and the minimum brightness values of the pixel point at the same position in the previous frame and the adjacent pixel points around the pixel point; if the difference value is smaller than a preset brightness threshold value, determining the pixel point as a non-noise point, and if not, determining the pixel point as a noise point;
the luminance value of the noise point is replaced.
In an embodiment of the present invention, the replacing the luminance value of the noise point includes:
replacing the brightness value of the noise point with the average value of adjacent pixel points around the noise point; or,
if the noise point has the maximum brightness value, replacing the brightness value of the noise point with the weighted average value of the maximum brightness value and the second maximum brightness value of the pixel point and the adjacent pixel points around the pixel point, and if the noise point has the minimum brightness value, replacing the brightness value of the noise point with the weighted average value of the minimum brightness value and the second minimum brightness value of the pixel point and the adjacent pixel points around the pixel point.
In an embodiment of the present invention, the preset brightness threshold is:
a maximum brightness error value, wherein the brightness error is a maximum picture brightness value which cannot be seen by naked eyes and is different from black; or,
if the noise point has the maximum brightness value, the noise point is 1/N of the next largest brightness value of the pixel point and the adjacent pixel points around the pixel point; if the noise point has the minimum brightness value, the noise point is 1/N of the next minimum brightness value of the pixel point and the adjacent pixel points around the pixel point, and N is the number of the adjacent pixel points around the pixel point; or,
if the noise point has the maximum brightness value, the geometric average value of the brightness value of the pixel point and the maximum brightness error value is obtained; and if the noise point has the minimum brightness value, the geometric average value of the brightness value of the pixel point and the minimum brightness error value is obtained.
In one embodiment of the present invention, the method further comprises performing color noise correction on the image, including:
converting image frames of the video stream to an HSI color space;
if the difference between the chromatic value of a pixel point and the chromatic values of adjacent pixel points around the pixel point and the same pixel point in the previous frame is larger than a chromatic threshold, the brightness value of the pixel point is smaller than the preset brightness threshold, and the color saturation of the pixel point is larger than a color saturation threshold, the pixel point is determined as a color noise point;
and carrying out color correction on the color noise points.
In one embodiment of the present invention, the color correcting the color noise point comprises:
and replacing the chromatic value of the color noise point with the chromatic value which is closest to the chromatic value of the color noise point in the adjacent pixel points around the color noise point and the pixel points at the same position of the previous frame.
In one embodiment of the invention, the chroma threshold is pi/2 and the color saturation threshold is 0.5.
In one embodiment of the present invention, the following algorithm is used for detecting and replacing non-key frames of the video stream:
acquiring the maximum brightness value of each pixel point of the key frame and the adjacent pixel points around the pixel point, and storing the maximum brightness value and the corresponding coordinates into a first two-dimensional array;
calculating the maximum brightness value and the secondary maximum brightness value of each pixel point in the non-key frame and adjacent pixel points around the pixel point, and storing the maximum brightness value and the corresponding coordinates into a second two-dimensional array;
judging whether the brightness value of each pixel point in the non-key frame is equal to the maximum brightness value of the pixel point and the adjacent pixel points around the pixel point, if so, judging whether the difference between the maximum brightness value and the next-largest brightness value of the adjacent pixel points around the pixel point in the non-key frame is greater than a brightness threshold value, and if so, determining the pixel point as a noise point;
and exchanging the pointers of the first two-dimensional array and the second two-dimensional array.
In one embodiment of the present invention, the following algorithm is used for detecting and replacing non-key frames of the video stream:
acquiring the minimum brightness value of each pixel point of the key frame and adjacent pixel points around the pixel point, and storing the minimum brightness value and corresponding coordinates into a first two-dimensional array;
calculating the minimum brightness value and the next minimum brightness value of each pixel point in the non-key frame and adjacent pixel points around the pixel point, and storing the minimum brightness value and the corresponding coordinates into a second two-dimensional array;
judging whether the brightness value of each pixel point in the non-key frame is equal to the minimum brightness value of the pixel point and the adjacent pixel points around the pixel point, if so, judging whether the difference between the minimum brightness value and the next minimum brightness value of the adjacent pixel points around the pixel point in the non-key frame is greater than a brightness threshold value, and if so, determining the pixel point as a noise point;
and exchanging the pointers of the first two-dimensional array and the second two-dimensional array.
In an embodiment of the present invention, the surrounding neighboring pixels are neighboring 8-neighbor pixels.
According to another aspect of the present invention, there is also provided an apparatus for rapidly denoising a picture using an inter-frame difference of a video image, the apparatus comprising: a processor and a memory for storing a computer program capable of running on the processor; when the processor is used for running the computer program, the method for quickly reducing the noise of the picture by utilizing the difference between the video image frames is executed.
Compared with the prior art, the invention has the following beneficial effects:
(1) different from the traditional noise reduction method, the method provided by the invention can quickly and efficiently identify the noise points by considering the similarity of the video images through the dependency relationship between the front and the back of the video images, and greatly improve the noise reduction speed and efficiency;
(2) in addition, for the low-brightness environment, noise is particularly easy to appear, algorithm optimization is specially performed on the scene, and the effect is obvious;
(3) the general noise reduction needs to depend on optical hardware, but the invention is based on digital image processing and does not need to depend on optical hardware, thereby greatly reducing the cost.
Drawings
FIG. 1 is a flowchart illustrating a method for fast denoising an image using inter-frame difference of a video image according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a device for fast denoising a picture by using a difference between video frames according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment of the invention provides a method for rapidly reducing noise of a picture by utilizing video image interframe difference, which specifically comprises the following steps of:
s101, collecting a video stream in a low-brightness environment at night, wherein the video stream comprises multiple frames of continuous images;
the method is suitable for video acquisition in a low-brightness environment at night, has more noise points in the low-brightness environment at night, and can effectively and quickly solve the problem of noise points by using the targeted algorithm.
S102, detecting a non-key frame of the video stream by taking a key frame of the video stream as a starting point, acquiring the maximum brightness values of pixel points in the non-key frame and adjacent pixel points around the pixel points, and comparing the brightness values of the pixel points with the maximum brightness values;
embodiments of the present invention start with a key frame of a video stream and only check for noise in non-key frames, since a key frame does not necessarily have a similar relationship to the previous frame.
The maximum brightness value of the non-key frame pixel point and the surrounding adjacent pixel points is obtained, the maximum brightness value can be obtained by using a bubbling algorithm, and the surrounding adjacent pixel points can be 8 surrounding points with the pixel point as the center. For noise, the brightness value of the noise is much larger than the surrounding brightness value, so that the comparison between the brightness value of each point in the non-key frame image and the maximum brightness value of the surrounding adjacent pixel points needs to be sequentially traversed, so as to determine whether the point is suspected to be noise.
Otherwise, if the brightness value of the pixel point is not the maximum brightness value, the point which is not suspected noise point can be eliminated.
S103, if the brightness value of the pixel point is the maximum brightness value, calculating the difference between the maximum brightness value and the next largest brightness value between the pixel point and the adjacent pixel points around, and if the brightness difference value is greater than a brightness threshold value, determining the pixel point as a suspected noise point;
s103 may specifically be:
and acquiring the maximum brightness value of each point of the key frame and 8 adjacent points around the key frame, storing the maximum brightness value and the corresponding coordinates into a first two-dimensional array, and recording the maximum brightness value and the corresponding coordinates into a lumax [ width ] [ height ], wherein the width and the height are the width and the height of the video respectively. (ii) a
Calculating the maximum brightness value max and the next maximum value next of the pixel point and 8 surrounding adjacent points in the non-key frame by using a bubbling algorithm, storing the maximum brightness value and the corresponding coordinates into a second two-dimensional array, and recording as lumaxtemp [ x ] [ y ], wherein x and y are point coordinates;
judging whether the brightness value of the pixel point in the non-key frame is equal to the maximum brightness value or not, if so, judging whether the difference between the maximum brightness value and the next largest brightness value of adjacent pixel points around the pixel point in the non-key frame is greater than a brightness threshold value or not, and if so, determining the pixel point as a suspected noise point; that is, it is judged whether (max-next) < threshold & (max-lumax [ x ] [ y ]) < threshold is true, and if true, the replacement current luminance value is an average value of surrounding eight-point luminances. threshold is the brightness threshold.
And exchanging pointers of the first two-dimensional array and the second two-dimensional array, and repeating the bubbling algorithm calculating step and the brightness threshold judging step.
Wherein, the brightness threshold value can take a fixed value or a floating value. The fixed value can be the maximum brightness error value (the brightness error is the maximum picture brightness value which cannot be seen by naked eyes and is different from black), and the floating value can be 1/8 of the brightness value of the pixel point or the geometric mean value (the square root of the multiplication of the two) of the brightness value of the pixel point and the maximum brightness value error.
If the difference between the pixel point and the next largest brightness value is less than or equal to the brightness threshold, the point is not considered as a suspected noise point, and the brightness judgment of the next point is continued until all points are completely traversed.
And S104, replacing the brightness value of the suspected noise point.
Step S104 may specifically be: and replacing the brightness value of the suspected noise point with the average value of the surrounding adjacent pixel points, or replacing the brightness value of the suspected noise point with the weighted average value of the maximum brightness value and the secondary maximum brightness value. For night scenes, consider the case where the background moves rapidly alone causing the spot to always move out of our detection range (only one pixel wide), such as distant car lights, or distant road lights. Considering that the influence of a far automobile on driving is not large, most lamplight is not always only 1 pixel wide, and can be ignored, if such a situation needs to be considered, the suspected noise point can be replaced by a weighted average value of the brightest point and the secondary bright point.
Further, in step s103, if the luminance difference is greater than the luminance threshold, the method may further include: calculating the difference value between the brightness value of the pixel point and the maximum brightness values of the pixel point at the same position in the previous frame and the adjacent pixel points around the pixel point; if the difference value is smaller than a preset brightness threshold value, determining the pixel point as a non-noise point, and if not, determining the pixel point as a noise point;
after the comparison with the neighborhood pixel points in the same frame, the comparison with the brightness value of the pixel points in the same position of the previous frame is carried out so as to further determine whether the pixel points are noise points.
Further optionally, color noise correction may also be done for low brightness video-brightened images. The method further comprises:
converting the video frame image to an HSI color space;
and if the difference between the chromatic value H of the pixel point and 8 surrounding points is greater than a chromatic threshold, the chromatic difference between the pixel point and the point at the same position of the previous frame is greater than the chromatic threshold, the brightness value I of the pixel point is less than the preset brightness threshold, and the color saturation S is greater than a color saturation threshold, determining the pixel point as a color noise point. Wherein, the chroma threshold value can be pi/2, and the color saturation threshold value can be 0.5;
and replacing the chromatic value of the color noise point with the chromatic value of the point which is closest to the current chromatic value in 8 points around the pixel point and the same position points of the previous frame.
Further, in the above embodiment, only the case where the noise is the maximum brightness value is described, and for the minimum brightness value, the corresponding process may be repeated, that is, the comparison and determination are performed with the minimum brightness value as the standard.
Different from the traditional noise reduction method, the method provided by the invention considers the similarity of the video images through the dependency relationship between the front and the back of the video images, can quickly and efficiently identify the noise points, greatly improves the noise reduction speed and efficiency, and particularly easily generates the noise points in a low-brightness environment. Meanwhile, general noise reduction needs to depend on optical hardware, but the method is based on digital image processing and does not need to depend on optical hardware, so that the cost can be greatly reduced.
The embodiment of the invention also provides a device for quickly reducing noise of a picture by utilizing the inter-frame difference of the video image, which comprises the following steps: the processor is used for executing the method for rapidly reducing the noise of the picture by utilizing the video image inter-frame difference when the computer program is run.
The embodiment of the invention also provides a storage medium, which stores computer instructions, and the instructions are executed by a processor to realize the method for quickly reducing the noise of the picture by utilizing the difference between the video image frames.
Fig. 2 is a schematic structural diagram of a server according to an embodiment of the present invention. The server 200 may include one or more Central Processing Units (CPUs) 210 (e.g., one or more processors) and memory 220, one or more storage media 230 (e.g., one or more mass storage devices) that store applications 232 or data 234. Memory 220 and storage medium 230 may be, among other things, transient or persistent storage. The program stored in the storage medium 230 may include one or more modules (not shown), each of which may include a series of instruction operations for the server. Further, the central processor 210 may be configured to communicate with the storage medium 230, and execute a series of instruction operations in the storage medium 230 on the server 200. Server 200 may also include one or more power supplies 240, one or more wired or wireless network interfaces 250, one or more input-output interfaces 260, and/or one or more operating systems 270, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so forth. The steps performed by the above-described method embodiments may be based on the server architecture shown in fig. 2.
It should be understood that, in the various embodiments of the present application, the size of the serial number of each process does not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative modules and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the apparatus and the module described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
All parts of the specification are described in a progressive mode, the same and similar parts of all embodiments can be referred to each other, and each embodiment is mainly introduced to be different from other embodiments. In particular, as to the apparatus and system embodiments, since they are substantially similar to the method embodiments, the description is relatively simple and reference may be made to the description of the method embodiments in relevant places.
Finally, it is to be noted that: the above description is only a preferred embodiment of the present disclosure, and is not intended to limit the scope of the present disclosure. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. To the extent that such modifications and variations of the present application fall within the scope of the claims and their equivalents, they are intended to be included within the scope of the present application.