技术领域technical field
本发明涉及图像处理领域，尤其涉及一种生成RGB图像的方法、装置和系统。The present invention relates to the field of image processing, in particular to a method, device and system for generating RGB images.
背景技术Background technique
RGBIR传感器是一种新型的传感器，是配置有RGBIR滤镜的图像传感器。RGBIR滤镜的阵列排列方式与传统彩色滤镜阵列(Color Filter Array，CFA)，比如拜耳彩色滤镜阵列(Bayer CFA)的排列方式不同，如图1和图2所示。图1为Bayer传感器的阵列示意图，R表示图像传感器上所有覆盖红色滤镜的像素，称为红色(Red，R)通道；G表示图像传感器上所有覆盖绿色滤镜的像素，称为绿色(Green，G)通道，B表示覆盖蓝色的像素，称为蓝色(Blue，B)通道。图2为RGBIR传感器的阵列示意图，RGBIR滤镜的阵列将Bayer彩色滤镜阵列中的一半的绿色分量由红外光(infrared)分量取代，该部分像素称为红外(Infrared，IR)通道，且RGBIR滤镜可以同时感应可见光和不可见光，这里将不可见光用红外光IR表示。为了基于RGBIR传感器得到正常的RGB彩色图像，需要将RGBIR传感器感知到的R、G、B通道像素中的IR成分去除。The RGBIR sensor is a new type of sensor, which is an image sensor equipped with an RGBIR filter. The array arrangement of the RGBIR filter is different from that of a traditional color filter array (Color Filter Array, CFA), such as a Bayer CFA, as shown in FIG. 1 and FIG. 2 . Figure 1 is a schematic diagram of the Bayer sensor array, R represents all pixels covered with red filters on the image sensor, called red (Red, R) channel; G represents all pixels covered with green filters on the image sensor, called green (Green) , G) channel, and B represents the pixel covering blue, which is called the blue (Blue, B) channel. Figure 2 is a schematic diagram of an array of RGBIR sensors. The array of RGBIR filters replaces half of the green components in the Bayer color filter array with infrared (infrared) components. This part of the pixels is called an infrared (Infrared, IR) channel, and RGBIR The filter can sense visible light and invisible light at the same time, here the invisible light is represented by infrared light IR. In order to obtain a normal RGB color image based on the RGBIR sensor, it is necessary to remove the IR components in the R, G, and B channel pixels perceived by the RGBIR sensor.
现有技术中，可以通过对输入的RGBIR图像进行图像插值，得到原始图像中每个像素点位置上的缺失的像素，计算出输出图像中的RGB通道和IR通道的数值，确定校正系数，根据校正系数分离RGB通道和红外IR通道，并校正RGB通道，最后输出RGB图像。In the prior art, the missing pixel at each pixel position in the original image can be obtained by image interpolation on the input RGBIR image, and the values of the RGB channel and the IR channel in the output image can be calculated to determine the correction coefficient, according to The correction coefficient separates the RGB channel and the infrared IR channel, and corrects the RGB channel, and finally outputs the RGB image.
但通过现有技术中的获取基于RGBIR传感器获取的RGBIR图像会造成输出的RGB图像的信噪比较低，尤其是在色温较高或者光照度较低的情况下。However, acquiring the RGBIR image acquired based on the RGBIR sensor in the prior art will result in a low signal-to-noise ratio of the output RGB image, especially when the color temperature is high or the illuminance is low.
发明内容Contents of the invention
本发明提供了一种生成RGB图像的方法、装置和系统，以实现在色温较高或者光照度比较低的情况下，提高RGB图像的信噪比。The invention provides a method, device and system for generating an RGB image, so as to improve the signal-to-noise ratio of the RGB image under the condition of high color temperature or relatively low illuminance.
第一方面，本发明提供了一种生成RGB图像的方法，该方法包括：接收RGBIR传感器发送的RGBIR图像信号；通过内插法对输入RGBIR图像信号进行插值得到信噪比比较高的第一图像信号，第一图像信号可以理解为每个像素点位置上带有红外IR分量的拜耳(Bayer)图像信号；去除第一图像中每个像素点位置上的红外IR分量，并对RGB通道进行校正，得到第二图像信号，其中，第二图像信号可以理解为相对RGBIR图像信号去除每个像素点位置上的IR分量后的Bayer图像；将第一图像信号和第二信号图像进行联合滤波得到滤波后的RGB图像。In a first aspect, the present invention provides a method for generating an RGB image, the method comprising: receiving an RGBIR image signal sent by an RGBIR sensor; interpolating the input RGBIR image signal by an interpolation method to obtain a first image with a relatively high signal-to-noise ratio signal, the first image signal can be understood as a Bayer (Bayer) image signal with an infrared IR component at each pixel position; remove the infrared IR component at each pixel position in the first image, and correct the RGB channel , to obtain the second image signal, wherein the second image signal can be understood as the Bayer image after removing the IR component at each pixel position relative to the RGBIR image signal; the first image signal and the second signal image are jointly filtered to obtain the filtered After the RGB image.
本发明提供的联合滤波，可以通过例如指导性滤波器(Guided Filter)、联合双边滤波器(Joint Bilateral Filter)等滤波器，依据第一图像信号与第二图像信号的相关性，对第二图像信号进行滤波得到滤波后的RGB图像。The joint filtering provided by the present invention can filter the second image according to the correlation between the first image signal and the second image signal through filters such as guided filter (Guided Filter) and joint bilateral filter (Joint Bilateral Filter). The signal is filtered to obtain a filtered RGB image.
通过本发明实施例提供的生成RGB图像的方法，利用未去除IR分量的RGBIR图像信噪比高的特点，在噪声去除的过程中，利用未去除IR分量的RGBIR图像信号与去除IR分量的RGBIR图像信号的相关性，对去除IR分量的低信噪比RGBIR图像信号进行联合滤波，实现了对基于RGBIR传感器获取的图像的处理，提高了处理后得到的RGB图像的信噪比；同时，也明显改善了在低色温或低照度环境下得到的图像的信噪比。The method for generating RGB images provided by the embodiments of the present invention utilizes the high signal-to-noise ratio of the RGBIR images without the IR components removed, and uses the RGBIR image signals without the IR components removed and the RGBIR images without the IR components removed during the noise removal process. The correlation of the image signal, the low signal-to-noise ratio RGBIR image signal that removes the IR component is jointly filtered, and the image processing based on the RGBIR sensor is realized, and the signal-to-noise ratio of the RGB image obtained after processing is improved; at the same time, it is also Significantly improved the signal-to-noise ratio of images obtained under low color temperature or low illumination environments.
结合第一方面，在第一方面的第一种可能实现的方式中，通过内插法对RGBIR图像信号进行插值以得到第一图像信号，包括：通过内插法计算RGBIR图像信号中每个像素点IR通道上的G像素值以得到第一图像信号。With reference to the first aspect, in the first possible implementation manner of the first aspect, the interpolation method is used to interpolate the RGBIR image signal to obtain the first image signal, including: calculating the value of each pixel in the RGBIR image signal by interpolation method Point the G pixel value on the IR channel to get the first image signal.
结合第一方面，在第一方面的第二种可能实现的方式中，通过内插法对所述RGBIR图像信号进行插值以得到第一图像信号，包括：With reference to the first aspect, in a second possible implementation manner of the first aspect, the RGBIR image signal is interpolated to obtain the first image signal by an interpolation method, including:
通过内插法计算RGBIR图像信号中每个像素点上R、G、B、IR通道中的R像素值，G像素值和B像素值，以得到第一图像信号；Calculate the R pixel value in the R, G, B, and IR channels of each pixel in the RGBIR image signal by interpolation, the G pixel value and the B pixel value, to obtain the first image signal;
其中，第一图像信号包括包括三个图像信号，一个图像信号的每个像素点的R、G、B和IR通道均为R像素值和IR像素值的图像信号；一个图像信号的每个像素点的R、G、B和IR通道均为G像素值和IR像素值的图像信号，另一个图像信号的每个像素点的R、G、B和IR通道均为B像素值和IR像素值的图像信号。Wherein, the first image signal includes three image signals, the R, G, B and IR channels of each pixel of an image signal are image signals with R pixel values and IR pixel values; each pixel of an image signal The R, G, B and IR channels of a point are image signals of G pixel value and IR pixel value, and the R, G, B and IR channels of each pixel of another image signal are B pixel value and IR pixel value image signal.
结合第一方面的第一种可能实现的方式或者第一方面的第二种可能实现的方式中，在第一方面的第三种可能实现的方式中，去除第一图像信号中每个像素点位置上的红外IR分量以得到第二图像信号，包括：通过内插法计算RGBIR图像信号中每个像素点位置上的IR像素值以得到第三图像信号，第三图像信号为每个像素点的R、G、B和IR通道均为IR像素值的图像信号；将第一图像信号和第三图像信号相减以得到第二图像信号。通过该设计方案，可以将RGBIR传感器获取的RGBIR图像信号转换为传统的Bayer图像信号，易于以传统的图像信号处理器(Image Signal Processor，ISP)对接，可以采用传统算法进行处理，减少了开发的成本。In combination with the first possible implementation of the first aspect or the second possible implementation of the first aspect, in the third possible implementation of the first aspect, removing each pixel in the first image signal Infrared IR components on the position to obtain the second image signal, including: calculating the IR pixel value on the position of each pixel point in the RGBIR image signal by interpolation to obtain the third image signal, the third image signal is each pixel point The R, G, B and IR channels are image signals of IR pixel values; the first image signal and the third image signal are subtracted to obtain a second image signal. Through this design scheme, the RGBIR image signal acquired by the RGBIR sensor can be converted into a traditional Bayer image signal, which is easy to interface with a traditional Image Signal Processor (ISP), and can be processed by traditional algorithms, reducing the development effort. cost.
结合第一方面的第三种可能实现的方式，在第一方面的第四种可能实现的方式中，将第一图像信号和第二图像信号进行联合滤波以得到滤波后的RGB图像，包括：In combination with the third possible implementation manner of the first aspect, in the fourth possible implementation manner of the first aspect, jointly filtering the first image signal and the second image signal to obtain a filtered RGB image includes:
根据第一图像信号中的第一像素点和第一参考像素点的距离或者像素值的差值确定权重，其中，第一参考像素点为所述第一像素点周围的像素点；Determine the weight according to the distance between the first pixel in the first image signal and the first reference pixel or the difference in pixel value, where the first reference pixel is a pixel around the first pixel;
确定第二图像信号中与第一像素点对应的第二像素点和与第一参考像素点对应的第二参考像素点；determining a second pixel point corresponding to the first pixel point and a second reference pixel point corresponding to the first reference pixel point in the second image signal;
根据权重对第二图像信号中的第二参考像素点进行加权平均，确定第二像素点的像素值，以得到滤波后的RGB图像。A weighted average is performed on the second reference pixel points in the second image signal according to the weights, and the pixel values of the second pixel points are determined to obtain a filtered RGB image.
结合第一方面至第一方面的第四种可能实现的方式，在第一方面的第五种可能实现的方式中，滤波后的RGB图像用于提供给图像信号处理器进行处理以得到输出的RGB图像。In combination with the first aspect to the fourth possible implementation manner of the first aspect, in the fifth possible implementation manner of the first aspect, the filtered RGB image is provided to an image signal processor for processing to obtain an output RGB image.
结合第一方面，在第一方面的第六种可能实现的方式中，通过图像信号处理器对滤波后的RGB图像进行处理以得到RGB图像，包括：With reference to the first aspect, in the sixth possible implementation manner of the first aspect, the filtered RGB image is processed by an image signal processor to obtain an RGB image, including:
对滤波后的RGB图像进行颜色校正矩阵、镜头阴影校正、自动白平衡或马赛克处理中的至少一个处理得到输出的RGB图像。Perform at least one of color correction matrix, lens shading correction, automatic white balance or mosaic processing on the filtered RGB image to obtain an output RGB image.
结合第一方面，在第一方面的第七种可能实现的方式中，图像信号处理器对滤波后的RGB图像进行处理以得到输出的RGB图像，包括：With reference to the first aspect, in a seventh possible implementation manner of the first aspect, the image signal processor processes the filtered RGB image to obtain an output RGB image, including:
通过内插法计算所述滤波后的RGB图像中每个像素点R、G和B通道上的R、G和B像素值，以得到第四图像信号；Calculating the R, G and B pixel values on the R, G and B channels of each pixel in the filtered RGB image by interpolation to obtain a fourth image signal;
对第四图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡处理得到所述输出的RGB图像；Performing color correction matrix, lens shading correction, and automatic white balance processing on the fourth image signal to obtain the output RGB image;
其中，第四图像信号包括每个像素点的R、G、B和IR通道均为R像素值的图像信号，每个像素点的R、G、B和IR通道均为G像素值的图像信号，以及每个像素点的R、G、B和IR通道均为B像素值的图像信号。Wherein, the fourth image signal includes an image signal in which the R, G, B and IR channels of each pixel are all R pixel values, and the R, G, B and IR channels of each pixel are all image signals in G pixel values , and the R, G, B and IR channels of each pixel are image signals with B pixel values.
结合第一方面至第一方面中的任一可能实现的方式中的任一实现方式，在第一方面的第八种可能实现的方式中，内插法包括双线性插值方法。With reference to any implementation manner of any possible implementation manner of the first aspect to the first aspect, in an eighth possible implementation manner of the first aspect, the interpolation method includes a bilinear interpolation method.
第二方面，本发明提供了一种生成RGB图像的装置，该装置包括：接收单元、插值单元、去除分量单元和滤波单元。接收单元用于接收RGBIR传感器发送的RGBIR图像信号；插值单元，用于通过内插法对RGBIR图像信号进行插值以得到第一图像信号；去除分量单元，用于去除第一图像信号中每个像素点位置上的红外IR分量以得到第二图像信号；滤波单元，用于将第一图像信号和第二图像信号进行联合滤波以得到滤波后的RGB图像。In a second aspect, the present invention provides a device for generating an RGB image, the device comprising: a receiving unit, an interpolation unit, a component removal unit and a filtering unit. The receiving unit is used to receive the RGBIR image signal sent by the RGBIR sensor; the interpolation unit is used to interpolate the RGBIR image signal by interpolation to obtain the first image signal; the component removal unit is used to remove each pixel in the first image signal The infrared IR component at the point position to obtain a second image signal; the filtering unit is used to jointly filter the first image signal and the second image signal to obtain a filtered RGB image.
其中，第一图像信号可以理解为每个像素点位置上带有IR分量的Bayer图像信号；第二图像信号可以理解为RGBIR图像去除每个像素点位置上的IR分量后的Bayer图像信号。Wherein, the first image signal can be understood as a Bayer image signal with an IR component at each pixel position; the second image signal can be understood as a Bayer image signal after removing the IR component at each pixel position of an RGBIR image.
在本发明联合滤波可以通过例如Guided Filter、Joint Bilateral Filter等滤波器，依据第一图像信噪比的相关性，对第二图像进行滤波，得到滤波后的RGB图像。In the joint filtering of the present invention, filters such as Guided Filter and Joint Bilateral Filter can be used to filter the second image according to the correlation of the signal-to-noise ratio of the first image to obtain a filtered RGB image.
通过本发明实施例提供的装置，利用未去除IR分量的RGBIR图像信噪比高的特点，在噪声去除的过程中，利用未去除IR分量的RGBIR图像信号与去除IR分量的RGBIR图像信号的相关性，对去除IR分量的低信噪比RGBIR图像信号进行联合滤波，实现了对基于RGBIR传感器获取的图像的处理，提高了处理后得到的RGB图像的信噪比；同时，也明显改善了在低色温或低照度环境下得到的图像的信噪比。Through the device provided by the embodiment of the present invention, the RGBIR image signal without the IR component removed has a high signal-to-noise ratio, and in the process of noise removal, the correlation between the RGBIR image signal without the IR component removed and the RGBIR image signal removed from the IR component is used. performance, and jointly filter the low signal-to-noise ratio RGBIR image signal that removes the IR component, realizes the processing of the image acquired based on the RGBIR sensor, and improves the signal-to-noise ratio of the processed RGB image; at the same time, it also significantly improves the The signal-to-noise ratio of images obtained in low color temperature or low-light environments.
结合第二方面，在第二方面的第一种可能实现的方式中，插值单元具体用于，通过内插法计算RGBIR图像信号中每个像素点IR通道上的G像素进行插值以得到第一图像。With reference to the second aspect, in the first possible implementation manner of the second aspect, the interpolation unit is specifically configured to calculate the G pixel on the IR channel of each pixel in the RGBIR image signal by interpolation and perform interpolation to obtain the first image.
结合第二方面，在第二方面的第二种可能实现的方式中，插值单元具体用于，With reference to the second aspect, in a second possible implementation manner of the second aspect, the interpolation unit is specifically configured to:
通过内插法计算RGBIR图像信号中每个像素点上R、G、B、IR通道中的R像素值，G像素值和B像素值，以得到第一图像信号；Calculate the R pixel value in the R, G, B, and IR channels of each pixel in the RGBIR image signal by interpolation, the G pixel value and the B pixel value, to obtain the first image signal;
其中，第一图像信号包括每个像素点的R、G、B和IR通道均为R像素值和IR像素值的图像信号，每个像素点的R、G、B和IR通道均为G像素值和IR像素值的图像信号，以及每个像素点的R、G、B和IR通道均为B像素值和IR像素值的图像信号。Wherein, the first image signal includes an image signal in which the R, G, B and IR channels of each pixel are R pixel values and IR pixel values, and the R, G, B and IR channels of each pixel are G pixels The image signal of value and IR pixel value, and the R, G, B and IR channels of each pixel are image signals of B pixel value and IR pixel value.
结合第二方面的第一种可能实现的方式或者第二方面的第二种可能实现的方式，在第二方面的第三种可能实现的方式中，去除分量单元具体用于，通过内插法极端所述RGBIR图像信号中每个像素点位置上的IR像素值，以得到第三图像信号，第三图像信号为每个像素点的R、G、B和IR通道均为IR像素值的图像信号；将第一图像信号和第三图像信号进行相减，以得到第二图像信号。In combination with the first possible implementation manner of the second aspect or the second possible implementation manner of the second aspect, in the third possible implementation manner of the second aspect, the component removal unit is specifically configured to use an interpolation method Extremely the IR pixel value on each pixel position in the RGBIR image signal to obtain a third image signal, the third image signal is an image in which the R, G, B and IR channels of each pixel point are all IR pixel values signal; subtracting the first image signal from the third image signal to obtain a second image signal.
结合第二方面的第三种可能实现的方式，在第二方面的第四中可能实现的方式中，滤波单元具体用于，In combination with the third possible implementation manner of the second aspect, in the fourth possible implementation manner of the second aspect, the filtering unit is specifically configured to:
根据第一图像信号中的第一像素点和第一参考像素点的距离或者像素值的差值确定权重，其中，第一参考像素点为第一像素点周围的像素点；Determine the weight according to the distance between the first pixel in the first image signal and the first reference pixel or the difference in pixel value, where the first reference pixel is a pixel around the first pixel;
确定第二图像信号中与第一像素点对应的第二像素点和与第一参考像素点对应的第二参考像素点；determining a second pixel point corresponding to the first pixel point and a second reference pixel point corresponding to the first reference pixel point in the second image signal;
根据权重对第二图像信号中的第二参考像素点进行加权平均，确定第二像素点的像素值，以得到滤波后的RGB图像。A weighted average is performed on the second reference pixel points in the second image signal according to the weights, and the pixel values of the second pixel points are determined to obtain a filtered RGB image.
结合第二方面至第二方面的第五种可能实现的方式中，滤波后的图像用于提供给图像信号处理器进行处理以得到输出的RGB图像。In a fifth possible implementation manner combining the second aspect to the second aspect, the filtered image is provided to an image signal processor for processing to obtain an output RGB image.
结合第二方面，在第二方面的第六种可能实现的方式中，图像信号处理器具体用于，With reference to the second aspect, in a sixth possible implementation manner of the second aspect, the image signal processor is specifically configured to:
对滤波后的RGB图像进行颜色校正矩阵、镜头阴影校正、自动白平衡或马赛克处理中的至少一个处理得到输出的RGB图像。Perform at least one of color correction matrix, lens shading correction, automatic white balance or mosaic processing on the filtered RGB image to obtain an output RGB image.
结合第二方面，在第二方面的第七种可能实现的方式中，图像信号处理器具体用于，With reference to the second aspect, in a seventh possible implementation manner of the second aspect, the image signal processor is specifically configured to:
通过内插法计算滤波后的RGB图像中每个像素点R、G和B通道上的R、G和B像素值，以得到第四图像信号；R, G and B pixel values on each pixel point R, G and B channels in the filtered RGB image are calculated by interpolation to obtain a fourth image signal;
对第四图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡处理得到输出的RGB图像；Performing color correction matrix, lens shading correction, and automatic white balance processing on the fourth image signal to obtain an output RGB image;
其中，第四图像信号包括三个图像信号，一个图像信号中每个像素点的R、G、B和IR通道均为R像素值的图像信号；一个图像信号中每个像素点的R、G、B和IR通道均为G像素值的图像信号；另一个图像信号中每个像素点的R、G、B和IR通道均为B像素值的图像信号。Wherein, the fourth image signal includes three image signals, the R, G, B and IR channels of each pixel in an image signal are image signals with R pixel values; the R, G channels of each pixel in an image signal , B and IR channels are image signals with G pixel values; the R, G, B and IR channels of each pixel in the other image signal are image signals with B pixel values.
结合第二方面至第二方面的任一种可能实现方式中的任一可能实现的方式，在第二方面的第八种可能实现的方式中，内插法包括双线性插值方法。In combination with any possible implementation manner of any one of the possible implementation manners of the second aspect to the second aspect, in an eighth possible implementation manner of the second aspect, the interpolation method includes a bilinear interpolation method.
第三方面，本发明实施例提供一种系统，该系统包括RGBIR传感器、存储器和处理器。In a third aspect, an embodiment of the present invention provides a system, and the system includes an RGBIR sensor, a memory, and a processor.
RGBIR传感器，用于接收RGBIR图像信号；RGBIR sensor, for receiving RGBIR image signal;
存储器，用于存储指令和RGBIR图像信号；A memory for storing instructions and RGBIR image signals;
处理器，用于调用所述存储器中的指令，并执行完成第一方面中任一种可能实现的方式以及第二方面中任一项可能实现的方式。A processor, configured to invoke instructions in the memory, and execute any possible implementation manner in the first aspect and any possible implementation manner in the second aspect.
基于本发明提供的一种生成RGB图像的方法、装置和系统，通过利用原始RGBIR图像信号信噪比高的特点，也就是未去除每个像素点位置上的IR分量的图像信号(第一图像信号)信噪比高的特点，将其与去除每个像素点位置上的IR分量的图像信号(第二图像信号)进行联合滤波，在噪声去除的过程中，利用第一图像信号的高信噪比信号的相关性对第二图像信号的低信噪比信号滤波，实现了对基于RGBIR传感器获取的图像的处理，提高了处理后得到的RGB图像的信噪比。Based on a method, device and system for generating RGB images provided by the present invention, by utilizing the high signal-to-noise ratio of the original RGBIR image signal, that is, the image signal without removing the IR component at each pixel position (the first image signal) with a high signal-to-noise ratio, it is jointly filtered with the image signal (second image signal) that removes the IR component at each pixel position, and in the process of noise removal, the high signal of the first image signal is used The correlation of the noise ratio signal filters the low signal-to-noise ratio signal of the second image signal, realizes the processing of the image acquired based on the RGBIR sensor, and improves the signal-to-noise ratio of the processed RGB image.
附图说明Description of drawings
为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例中所需要使用的附图作简单地介绍，显而易见地，下面所描述的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required in the embodiments of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.
图1为Bayer传感器的阵列示意图；Fig. 1 is the array schematic diagram of Bayer sensor;
图2为RGBIR传感器的阵列示意图；Figure 2 is a schematic diagram of an array of RGBIR sensors;
图3为本发明实施例提供的一种生成RGB图像的装置结构示意图；3 is a schematic structural diagram of a device for generating RGB images provided by an embodiment of the present invention;
图4为本发明实施例提供的一种生成RGB图像的方法流程图；FIG. 4 is a flow chart of a method for generating an RGB image provided by an embodiment of the present invention;
图5为本发明实施例提供的另一种生成RGB图像的方法流程图；FIG. 5 is a flow chart of another method for generating an RGB image provided by an embodiment of the present invention;
图6为生成RGB图像的处理过程简图；Fig. 6 is a schematic diagram of the process of generating an RGB image;
图7为本发明实施例提供的另一种生成RGB图像的方法流程图；FIG. 7 is a flow chart of another method for generating an RGB image provided by an embodiment of the present invention;
图8为本发明实施例提供的另一种生成RGB图像的处理过程简图；FIG. 8 is a schematic diagram of another processing procedure for generating an RGB image provided by an embodiment of the present invention;
图9为本发明实施例提供的又一种生成RGB图像的处理过程简称；FIG. 9 is an abbreviation for another processing procedure for generating an RGB image provided by an embodiment of the present invention;
图10为本发明实施例提供的一种生成RGB图像的装置结构示意图。FIG. 10 is a schematic structural diagram of an apparatus for generating RGB images provided by an embodiment of the present invention.
具体实施方式Detailed ways
为了让RGBIR传感器得到正常的RGB图像，需要将RGBIR传感器中红色(R)通道、绿色(G)通道和蓝色(B)通道中感知到的不可见光部分去除。在本发明中将RGBIR传感器感知到的不可见光写为红外光(infrared light，IR)。本发明提供了一种新的架构来处理RGBIR传感器获取的图像，使最后得到的RGB图像的信噪比提高。In order for the RGBIR sensor to obtain a normal RGB image, it is necessary to remove the invisible light perceived in the red (R) channel, green (G) channel and blue (B) channel of the RGBIR sensor. In the present invention, the invisible light sensed by the RGBIR sensor is written as infrared light (infrared light, IR). The invention provides a new framework to process images acquired by RGBIR sensors, so that the signal-to-noise ratio of the finally obtained RGB images is improved.
本发明提供的一种提高RGBIR传感器信噪比的方法和装置，根据RGBIR传感器获取的RGBIR图像信号，利用高信噪比RGBIR图像信号中信号的边缘锐度，对去除RGBIR图像中每个像素点位置上的IR分量后的低信噪比图像进行滤波，从而提高最后得到的RGB图像的信噪比。A method and device for improving the signal-to-noise ratio of an RGBIR sensor provided by the present invention, according to the RGBIR image signal obtained by the RGBIR sensor, utilizes the edge sharpness of the signal in the RGBIR image signal with a high signal-to-noise ratio to remove each pixel in the RGBIR image The low signal-to-noise ratio image after the IR component on the position is filtered, thereby improving the signal-to-noise ratio of the final RGB image.
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都应属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.
图3为本发明实施例提供的一种生成RGB图像系统的结构示意图。如图3所示，该系统100可以包括RGBIR传感器110、处理装置120、图像信号处理器130、显示器140和通信总线150，RGBIR传感器110、处理装置120、图像信号处理器(Image signal Processor，ISP)130、显示器140通过通信总线150相互连接，并完成相互之间的通信。FIG. 3 is a schematic structural diagram of a system for generating an RGB image provided by an embodiment of the present invention. As shown in Figure 3, this system 100 can comprise RGBIR sensor 110, processing device 120, image signal processor 130, display 140 and communication bus 150, RGBIR sensor 110, processing device 120, image signal processor (Image signal Processor, ISP ) 130 and the display 140 are connected to each other through the communication bus 150, and complete the mutual communication.
RGBIR传感器110用于接收RGBIR图像信号，并将RGBIR图像信号发送给处理装置120进行图像信号处理。处理装置120用于对RGBIR图像信号进行处理以得到相对现有技术信噪比高的RGB图像信号。图像信号处理器ISP130用于将得到的RGB图像信号进一步处理得到用户可视的RGB图像，并通过显示器140向用户提供显示的RGB图像。The RGBIR sensor 110 is used for receiving RGBIR image signals, and sending the RGBIR image signals to the processing device 120 for image signal processing. The processing device 120 is used for processing the RGBIR image signal to obtain an RGB image signal with a higher signal-to-noise ratio than the prior art. The image signal processor ISP130 is used to further process the obtained RGB image signal to obtain a RGB image visible to the user, and provide the displayed RGB image to the user through the display 140 .
该处理装置120可以包括接收器121、处理器122和存储器123。接收器121用于接收RGBIR传感器发送的RGBIR图像信号。存储器123用来存储指令和接收到的RGBIR图像信号以及经过处理后的RGBIR图像信号。处理器122用于调用存储器中的指令，并执行以下的操作：The processing device 120 may include a receiver 121 , a processor 122 and a memory 123 . The receiver 121 is used for receiving the RGBIR image signal sent by the RGBIR sensor. The memory 123 is used to store instructions, received RGBIR image signals and processed RGBIR image signals. The processor 122 is used to call instructions in the memory, and perform the following operations:
通过内插法对接收到的RGBIR图像信号进行插值以得到第一图像信号；Interpolating the received RGBIR image signal by an interpolation method to obtain a first image signal;
去除第一图像信号中每个像素点位置上的红外IR分量以得到第二图像信号；removing the infrared IR component at each pixel position in the first image signal to obtain a second image signal;
将第一图像信号和第二图像信号进行联合滤波以得到滤波后的RGB图像。The first image signal and the second image signal are jointly filtered to obtain a filtered RGB image.
在本发明实施例中，处理器122可以是中央处理单元(Central Processing Unit，CPU)，该处理器120还可以是其他通用处理器、数字信号处理器(Digital SignalProcessor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。In the embodiment of the present invention, the processor 122 may be a central processing unit (Central Processing Unit, CPU), and the processor 120 may also be other general processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits ( Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
存储器123可以包括只读存储器和随机存取存储器，并向处理器122提供指令和数据。存储器的一部分还可以包括非易失性随机存取存储器。例如，存储器还可以存储设备类型的信息。Memory 123 may include read-only memory and random-access memory, and provides instructions and data to processor 122 . A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
通信总线150除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为通信总线150。In addition to the data bus, the communication bus 150 may also include a power bus, a control bus, a status signal bus, and the like. However, for clarity of illustration, the various buses are labeled as communication bus 150 in the figure.
另外，在本发明实施例中，该系统还可以中还可以包括图像信号处理器130，以用于将滤波后的RGB图像进行处理得到数据的RGB图像，该处理过程可以包括对滤波后的RGB图像进行颜色校正矩阵、镜头阴影校正、自动白平衡或马赛克处理中的至少一个处理得到输出的RGB图像。In addition, in the embodiment of the present invention, the system can also include an image signal processor 130, which is used to process the filtered RGB image to obtain a data RGB image, and the processing process can include filtering the filtered RGB image. The image is processed by at least one of color correction matrix, lens shading correction, automatic white balance or mosaic processing to obtain an output RGB image.
需要说明的是，图像信号处理器130可以设置在处理装置120中，对此在本发明实施例中不作任何限制。It should be noted that the image signal processor 130 may be disposed in the processing device 120, which is not limited in this embodiment of the present invention.
图4为本发明实施例提供的一种提高RGBIR传感器信噪比的方法流程图。如图4所示，该方法200的执行主体可以为图3中所示的系统，该方法200可以具体包括以下步骤：FIG. 4 is a flowchart of a method for improving the signal-to-noise ratio of an RGBIR sensor provided by an embodiment of the present invention. As shown in FIG. 4, the subject of execution of the method 200 may be the system shown in FIG. 3, and the method 200 may specifically include the following steps:
S210，接收RGBIR传感器发送的RGBIR图像信号。S210. Receive an RGBIR image signal sent by the RGBIR sensor.
S220，通过内插法对RGBIR图像信号进行插值以得到第一图像信号。S220. Interpolate the RGBIR image signal by using an interpolation method to obtain a first image signal.
具体的，第一图像信号为对RGBIR传感器采集到的原始RGBIR图像信号进行插值后得到的图像，该图像信号可以理解为每个像素点位置上带有IR分量的拜耳(Bayer)图像信号。Specifically, the first image signal is an image obtained by interpolating the original RGBIR image signal collected by the RGBIR sensor, and the image signal can be understood as a Bayer (Bayer) image signal with an IR component at each pixel position.
需要说明的是，在S220中，对原始RGBIR图像进行插值计算过程中，可以对每个像素点位置上的缺少的颜色和红外光IR分量进行插值。It should be noted that, in S220, during the interpolation calculation process of the original RGBIR image, the missing color and infrared light IR components at each pixel position may be interpolated.
具体的可以获取RGBIR图像信号中缺失颜色对应的颜色通道上该颜色的像素值，如每个像素点位置上的R、G、B、IR通道中R通道上确实G、B、IR的像素值，可以分别从G通道上获取G像素值，B通道上获取B像素值，IR通道上获取IR像素值，并添加到R通道上，简单讲可以将G通道上的G像素值，B通道上的B像素值和IR通道上的IR像素值拷贝(copy)到R通道上。G、B、IR通道上也可以通过该插值方法进行插值，以得到第一图像信号。Specifically, the pixel value of the color on the color channel corresponding to the missing color in the RGBIR image signal can be obtained, such as the pixel values of G, B, and IR on the R channel of the R, G, B, and IR channels at each pixel position , the G pixel value can be obtained from the G channel, the B pixel value can be obtained from the B channel, and the IR pixel value can be obtained from the IR channel, and added to the R channel. Simply put, the G pixel value on the G channel can be obtained from the B channel Copy the B pixel value and the IR pixel value on the IR channel to the R channel. The G, B, and IR channels can also be interpolated by the interpolation method to obtain the first image signal.
S230，去除第一图像信号中每个像素点位置上的红外IR分量以得到第二图像信号。S230. Remove the infrared IR component at each pixel position in the first image signal to obtain a second image signal.
在S230中，可以通过根据第一图像信号确定校正RGBIR图像中的校正系数，根据校正系数校正RGBIR图像，得到去除每个像素点位置上的IR分量的图像，也就是第二图像信号。In S230, the correction coefficient in the corrected RGBIR image can be determined according to the first image signal, and the RGBIR image can be corrected according to the correction coefficient to obtain an image without the IR component at each pixel position, that is, the second image signal.
在本发明实施例中，根据第一图像信号确定校正RGBIR图像的校正系数，可以具体包括：步骤1：在某一光照条件下，也就是某一光照度下，采用红外截止滤光片拍摄标准对色卡，计算对色卡各色块四通道(R、G、B、IR通道)像素的平均值；步骤2：在与步骤1同一光照条件下，搭配可以通过可见光和红外光的滤光片拍摄对色卡，并计算对色卡各色块四通道像素的平均值；步骤3：根据步骤1和步骤2得到的平均值，利用如公式(1)所示的4×4矩阵，将步骤1中得到的平均值作为公式(1)右侧中RGBIR的某一通道的数值代入公式(1)，将步骤2中得到的平均值，作为公式(1)左侧中RGBIR的某一通道的数值代入公式(1)，使用最小二乘法计算得到公式1中的Axx；步骤4：在不同的光照条件下重复步骤1至步骤3的过程，完成公式(1)中不同光照下的校正系数。In the embodiment of the present invention, determining the correction coefficient for correcting the RGBIR image according to the first image signal may specifically include: Step 1: Under a certain lighting condition, that is, under a certain light intensity, use an infrared cut filter to shoot a standard pair Color card, calculate the average value of the four-channel (R, G, B, IR channel) pixels of each color block of the color card; Step 2: Under the same lighting conditions as in step 1, shoot with a filter that can pass through visible light and infrared light Match the color card, and calculate the average value of the four-channel pixels of each color block of the color card; Step 3: According to the average value obtained in step 1 and step 2, use the 4×4 matrix shown in formula (1) to convert the The obtained average value is substituted into formula (1) as the value of a certain channel of RGBIR on the right side of formula (1), and the average value obtained in step 2 is substituted into the value of a certain channel of RGBIR on the left side of formula (1) Formula (1), using the least square method to calculate Axx in Formula 1; Step 4: Repeat the process from Step 1 to Step 3 under different lighting conditions to complete the correction coefficients in Formula (1) under different lighting conditions.
需要说明的是，在本发明实施例中，该校正系数也可以为预先设置好的校正系数表，在需要根据第一图像确定校正系数时，需要统计RGBIR图像中统计区内所有点的RGB通道像素的平均值和IR通道像素的平均值，根据RGB通道像素的平均值和IR通道像素的平均值查询校正系数表，确定与RGBIR图像相匹配的一组校正系数。It should be noted that, in the embodiment of the present invention, the correction coefficient can also be a preset correction coefficient table. When the correction coefficient needs to be determined according to the first image, it is necessary to count the RGB channels of all points in the statistical area in the RGBIR image The average value of pixels and the average value of IR channel pixels, query the correction coefficient table according to the average value of RGB channel pixels and the average value of IR channel pixels, and determine a set of correction coefficients that match the RGBIR image.
然后通过确定的校正系数计算RGBIR图像，在校正过程中去除RGBIR图像信号中每个像素点的IR分量，得到去除IR分量的Bayer图像信号。Then the RGBIR image is calculated by the determined correction coefficient, and the IR component of each pixel in the RGBIR image signal is removed during the correction process to obtain the Bayer image signal with the IR component removed.
可选地，在本发明实施例中，可以采用内插法对RGBIR图像信号进行插值计算，得到RGBIR图像信号中每个像素点的R通道、G通道、B通道和IR通道上的IR分量，得到RGBIR图像信号中每个像素点的R、G、B、IR通道上均为IR分量的图像信号，在与对S220中得到的RGBIR图像信号中每个像素点位置上带有IR分量的拜耳(Bayer)图像信号进行相减得到去除IR分量的Bayer图像信号。Optionally, in the embodiment of the present invention, an interpolation method can be used to interpolate the RGBIR image signal to obtain the IR components on the R channel, G channel, B channel and IR channel of each pixel in the RGBIR image signal, Obtain the image signal of the IR component on the R, G, B, and IR channels of each pixel in the RGBIR image signal, and the Bayer with the IR component on each pixel position in the RGBIR image signal obtained in S220 The (Bayer) image signal is subtracted to obtain the Bayer image signal with the IR component removed.
S240，将第一图像信号和第二图像信号进行联合滤波以滤波后的RGB图像。S240. Jointly filter the first image signal and the second image signal to obtain a filtered RGB image.
可选地，在本发明实施例中，根据第一图像信号对第二图像信号进行联合滤波以得到输出图像信号，可以包括：Optionally, in this embodiment of the present invention, performing joint filtering on the second image signal according to the first image signal to obtain an output image signal may include:
根据第一图像信号中的第一像素点和第一参考像素点的距离或者像素值的差值确定权重，其中，第一参考像素点为第一像素点周围的像素点；确定第二图像信号中与第一像素点对应的第二像素点和与第一参考像素点对应的第二参考像素点；根据权重对所述第二图像信号中的第二参考像素点进行加权平均，确定第二像素点的像素值，以得到滤波后的RGB图像。Determine the weight according to the distance between the first pixel in the first image signal and the first reference pixel or the difference in pixel value, where the first reference pixel is the pixels around the first pixel; determine the second image signal The second pixel point corresponding to the first pixel point and the second reference pixel point corresponding to the first reference pixel point in the second image signal; carry out weighted average on the second reference pixel point in the second image signal according to the weight, and determine the second The pixel value of the pixel to get the filtered RGB image.
简单讲，将S220中得到的每个像素点位置上带有IR分量的Bayer图像信号和去除RGBIR图像信号中每个像素点位置上的IR分量的Bayer图像信号进行联合滤波，该联合滤波过程可以根据未去除IR分量的Bayer图像信号中的信号边缘锐度确定差值权重，根据确定的差值权重确定去除IR分量的Bayer图像信号中每个像素点的像素值：将去除IR分量的Bayer图像信号中每个像素点的像素值用该像素点的像素值与其周围的像素值的加权平均确定，以实现对去除IR分量的Bayer图像信号的滤波。其过程例如：当在图像边缘处，像素间差值较大时，将其做加权平均的权重值减小，或者说尽量保持原始像素值，以保持图像信号的边缘锐度。Simply speaking, the Bayer image signal with the IR component at each pixel position obtained in S220 and the Bayer image signal that removes the IR component at each pixel position in the RGBIR image signal are jointly filtered, and the joint filtering process can be The difference weight is determined according to the signal edge sharpness in the Bayer image signal without the IR component, and the pixel value of each pixel in the Bayer image signal with the IR component removed is determined according to the determined difference weight: the Bayer image with the IR component removed The pixel value of each pixel in the signal is determined by the weighted average of the pixel value of the pixel and its surrounding pixel values, so as to realize the filtering of the Bayer image signal that removes the IR component. The process is, for example: when the difference between pixels is large at the edge of the image, the weight value of the weighted average is reduced, or the original pixel value is kept as far as possible to maintain the edge sharpness of the image signal.
在本发明实施例中，根据未去除IR分量的图像对去除IR分量的图像进行联合滤波，还可以采用指导像滤波器(Guided Filter)，或者采用联合双边滤波器，在本发明实施例中，对此不作任何限制。In the embodiment of the present invention, the image without the IR component is jointly filtered according to the image without the IR component removed, and a guided image filter (Guided Filter) can also be used, or a joint bilateral filter can be used. In the embodiment of the present invention, There are no restrictions on this.
通过本发明实施例提供的生成RGB图像的方法，利用原始RGBIR图像信噪比高的特点，在噪声去除的过程中，利用未去除IR分量的高信噪比图像信号与去除IR分量的低信噪比图像信号的相关性，对去除IR分量的RGBIR图像信号进行联合滤波，实现了对基于RGBIR传感器获取的图像的处理，提高了处理后得到的RGB图像的信噪比；同时，也明显改善了在低色温或低照度环境下得到的图像的信噪比。The method for generating RGB images provided by the embodiments of the present invention utilizes the high signal-to-noise ratio of the original RGBIR image. The correlation of the noise ratio image signal, the joint filtering of the RGBIR image signal with the IR component removed, realizes the processing of the image obtained based on the RGBIR sensor, and improves the signal-to-noise ratio of the processed RGB image; at the same time, it also significantly improves It improves the signal-to-noise ratio of images obtained in low color temperature or low illumination environments.
可选地，作为本发明的另一实施例，该方法200还可以包括：Optionally, as another embodiment of the present invention, the method 200 may also include:
S250，将滤波后的RGB图像提供给图像信号处理器进行处理以得到输出的RGB图像。S250. Provide the filtered RGB image to the image signal processor for processing to obtain an output RGB image.
具体的，根据未去除IR分量的高信噪比RGBIR图像信号，对去除IR分量的低信噪比图像信号进行联合滤波后的图像信号是不能被识别的，需要通过图像信号处理器(ImageSignal Processor，ISP)进行处理，得到可以识别的图像。Specifically, according to the high signal-to-noise ratio RGBIR image signal without the IR component removed, the image signal after the joint filtering of the low signal-to-noise ratio image signal without the IR component cannot be recognized, and needs to be processed by an image signal processor (ImageSignal Processor). , ISP) for processing to obtain recognizable images.
图像信号处理器需要对输出图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡以及马赛克等处理得到输出的RGB图像。The image signal processor needs to perform color correction matrix, lens shading correction, automatic white balance and mosaic processing on the output image signal to obtain the output RGB image.
下面根据图5和图9对本发明的技术方案进行详细的说明。The technical solution of the present invention will be described in detail below with reference to FIG. 5 and FIG. 9 .
图5为本发明实施例提供的一种提高RGB图像信噪比的方法的流程图。如图5所示，该方法300的执行主体可以为如图3所示的系统，该方法300具体包括以下步骤：FIG. 5 is a flow chart of a method for improving the signal-to-noise ratio of an RGB image provided by an embodiment of the present invention. As shown in FIG. 5, the subject of execution of the method 300 may be the system shown in FIG. 3, and the method 300 specifically includes the following steps:
S310，通过内插法计算接收到的RGBIR图像信号中每个像素点IR通道上的G像素值得到第一图像信号。S310. Calculate the G pixel value on the IR channel of each pixel in the received RGBIR image signal by an interpolation method to obtain a first image signal.
具体的，在本发明实施例中，可以通过双线性插值的方法对RGBIR图像信号中的IR通道的像素进行插值计算，得到IR通道位置的G像素值，以得到第一图像信号，如图6所示。Specifically, in the embodiment of the present invention, the pixels of the IR channel in the RGBIR image signal can be interpolated by bilinear interpolation to obtain the G pixel value at the position of the IR channel to obtain the first image signal, as shown in the figure 6.
需要说明的是，在本发明实施例中，通过双线性插值的方法得到IR通道位置上的G像素的数值只是一种实现方式，在本发明实施例中，还可以通过其他方式实现RGBIR图像的IR通道位置上的G像素的数值，例如：在计算IR通道上的G像素值时，可以将G通道中的G像素的数值拷贝(copy)到IR通道的位置，得到IR通道位置上的G像素值，在本发明实施例中对此不作任何限制。It should be noted that, in the embodiment of the present invention, obtaining the value of the G pixel at the position of the IR channel through the method of bilinear interpolation is only one way of realization. In the embodiment of the present invention, RGBIR images can also be realized in other ways The value of the G pixel at the position of the IR channel, for example: when calculating the value of the G pixel on the IR channel, the value of the G pixel in the G channel can be copied (copy) to the position of the IR channel to obtain the value of the G pixel at the position of the IR channel The G pixel value is not limited in this embodiment of the present invention.
S320，通过内插法计算RGBIR图像信号中每个像素点位置上的IR像素值，以得到第三图像信号。S320. Calculate the IR pixel value at each pixel position in the RGBIR image signal by an interpolation method to obtain a third image signal.
在S320中，对RGBIR图像进行插值计算，得到每个像素点R、G、B、IR通道上均为IR像素值的图像信号，即第三图像信号，也可以称为IR全幅图如图6所示。In S320, the RGBIR image is interpolated to obtain an image signal with IR pixel values on the R, G, B, and IR channels of each pixel point, that is, the third image signal, which can also be called an IR full-frame image as shown in Figure 6 shown.
S330，将第一图像信号和第三图像信号相减得到第二图像信号。S330. Subtract the first image signal and the third image signal to obtain a second image signal.
在本发明实施例中，可以通过将S310中得到的未去除IR分量的Bayer图像信号，与S320步骤中得到的第三图像信号相减，即将第一图像信号中每个像素点位置上的IR分量去掉，得到第二图像信号，即去除IR分量的Bayer图像信号，如图6所示。In the embodiment of the present invention, by subtracting the Bayer image signal obtained in S310 without removing the IR component from the third image signal obtained in step S320, that is, the IR at each pixel position in the first image signal The component is removed to obtain the second image signal, that is, the Bayer image signal with the IR component removed, as shown in FIG. 6 .
需要说明的是，S330中得到的去除IR分量的Bayer图像信号的信噪比相对S310中得到的未去除IR分量的Bayer图像信号的信噪比低。It should be noted that the signal-to-noise ratio of the Bayer image signal with the IR component removed obtained in S330 is lower than the signal-to-noise ratio of the Bayer image signal without the IR component removed obtained in S310 .
S340，根据第一图像信号对第二图像信号进行联合滤波。S340. Perform joint filtering on the second image signal according to the first image signal.
在S340中，根据信噪比高的未去除IR分量的Bayer图像信号与去除IR分量的Bayer图像信号的相关性，对去除IR分量的Bayer图像信号进行滤波处理，得到处理后的输出图像信号。In S340, according to the correlation between the Bayer image signal without the IR component and the Bayer image signal without the IR component with a high signal-to-noise ratio, filter the Bayer image signal with the IR component removed to obtain a processed output image signal.
在本发明实施例中，在去噪过程中，可以对已去除IR分量的图像信号中预划分区块中各个像素点的数值求平均值，并将该平均值取代区块中各个像素点的数值，得到输出图像信号。In the embodiment of the present invention, in the denoising process, the value of each pixel in the pre-divided block in the image signal from which the IR component has been removed can be averaged, and the average value can be used to replace the value of each pixel in the block value to get the output image signal.
需要说明的是，经过联合滤波后的图像信号是不能被用户所识别的，需要通过S350进行处理，得到用户可以识别的RGB图像。It should be noted that the image signal after joint filtering cannot be recognized by the user, and needs to be processed by S350 to obtain an RGB image that can be recognized by the user.
S350，通过图像信号处理器对输出图像信号进行处理得到RGB图像。S350. Process the output image signal by an image signal processor to obtain an RGB image.
在S350中，需要通过内插法计算输出图像信号中每个像素点R、G和B通道上的R、G和B像素值，以得到第四图像信号，其中，第四图像信号包括每个像素点的R、G、B和IR通道均为R像素值的图像信号，每个像素点的R、G、B和IR通道均为G像素值的图像信号，以及每个像素点的R、G、B和IR通道均为B像素值的图像信号，如图6所示；对第四图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡处理得到RGB图像。第四图像信号为人眼不能识别的图像信号，需要通过图像信号处理器(Image Signal Processor，ISP)得到用户可以识别的RGB图像。In S350, it is necessary to calculate the R, G and B pixel values on the R, G and B channels of each pixel point in the output image signal by interpolation to obtain the fourth image signal, wherein the fourth image signal includes each The R, G, B and IR channels of each pixel are image signals of R pixel values, the R, G, B and IR channels of each pixel are image signals of G pixel values, and the R, The G, B, and IR channels are all image signals with B pixel values, as shown in Figure 6; the fourth image signal is processed by color correction matrix, lens shading correction, and automatic white balance processing to obtain an RGB image. The fourth image signal is an image signal that cannot be recognized by human eyes, and an RGB image that can be recognized by a user needs to be obtained through an image signal processor (Image Signal Processor, ISP).
经过S310至S350处理的RGBIR传感器获取的RGBIR图像信号可以得到颜色正常的RGB图像。The RGBIR image signal acquired by the RGBIR sensor processed in S310 to S350 can obtain an RGB image with normal color.
在本发明实施例中，通过S310至S340的处理，可以将RGBIR传感器获取的RGBIR图像信号转变为传统的Bayer图像信号，易于与S350中的传统ISP对接，使得传统算法可以重用，减少开发的成本。In the embodiment of the present invention, through the processing from S310 to S340, the RGBIR image signal acquired by the RGBIR sensor can be converted into a traditional Bayer image signal, which is easy to interface with the traditional ISP in S350, so that the traditional algorithm can be reused, reducing the cost of development .
图7为本发明实施例提供的另一提高RGB图像信噪比的方法流程图。如图7所示，该方法400的执行主体可以为图3所示的系统，该方法400可以具体包括以下步骤：FIG. 7 is a flow chart of another method for improving the signal-to-noise ratio of an RGB image provided by an embodiment of the present invention. As shown in FIG. 7, the subject of execution of the method 400 may be the system shown in FIG. 3, and the method 400 may specifically include the following steps:
S410，通过内插法计算接收到的RGBIR图像信号中每个像素点上R、G、B、IR通道中的R像素值、G像素值和B像素值以得到第一图像信号S410, calculate the R pixel value, G pixel value and B pixel value in the R, G, B, and IR channels of each pixel in the received RGBIR image signal by interpolation to obtain the first image signal
具体的，对RGBIR传感器获取的RGBIR图像信号中每个像素点上进行插值，通过内插法计算RGBIR图像信号中每个像素点上R、G、B、IR通道中的R像素值、G像素值和B像素值以得到第一图像信号。即通过内插法得到RGBIR图像信号中每个像素点上的R、G、B和IR通道均为R像素值和IR像素值的图像信号，每个像素点的R、G、B和IR通道均为G像素值和IR像素值的图像信号，以及每个像素点的R、G、B和IR通道均为B像素值和IR像素值的图像信号，或者说得到的每个第一图像信号分别为带有IR分量R、G、B、IR全幅图，即第一图像，如图8所示。Specifically, interpolation is performed on each pixel in the RGBIR image signal acquired by the RGBIR sensor, and the R pixel value and G pixel value in the R, G, B, and IR channels of each pixel in the RGBIR image signal are calculated by interpolation. value and B pixel value to get the first image signal. That is, the R, G, B and IR channels of each pixel in the RGBIR image signal are obtained by interpolation. The image signals are G pixel values and IR pixel values, and the R, G, B and IR channels of each pixel are image signals B pixel values and IR pixel values, or each first image signal obtained They are full images with IR components R, G, B, and IR respectively, that is, the first image, as shown in FIG. 8 .
S420，去除第一图像信号中每个像素点位置上的IR分量，校正RGB通道，得到第二图像信号。S420. Remove the IR component at each pixel position in the first image signal, correct RGB channels, and obtain a second image signal.
将S410步骤中得到的第一图像信号中每个像素点位置上的IR分量去除，可以通过图中S320和S330的过程得到第二图像信号，为简洁描述在这里不再赘述，如图8所示。The IR component at each pixel point position in the first image signal obtained in step S410 is removed, and the second image signal can be obtained through the processes of S320 and S330 in the figure, which will not be repeated here for brevity, as shown in Figure 8 Show.
在本发明实施例中，去除第一图像中每个像素点位置上的IR分量可以通过方法200中提到的去除每个像素点位置上的IR分量的方法，例如：根据第一图像确定校正系数；根据校正系数去除第一图像中每个像素点位置上的IR分量，并校正RGB通道，确定第二图像信号，如图9所示，为简洁描述，其具体过程在此不再赘述。In the embodiment of the present invention, removing the IR component at each pixel position in the first image may be performed by the method of removing the IR component at each pixel position mentioned in method 200, for example: determining the correction according to the first image Coefficient: remove the IR component at each pixel point position in the first image according to the correction coefficient, and correct the RGB channels to determine the second image signal, as shown in Figure 9, which is a brief description, and its specific process will not be repeated here.
S430，根据第一图像信号对第二图像信号进行联合滤波得到输出图像信号。S430. Perform joint filtering on the second image signal according to the first image signal to obtain an output image signal.
该具体的过程可以通过方法200中提到的根据第一图像信号对第二图像信号进行联合滤波的处理方式进行处理，为简洁描述，在这里不再赘述。The specific process may be processed by the method 200 of performing joint filtering on the second image signal according to the first image signal, which is not repeated here for brevity.
S440，通过ISP图像信号处理器对输出图像进行处理，得到RGB。S440. Process the output image through an ISP image signal processor to obtain RGB.
在本发明实施例中，ISP需要对人眼不可识别的输出图像信号进行处理，例如需要对对输出图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡以及马赛克等处理得到人眼可以识别的RGB图像。In the embodiment of the present invention, the ISP needs to process the output image signal that cannot be recognized by the human eye. RGB image.
具体的，S440的具体实现可以依方法200中的S250或方法300中的S350对第四图像信号的处理方式来实现，为简洁描述，其具体过程在此不再赘述。Specifically, the specific implementation of S440 may be implemented according to the processing method of the fourth image signal in S250 in the method 200 or S350 in the method 300. For the sake of concise description, the specific process will not be repeated here.
上文中结合图2至图9，详细描述了根据本发明实施例的提高RGBIR传感器信噪比的方法，下面将结合图10，详细描述根据本发明实施例的装置。The method for improving the signal-to-noise ratio of an RGBIR sensor according to an embodiment of the present invention is described in detail above with reference to FIGS. 2 to 9 . The device according to an embodiment of the present invention will be described in detail below in conjunction with FIG. 10 .
图10为本发明实施例提供的一种装置的结构示意图。如图10所示，该装置500可以包括接收单元510、插值单元520、去除分量单元530、滤波单元540和图像信号处理单元550。Fig. 10 is a schematic structural diagram of a device provided by an embodiment of the present invention. As shown in FIG. 10 , the apparatus 500 may include a receiving unit 510 , an interpolation unit 520 , a component removal unit 530 , a filtering unit 540 and an image signal processing unit 550 .
接收单元510，用于接收RGBIR传感器发送的RGBIR图像信号。The receiving unit 510 is configured to receive the RGBIR image signal sent by the RGBIR sensor.
插值单元520，用于通过内插法对RGBIR图像进行插值以得到第一图像信号。The interpolation unit 520 is configured to interpolate the RGBIR image to obtain a first image signal by an interpolation method.
去除分量单元530，用于去除第一图像信号中每个像素点位置上的红外IR分量以得到第二图像信号。The component removal unit 530 is configured to remove an infrared IR component at each pixel position in the first image signal to obtain a second image signal.
滤波单元540，用于将第一图像信号和第二图像信号进行联合滤波以得到滤波后的RGB图像。The filtering unit 540 is configured to jointly filter the first image signal and the second image signal to obtain a filtered RGB image.
图像信号处理单元550，用于对输出图像信号进行处理以得到RGB图像。The image signal processing unit 550 is configured to process the output image signal to obtain an RGB image.
具体的，装置中的插值单元520通过内插法对RGBIR图像进行插值，在本发明实施例中，内插法可以采用双线插值方法。例如对IR通道进行插值，得到IR通道的G像素，得到每个像素点位置上带有IR分量的Bayer图像信号；再将带有IR分量的Bayer图像信号中每个像素点位置上的IR分量去除，并校正RGB通道，得到去除IR分量的Bayer图像信号；根据未去除IR分量的Bayer图像对去除IR分量的Bayer图像进行联合滤波，也就是在去噪的过程，得到输出图像信号，并通过图像信号处理器对输出图像信号进行处理得到正常颜色的RGB图像。Specifically, the interpolation unit 520 in the device performs interpolation on the RGBIR image. In the embodiment of the present invention, the interpolation method may use a bilinear interpolation method. For example, the IR channel is interpolated to obtain the G pixel of the IR channel, and the Bayer image signal with the IR component is obtained at each pixel position; then the IR component at each pixel position in the Bayer image signal with the IR component is obtained Remove and correct the RGB channel to obtain the Bayer image signal with the IR component removed; perform joint filtering on the Bayer image with the IR component removed according to the Bayer image without the IR component removed, that is, in the denoising process, obtain the output image signal, and pass The image signal processor processes the output image signal to obtain an RGB image with normal colors.
通过本发明实施例提供的提高RGB图像信噪比的方法，利用原始RGBIR图像信噪比高的特点，在噪声去除的过程中，利用未去除IR分量的高信噪比图像的相关性对去除IR分量的低信噪比图像进行联合滤波，实现了对基于RGBIR传感器获取的图像的处理，提高了最后得到的RGB图像的信噪比；同时，也明显改善了在低色温或低照度环境下得到的图像的信噪比。Through the method for improving the signal-to-noise ratio of RGB images provided by the embodiments of the present invention, the original RGBIR image has a high signal-to-noise ratio, and in the process of noise removal, the correlation of the high-signal-to-noise ratio image without IR components is used to remove the noise. The low signal-to-noise ratio image of the IR component is jointly filtered to realize the processing of the image acquired based on the RGBIR sensor, which improves the signal-to-noise ratio of the final RGB image; The signal-to-noise ratio of the resulting image.
可选地，作为本发明另一实施例，插值单元520具体用于，通过内插法计算RGBIR图像信号中每个像素点IR通道上的G像素以得到第一图像信号。Optionally, as another embodiment of the present invention, the interpolation unit 520 is specifically configured to calculate the G pixel on the IR channel of each pixel in the RGBIR image signal through an interpolation method to obtain the first image signal.
可选地，作为本发明另一实施例，插值单元520具体用于，Optionally, as another embodiment of the present invention, the interpolation unit 520 is specifically configured to:
通过内插法计算RGBIR图像信号中每个像素点上R、G、B、IR通道中的R像素值，G像素值和B像素值，以得到第一图像信号；其中，第一图像信号包括每个像素点的R、G、B和IR通道均为R像素值和IR像素值的图像信号，每个像素点的R、G、B和IR通道均为G像素值和IR像素值的图像信号，以及每个像素点的R、G、B和IR通道均为B像素值和IR像素值的图像信号。Calculate the R pixel value in the R, G, B, and IR channels of each pixel in the RGBIR image signal by interpolation, the G pixel value and the B pixel value to obtain the first image signal; wherein the first image signal includes The R, G, B and IR channels of each pixel are image signals of R pixel values and IR pixel values, and the R, G, B and IR channels of each pixel are images of G pixel values and IR pixel values The signal, and the R, G, B and IR channels of each pixel are image signals of B pixel value and IR pixel value.
可选地，在本发明实施例中，去除分量单元530具体用于，通过内插法计算RGBIR图像信号中每个像素点位置上的IR像素值以得到第三图像，该第三图像信号为每个像素点的R、G、B和IR通道均为IR像素值的图像信号；将第一图像信号和第三图像相减得到第二图像信号。Optionally, in this embodiment of the present invention, the component removal unit 530 is specifically configured to calculate the IR pixel value at each pixel position in the RGBIR image signal by interpolation to obtain a third image, and the third image signal is The R, G, B and IR channels of each pixel are image signals of IR pixel values; the second image signal is obtained by subtracting the first image signal from the third image.
其中第二图像信号为去除IR分量后的图像信号。Wherein the second image signal is an image signal after removing the IR component.
可选地，作为本发明另一实施例中，滤波单元540具体用于，Optionally, as another embodiment of the present invention, the filtering unit 540 is specifically used to:
根据第一图像信号中的第一像素点和第一参考像素点的距离或者像素值的差值确定权重，其中，第一参考像素点为第一像素点周围的像素点；确定第二图像信号中与第一像素点对应的第二像素点和与第一参考像素点对应的第二参考像素点；根据权重对第二图像信号中的第二参考像素点进行加权平均，确定第二像素点的像素值，以得到输出图像信号。Determine the weight according to the distance between the first pixel in the first image signal and the first reference pixel or the difference in pixel value, where the first reference pixel is the pixels around the first pixel; determine the second image signal The second pixel point corresponding to the first pixel point and the second reference pixel point corresponding to the first reference pixel point in the second image signal; the second reference pixel point in the second image signal is weighted and averaged according to the weight to determine the second pixel point The pixel value to get the output image signal.
可选地，在一个实施例中，图像信号处理单元550具体用于，Optionally, in one embodiment, the image signal processing unit 550 is specifically configured to:
对输出图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡或马赛克处理中的至少一个处理得到输出RGB图像。At least one of color correction matrix, lens shading correction, automatic white balance or mosaic processing is performed on the output image signal to obtain an output RGB image.
可选地，在另一个实施例中，图像信号处理单元550具体用于，Optionally, in another embodiment, the image signal processing unit 550 is specifically configured to:
通过内插法计算输出图像信号中每个像素点R、G和B通道上的R、G和B像素值，以得到第四图像信号；对第四图像信号进行颜色校正矩阵、镜头阴影校正、自动白平衡处理得到所述RGB图像。Calculate the R, G and B pixel values on the R, G and B channels of each pixel in the output image signal by interpolation to obtain the fourth image signal; perform color correction matrix, lens shading correction, The RGB image is obtained by automatic white balance processing.
其中，第四图像信号包括每个像素点的R、G、B和IR通道均为R像素值的图像信号，每个像素点的R、G、B和IR通道均为G像素值的图像信号，以及每个像素点的R、G、B和IR通道均为B像素值的图像信号。Wherein, the fourth image signal includes an image signal in which the R, G, B and IR channels of each pixel are all R pixel values, and the R, G, B and IR channels of each pixel are all image signals in G pixel values , and the R, G, B and IR channels of each pixel are image signals with B pixel values.
需要说明的是，本发明实施例提供的装置500中的各个单元可以实现图4至图6所示的方法200至400的方法或步骤，为简洁描述，在此不再赘述其详细过程。It should be noted that each unit in the device 500 provided by the embodiment of the present invention can implement the methods or steps of the methods 200 to 400 shown in FIG. 4 to FIG. 6 , and for the sake of brevity, the detailed process will not be repeated here.
结合本文中所公开的实施例描述的方法或算法的步骤可以用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.
以上所述的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Protection scope, within the spirit and principles of the present invention, any modification, equivalent replacement, improvement, etc., shall be included in the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610859097.0ACN106375740B (en) | 2016-09-28 | 2016-09-28 | Generate the methods, devices and systems of RGB image |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610859097.0ACN106375740B (en) | 2016-09-28 | 2016-09-28 | Generate the methods, devices and systems of RGB image |
| Publication Number | Publication Date |
|---|---|
| CN106375740A CN106375740A (en) | 2017-02-01 |
| CN106375740Btrue CN106375740B (en) | 2018-02-06 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610859097.0AActiveCN106375740B (en) | 2016-09-28 | 2016-09-28 | Generate the methods, devices and systems of RGB image |
| Country | Link |
|---|---|
| CN (1) | CN106375740B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107590840B (en)* | 2017-09-21 | 2019-12-27 | 长沙全度影像科技有限公司 | Color shadow correction method based on grid division and correction system thereof |
| CN107731199B (en)* | 2017-10-27 | 2020-12-29 | 努比亚技术有限公司 | Screen color temperature adjusting method, terminal and computer readable storage medium |
| CN108965654B (en)* | 2018-02-11 | 2020-12-25 | 浙江宇视科技有限公司 | Double-spectrum camera system based on single sensor and image processing method |
| CN108391067B (en)* | 2018-05-10 | 2023-09-29 | 浙江芯劢微电子股份有限公司 | Denoising enhancement device and method based on RGB-IR sensor |
| CN109040720B (en)* | 2018-07-24 | 2019-11-19 | 浙江大华技术股份有限公司 | A kind of method and device generating RGB image |
| CN113039417B (en)* | 2018-11-05 | 2024-04-23 | 索尼集团公司 | Temperature estimation device, temperature estimation method, computer-readable storage medium, and computer program product |
| CN110490187B (en)* | 2019-05-31 | 2022-04-15 | 杭州海康威视数字技术股份有限公司 | License plate recognition device and method |
| CN112399163B (en)* | 2019-08-19 | 2022-12-09 | 瑞昱半导体股份有限公司 | Image sensing device and automatic white balance method |
| CN112887513B (en)* | 2019-11-13 | 2022-08-30 | 杭州海康威视数字技术股份有限公司 | Image noise reduction method and camera |
| CN113115010B (en)* | 2020-01-11 | 2023-04-18 | 晋城三赢精密电子有限公司 | Image format conversion method and device based on RGB-IR image sensor |
| CN113365034B (en)* | 2020-03-04 | 2022-09-06 | 合肥君正科技有限公司 | Color calibration method based on RGB-IR image sensor |
| CN113365035B (en)* | 2020-03-04 | 2022-10-21 | 合肥君正科技有限公司 | Calibration system for image color restoration |
| KR20220081532A (en)* | 2020-12-09 | 2022-06-16 | 에스케이하이닉스 주식회사 | Image signal processor and image processing system |
| CN113074627B (en)* | 2021-03-12 | 2022-06-10 | 中国科学院生物物理研究所 | Imaging method and device of direct electronic detection camera and computer equipment |
| CN117692615B (en)* | 2024-02-01 | 2024-04-26 | 东北大学秦皇岛分校 | RGB image communication method based on infrared wavelength |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102665030A (en)* | 2012-05-14 | 2012-09-12 | 浙江大学 | Improved bilinear Bayer format color interpolation method |
| CN102778201A (en)* | 2011-05-13 | 2012-11-14 | 索尼公司 | Image processing device, image processing method, and program |
| CN103686111A (en)* | 2013-12-31 | 2014-03-26 | 上海富瀚微电子有限公司 | Method and device for correcting color based on RGBIR (red, green and blue, infra red) image sensor |
| CN104639923A (en)* | 2015-02-04 | 2015-05-20 | 华为技术有限公司 | Method and device for processing image data, and terminal |
| WO2016002283A1 (en)* | 2014-06-30 | 2016-01-07 | ソニー株式会社 | Image processing device, image pickup device, information processing device and program |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7821552B2 (en)* | 2005-12-27 | 2010-10-26 | Sanyo Electric Co., Ltd. | Imaging apparatus provided with imaging device having sensitivity in visible and infrared regions |
| KR20130031572A (en)* | 2011-09-21 | 2013-03-29 | 삼성전자주식회사 | Image processing method and image processing apparatus |
| JP6231284B2 (en)* | 2013-02-21 | 2017-11-15 | クラリオン株式会社 | Imaging device |
| US20140307055A1 (en)* | 2013-04-15 | 2014-10-16 | Microsoft Corporation | Intensity-modulated light pattern for active stereo |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102778201A (en)* | 2011-05-13 | 2012-11-14 | 索尼公司 | Image processing device, image processing method, and program |
| CN102665030A (en)* | 2012-05-14 | 2012-09-12 | 浙江大学 | Improved bilinear Bayer format color interpolation method |
| CN103686111A (en)* | 2013-12-31 | 2014-03-26 | 上海富瀚微电子有限公司 | Method and device for correcting color based on RGBIR (red, green and blue, infra red) image sensor |
| WO2016002283A1 (en)* | 2014-06-30 | 2016-01-07 | ソニー株式会社 | Image processing device, image pickup device, information processing device and program |
| CN104639923A (en)* | 2015-02-04 | 2015-05-20 | 华为技术有限公司 | Method and device for processing image data, and terminal |
| Publication number | Publication date |
|---|---|
| CN106375740A (en) | 2017-02-01 |
| Publication | Publication Date | Title |
|---|---|---|
| CN106375740B (en) | Generate the methods, devices and systems of RGB image | |
| CN113228094B (en) | Image Processor | |
| TWI407800B (en) | Improved processing of mosaic images | |
| US11949995B2 (en) | Universal and adaptive de-mosaicing (CFA) system | |
| CN105809630B (en) | A kind of picture noise filter method and system | |
| CN110557584B (en) | Image processing method and device, and computer readable storage medium | |
| CN110930301A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| US8238685B2 (en) | Image noise reduction method and image processing apparatus using the same | |
| CN113168669B (en) | Image processing method, device, electronic device and readable storage medium | |
| JP5767064B2 (en) | Image edge enhancement method | |
| US9936172B2 (en) | Signal processing device, signal processing method, and signal processing program for performing color reproduction of an image | |
| CN111784603A (en) | RAW domain image denoising method, computer device and computer readable storage medium | |
| CN113658043B (en) | Image processing method, device, electronic device and readable storage medium | |
| CN103905802A (en) | Method and device for mosaic removal based on P-mode color filter array | |
| CN110493583A (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN102377911B (en) | Image processing apparatus, image processing method, and camera module | |
| EP3834170B1 (en) | Apparatus and methods for generating high dynamic range media, based on multi-stage compensation of motion | |
| CN110290370A (en) | Image processing method and device | |
| CN108307162B (en) | Efficient and flexible color processor | |
| CN101511032A (en) | Image signal processing device, image signal processing method and imaging device | |
| CN108810320B (en) | Image quality improving method and device | |
| CN101401442B (en) | Apparatus for color interpolation using adjustable threshold | |
| CN106131462A (en) | A kind of method and apparatus to correct image | |
| JP4596069B2 (en) | Noise removal filter processing circuit, image processing circuit, imaging device, and noise removal program | |
| TW201336303A (en) | Image capture system and image processing method applied to an image capture system |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |