
技术领域technical field
本发明属于图像处理领域,涉及的是一种基于色彩和亮度传递的可见光图像与红外图像融合方法。The invention belongs to the field of image processing, and relates to a fusion method of a visible light image and an infrared image based on color and brightness transfer.
背景技术Background technique
红外与可见光异源图像融合能提高目标探测能力,揭示出目标在单一图像中观察时无法探测特征,在安防、辅助驾驶等领域有重要的应用价值。尤其是夜间驾驶辅助系统,灯光图像地有效处理可大大扩展驾驶员的可视距离,提高安全性。改善图像的颜色信息,将融合图像显示成适合人眼观察的自然颜色,可明显改善人眼的识别性能,减小操作者的疲劳感。The fusion of infrared and visible light heterogeneous images can improve the target detection ability and reveal the features of the target that cannot be detected when observed in a single image, which has important application value in the fields of security and assisted driving. Especially in the night driving assistance system, the effective processing of light images can greatly expand the driver's visual distance and improve safety. Improve the color information of the image and display the fused image in a natural color suitable for human observation, which can significantly improve the recognition performance of the human eye and reduce the operator's fatigue.
基于颜色空间变换的专利有:基于颜色传递的多色调图像统一调整方法(CN200810162135.2)公开了一种基于颜色传递的多色调彩色图像统一调整方法,基于自适应聚类颜色传递的雾天图像清晰化方法(CN200810018174.5)公开了一种基于自适应聚类颜色传递的雾天图像清晰化方法,但上述两种方法只适用于单可见光图像。一种基于彩色传递与熵信息的红外与彩色可见光图像融合方法(CN200810017443.6)提出一种非采样Contourlet变换对灰度可见光图像及红外图像进行分解,采用基于lαβ颜色空间的彩色传递方法将可见光图像的彩色信息传递到融合图像中,得到彩色融合图像。该方法需要利用可见光的色彩,因此不适用于在汽车灯光下的夜视弱光图像与红外图像的彩色融合。Patents based on color space transformation include: Unified adjustment method for multi-tone color images based on color transfer (CN200810162135.2) discloses a unified adjustment method for multi-tone color images based on color transfer, foggy images based on adaptive clustering color transfer The clearing method (CN200810018174.5) discloses a foggy image clearing method based on adaptive clustering color transfer, but the above two methods are only applicable to single visible light images. A fusion method of infrared and color visible light images based on color transfer and entropy information (CN200810017443.6) proposes a non-sampling Contourlet transform to decompose grayscale visible light images and infrared images, and adopts a color transfer method based on lαβ color space to convert visible light The color information of the image is transferred to the fused image to obtain a color fused image. This method needs to use the color of visible light, so it is not suitable for the color fusion of night vision low-light images and infrared images under car lights.
发明内容Contents of the invention
本发明针对现有技术的不足,提供了一种基于色彩传递的夜视弱光图像与红外图像的彩色融合方法。Aiming at the deficiencies of the prior art, the present invention provides a color fusion method of night vision weak light images and infrared images based on color transfer.
本发明方法的步骤如下:The steps of the inventive method are as follows:
步骤1.将参考图像T转换到YUV空间。Step 1. Convert the reference imageT toYUV space.
YUV空间是另一种颜色表示法,Y为亮度,U、V为图像的色度,分别表示蓝色和红色与Y的差异R-Y、B-Y,也称色差信号。利用YUV空间和RGB之间的转换关系得到参考图像T的YUV空间:YUV space is another color representation,Y is the brightness,U ,V are the chromaticity of the image, respectively represent the difference between blue and red andYRY ,BY , also known as the color difference signal. Use the conversion relationship betweenYUV space andRGB to get theYUV space of the reference imageT :
(1) (1)
步骤2.夜视弱光图像与红外图像融合得到融合图像P。Step 2. The night vision low-light image is fused with the infrared image to obtain a fused imageP .
首先将夜视弱光图像进行单色化,得到单色可见光图像的灰度值First, the night vision low-light image is monochromatized to obtain the gray value of the monochromatic visible light image
(2) (2)
式中、、是夜视弱光图像的R、G、B分量的灰度值。再将单色可见光图像的灰度值和红外图像的灰度值融合得到融合图像P的伪Y、U、V分量,方法为:In the formula , , is the gray value ofthe R ,G , andB components of the night vision low-light image. Then the gray value of the monochromatic visible light image and the gray value of the infrared image Fusion to obtain the pseudoY ,U ,V components of the fusion imageP , the method is:
(3) (3)
式中,为正有理数,通常的取值范围为,,。In the formula , is a positive rational number, and the usual value range is , , .
步骤3.根据参考图像T的Y、U、V分量的均值和方差,调整融合图像P伪Y、U、V分量(3)的均值和方差,将参考图像的亮度分布传递到融合图像P中,得到彩色融合图像C。具体包括以下步骤:Step 3. According to the mean and variance ofthe Y ,U ,V components of the reference imageT , adjust the mean and variance of the pseudoY ,U ,V components (3) of the fused imageP , and transfer the brightness distribution of the reference image to the fused imageP , get the color fusion imageC . Specifically include the following steps:
3-1.将融合图像P的Y、U、V分量分别减去其均值,消除背景对传递效果的影响。3-1.The Y ,U , andV components of the fused imageP are respectively subtracted from their mean values to eliminate the influence of the background on the transfer effect.
3-2.对处理后的图像像素值根据其与参考图像的标准方差之比进行缩放,并加上参考图像的均值,即:3-2. Scale the pixel value of the processed image according to its ratio to the standard deviation of the reference image, and add the mean value of the reference image, namely:
(4) (4)
(5) (5)
(6) (6)
其中、、表示融合图像P每个像素的、、分量。,表示参考图像T的Y、U、V分量的标准偏差和均值。,表示融合图像P的Y、U、V分量的标准偏差和均值。为比例缩放系数,用于调节融合后图像的亮度,通常取值范围为。、、表示彩色融合图像C每个像素的、、分量。in , , Indicates the value of each pixel of the fused imageP , , portion. , Indicates the standard deviation and mean of the Y,U , andV componentsof the reference image T. , Indicates the standard deviation and mean ofthe Y ,U , andV components of the fused imageP. is a scaling factor, used to adjust the brightness of the fused image, usually the value range is . , , represents the color fusion imageC for each pixel , , portion.
步骤4.通过YUV逆变换获得彩色融合图像C的R、G、B值,逆变换方法:Step 4. Obtain theR ,G ,B values of color fusion imageC byYUV inverse transformation, inverse transformation method:
本发明的有益效果是:The beneficial effects of the present invention are:
1.利用求图像伪Y、U、V分量的方法,将彩色信息量很少的夜视微光图像和黑白的红外图像进行融合,建立了弱彩色图像、黑白图像和正常的彩色图像之间的颜色联系,为弱彩色图像、黑白图像进行彩色增强提供了必要条件。1. Using the method of calculating the pseudoY ,U andV components of the image, the night vision low-light image with little color information and the black and white infrared image are fused, and the relationship between the weak color image, the black and white image and the normal color image is established. The color connection provides the necessary conditions for color enhancement of weak color images and black and white images.
2.根据参考图像的Y、U、V各分量对融合后图像进行了色彩增强处理,使融合后图像具有与参考图像相同分布的颜色分量。2. According tothe Y ,U ,V components of the reference image, the color enhancement processing is carried out on the fused image, so that the fused image has the same distribution of color components as the reference image.
3.利用比例缩放系数可以根据实际情况对融合图像的亮度进行相应调整,具有更强的适应性。3. Use The scaling factor can adjust the brightness of the fusion image according to the actual situation, which has stronger adaptability.
附图说明Description of drawings
图1是本发明方法流程图。Fig. 1 is a flow chart of the method of the present invention.
具体实施方式Detailed ways
以下结合附图对本发明作进一步说明。The present invention will be further described below in conjunction with accompanying drawing.
如图1所示,本发明方法包括以下步骤:As shown in Figure 1, the inventive method comprises the following steps:
步骤(1)将参考图像T由RGB空间转换为YUV空间,得到参考图像T的Y、U、V空间:Step (1) Convert the reference imageT fromthe RGB space to theYUV space, and obtainthe Y ,U , andV spaces of the reference imageT :
步骤(2)将夜视弱光图像与红外图像的融合得到融合图像P伪Y、U、V分量,具体方法为:先将夜视弱光图像进行单色化,得到单色可见光图像的灰度值,再单色可见光图像的灰度值和红外图像的灰度值融合得到融合图像P的伪Y、U、V分量,方法为:Step (2) Fusion the night vision weak light image and the infrared image to obtainthe pseudoY ,U andV components of the fusion image. The specific method is: first monochromatize the night vision weak light image to obtain the grayscale degree value , and then the gray value of the monochromatic visible light image and the gray value of the infrared image Fusion to obtain the pseudoY ,U ,V components of the fusion imageP , the method is:
式中、、是夜视弱光图像的R、G、B分量的灰度值;In the formula , , is the gray value ofthe R ,G , andB components of the night vision low-light image;
,为正有理数,通常的取值范围为,,。 , is a positive rational number, and the usual value range is , , .
步骤(3).根据参考图像调节融合后图像的色彩分量,具体方法为:先将融合图像P的伪Y、U、V分量分别减去其均值,消除背景对传递效果的影响;再对处理后的图像像素值根据其与参考图像的标准方差之比进行缩放,并加上参考图像的均值,即:Step (3). Adjust the color component of the fused image according to the reference image. The specific method is: first subtract its mean value from the pseudoY ,U , andV components of the fused imageP to eliminate the influence of the background on the transfer effect; The resulting image pixel values are scaled according to their ratio to the standard deviation of the reference image and added to the mean of the reference image, namely:
其中、、表示融合图像P每个像素的、、分量。,表示参考图像T的Y、U、V分量的标准偏差和均值。,表示融合图像P的Y、U、V分量的标准偏差和均值。为比例缩放系数,用于调节融合后图像的亮度,通常取值范围为。、、表示彩色融合图像C每个像素的、、分量。in , , Indicates the value of each pixel of the fused imageP , , portion. , Indicates the standard deviation and mean of the Y,U , andV componentsof the reference image T. , Indicates the standard deviation and mean ofthe Y ,U , andV components of the fused imageP. is a scaling factor, used to adjust the brightness of the fused image, usually the value range is . , , represents the color fusion imageC for each pixel , , portion.
步骤(4).通过YUV逆变换获得彩色融合图像C的R、G、B值。逆变换方法:Step (4). Obtain theR ,G , andB values of the color fusion imageC throughYUV inverse transformation. Inverse transformation method:
。 .
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101559584ACN102298769A (en) | 2011-06-11 | 2011-06-11 | Colored fusion method of night vision low-light image and infrared image based on color transmission |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101559584ACN102298769A (en) | 2011-06-11 | 2011-06-11 | Colored fusion method of night vision low-light image and infrared image based on color transmission |
| Publication Number | Publication Date |
|---|---|
| CN102298769Atrue CN102298769A (en) | 2011-12-28 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011101559584APendingCN102298769A (en) | 2011-06-11 | 2011-06-11 | Colored fusion method of night vision low-light image and infrared image based on color transmission |
| Country | Link |
|---|---|
| CN (1) | CN102298769A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547063A (en)* | 2012-02-08 | 2012-07-04 | 南京航空航天大学 | Natural sense color fusion method based on color contrast enhancement |
| CN104601953A (en)* | 2015-01-08 | 2015-05-06 | 中国航空无线电电子研究所 | Video image fusion-processing system |
| CN104796577A (en)* | 2015-03-20 | 2015-07-22 | 南京理工大学 | Colored night vision imaging device and method based on EMCCD and single-color CCD |
| CN105046869A (en)* | 2015-07-06 | 2015-11-11 | 北京理工大学 | Forest fire prevention monitoring system based on double-wave-band fusion theory |
| CN105701765A (en)* | 2015-09-23 | 2016-06-22 | 河南科技学院 | Image-processing method and mobile terminal |
| CN105917641A (en)* | 2013-08-01 | 2016-08-31 | 核心光电有限公司 | Slim multi-aperture imaging system with autofocus and method of use |
| CN104156992B (en)* | 2014-07-18 | 2017-02-15 | 北京理工大学 | Visible light black and white video image natural colorization method based on color transferring |
| CN106570850A (en)* | 2016-10-12 | 2017-04-19 | 成都西纬科技有限公司 | Image fusion method |
| CN106815826A (en)* | 2016-12-27 | 2017-06-09 | 上海交通大学 | Night vision image Color Fusion based on scene Recognition |
| CN107005639A (en)* | 2014-12-10 | 2017-08-01 | 索尼公司 | Image pick up equipment, image pickup method, program and image processing equipment |
| CN107169950A (en)* | 2017-06-02 | 2017-09-15 | 江苏北方湖光光电有限公司 | A kind of high-definition picture fusion treatment circuit |
| CN104143183B (en)* | 2014-08-07 | 2017-12-12 | 北京理工大学 | The gray scale fusion method of visible ray and infrared black and white video image is transmitted based on brightness |
| CN108040243A (en)* | 2017-12-04 | 2018-05-15 | 南京航空航天大学 | Multispectral 3-D visual endoscope device and image interfusion method |
| CN110211083A (en)* | 2019-06-10 | 2019-09-06 | 北京宏大天成防务装备科技有限公司 | A kind of image processing method and device |
| CN110651301A (en)* | 2017-05-24 | 2020-01-03 | 黑拉有限责任两合公司 | Method and system for automatically coloring night vision images |
| CN111402306A (en)* | 2020-03-13 | 2020-07-10 | 中国人民解放军32801部队 | Low-light-level/infrared image color fusion method and system based on deep learning |
| CN111476732A (en)* | 2020-04-03 | 2020-07-31 | 江苏宇特光电科技股份有限公司 | Image fusion and denoising method and system |
| CN112217962A (en)* | 2019-07-10 | 2021-01-12 | 杭州海康威视数字技术股份有限公司 | Camera and image generation method |
| CN112767298A (en)* | 2021-03-16 | 2021-05-07 | 杭州海康威视数字技术股份有限公司 | Method and device for fusing visible light image and infrared image |
| CN113362261A (en)* | 2020-03-04 | 2021-09-07 | 杭州海康威视数字技术股份有限公司 | Image fusion method |
| CN115034974A (en)* | 2022-05-06 | 2022-09-09 | 电子科技大学 | Method, device and storage medium for natural color restoration of visible light and infrared fusion images |
| CN118828146A (en)* | 2024-08-09 | 2024-10-22 | 沈阳航空航天大学 | Color night vision method based on improved GAN network and color contrast enhancement |
| CN119887538A (en)* | 2024-12-26 | 2025-04-25 | 北京泰岳天成科技有限公司 | Infrared image and visible light image fusion method |
| CN119887538B (en)* | 2024-12-26 | 2025-10-17 | 北京泰岳天成科技有限公司 | Infrared image and visible light image fusion method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7620265B1 (en)* | 2004-04-12 | 2009-11-17 | Equinox Corporation | Color invariant image fusion of visible and thermal infrared video |
| CN101673396A (en)* | 2009-09-07 | 2010-03-17 | 南京理工大学 | Image fusion method based on dynamic object detection |
| CN101853492A (en)* | 2010-05-05 | 2010-10-06 | 浙江理工大学 | A Fusion Method of Night Vision Low Light Image and Infrared Image |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7620265B1 (en)* | 2004-04-12 | 2009-11-17 | Equinox Corporation | Color invariant image fusion of visible and thermal infrared video |
| CN101673396A (en)* | 2009-09-07 | 2010-03-17 | 南京理工大学 | Image fusion method based on dynamic object detection |
| CN101853492A (en)* | 2010-05-05 | 2010-10-06 | 浙江理工大学 | A Fusion Method of Night Vision Low Light Image and Infrared Image |
| Title |
|---|
| 史世明等: "基于YUV空间色彩传递的可见光/热成像双通道彩色成像系统", 《兵工学报》, vol. 30, no. 1, 15 January 2009 (2009-01-15)* |
| 王岭雪等: "基于YUV空间的双通道视频图像色彩传递及实时系统", 《北京理工大学学报》, vol. 30, no. 1, 30 March 2007 (2007-03-30)* |
| 金伟其等: "彩色夜视成像处理算法的新进展", 《红外与激光工程》, vol. 37, no. 1, 25 February 2008 (2008-02-25)* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547063B (en)* | 2012-02-08 | 2014-06-11 | 南京航空航天大学 | Natural sense color fusion method based on color contrast enhancement |
| CN102547063A (en)* | 2012-02-08 | 2012-07-04 | 南京航空航天大学 | Natural sense color fusion method based on color contrast enhancement |
| CN105917641A (en)* | 2013-08-01 | 2016-08-31 | 核心光电有限公司 | Slim multi-aperture imaging system with autofocus and method of use |
| CN108989649A (en)* | 2013-08-01 | 2018-12-11 | 核心光电有限公司 | With the slim multiple aperture imaging system focused automatically and its application method |
| CN104156992B (en)* | 2014-07-18 | 2017-02-15 | 北京理工大学 | Visible light black and white video image natural colorization method based on color transferring |
| CN104143183B (en)* | 2014-08-07 | 2017-12-12 | 北京理工大学 | The gray scale fusion method of visible ray and infrared black and white video image is transmitted based on brightness |
| CN107005639B (en)* | 2014-12-10 | 2020-04-14 | 索尼公司 | Image pickup apparatus, image pickup method, and image processing apparatus |
| CN107005639A (en)* | 2014-12-10 | 2017-08-01 | 索尼公司 | Image pick up equipment, image pickup method, program and image processing equipment |
| CN104601953B (en)* | 2015-01-08 | 2017-12-15 | 中国航空无线电电子研究所 | A kind of video image fusion processing system |
| CN104601953A (en)* | 2015-01-08 | 2015-05-06 | 中国航空无线电电子研究所 | Video image fusion-processing system |
| CN104796577A (en)* | 2015-03-20 | 2015-07-22 | 南京理工大学 | Colored night vision imaging device and method based on EMCCD and single-color CCD |
| CN104796577B (en)* | 2015-03-20 | 2017-10-24 | 南京理工大学 | Color night vision imaging device and method based on EMCCD and monochrome CCD |
| CN105046869A (en)* | 2015-07-06 | 2015-11-11 | 北京理工大学 | Forest fire prevention monitoring system based on double-wave-band fusion theory |
| CN105701765A (en)* | 2015-09-23 | 2016-06-22 | 河南科技学院 | Image-processing method and mobile terminal |
| CN106570850A (en)* | 2016-10-12 | 2017-04-19 | 成都西纬科技有限公司 | Image fusion method |
| CN106570850B (en)* | 2016-10-12 | 2019-06-04 | 成都西纬科技有限公司 | A kind of image interfusion method |
| CN106815826A (en)* | 2016-12-27 | 2017-06-09 | 上海交通大学 | Night vision image Color Fusion based on scene Recognition |
| CN110651301A (en)* | 2017-05-24 | 2020-01-03 | 黑拉有限责任两合公司 | Method and system for automatically coloring night vision images |
| CN107169950A (en)* | 2017-06-02 | 2017-09-15 | 江苏北方湖光光电有限公司 | A kind of high-definition picture fusion treatment circuit |
| CN108040243A (en)* | 2017-12-04 | 2018-05-15 | 南京航空航天大学 | Multispectral 3-D visual endoscope device and image interfusion method |
| CN110211083A (en)* | 2019-06-10 | 2019-09-06 | 北京宏大天成防务装备科技有限公司 | A kind of image processing method and device |
| CN112217962A (en)* | 2019-07-10 | 2021-01-12 | 杭州海康威视数字技术股份有限公司 | Camera and image generation method |
| CN112217962B (en)* | 2019-07-10 | 2022-04-05 | 杭州海康威视数字技术股份有限公司 | Camera and image generation method |
| CN113362261A (en)* | 2020-03-04 | 2021-09-07 | 杭州海康威视数字技术股份有限公司 | Image fusion method |
| CN113362261B (en)* | 2020-03-04 | 2023-08-11 | 杭州海康威视数字技术股份有限公司 | Image fusion method |
| CN111402306A (en)* | 2020-03-13 | 2020-07-10 | 中国人民解放军32801部队 | Low-light-level/infrared image color fusion method and system based on deep learning |
| CN111476732A (en)* | 2020-04-03 | 2020-07-31 | 江苏宇特光电科技股份有限公司 | Image fusion and denoising method and system |
| CN112767298A (en)* | 2021-03-16 | 2021-05-07 | 杭州海康威视数字技术股份有限公司 | Method and device for fusing visible light image and infrared image |
| CN112767298B (en)* | 2021-03-16 | 2023-06-13 | 杭州海康威视数字技术股份有限公司 | Fusion method and device of visible light image and infrared image |
| CN115034974A (en)* | 2022-05-06 | 2022-09-09 | 电子科技大学 | Method, device and storage medium for natural color restoration of visible light and infrared fusion images |
| CN118828146A (en)* | 2024-08-09 | 2024-10-22 | 沈阳航空航天大学 | Color night vision method based on improved GAN network and color contrast enhancement |
| CN119887538A (en)* | 2024-12-26 | 2025-04-25 | 北京泰岳天成科技有限公司 | Infrared image and visible light image fusion method |
| CN119887538B (en)* | 2024-12-26 | 2025-10-17 | 北京泰岳天成科技有限公司 | Infrared image and visible light image fusion method |
| Publication | Publication Date | Title |
|---|---|---|
| CN102298769A (en) | Colored fusion method of night vision low-light image and infrared image based on color transmission | |
| CN102231206B (en) | Colorized night vision image brightness enhancement method applicable to automotive assisted driving system | |
| CN102186089B (en) | Simple-component video image rain field removing method | |
| CN106056559B (en) | Nonuniform illumination Underwater Target Detection image enchancing method based on dark channel prior | |
| CN110443771B (en) | Method for adjusting brightness and color consistency of surround view in vehicle surround view camera system | |
| CN105069768A (en) | Visible-light image and infrared image fusion processing system and fusion method | |
| CN103491357B (en) | A kind of auto white balance treatment method of image sensor | |
| CN106897981A (en) | A kind of enhancement method of low-illumination image based on guiding filtering | |
| CN105205794B (en) | A kind of synchronous enhancing denoising method of low-light (level) image | |
| CN103886565B (en) | Nighttime color image enhancement method based on purpose optimization and histogram equalization | |
| CN103714520A (en) | Digital video image enhancement achieving system and method based on FPGA | |
| CN101930596A (en) | A two-step color constancy method under complex lighting | |
| CN102547063A (en) | Natural sense color fusion method based on color contrast enhancement | |
| CN102542526B (en) | Image decolorizing method | |
| CN111107330B (en) | A Color Cast Correction Method in Lab Space | |
| CN109658367A (en) | Image interfusion method based on Color transfer | |
| CN102306295A (en) | Natural color night vision realization method based on single band infrared image | |
| CN106485674A (en) | A kind of low light image Enhancement Method based on integration technology | |
| CN105405110A (en) | Uneven light compensation method | |
| CN105118032B (en) | A kind of wide method for dynamically processing of view-based access control model system | |
| CN208353496U (en) | A kind of white balance correction device based on RGB-IR image data | |
| CN111223051B (en) | Lane line image enhancement method based on A-law 13 fold line | |
| WO2019085053A1 (en) | Method and system for improving color gamut of display | |
| CN104112258A (en) | Method and system for realizing RGB color image processing | |
| CN102800061B (en) | The quick self-adapted optimization method of digital picture under high illumination |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication | Application publication date:20111228 |