技术领域Technical field
本发明涉及图像处理技术领域，具体涉及一种基于卷积神经网络的图像去雾方法。The present invention relates to the technical field of image processing, and in particular to an image defogging method based on a convolutional neural network.
背景技术Background technique
图像是人类从现实世界获取信息的视觉基础，是传递信息的重要媒介，是表达信息的重要手段。图像处理技术应运而生，其在医疗、交通、考古、农业、工业、建筑业等很多领域发挥着重要的作用。图像处理是一种使用计算机对图像进行相关分析、处理等技术，其目的是为了将图像处理成所需要达成的结果。图像处理技术一般包括图像压缩、图像增强、图像复原、图像识别等方法，本发明提出的图像去雾方法属于图像复原技术。Images are the visual basis for humans to obtain information from the real world, an important medium for transmitting information, and an important means of expressing information. Image processing technology came into being and plays an important role in many fields such as medical care, transportation, archaeology, agriculture, industry, and construction. Image processing is a technology that uses computers to perform relevant analysis and processing of images. Its purpose is to process the image into the desired result. Image processing technology generally includes image compression, image enhancement, image restoration, image recognition and other methods. The image defogging method proposed by the present invention belongs to image restoration technology.
许多专业领域以及日常的生活领域都依赖于光学成像仪器系统的相关工作，比如说实时监控系统、目标追踪系统、自动驾驶、卫星遥感技术以及日常生活照片的成像等等。在雾天天气下的影响下，所采集到的图像，因为漂浮在空中的微小物质对光线造成散射、反射、吸收等物理因素的影响，降低了图像的对比度和识别度，颜色严重衰减，细节信息丢失，从而引起图像失真，视觉系统受到极大的影响，无法直接观测到与图像相关的特征，不利于图像的分析与应用。因此，有必要使用高效的、快速的图像去雾方法对雾天图像进行去雾处理，恢复图像的清晰度，从而改善图像的质量和可见度。Many professional fields and daily life areas rely on the work related to optical imaging instrument systems, such as real-time monitoring systems, target tracking systems, autonomous driving, satellite remote sensing technology, and imaging of daily life photos, etc. Under the influence of foggy weather, the images collected are affected by physical factors such as scattering, reflection, and absorption of light caused by tiny substances floating in the air, which reduces the contrast and recognition of the image, and the color is seriously attenuated and the details Information is lost, causing image distortion. The visual system is greatly affected, and features related to the image cannot be directly observed, which is not conducive to image analysis and application. Therefore, it is necessary to use efficient and fast image defogging methods to dehaze images in foggy weather and restore the clarity of the image, thereby improving the quality and visibility of the image.
二十一世纪后，深度学习理论的提出，以及计算机技术和设备的改进，卷积神经网络得到了快速发展，之后被广泛用于计算机视觉、自然语言处理、图像分割、目标检测等领域中。其基本思想为利用卷积神经网络中的卷积层和池化层学习原始数据的特征，通过共享卷积核和权值，降低了学习的复杂度，大大减少了计算量，便于训练模型，再回归一些或所有物理参数，进一步用于恢复干净的图像。基于多尺度卷积神经网络的单图像去雾方法(Single Image Dehazing via Multi-Scale Convolutional Neural Networks，MSCNN)虽然表现优于传统的方法，在天空等亮白区域中表现的良好，性能在深度区域中也有所降低，但是可能由于这些方法对所含有的雾度水平的估计低于其真实量，导致去雾不彻底，输出的干净图像中仍然包含一些没有去干净的雾霾，并且存在雾后的图像颜色偏暗问题，不能取得比较理想的去雾结果。After the 21st century, with the introduction of deep learning theory and the improvement of computer technology and equipment, convolutional neural networks have developed rapidly and have since been widely used in fields such as computer vision, natural language processing, image segmentation, and target detection. The basic idea is to use the convolution layer and pooling layer in the convolutional neural network to learn the characteristics of the original data. By sharing the convolution kernel and weights, it reduces the complexity of learning, greatly reduces the amount of calculation, and facilitates model training. Some or all physical parameters are then regressed and further used to recover clean images. Although the single image dehazing method (Single Image Dehazing via Multi-Scale Convolutional Neural Networks, MSCNN) based on multi-scale convolutional neural networks performs better than traditional methods, it performs well in bright white areas such as the sky, and its performance is poor in depth areas. has also been reduced, but it may be because these methods estimate the haze level lower than its true amount, resulting in incomplete dehazing. The output clean image still contains some haze that has not been cleaned, and there are some haze after haze. The image color is dark and the ideal dehazing result cannot be achieved.
发明内容Contents of the invention
本发明的目的就是为了上述问题，提供一种基于卷积神经网络的图像去雾方法，本发明采用如下技术方案。The purpose of the present invention is to provide an image defogging method based on a convolutional neural network to solve the above problems. The present invention adopts the following technical solutions.
在现有方法的基础上，本发明提出基于卷积神经网络的图像去雾技术，将一个失真的有雾图像经过图像去雾恢复处理成比较清晰的图像。首先对大气散射模型进行变换，然后搭建Encoder-decoder网络估计中间传输层，再对图像复原问题进行相关处理，最后搭建Dehazer网络输出去雾图像。On the basis of existing methods, the present invention proposes an image defogging technology based on convolutional neural network, which processes a distorted foggy image into a relatively clear image through image dehazing and restoration. First, the atmospheric scattering model is transformed, then an Encoder-decoder network is built to estimate the intermediate transmission layer, and then the image restoration problem is processed, and finally a Dehazer network is built to output the dehazed image.
本发明的具体步骤Specific steps of the present invention
1）分析大气散射模型得知，变换大气散射模型。1) Analyze the atmospheric scattering model and transform the atmospheric scattering model.
2）搭建Encoder-decoder网络，估计出较为准确的中间传输图。2) Build an Encoder-decoder network and estimate a more accurate intermediate transmission graph.
3）在步骤1）的基础上，将变换后的大气散射模型作为图像复原问题进行处理，得到Dehazer函数。3) Based on step 1), treat the transformed atmospheric scattering model as an image restoration problem to obtain the Dehazer function.
4）搭建Dehazer网络实现Dehazer函数，输出去雾图像。4) Build a Dehazer network to implement the Dehazer function and output the dehazed image.
所述步骤1）根据大气散射模型，对有雾图像进行问题分析，图像去雾需要解决问题是中间传输图和大气光值未知，大气散射模型公式如下：The step 1) performs problem analysis on hazy images based on the atmospheric scattering model. The problem that needs to be solved for image dehazing is that the intermediate transmission map and atmospheric light value are unknown. The formula of the atmospheric scattering model is as follows:
其中，为输入图像的像素点；为相机在浓雾天气下得到的有雾图像，即输入图像；为去雾之后得到的清晰图像，即无雾图像；表示目标景物中所含场景光线的透射率，即中间传输图；是一个常量，表示全局大气光值。假设在大气中，雾的浓度是均匀的，则中间传输图计算公式如公式(2)所示。为大气散射系数，表示单位体积的大气对光线的散射能力，一般取数值比较小的常数。是物体和相机之间的距离,即景深。其中，中间传输图与景深的关系为，随着景深的不断增大，呈指数型衰减。根据大气散射模型可知，解决去雾问题的难点在于中间传输图和大气光值未知，需要通过一定的方法对其进行较为准确的估计。许多论文也是基于此难点，构建和改进相关方法，以实现图像去雾。in, are the pixels of the input image; It is the foggy image obtained by the camera in dense fog weather, that is, the input image; It is the clear image obtained after haze removal, that is, a haze-free image; Represents the transmittance of scene light contained in the target scene, that is, the intermediate transmission map; Is a constant representing the global atmospheric light value. Assuming that the concentration of fog is uniform in the atmosphere, the calculation formula of the intermediate transmission diagram is as shown in formula (2). It is the atmospheric scattering coefficient, which represents the scattering ability of light per unit volume of the atmosphere. It is generally a relatively small constant. It is the distance between the object and the camera, which is the depth of field. Among them, the relationship between the intermediate transmission map and the depth of field is, with the depth of field continues to increase, Decays exponentially. According to the atmospheric scattering model, it can be seen that the difficulty in solving the dehazing problem lies in the intermediate transmission map and atmospheric light value It is unknown and needs to be estimated more accurately through certain methods. Many papers are also based on this difficulty to construct and improve related methods to achieve image dehazing.
将公式进一步变换为：The formula is further transformed into:
令，，(1)可被重写为：make , , (1) can be rewritten as:
其中，公式(4)中的表示清晰的图像，表示有雾图像与中间传输图的比值，表示各种因素引起图像降质的汇总的雾或雾霾等噪声，即残差图像。Among them, in formula (4) represents a clear image, Represents foggy image with intermediate transfer diagram The ratio of It represents the summary of fog or haze and other noise caused by various factors that cause image degradation, that is, the residual image.
所述步骤2）搭建Encoder-decoder网络，估计出较为准确的中间传输图。Step 2) Build an Encoder-decoder network and estimate a more accurate intermediate transmission graph.
所述步骤3）利用最大后验概率估计技术（MAP）来获取图像复原问题的解决方案，公式(4)可以被表述为：The step 3) uses the maximum a posteriori probability estimation technique (MAP) to obtain the solution to the image restoration problem. Formula (4) can be expressed as:
其中表示对数似然项，表示有雾图像与中间传输图的比值，项中所提供的先验的与无关，所以式(5)可以重新被表述为：in represents the log-likelihood term, Represents foggy image with intermediate transfer diagram The ratio of a priori provided in and has nothing to do, so equation (5) can be reformulated as:
其中，表示能量函数中观测的有雾图像和重构的图像之间的残差，表示保真项，表示权衡参数，表示正则化项。in, represents the residual between the observed hazy image and the reconstructed image in the energy function, Represents the fidelity item, represents the trade-off parameter, represents the regularization term.
本发明通过优化训包含清晰图像对的训练集上的损失函数来学习，用一个预定义的非线性函数替换MAP推理，将公式(6)变换为对先验参数的学习，公式(6)重新表述为目标函数：The present invention learns by optimizing the loss function on a training set containing clear image pairs, using a predefined nonlinear function Replace MAP reasoning, transform formula (6) into learning of a priori parameters, and reformulate formula (6) into an objective function:
通过半二次方分裂方法(Half Quadratic Splitting，HQS)可以对保真项和正则化项进行解耦处理，简化计算，引入辅助变量，即将公式(6)中的正则化项进行变量的替换，公式(6)可以重新表述为约束优化问题：Through the Half Quadratic Splitting (HQS) method, the fidelity term and the regularization term can be decoupled, the calculation can be simplified, and auxiliary variables can be introduced , that is, the regularization term in formula (6) is replaced by variables, and formula (6) can be reformulated as a constrained optimization problem:
然后，损失函数可以通过HQS方法实现最小化：The loss function can then be minimized via the HQS method:
其中，表示惩罚参数，以非降序迭代的方式变化。公式(9)可以进一步被以下的迭代方法所解决：in, Represents the penalty parameter, changing in a non-descending iterative manner. Equation (9) can be further solved by the following iterative method:
通过以上的公式推导，可以看出，保真项和正则化项中的相同变量被拆分开来，独立的分为两个子问题，与公式(10a)具体公式可以由以下公式进行求解：Through the above formula derivation, it can be seen that the same variables in the fidelity term and the regularization term is split into two independent sub-problems, and the specific formula of formula (10a) can be solved by the following formula:
正则化项方程式(10b)可以改写为：The regularization term equation (10b) can be rewritten as:
根据贝叶斯概率，公式(11)可以通过具有噪声水平的Dehazer函数对有雾图像进行去雾，公式(12)可以重新表述为：According to Bayesian probability, formula (11) can be obtained by having the noise level Dehazer function for hazy images For defogging, formula (12) can be reformulated as:
。 .
所述步骤4），搭建Dehazer网络实现Dehazer函数，输出去雾图像。Step 4), build a Dehazer network to implement the Dehazer function and output the dehazed image.
本发明的有益效果Beneficial effects of the invention
（1）本发明搭建的Encoder-decoder网络可以估计中间传输图。Encoder-decoder结构在本层中发挥着重要的作用，使用该结构具有减轻噪声和抖动的性能，不需要改变网络内部的具体结构以及与之相关的参数，可以在极短的时间内捕获到更多重要的信息，保留关键的特征，丢弃不重要的特征，方便较为准确的估计中间传输图。(1) The Encoder-decoder network built by this invention can estimate the intermediate transmission graph. The Encoder-decoder structure plays an important role in this layer. Using this structure has the performance of reducing noise and jitter. There is no need to change the specific structure inside the network and its related parameters, and more information can be captured in a very short time. Multiple important information, retain key features, and discard unimportant features to facilitate a more accurate estimation of the intermediate transmission map.
（2）本发明搭建的Dehazer网络可以实现Dehazer函数，输出去雾图像。网络模型搭建结构简单，网络性能稳定可靠，Dehazer网络中的参数共享，避免了设置多个参数，大大减少了相关的计算量，方便对其进行训练，便于迅速得出去雾图像。(2) The Dehazer network built by the present invention can implement the Dehazer function and output dehazed images. The network model has a simple structure, and the network performance is stable and reliable. The parameter sharing in the Dehazer network avoids setting multiple parameters, greatly reduces the related calculation amount, facilitates its training, and quickly obtains dehazed images.
（3）本发明通过卷积神经网络模型以及分离技术训练出一系列快速而有效的Dehazer网络，这些Dehazer网络可以作为先验知识运用于基于模型的方法中，即Dehazer网络可以作为模块插入在基于模型的优化方法中，在其他更高级的领域中以用来解决相关问题。(3) The present invention trains a series of fast and effective Dehazer networks through the convolutional neural network model and separation technology. These Dehazer networks can be used as prior knowledge in model-based methods, that is, the Dehazer network can be inserted as a module in the Model optimization methods are used to solve related problems in other more advanced fields.
（4）本发明的方法在亮白区域和白色物体等内容信息中，估计的传输图是较为准确的，能去除光晕（光晕是围绕在光源的明亮的光环），能保持较高的鲁棒性，调节了图像的亮度，突出了图像细节，很小的细节特征能够得到很好处理，去雾后的视觉效果自然，更接近于真实场景(Ground truth)。(4) In content information such as bright white areas and white objects, the method of the present invention estimates the transmission map more accurately, can remove halos (halo is a bright halo surrounding a light source), and can maintain a high Robustness, it adjusts the brightness of the image and highlights the details of the image. Small details can be well processed. The visual effect after dehazing is natural and closer to the real scene (Ground truth).
附图说明Description of the drawings
图1是本发明流程示意图。Figure 1 is a schematic flow diagram of the present invention.
图2是本发明搭建的Encoder-decoder网络图。Figure 2 is an Encoder-decoder network diagram built by the present invention.
图3是本发明搭建的Dehazer网络图。Figure 3 is a Dehazer network diagram built by the present invention.
图4是本发明应用于室内图像的去雾结果。Figure 4 is the dehazing result of the present invention applied to indoor images.
图5是本发明应用于室外图像的去雾结果。Figure 5 is the dehazing result of the present invention applied to outdoor images.
具体实施方式Detailed ways
下面结合附图与实施例对本发明作进一步说明。The present invention will be further described below in conjunction with the accompanying drawings and examples.
如图1所示，首先变换大气散射模型，然后搭建Encoder-decoder网络估计中间传输图，再对图像复原问题进行处理，最后搭建Dehazer网络实现Dehazer函数，输出去雾图像，具体步骤如下：As shown in Figure 1, the atmospheric scattering model is first transformed, then an Encoder-decoder network is built to estimate the intermediate transmission map, and then the image restoration problem is processed. Finally, a Dehazer network is built to implement the Dehazer function and output the dehazed image. The specific steps are as follows:
1）根据大气散射模型，对有雾图像进行问题分析，图像去雾需要解决问题是中间传输图和大气光值未知，大气散射模型公式如下：1) Based on the atmospheric scattering model, analyze the problem of hazy images. The problem that needs to be solved for image dehazing is that the intermediate transmission map and atmospheric light value are unknown. The formula of the atmospheric scattering model is as follows:
其中，为输入图像的像素点；为相机在浓雾天气下得到的有雾图像，即输入图像；为去雾之后得到的清晰图像，即无雾图像；表示目标景物中所含场景光线的透射率，即中间传输图；是一个常量，表示全局大气光值。假设在大气中，雾的浓度是均匀的，则中间传输图计算公式如公式(2)所示。为大气散射系数，表示单位体积的大气对光线的散射能力，一般取数值比较小的常数。是物体和相机之间的距离,即景深。其中，中间传输图与景深的关系为，随着景深的不断增大，呈指数型衰减。根据大气散射模型可知，解决去雾问题的难点在于中间传输图和大气光值未知，需要通过一定的方法对其进行较为准确的估计。许多论文也是基于此难点，构建和改进相关方法，以实现图像去雾。in, are the pixels of the input image; It is the foggy image obtained by the camera in dense fog weather, that is, the input image; It is the clear image obtained after haze removal, that is, a haze-free image; Represents the transmittance of scene light contained in the target scene, that is, the intermediate transmission map; Is a constant representing the global atmospheric light value. Assuming that the concentration of fog is uniform in the atmosphere, the calculation formula of the intermediate transmission diagram is as shown in formula (2). It is the atmospheric scattering coefficient, which represents the scattering ability of light per unit volume of the atmosphere. It is generally a relatively small constant. It is the distance between the object and the camera, which is the depth of field. Among them, the relationship between the intermediate transmission map and the depth of field is, with the depth of field continues to increase, Decays exponentially. According to the atmospheric scattering model, it can be seen that the difficulty in solving the dehazing problem lies in the intermediate transmission map and atmospheric light value It is unknown and needs to be estimated more accurately through certain methods. Many papers are also based on this difficulty to construct and improve related methods to achieve image dehazing.
将公式进一步变换为：The formula is further transformed into:
令，，(1)可被重写为：make , , (1) can be rewritten as:
其中，公式(4)中的表示清晰的图像，表示有雾图像与中间传输图的比值，表示各种因素引起图像降质的汇总的雾或雾霾等噪声，即残差图像。Among them, in formula (4) represents a clear image, Represents foggy image with intermediate transfer diagram The ratio of It represents the summary of fog or haze and other noise caused by various factors that cause image degradation, that is, the residual image.
2）如图2所示，搭建Encoder-decoder网络，估计出较为准确的中间传输图，本层网络搭建的各组成网络层详细的介绍如下：2) As shown in Figure 2, build the Encoder-decoder network and estimate a more accurate intermediate transmission graph. The detailed introduction of each component network layer of this layer of network is as follows:
（1）第一层使用一层ConV+Leaky ReLU，初步处理有雾的图像，卷积核的大小为，ConV通过卷积核和权值共享进行特征提取，减少了参数的使用，简化了模型的复杂度，提高训练网络的速度和精确度。它可以多个提取图像的浅层特征，比如，图像的形状、颜色、边缘等。(1) The first layer uses a layer of ConV+Leaky ReLU to initially process foggy images. The size of the convolution kernel is. ConV performs feature extraction through convolution kernel and weight sharing, reducing the use of parameters and simplifying the model. complexity, improving the speed and accuracy of training the network. It can extract multiple shallow features of the image, such as the shape, color, edge, etc. of the image.
（2）中间层使用Encoder-decoder结构，采用四层ConV+BN+Leaky ReLU与四层DConV+BN+Leaky ReLU作为Encoder-decoder结构。其中所用到的ConV层以及DConv卷积核的大小分别、、、、、、、，使用不同卷积核大小的卷积层是为了进一步提取图像的深层特征，更如纹理等特征。编码器结构使用ConV，即卷积层，将高维的图像特征映射为低维，保留图像中重要的特征，得到多个特征图。解码器部分使用DConV层，即反卷积层，反卷积与卷积的作用相反，将低维的图像特征映射为高维，通过反卷积的还原作用，使得处理后的特征图更加可视化。为了避免图像重要特征的丢失，在Encoder-decoder结构中使用了跳跃连接（Skip connection）。(2) The middle layer uses an Encoder-decoder structure, using four layers of ConV+BN+Leaky ReLU and four layers of DConV+BN+Leaky ReLU as the Encoder-decoder structure. The sizes of the ConV layer and DConv convolution kernel used are respectively , , , , , , , , The purpose of using convolutional layers with different convolution kernel sizes is to further extract the deep features of the image, such as texture and other features. The encoder structure uses ConV, which is a convolutional layer, to map high-dimensional image features into low-dimensional features, retain important features in the image, and obtain multiple feature maps. The decoder part uses the DConV layer, which is the deconvolution layer. The effect of deconvolution is opposite to that of convolution. It maps low-dimensional image features to high dimensions. Through the reduction effect of deconvolution, the processed feature map is more visualized. . In order to avoid the loss of important features of the image, skip connections are used in the Encoder-decoder structure.
（3）最后一层使用ConV+Leaky ReLU结构， ConV层卷积核的大小为，将之前得到关于图像的特征映射组合起来，最后通过非线性回归，获得估计的中间传输图。(3) The last layer uses the ConV+Leaky ReLU structure, and the size of the ConV layer convolution kernel is , combine the previously obtained feature maps about the image, and finally obtain the estimated intermediate transmission map through nonlinear regression.
3）利用最大后验概率估计技术（MAP）来获取图像复原问题的解决方案，公式(4)可以被表述为：3) Use the maximum a posteriori probability estimation technique (MAP) to obtain the solution to the image restoration problem. Formula (4) can be expressed as:
其中表示对数似然项，表示有雾图像与中间传输图的比值，项中所提供的先验的与无关，所以式(5)可以重新被表述为：in represents the log-likelihood term, Represents foggy image with intermediate transfer diagram The ratio of a priori provided in and has nothing to do, so equation (5) can be reformulated as:
其中，表示能量函数中观测的有雾图像和重构的图像之间的残差，表示保真项，表示权衡参数，表示正则化项。in, represents the residual between the observed hazy image and the reconstructed image in the energy function, Represents the fidelity item, represents the trade-off parameter, represents the regularization term.
本发明通过优化训包含清晰图像对的训练集上的损失函数来学习，用一个预定义的非线性函数替换MAP推理，将公式(6)变换为对先验参数的学习，公式(6)重新表述为目标函数：The present invention learns by optimizing the loss function on a training set containing clear image pairs, using a predefined nonlinear function Replace MAP reasoning, transform formula (6) into learning of a priori parameters, and reformulate formula (6) into an objective function:
通过半二次方分裂方法对保真项和正则化项进行解耦处理，简化计算，引入辅助变量，即将公式(6)中的正则化项进行变量的替换，公式(6)可以重新表述为约束优化问题：The fidelity term and the regularization term are decoupled through the semi-quadratic splitting method, the calculation is simplified, and auxiliary variables are introduced , that is, the regularization term in formula (6) is replaced by variables, and formula (6) can be reformulated as a constrained optimization problem:
然后，损失函数可以通过HQS方法实现最小化：The loss function can then be minimized via the HQS method:
其中，表示惩罚参数，以非降序迭代的方式变化。公式(9)可以进一步被以下的迭代方法所解决：in, Represents the penalty parameter, changing in a non-descending iterative manner. Equation (9) can be further solved by the following iterative method:
通过以上的公式推导，可以看出，保真项和正则化项中的相同变量被拆分开来，独立的分为两个子问题，与公式(10a)具体公式可以由以下公式进行求解：Through the above formula derivation, it can be seen that the same variables in the fidelity term and the regularization term is split into two independent sub-problems, and the specific formula of formula (10a) can be solved by the following formula:
正则化项方程式(10b)可以改写为：The regularization term equation (10b) can be rewritten as:
根据贝叶斯概率，公式(11)可以通过具有噪声水平的Dehazer函数对有雾图像进行去雾，公式(12)可以重新表述为：According to Bayesian probability, formula (11) can be obtained by having the noise level Dehazer function for hazy images For defogging, formula (12) can be reformulated as:
。 .
4）如图3所示，搭建Dehazer网络实现Dehazer函数，输出去雾图像，完成对图像进行去雾处理，得到去雾图像。本层网络搭建的各组成网络层详细的介绍如下：4) As shown in Figure 3, build a Dehazer network to implement the Dehazer function, output the dehazed image, complete the dehazing process on the image, and obtain the dehazed image. The detailed introduction of each component network layer built in this layer of network is as follows:
（1）首层使用Conv+Leaky ReLU，该层为特征提取层，ConV层卷积核的大小为，该层根据上层所得到的中间传输图，将有雾图像与中间传输图的比值作为该层的输入，初步提取网络中相关的目标图像的特征。(1) The first layer uses Conv+Leaky ReLU. This layer is a feature extraction layer. The size of the convolution kernel of the ConV layer is , This layer uses the ratio of the foggy image to the intermediate transmission map as the input of this layer based on the intermediate transmission map obtained by the upper layer, and initially extracts the features of the relevant target image in the network.
（2）第二层使用ConV+BN+Leaky ReLU，该层为第一特征转换层，ConV层卷积核的大小为 ，并且加入了BN层，将上层的特征映射作为本层的输入，转换上层结果的相关特征。(2) The second layer uses ConV+BN+Leaky ReLU. This layer is the first feature conversion layer. The size of the ConV layer convolution kernel is , and a BN layer is added, using the feature map of the upper layer as the input of this layer, and converting the relevant features of the upper layer results.
（3）第三层使用Feature 1 conversion layer，该层对上层转换得到的特征映射结果进行相关的特征转换。(3) The third layer uses Feature 1 conversion layer, which performs relevant feature conversion on the feature mapping results obtained by the upper layer conversion.
（4）第四层使用ConV+Leaky ReLU，该层为特征提取层，ConV层卷积核的大小为，本层的主要用于多层提取相关的特征，通过该层对图像进行各方面的特征提取，更加细致地提取目标图像的特征。(4) The fourth layer uses ConV+Leaky ReLU. This layer is a feature extraction layer. The size of the ConV layer convolution kernel is , This layer is mainly used to extract relevant features in multiple layers. Through this layer, all aspects of the image are extracted, and the features of the target image are extracted in more detail.
（5）第五层使用ConV+BN+Leaky ReLU，该层为第二类特征转换层，ConV层卷积核的大小为，为下一步的处理操作提供服务。(5) The fifth layer uses ConV+BN+Leaky ReLU. This layer is the second type of feature conversion layer. The size of the ConV layer convolution kernel is , to provide services for the next processing operation.
（6）第六层使用Feature 2 conversion layer，该层对上层转换得到的特征映射结果进行更深层次的特征转换。(6) The sixth layer uses Feature 2 conversion layer, which performs deeper feature conversion on the feature mapping results obtained by the upper layer conversion.
（7）第七层使用ConV+Leaky ReLU，该层为特征提取层，ConV层卷积核的大小为，多层次的进行特征提取，便于很细致、很全面的学习到目标图像的重要特征。(7) The seventh layer uses ConV+Leaky ReLU. This layer is a feature extraction layer. The size of the ConV layer convolution kernel is , perform feature extraction at multiple levels to facilitate detailed and comprehensive learning of important features of the target image.
（8）第八层使用ConV+BN+Leaky ReLU，该层为第一类特征转换层，ConV层卷积核的大小为，提高特征转换的精度，可以很好的预测目标图像的其他相关信息特征。(8) The eighth layer uses ConV+BN+Leaky ReLU. This layer is the first type of feature conversion layer. The size of the ConV layer convolution kernel is , improve the accuracy of feature conversion, and can well predict other related information features of the target image.
（9）第九层使用Feature 1 conversion layer，回归于之前使用的特征转换层，更加精细地对上层转换得到的特征映射结果进行特征转换，以降低网络训练过程中产生的误差。(9) The ninth layer uses the Feature 1 conversion layer, which returns to the previously used feature conversion layer, and more precisely performs feature conversion on the feature mapping results obtained by the upper layer conversion to reduce errors generated during the network training process.
（10）第十层使用Conv+Leaky ReLU，ConV层卷积核的大小为，通过非线性回归，得出输出去雾图像。(10) The tenth layer uses Conv+Leaky ReLU, and the size of the ConV layer convolution kernel is , through nonlinear regression, the output dehazed image is obtained.
本发明的可以通过以下的实验结果进一步进行说明。The present invention can be further illustrated by the following experimental results.
1、实验内容：REalistic Single Image Dehazing（RESIDE）真实单图像去雾数据集是一个综合的大规模的数据集，使用RESIDE数据集对本发明提出的网络进行训练。RESIDE数据集所收集图像包含成对的清晰图像和有雾图像，室内和室外场景该数据集中所收集到的大规模数据完全足够本发明作为训练集和测试集，并且所训练出来的网络是基本上是可靠的，并且训练出来的网络具有很好的性能。1. Experimental content: REalistic Single Image Dehazing (RESIDE) real single image dehazing data set is a comprehensive large-scale data set. The RESIDE data set is used to train the network proposed by the present invention. The images collected in the RESIDE data set include pairs of clear images and hazy images, indoor and outdoor scenes. The large-scale data collected in this data set is completely sufficient for this invention to be used as a training set and a test set, and the trained network is basically It is reliable and the trained network has good performance.
2、实验结果2. Experimental results
图4为本发明方法应用于室内图像的去雾结果。其中图4（a）为室内第一幅有雾图像，图4（b）、图4（c）以及图4（d）分别为图4（a）的MSCNN方法的去雾结果，本发明方法的去雾结果和Ground Truth图像；图4（e）为室内第二幅有雾图像，图4（f）、图4（g）以及图4（h）分别为图4（e）的MSCNN方法的去雾结果，本发明方法的去雾结果和Ground Truth（真实场景）图像。Figure 4 shows the dehazing results of the method of the present invention applied to indoor images. Figure 4(a) is the first indoor hazy image, Figure 4(b), Figure 4(c) and Figure 4(d) are the dehazing results of the MSCNN method in Figure 4(a) respectively. The method of the present invention The dehazing results and Ground Truth image; Figure 4(e) is the second hazy image indoors, Figure 4(f), Figure 4(g) and Figure 4(h) are the MSCNN method of Figure 4(e) respectively. The dehazing results, the dehazing results of the method of the present invention and the Ground Truth (real scene) image.
从图4可以看出，MSCNN方法可能由于对所含有的雾度水平的估计低于其真实量，导致去雾不彻底，输出的去雾图像中仍然包含一些没有去干净的雾霾。本发明方法去雾比较彻底，整体的去雾视觉效果自然，比较接近Ground Truth。As can be seen from Figure 4, the MSCNN method may have incomplete haze removal because the estimate of the haze level contained is lower than its true amount, and the output dehazed image still contains some haze that has not been cleaned. The method of the present invention is relatively thorough in dehazing, and the overall visual effect of dehazing is natural and closer to Ground Truth.
图5为本发明方法应用于室外图像的去雾结果。其中图5（a）为室内第一幅有雾图像，图5（b）、图5（c）以及图5（d）分别为图5（a）的MSCNN方法的去雾结果，本发明方法的去雾结果和Ground Truth；图5（e）为室内第二幅有雾图像，图5（f）、图5（g）以及图5（h）分别为图5（e）的MSCNN方法的去雾结果，本发明方法的去雾结果和Ground Truth。Figure 5 shows the dehazing results of the method of the present invention applied to outdoor images. Figure 5(a) is the first indoor hazy image, Figure 5(b), Figure 5(c) and Figure 5(d) are the dehazing results of the MSCNN method in Figure 5(a) respectively. The method of the present invention The dehazing results and Ground Truth; Figure 5(e) is the second indoor hazy image, Figure 5(f), Figure 5(g) and Figure 5(h) are the results of the MSCNN method in Figure 5(e) respectively. Dehazing results, dehazing results and Ground Truth of the method of the present invention.
从图5可以看出，MSCNN方法在天空等亮白区域中表现的一般，去雾图像的亮白区域出现颜色偏暗的现象，仍然有雾残留的现象，去雾不彻底。本发明方法去雾比较彻底，基本上没有雾的残留，能有效的处理天空区域，突出图像的细节特征，在图像色彩饱和度、对比度等方面处理基本上也与Ground Truth大致相似。As can be seen from Figure 5, the MSCNN method performs generally in bright white areas such as the sky. The bright white areas of the dehazed image appear darker, and there is still fog remaining, so the dehazing is incomplete. The method of the present invention is relatively thorough in dehazing, with basically no fog residue, and can effectively process the sky area and highlight the detailed features of the image. The processing of image color saturation, contrast, etc. is basically similar to Ground Truth.
综上，本发明通过搭建两层网络模型来实现图像去雾处理，Encoder-decoder网络估计中间传输图。Dehazer网络根据相关的训练，实现Dehazer函数，输出去雾图像。本发明搭建的网络模型简单，便于实现，运行时间少，执行效率高，能够很好的对有雾图像进行去雾处理。In summary, the present invention realizes image defogging by building a two-layer network model, and the Encoder-decoder network estimates the intermediate transmission graph. The Dehazer network implements the Dehazer function based on relevant training and outputs dehazed images. The network model built by the present invention is simple, easy to implement, has short running time, high execution efficiency, and can effectively dehaze foggy images.
本发明的具体实施方法通过上述内容以及结合附图部分进行了详细的描述，但并没有限制本发明的保护范围，本领域技术人员可以在本发明的技术方案基础上，对本发明做出各种修改或变形，这仍然在本发明的保护范围之内。The specific implementation method of the present invention is described in detail through the above content and in conjunction with the accompanying drawings, but this does not limit the protection scope of the present invention. Those skilled in the art can make various modifications to the present invention based on the technical solutions of the present invention. Modifications or deformations are still within the scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911020206.XACN110807744B (en) | 2019-10-25 | 2019-10-25 | An image defogging method based on convolutional neural network |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911020206.XACN110807744B (en) | 2019-10-25 | 2019-10-25 | An image defogging method based on convolutional neural network |
| Publication Number | Publication Date |
|---|---|
| CN110807744A CN110807744A (en) | 2020-02-18 |
| CN110807744Btrue CN110807744B (en) | 2023-09-08 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911020206.XAActiveCN110807744B (en) | 2019-10-25 | 2019-10-25 | An image defogging method based on convolutional neural network |
| Country | Link |
|---|---|
| CN (1) | CN110807744B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022011661A1 (en)* | 2020-07-17 | 2022-01-20 | 南京理工大学 | Progressive feature fusion-based image smog removal method and system |
| CN114331853B (en)* | 2020-09-30 | 2023-05-12 | 四川大学 | Single image restoration iteration framework based on target vector updating module |
| CN112150395A (en)* | 2020-10-15 | 2020-12-29 | 山东工商学院 | An Encoder-Decoder Network Image Dehazing Method Combining Residual Blocks and Dense Blocks |
| CN112634171B (en)* | 2020-12-31 | 2023-09-29 | 上海海事大学 | Image defogging method and storage medium based on Bayesian convolutional neural network |
| CN114372929A (en)* | 2021-12-30 | 2022-04-19 | 新疆大学 | Image rapid defogging method and device based on model optimization and compression |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105719247A (en)* | 2016-01-13 | 2016-06-29 | 华南农业大学 | Characteristic learning-based single image defogging method |
| CN107295261A (en)* | 2017-07-27 | 2017-10-24 | 广东欧珀移动通信有限公司 | Image defogging processing method, device, storage medium and mobile terminal |
| CN107749052A (en)* | 2017-10-24 | 2018-03-02 | 中国科学院长春光学精密机械与物理研究所 | Image defogging method and system based on deep learning neutral net |
| CN109389569A (en)* | 2018-10-26 | 2019-02-26 | 大象智能科技(南京)有限公司 | Based on the real-time defogging method of monitor video for improving DehazeNet |
| CN109410135A (en)* | 2018-10-02 | 2019-03-01 | 复旦大学 | It is a kind of to fight learning-oriented image defogging plus mist method |
| CN109544470A (en)* | 2018-11-08 | 2019-03-29 | 西安邮电大学 | A kind of convolutional neural networks single image to the fog method of boundary constraint |
| CN109903232A (en)* | 2018-12-20 | 2019-06-18 | 江南大学 | An Image Dehazing Method Based on Convolutional Neural Network |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9305339B2 (en)* | 2014-07-01 | 2016-04-05 | Adobe Systems Incorporated | Multi-feature image haze removal |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105719247A (en)* | 2016-01-13 | 2016-06-29 | 华南农业大学 | Characteristic learning-based single image defogging method |
| CN107295261A (en)* | 2017-07-27 | 2017-10-24 | 广东欧珀移动通信有限公司 | Image defogging processing method, device, storage medium and mobile terminal |
| CN107749052A (en)* | 2017-10-24 | 2018-03-02 | 中国科学院长春光学精密机械与物理研究所 | Image defogging method and system based on deep learning neutral net |
| CN109410135A (en)* | 2018-10-02 | 2019-03-01 | 复旦大学 | It is a kind of to fight learning-oriented image defogging plus mist method |
| CN109389569A (en)* | 2018-10-26 | 2019-02-26 | 大象智能科技(南京)有限公司 | Based on the real-time defogging method of monitor video for improving DehazeNet |
| CN109544470A (en)* | 2018-11-08 | 2019-03-29 | 西安邮电大学 | A kind of convolutional neural networks single image to the fog method of boundary constraint |
| CN109903232A (en)* | 2018-12-20 | 2019-06-18 | 江南大学 | An Image Dehazing Method Based on Convolutional Neural Network |
| Title |
|---|
| JINJIANG LI，et al.Image Dehazing Using Residual-Based Deep CNN.《IEEE Access》.2018,正文第23831-26841页.* |
| Publication number | Publication date |
|---|---|
| CN110807744A (en) | 2020-02-18 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110807744B (en) | An image defogging method based on convolutional neural network | |
| Zhou et al. | Fsad-net: feedback spatial attention dehazing network | |
| CN111209952B (en) | Underwater target detection method based on improved SSD and migration learning | |
| CN111062880B (en) | Underwater image real-time enhancement method based on condition generation countermeasure network | |
| CN116797488A (en) | A low-light image enhancement method based on feature fusion and attention embedding | |
| CN111079764B (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN116757986A (en) | Infrared and visible light image fusion method and device | |
| CN111861939B (en) | Single image defogging method based on unsupervised learning | |
| CN106157267A (en) | A kind of image mist elimination absorbance optimization method based on dark channel prior | |
| CN110503613A (en) | Single Image-Oriented Rain Removal Method Based on Cascaded Atrous Convolutional Neural Network | |
| CN114387195A (en) | Infrared image and visible light image fusion method based on non-global pre-enhancement | |
| CN110517203A (en) | A Dehazing Method Based on Reference Image Reconstruction | |
| CN115527159B (en) | Counting system and method based on inter-modal scale attention aggregation features | |
| CN103077506A (en) | Local and non-local combined self-adaption image denoising method | |
| CN115953311B (en) | Image dehazing method based on multi-scale feature representation of Transformer | |
| CN104657951A (en) | Multiplicative noise removal method for image | |
| Sun et al. | Underwater image enhancement with encoding-decoding deep CNN networks | |
| CN112907478A (en) | Feature domain single image defogging method based on domain conversion | |
| CN116563146A (en) | Image Enhancement Method and System Based on Learnable Curvature Map | |
| CN111340718A (en) | Image defogging method based on progressive guiding strong supervision neural network | |
| CN110738624A (en) | area self-adaptive image defogging system and method | |
| CN117974716A (en) | A nighttime drone target tracking method integrating hybrid attention mechanism | |
| CN111723934B (en) | Image processing method and system, electronic device and storage medium | |
| CN116228550A (en) | Image self-enhancement defogging algorithm based on generation of countermeasure network | |
| CN119850467A (en) | Polarized image defogging method based on double-iteration regular separation |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |