技术领域technical field
本发明涉及变化检测领域，尤其涉及一种基于多尺度深度特征差值融合网络的变化检测方法。The invention relates to the field of change detection, in particular to a change detection method based on a multi-scale depth feature difference fusion network.
背景技术Background technique
变化检测研究同一场景不同时刻得到的图像所发生的变化，被广泛应用到资源监控、异常检测、视频监控、以及自动驾驶等领域。Change detection studies the changes in images obtained at different times in the same scene, and is widely used in resource monitoring, anomaly detection, video monitoring, and automatic driving.
传统的变化检测方法，最常用的是差值法，它求得变化前后两图像的差值图再用阈值分割得到变化和未变化区域。此外还有比值法，变化向量分析法等。这类方法简单直接易于理解，但是易受到光照变化和相机位姿差异等外在噪音的影响从而影响变化检测结果。Feng[1]等人提出了一个联合优化方法，对相机位姿，光照和变化结果进行联合优化，在微变检测领域取得良好的效果。The traditional change detection method, the most commonly used is the difference method, which obtains the difference map of the two images before and after the change, and then uses threshold segmentation to obtain the changed and unchanged areas. In addition, there are ratio method, change vector analysis method and so on. This type of method is simple, straightforward and easy to understand, but it is susceptible to external noise such as illumination changes and camera pose differences, which will affect the change detection results. Feng[1] et al. proposed a joint optimization method to jointly optimize the camera pose, illumination and change results, and achieved good results in the field of small change detection.
目前较为流行的是基于深度学习的变化检测方法，该方法用深度网络提取图像特征并对原始图像与变化真值图进行端到端的训练学习，能够有效克服由于光照变化和相机位姿差异所引入的变化噪声。Sakurada[2]等人提出将变化前后的图像对和光流图输入CNN(Convolutional neural network,卷积神经网络)以克服光照和相机影响。Huang[3]等人提出一个由相机位姿校正网络和变化检测网络相结合的微变检测模型，有效克服光照与相机位姿影响。Currently more popular is the change detection method based on deep learning. This method uses a deep network to extract image features and conducts end-to-end training and learning on the original image and the change truth map, which can effectively overcome the changes caused by illumination changes and camera pose differences. change noise. Sakurada[2] and others proposed to input the image pair and optical flow map before and after the change into CNN (Convolutional neural network, Convolutional Neural Network) to overcome the influence of illumination and camera. Huang[3] et al. proposed a slight change detection model combining the camera pose correction network and the change detection network, which can effectively overcome the influence of illumination and camera pose.
现有的基于深度学习的变化检测方法可以有效克服光照等噪音影响，但是大多只是使用了深度网络最后一层提取的图像特征，并没有充分利用图像的多尺度信息，导致传统变化检测方法易受到光照变化和相机位姿差异的影响产生错误的变化检测结果。The existing change detection methods based on deep learning can effectively overcome the influence of noise such as illumination, but most of them only use the image features extracted by the last layer of the deep network, and do not make full use of the multi-scale information of the image, which makes the traditional change detection methods vulnerable to The effects of illumination changes and camera pose differences produce erroneous change detection results.
参考文献references
[1]W.Feng,F.-P.Tian,Q.Zhang,N.Zhang,L.Wan,J.Sun,Fine-grained changedetection of misaligned scenes with varied illuminations,in:Proceedings ofthe IEEE International Conference on Computer Vision,2015,pp.1260–1268.[1] W. Feng, F.-P. Tian, Q. Zhang, N. Zhang, L. Wan, J. Sun, Fine-grained change detection of misaligned scenes with varied illuminations, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp.1260–1268.
[2]K.Sakurada,W.Wang,N.Kawaguchi,R.Nakamura,Dense optical flow basedchange detection network robust to difference of camera viewpoints,arXivpreprint arXiv:1712.02941.[2]K.Sakurada,W.Wang,N.Kawaguchi,R.Nakamura,Dense optical flow based change detection network robust to difference of camera viewpoints,arXivpreprint arXiv:1712.02941.
[3]Huang R,Feng W,Wang Z,et al.Learning to detect fine-grained changeunder variant imaging condit ions[C]//Proceedings of the IEEE InternationalConference on Computer Vision.2017:2916-2924.[3]Huang R, Feng W, Wang Z, et al. Learning to detect fine-grained change under variant imaging conditions[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017:2916-2924.
发明内容Contents of the invention
本发明提供了一种基于多尺度深度特征差值融合网络的变化检测方法，本发明设计了有效的孪生卷积神经网络模型，采用内编码技术融合图像多尺度信息、用交叉编码技术捕获图像变化前后的多尺度差异特征，并融合了从较高层到较低层的变化差异特征，用多层损失函数训练网络，以解决变化检测中由光照和相机位姿引入的噪音问题，详见下文描述：The present invention provides a change detection method based on multi-scale depth feature difference fusion network. The present invention designs an effective twin convolutional neural network model, adopts internal encoding technology to fuse multi-scale information of images, and uses cross-encoding technology to capture image changes The multi-scale difference features before and after, and the change difference features from the higher layer to the lower layer are fused, and the network is trained with a multi-layer loss function to solve the noise problem introduced by illumination and camera pose in change detection. See the description below for details :
一种基于多尺度深度特征差值融合网络的变化检测方法，所述方法包括：A change detection method based on a multi-scale deep feature difference fusion network, the method comprising:
基于孪生卷积神经网络，对Dec阶段的图像深度特征用Enc模块进行内编码处理；对多尺度的图像深度特征进行交叉编码处理；Based on the twin convolutional neural network, the Enc module is used to internally encode the image depth features in the Dec stage; cross-encode the multi-scale image depth features;
使用两通道的卷积层作用于多尺度的深度特征融合差异信息，获取多尺度变化概率图并拼接，得到预测变化概率图；Use a two-channel convolutional layer to act on multi-scale depth features to fuse difference information, obtain multi-scale change probability maps and stitch them together, and obtain predicted change probability maps;
分别计算多尺度变化概率图以及预测变化概率图的交叉熵损失函数，对损失函数进行累加获取最终的损失函数；Calculate the cross-entropy loss function of the multi-scale change probability map and the predicted change probability map respectively, and accumulate the loss functions to obtain the final loss function;
使用真值图的变化区域为中心，对输入图像和真值图进行裁剪和翻转以此实现对训练数据的扩展，构建成多尺度深度特征差值的融合网络；Using the change area of the truth map as the center, crop and flip the input image and the truth map to expand the training data and build a fusion network of multi-scale depth feature differences;
根据扩展后的训练数据训练融合网络，输入查询图像和参考图像，输出变化检测结果。The fusion network is trained according to the expanded training data, the query image and the reference image are input, and the change detection results are output.
其中，所述对Dec阶段的图像深度特征用Enc模块进行内编码处理具体为：Wherein, the internal encoding process of the image depth feature in the Dec stage with the Enc module is specifically:
从Dec5阶段开始，使用Enc模块对每一阶段上采样并与前一阶段特征进行融合，分别由Enc5-Enc1阶段的内编码产生五种尺度的深度特征融合信息。Starting from the Dec5 stage, the Enc module is used to upsample each stage and fuse with the features of the previous stage, and the deep feature fusion information of five scales is generated by the internal coding of the Enc5-Enc1 stage respectively.
进一步地，所述对多尺度的图像深度特征进行交叉编码处理具体为：Further, the cross-coding processing of multi-scale image depth features is specifically:
对变化前后两张图像在内编码阶段产生的多尺度深度特征融合信息进行对应尺度求差，得到多尺度深度特征差异图；Calculate the corresponding scale difference of the multi-scale depth feature fusion information generated in the internal encoding stage of the two images before and after the change, and obtain the multi-scale depth feature difference map;
使用Enc模块对每种尺度的深度特征差异图进行上采样并与前一层深度特征差异图进行融合，分别由Encxy5-Encxy1阶段的交叉编码产生深度特征融合差异信息。The Enc module is used to upsample the depth feature difference map of each scale and fuse it with the depth feature difference map of the previous layer, and the depth feature fusion difference information is generated by the cross-encoding of the Encxy5-Encxy1 stage respectively.
其中，所述孪生卷积神经网络具体为：Wherein, the twin convolutional neural network is specifically:
将VGG16的全连层模块FC6和FC7移除，将CONV1-CONV5分别命名为Dec1-Dec5，分别从查询图像和参考图像上提取图像的深度特征。The fully connected layer modules FC6 and FC7 of VGG16 are removed, and CONV1-CONV5 are named Dec1-Dec5, respectively, and the depth features of the image are extracted from the query image and the reference image respectively.
本发明提供的技术方案的有益效果是：The beneficial effects of the technical solution provided by the invention are:
1、本发明采用孪生卷积神经网络模型得到的多尺度深度特征差值进行变化检测，在基准数据集上展示了优异的检测性能；1. The present invention uses the multi-scale depth feature difference obtained by the twin convolutional neural network model for change detection, and exhibits excellent detection performance on the benchmark data set;
2、本发明设计出内编码和交叉编码模块，融合了不同尺度的卷积特征和深度差异特征，充分的利用了图像的多尺度信息；2. The present invention designs the inner coding and cross coding modules, which integrate the convolution features and depth difference features of different scales, and make full use of the multi-scale information of the image;
3、本发明在预测模块时使用不同尺度的差异信息产生变化预测，得到多个监督信号来训练网络，有效提高网络鲁棒性。3. The present invention uses difference information of different scales to generate change predictions when predicting modules, obtains multiple supervisory signals to train the network, and effectively improves network robustness.
附图说明Description of drawings
图1为本发明提出的网络结构示意图；Fig. 1 is the network structure schematic diagram that the present invention proposes;
图2为本发明提出方法和其他方法在公共数据集PCD上的检测结果示意图；Fig. 2 is a schematic diagram of the detection results of the method proposed by the present invention and other methods on the public data set PCD;
图3为本发明提出方法和其他方法在公共数据集VL_CMU_CD上的检测结果示意图；Fig. 3 is a schematic diagram of the detection results of the method proposed by the present invention and other methods on the public data set VL_CMU_CD;
图4为本发明提出方法和其他方法在公共数据集CDnet2014上的检测结果示意图。Fig. 4 is a schematic diagram of the detection results of the method proposed by the present invention and other methods on the public dataset CDnet2014.
具体实施方式Detailed ways
为使本发明的目的、技术方案和优点更加清楚，下面对本发明实施方式作进一步地详细描述。In order to make the purpose, technical solution and advantages of the present invention clearer, the implementation manners of the present invention will be further described in detail below.
实施例1Example 1
一种基于多尺度深度特征差值融合网络的变化检测方法，参见图1，该方法包括以下步骤：A change detection method based on multi-scale deep feature difference fusion network, see Figure 1, the method includes the following steps:
一、网络基本架构1. Basic Network Architecture
参见图1，本发明实施例中的网络基本架构与VGG16网络结构相同(其中，VGG16网络结构主要包括：5个卷积层模块CONV1-CONV5、和两个全连层模块FC6、FC7，该VGG16网络结构为本领域技术人员所公知，本发明实施例对此不做赘述)，但本发明实施例将VGG16的全连层模块FC6和FC7移除。将CONV1-CONV5分别命名为Dec1-Dec5，分别从查询图像X和参考图像Y上提取图像的深度特征。Referring to Fig. 1, the basic network architecture in the embodiment of the present invention is the same as the VGG16 network structure (wherein, the VGG16 network structure mainly includes: 5 convolutional layer modules CONV1-CONV5, and two fully connected layer modules FC6, FC7, the VGG16 The network structure is well known to those skilled in the art, and will not be described in detail in the embodiment of the present invention), but the fully connected layer modules FC6 and FC7 of the VGG16 are removed in the embodiment of the present invention. Name CONV1-CONV5 as Dec1-Dec5 respectively, and extract the depth features of the image from the query image X and the reference image Y respectively.
由于在两张图像上使用了独立的特征提取模块形成了本发明实施例所提及的孪生卷积神经网络。The Siamese convolutional neural network mentioned in the embodiment of the present invention is formed by using independent feature extraction modules on the two images.
二、对Dec阶段得到的图像深度特征用Enc模块进行内编码操作2. Use the Enc module to perform internal encoding operations on the image depth features obtained in the Dec stage
其中，Enc模块包括卷积层、激活层、批标准化层和上采样层。内编码阶段对输入图像的卷积特征进行上采样及融合。内编码操作针对Dec1-Dec5阶段所产生的特征，从Dec5阶段开始，使用Enc模块对每一阶段进行上采样并与前一阶段特征进行融合，分别由Enc5-Enc1阶段的内编码产生五种尺度的深度特征融合信息。内编码公式如下：Among them, the Enc module includes convolutional layers, activation layers, batch normalization layers, and upsampling layers. In the inner encoding stage, the convolutional features of the input image are upsampled and fused. The inner encoding operation is aimed at the features generated in the Dec1-Dec5 stage. Starting from the Dec5 stage, the Enc module is used to upsample each stage and fuse with the features of the previous stage. Five scales are generated by the inner encoding of the Enc5-Enc1 stage deep feature fusion information. The internal encoding formula is as follows:
其中，为对查询图像X或参考图像Y提取到的第i个卷积层模块的图像深度特征；为对查询图像X或参考图像Y的第i+1个卷积层模块的图像深度特征编码后的特征；cat(·)为拼接函数，将特征在第三个维度上进行拼接；为内编码函数，由卷积层、激活层、批标准化层和上采样层一系列计算组成。in, is the image depth feature of the i-th convolutional layer module extracted from the query image X or the reference image Y; is the encoded feature of the image depth feature of the i+1th convolutional layer module of the query image X or reference image Y; cat( ) is a splicing function, splicing the features in the third dimension; It is an inner coding function, which consists of a series of calculations of convolutional layer, activation layer, batch normalization layer and upsampling layer.
上述公式(1)表示了内编码阶段对输入图像X和Y的卷积特征进行上采样及融合，得到Enc4-Enc1阶段的内编码特征，注意到Enc5阶段的内编码特征等于Dec5阶段的图像特征，即：The above formula (1) expresses the upsampling and fusion of the convolution features of the input images X and Y in the inner coding stage to obtain the inner coding features of the Enc4-Enc1 stage, and note that the inner coding features of the Enc5 stage are equal to the image features of the Dec5 stage ,which is:
三、对内编码阶段得到的多尺度的深度特征进行交叉编码操作3. Perform cross-coding operations on the multi-scale depth features obtained in the inner coding stage
其中，交叉编码模块Encxy包括卷基层，激活层，批标准化层和上采样层。首先对变化前后两张图像在内编码阶段Enc5-Enc1所产生的多尺度深度特征融合信息进行对应尺度求差，得到多尺度深度特征差异图，使用Enc模块对每种尺度的深度特征差异图进行上采样并与前一层深度特征差异图进行融合，分别由Encxy5-Encxy1阶段的交叉编码产生五种尺度的深度特征融合差异信息。交叉编码公式如下：Among them, the cross-encoding module Encxy includes volume base layer, activation layer, batch normalization layer and upsampling layer. First, the multi-scale depth feature fusion information generated by the internal encoding stage Enc5-Enc1 of the two images before and after the change is calculated for the corresponding scale difference to obtain the multi-scale depth feature difference map, and the Enc module is used to perform the depth feature difference map of each scale. It is upsampled and fused with the depth feature difference map of the previous layer, and the depth feature fusion difference information of five scales is generated by the cross-encoding of the Encxy5-Encxy1 stage respectively. The cross-coding formula is as follows:
其中，为对查询图像X在第i层上得到的内编码特征；为对参考图像Y在第i层上得到的内编码特征；为在第i+1层上得到的交叉编码特征；ψ()表示交叉编码模块的计算过程。in, is the internal coding feature obtained on the i-th layer for the query image X; is the inner coding feature obtained on the i-th layer of the reference image Y; is the cross-encoding feature obtained on the i+1th layer; ψ() represents the calculation process of the cross-encoding module.
上述公式(3)表示了对内编码阶段产生的多尺度特征进行求差并对差异图进行上采样及融合，得到Encxy4-Encxy1阶段的交叉编码特征，注意到Encxy5阶段的交叉编码特征等于输入图像X与Y在Enc5阶段的内编码特征之差，即：The above formula (3) expresses the difference between the multi-scale features generated in the inner encoding stage and the upsampling and fusion of the difference map to obtain the cross-encoding features of the Encxy4-Encxy1 stage. Note that the cross-encoding features of the Encxy5 stage are equal to the input image The difference between the internal coding features of X and Y in the Enc5 stage, namely:
四、多尺度变化结果预测4. Prediction of multi-scale change results
使用两通道的3x3的卷积层，分别作用于交叉编码阶段Encxy5-Encxy1产生的多尺度深度特征融合差异信息，得到多尺度变化概率图P5-P1。再将P5-P1拼接起来，经过一个两通道的3x3的卷积层得到最终变化概率图Pf，预测公式如下：Using the two-channel 3x3 convolutional layer, respectively act on the multi-scale depth feature fusion difference information generated by the cross-encoding stage Encxy5-Encxy1, and obtain the multi-scale change probability map P5-P1. Then splice P5-P1 together, and get the final change probability map Pf through a two-channel 3x3 convolutional layer. The prediction formula is as follows:
Pi＝conv(FEncxyi)，s.t.，i＝1，…，5， (5)Pi=conv(FEncxyi ), st, i=1,...,5, (5)
Pf＝conv(cat(P1,P2,P3,P4,P5)). (6)Pf＝conv(cat(P1,P2,P3,P4,P5)). (6)
上述公式(5)表示由交叉编码阶段Encxy5-Encxy1产生的多尺度深度特征差异信息，得到多尺度变化概率图，公式(6)表示将公式(5)产生的多尺度变化概率图融合得到最终的预测变化概率图。The above formula (5) represents the multi-scale depth feature difference information generated by the cross-encoding stage Encxy5-Encxy1 to obtain a multi-scale change probability map, and formula (6) represents the fusion of the multi-scale change probability maps generated by formula (5) to obtain the final Predicted Change Probability Plot.
五、多层交叉熵损失策略Five, multi-layer cross entropy loss strategy
对预测阶段产生的每一层变化概率图P1-P5以及最终的变化概率图Pf使用公知的交叉熵损失函数，本发明实施例中最终的损失函数由以上所有损失函数相加得到。其具体计算方法为：The known cross-entropy loss function is used for each layer of change probability maps P1-P5 generated in the prediction stage and the final change probability map Pf. In the embodiment of the present invention, the final loss function is obtained by adding all the above loss functions. Its specific calculation method is:
上述公式(7)表示最终损失函数L由P1-P5层变化概率图对应的损失L1-L5和最终变化概率图对应的损失Lf相加得到。The above formula (7) indicates that the final loss function L is obtained by adding the losses L1-L5 corresponding to the P1-P5 layer change probability map and the lossLf corresponding to the final change probability map.
六、扩展训练数据6. Expand training data
变化检测扩展后的图像要保证变化区域的存在性及合理性，因此在数据扩展时，使用真值图的变化区域为中心，对输入图像和真值图进行随机裁剪和翻转。The extended image of change detection needs to ensure the existence and rationality of the changed area. Therefore, when the data is expanded, the changed area of the truth map is used as the center, and the input image and the truth map are randomly cropped and flipped.
七、网络训练和测试7. Network training and testing
基于Pytorch深度学习网络框架，使用第六部中扩展的数据训练第一步至第五步所提出的网络，可以在相应的数据集上得到一个训练好的网络模型。使用该网络模型，输入查询图像X和参考图像Y，经过网络计算后产生变化检测结果Pf，流程结束。Based on the Pytorch deep learning network framework, use the expanded data in the sixth part to train the network proposed in the first step to the fifth step, and a trained network model can be obtained on the corresponding data set. Using the network model, the query image X and the reference image Y are input, and the change detection result Pf is generated after network calculation, and the process ends.
综上所述，本发明实施例用内编码技术融合图像多尺度信息，用交叉编码技术捕获图像变化前后的多尺度差异特征，并融合了从较高层到较低层的变化差异特征，用多层损失训练网络，以解决变化检测中由光照和相机位姿引入的噪音问题，满足了实际应用中的多种需要。To sum up, the embodiment of the present invention fuses the multi-scale information of the image with the intra-coding technology, captures the multi-scale difference features before and after the image change with the cross-coding technology, and fuses the change difference features from the higher layer to the lower layer, using multiple The layer loss trains the network to solve the noise problem introduced by lighting and camera pose in change detection, which meets various needs in practical applications.
实施例2Example 2
下面结合图1、具体实例对实施例1中的方案进行进一步地介绍，详见下文描述：Below in conjunction with Fig. 1, specific examples, the scheme in embodiment 1 is further introduced, see the following description for details:
本发明实施例在进行网络设计时，需要考虑如何有效利用卷积神经网络的不同尺度的特征，捕捉图像场景的变化区域。In the embodiment of the present invention, when designing the network, it is necessary to consider how to effectively use the features of different scales of the convolutional neural network to capture the changing area of the image scene.
具体来讲，本发明实施例设计的多尺度深度特征差值融合网络用内编码模块逐步将较高层的卷积特征融合到较低层的卷积特征，用交叉编码模块对内编码阶段得到的多尺度特征求差并逐步将较高层的差异特征融合到较低层的差异特征，最后产生多种尺度的变化差异图，即：Specifically, the multi-scale deep feature difference fusion network designed in the embodiment of the present invention uses the inner coding module to gradually fuse the convolutional features of the higher layer to the convolutional features of the lower layer, and uses the cross coding module to obtain the inner coding stage. The difference of multi-scale features is calculated and the higher-level difference features are gradually fused to the lower-level difference features, and finally a multi-scale change difference map is generated, namely:
1)内编码操作针对Dec1-Dec5阶段所产生的特征，从Dec5阶段开始，使用Enc模块对每一阶段进行上采样并与前一阶段特征进行融合，分别由Enc5-Enc1阶段的内编码产生五种尺度的深度特征融合信息。1) The internal coding operation is aimed at the features generated in the Dec1-Dec5 stage. Starting from the Dec5 stage, the Enc module is used to upsample each stage and fuse with the features of the previous stage, and the internal coding of the Enc5-Enc1 stage generates five Fusion information of deep features of various scales.
2)交叉编码操作首先对变化前后两张图像在内编码阶段Enc5-Enc1所产生的多尺度深度特征融合信息进行对应尺度求差，得到多尺度深度特征差异图，使用Enc模块对每种尺度的深度特征差异图进行上采样并与前一层深度特征差异图进行融合，分别由Encxy5-Encxy1阶段的交叉编码产生五种尺度的深度特征融合差异信息。2) The cross-coding operation firstly calculates the corresponding scale difference of the multi-scale depth feature fusion information generated by the internal encoding stage Enc5-Enc1 of the two images before and after the change, and obtains the multi-scale depth feature difference map. The depth feature difference map is upsampled and fused with the depth feature difference map of the previous layer, and the depth feature fusion difference information of five scales is generated by the cross-encoding of the Encxy5-Encxy1 stage respectively.
本发明实施例通过逐步结合较低层的图像内部多尺度卷积特征和图像间多尺度差异特征，实现了在物体边缘上进行更精确的变化轮廓检测。The embodiment of the present invention realizes more accurate change contour detection on the edge of an object by gradually combining lower-level multi-scale convolution features inside an image and multi-scale difference features between images.
本发明实施例使用多层交叉熵损失作为监督信号，共同训练网络，充分利用了多尺度的变化概率图损失信息，提高网络的鲁棒性。The embodiment of the present invention uses multi-layer cross-entropy loss as a supervisory signal to jointly train the network, fully utilizes the multi-scale change probability map loss information, and improves the robustness of the network.
本发明实施例将用于图像分类的VGG16的卷积网络作为基础架构，在VGG16上移除原FC6，FC7层并添加了新的网络层(内编码模块，交叉编码模块以及概率图预测模块)，并随机初始化新网络层的参数，对于原VGG16结构模块，采用在图像分类数据集上预训练好的模型参数进行初始化。The embodiment of the present invention uses the convolutional network of VGG16 for image classification as the basic architecture, removes the original FC6 and FC7 layers on VGG16 and adds new network layers (inner coding module, cross coding module and probability map prediction module) , and randomly initialize the parameters of the new network layer. For the original VGG16 structure module, the model parameters pre-trained on the image classification dataset are used for initialization.
本发明实施例针对变化检测数据的特点进行了基于变化区域的数据扩展方法，在数据扩展时，以真值图中的变化区域为中心，进行图像的剪切和镜面翻转。According to the characteristics of the change detection data, the embodiment of the present invention implements a data expansion method based on the change area. During the data expansion, the image is cut and mirror flipped centering on the change area in the truth map.
具体来说，首先得到变化区域的边界框(如果存在多个变化区域，分别得到每个区域的边界框)。对于每一个变化区域在左上角和右下角随机选取5个开始和结束位置剪切出5张包含变化区域的新的图像，再采用水平和垂直翻转进一步扩展数据。在剪切和翻转的同时，变化区域对应的原图像和真值图同时被裁剪和翻转。Specifically, the bounding box of the changed region is first obtained (if there are multiple changed regions, the bounding box of each region is obtained separately). For each change area, randomly select 5 start and end positions in the upper left corner and lower right corner to cut out 5 new images containing the change area, and then use horizontal and vertical flipping to further expand the data. At the same time of cutting and flipping, the original image and ground truth map corresponding to the changed region are cropped and flipped at the same time.
本发明实施例使用了VL_CMU_CD、PCD和CDnet3个变化检测基准数据集，根据实际需要，通过上述数据扩展方法对VL_CMU_CD数据集进行了扩展。The embodiment of the present invention uses three change detection reference data sets of VL_CMU_CD, PCD and CDnet, and the VL_CMU_CD data set is extended by the above-mentioned data extension method according to actual needs.
实施例3Example 3
下面结合图2-图4对实施例1和2中的方案进行可行性验证，详见下文描述：Below in conjunction with Fig. 2-Fig. 4, carry out feasibility verification to the scheme in embodiment 1 and 2, see the following description for details:
根据图1所示网络结构搭建本发明实施例中的网络，对待扩展数据进行扩展，产生相应的训练数据集，并进行网络训练。Build the network in the embodiment of the present invention according to the network structure shown in FIG. 1 , expand the data to be expanded, generate corresponding training data sets, and perform network training.
从图2、3、4中，可以看出本发明实施例得到的变化检测结果明显优于其它变化检测结果。图2、3、4中，第一列到第三列分别为参考图像、查询图像、真值图，第四列为本发明实例检测到的变化区域。其余为基于其它变化检测算法产生的检测结果。从图2、3、4的实验对比结果可以看出，相对于现有其它方法的变化检测结果，本发明实施例提出的方法得到的变化检测结果更加准确，变化区域的边界更加平滑，细节部分检测效果较好。From Figures 2, 3, and 4, it can be seen that the change detection results obtained in the embodiment of the present invention are obviously better than other change detection results. In Figures 2, 3, and 4, the first to third columns are the reference image, the query image, and the truth map, respectively, and the fourth column is the change area detected by the example of the present invention. The rest are detection results based on other change detection algorithms. From the experimental comparison results in Figures 2, 3, and 4, it can be seen that compared with the change detection results of other existing methods, the change detection results obtained by the method proposed in the embodiment of the present invention are more accurate, the boundaries of the change regions are smoother, and the details The detection effect is better.
本发明实施例对各器件的型号除做特殊说明的以外，其他器件的型号不做限制，只要能完成上述功能的器件均可。In the embodiments of the present invention, unless otherwise specified, the models of the devices are not limited, as long as they can complete the above functions.
本领域技术人员可以理解附图只是一个优选实施例的示意图，上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the above-mentioned embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910596151.0ACN110472634B (en) | 2019-07-03 | 2019-07-03 | Change detection method based on multi-scale depth feature difference fusion network |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910596151.0ACN110472634B (en) | 2019-07-03 | 2019-07-03 | Change detection method based on multi-scale depth feature difference fusion network |
| Publication Number | Publication Date |
|---|---|
| CN110472634Atrue CN110472634A (en) | 2019-11-19 |
| CN110472634B CN110472634B (en) | 2023-03-14 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910596151.0AActiveCN110472634B (en) | 2019-07-03 | 2019-07-03 | Change detection method based on multi-scale depth feature difference fusion network |
| Country | Link |
|---|---|
| CN (1) | CN110472634B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111161217A (en)* | 2019-12-10 | 2020-05-15 | 中国民航大学 | Fuzzy detection method based on Conv-LSTM multi-scale feature fusion |
| CN112308156A (en)* | 2020-11-05 | 2021-02-02 | 电子科技大学 | Two-stage image change detection method based on counterstudy |
| CN112365462A (en)* | 2020-11-06 | 2021-02-12 | 华雁智科(杭州)信息技术有限公司 | An Image-Based Change Detection Method |
| CN112766045A (en)* | 2020-12-28 | 2021-05-07 | 平安科技（深圳）有限公司 | Scene change detection method, system, electronic device and storage medium |
| CN113033454A (en)* | 2021-04-07 | 2021-06-25 | 桂林电子科技大学 | Method for detecting building change in urban video camera |
| CN113222016A (en)* | 2021-05-12 | 2021-08-06 | 中国民航大学 | Change detection method and device based on cross enhancement of high-level and low-level features |
| CN113240023A (en)* | 2021-05-19 | 2021-08-10 | 中国民航大学 | Change detection method and device based on change image classification and feature difference value prior |
| CN115511781A (en)* | 2022-07-27 | 2022-12-23 | 中国电子科技集团公司第二十八研究所 | Building change detection method based on multi-scale twin network |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520219A (en)* | 2018-03-30 | 2018-09-11 | 台州智必安科技有限责任公司 | A kind of multiple dimensioned fast face detecting method of convolutional neural networks Fusion Features |
| CN109360155A (en)* | 2018-08-17 | 2019-02-19 | 上海交通大学 | A single-frame image rain removal method based on multi-scale feature fusion |
| CN109447994A (en)* | 2018-11-05 | 2019-03-08 | 陕西师范大学 | In conjunction with the remote sensing image segmentation method of complete residual error and Fusion Features |
| CN109903276A (en)* | 2019-02-23 | 2019-06-18 | 中国民航大学 | A Convolutional Neural Network RGB-D Saliency Detection Method Based on Multilayer Fusion |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108520219A (en)* | 2018-03-30 | 2018-09-11 | 台州智必安科技有限责任公司 | A kind of multiple dimensioned fast face detecting method of convolutional neural networks Fusion Features |
| CN109360155A (en)* | 2018-08-17 | 2019-02-19 | 上海交通大学 | A single-frame image rain removal method based on multi-scale feature fusion |
| CN109447994A (en)* | 2018-11-05 | 2019-03-08 | 陕西师范大学 | In conjunction with the remote sensing image segmentation method of complete residual error and Fusion Features |
| CN109903276A (en)* | 2019-02-23 | 2019-06-18 | 中国民航大学 | A Convolutional Neural Network RGB-D Saliency Detection Method Based on Multilayer Fusion |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111161217B (en)* | 2019-12-10 | 2023-04-18 | 中国民航大学 | Conv-LSTM multi-scale feature fusion-based fuzzy detection method |
| CN111161217A (en)* | 2019-12-10 | 2020-05-15 | 中国民航大学 | Fuzzy detection method based on Conv-LSTM multi-scale feature fusion |
| CN112308156A (en)* | 2020-11-05 | 2021-02-02 | 电子科技大学 | Two-stage image change detection method based on counterstudy |
| CN112308156B (en)* | 2020-11-05 | 2022-05-03 | 电子科技大学 | A Two-Stage Image Change Detection Method Based on Adversarial Learning |
| CN112365462A (en)* | 2020-11-06 | 2021-02-12 | 华雁智科(杭州)信息技术有限公司 | An Image-Based Change Detection Method |
| CN112766045A (en)* | 2020-12-28 | 2021-05-07 | 平安科技（深圳）有限公司 | Scene change detection method, system, electronic device and storage medium |
| CN112766045B (en)* | 2020-12-28 | 2023-11-24 | 平安科技（深圳）有限公司 | Scene change detection method, system, electronic device and storage medium |
| CN113033454A (en)* | 2021-04-07 | 2021-06-25 | 桂林电子科技大学 | Method for detecting building change in urban video camera |
| CN113222016A (en)* | 2021-05-12 | 2021-08-06 | 中国民航大学 | Change detection method and device based on cross enhancement of high-level and low-level features |
| CN113222016B (en)* | 2021-05-12 | 2022-07-12 | 中国民航大学 | Change detection method and device based on cross enhancement of high-level and low-level features |
| CN113240023A (en)* | 2021-05-19 | 2021-08-10 | 中国民航大学 | Change detection method and device based on change image classification and feature difference value prior |
| CN113240023B (en)* | 2021-05-19 | 2022-09-09 | 中国民航大学 | Change detection method and device based on change image classification and feature difference prior |
| CN115511781A (en)* | 2022-07-27 | 2022-12-23 | 中国电子科技集团公司第二十八研究所 | Building change detection method based on multi-scale twin network |
| CN115511781B (en)* | 2022-07-27 | 2025-08-29 | 中国电子科技集团公司第二十八研究所 | A building change detection method based on multi-scale twin networks |
| Publication number | Publication date |
|---|---|
| CN110472634B (en) | 2023-03-14 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110472634B (en) | Change detection method based on multi-scale depth feature difference fusion network | |
| CN114330529A (en) | Real-time pedestrian shielding detection method based on improved YOLOv4 | |
| CN110532961B (en) | A Semantic Traffic Signal Detection Method Based on Multi-scale Attention Mechanism Network Model | |
| CN114863097A (en) | Infrared dim target detection method based on attention system convolutional neural network | |
| GB2629483A (en) | RGB-D salient object detection method | |
| CN111882620A (en) | Road drivable area segmentation method based on multi-scale information | |
| CN106971155B (en) | Unmanned vehicle lane scene segmentation method based on height information | |
| CN115641445B (en) | A remote sensing image shadow detection method based on fusion of asymmetric inner convolution and Transformer | |
| CN113033454B (en) | A detection method for building changes in urban video cameras | |
| CN113256546A (en) | Depth map completion method based on color map guidance | |
| CN112419325B (en) | A superpixel segmentation method based on deep learning | |
| Yang et al. | [Retracted] A Method of Image Semantic Segmentation Based on PSPNet | |
| CN116503709A (en) | Vehicle detection method based on improved YOLOv5 in haze weather | |
| CN117726954B (en) | A method and system for segmenting land and sea in remote sensing images | |
| CN113901897A (en) | Vehicle detection method in parking lot based on DARFNet model | |
| CN116229112A (en) | Twin network target tracking method based on multiple attentives | |
| CN112634289B (en) | A Fast Feasible Domain Segmentation Method Based on Asymmetric Atrous Convolution | |
| CN117115435A (en) | Attention and multi-scale feature extraction-based real-time semantic segmentation method | |
| CN111008608A (en) | A night vehicle detection method based on deep learning | |
| CN114882530A (en) | Pedestrian detection-oriented lightweight convolutional neural network model | |
| Li et al. | Symmnet: A symmetric convolutional neural network for occlusion detection | |
| CN113269734A (en) | Tumor image detection method and device based on meta-learning feature fusion strategy | |
| CN116222577A (en) | Closed loop detection method, training method, system, electronic equipment and storage medium | |
| CN114596205B (en) | Low illumination image enhancement method by fusing visible light and thermal imaging images | |
| CN110765864A (en) | Image pedestrian re-identification system and method based on resolution irrelevant features |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |