技术领域Technical field
本发明涉及图像处理技术领域，特别是涉及一种变电设备缺陷图像数据扩充与数据清洗方法。The present invention relates to the field of image processing technology, and in particular to a method for expanding and cleaning data of substation defect image data.
背景技术Background technique
随着新型电力系统建设步伐的加速，变电站运维模式开始由无人值守型向智慧型过渡，借助目标检测、语义分割等深度学习方法对现有变电设备缺陷图像进行集中处理分析，可以大幅缩短巡检周期，提高设备缺陷处理效率。一个准确率高、泛化性强的深度学习模型需要大量有代表性的标注样本，而现有的变电设备缺陷图像往往是在光照条件好、可见度高的环境下采集的，导致训练的深度学习模型在清晨、黄昏、黑夜、沙尘、暴雨、大雪、浓雾等场景中的泛化性较差，无法适应弱光及恶劣天气场景中的自动巡检需求，同时在实际变电站中收集大量存在天气、光照等各类复杂干扰变化的缺陷图像代价昂贵，少量样本又难以支撑深度学习模型的训练及定量评估。With the acceleration of the construction of new power systems, the operation and maintenance model of substations has begun to transition from unattended to intelligent. Deep learning methods such as target detection and semantic segmentation are used to centrally process and analyze existing substation equipment defect images, which can significantly Shorten the inspection cycle and improve the efficiency of equipment defect processing. A deep learning model with high accuracy and strong generalization requires a large number of representative annotated samples. However, existing defect images of substation equipment are often collected in environments with good lighting conditions and high visibility, resulting in in-depth training. The learning model has poor generalization in scenes such as early morning, dusk, dark night, sand and dust, heavy rain, heavy snow, dense fog, etc., and cannot adapt to the automatic inspection needs in low light and severe weather scenes. At the same time, a large number of data collected in actual substations Defect images with various complex interference changes such as weather and lighting are expensive, and a small number of samples are difficult to support the training and quantitative evaluation of deep learning models.
因此，如何依托平行视觉技术，利用有限的变电设备缺陷图像和各种弱光场景图像生成低光照变电设备缺陷图像，并保证样本分布的一致性，以提高深度学习模型的泛化性，是一个亟需解决的问题。Therefore, how to rely on parallel vision technology to use limited substation equipment defect images and various low-light scene images to generate low-light substation equipment defect images and ensure the consistency of sample distribution to improve the generalization of the deep learning model, It is a problem that needs to be solved urgently.
发明内容Contents of the invention
为了克服现有技术的不足，本发明的目的是提供一种变电设备缺陷图像数据扩充与数据清洗方法。In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a method for data expansion and data cleaning of defect images of substation equipment.
为实现上述目的，本发明提供了如下方案：In order to achieve the above objects, the present invention provides the following solutions:
一种变电设备缺陷图像数据扩充与数据清洗方法，包括：A method for data expansion and data cleaning of defect images of substation equipment, including:
利用预设数据集对风格迁移网络进行训练，得到训练好的网络权重参数；Use the preset data set to train the style transfer network and obtain the trained network weight parameters;
遍历原始变电设备缺陷图像和弱光场景图像，并利用加载所述网络权重参数的所述风格迁移网络，根据所述原始变电设备缺陷图像和弱光场景图像对缺陷图像数据进行扩充，得到低光照变电设备缺陷图像数据，并验证风格迁移效果；Traverse the original substation equipment defect images and low-light scene images, and use the style migration network loaded with the network weight parameters to expand the defect image data according to the original substation equipment defect images and low-light scene images to obtain Low-light substation equipment defect image data and verify the style transfer effect;
滤除所述低光照变电设备缺陷图像数据中的低质量生成图像，并依托对抗验证方法利用分类模型筛选出与真实图像集合样本分布近似的生成图像，得到最终扩充的低光照变电设备缺陷图像集合。Filter out the low-quality generated images in the low-light substation equipment defect image data, and rely on the adversarial verification method to use a classification model to screen out generated images that are similar to the sample distribution of the real image collection to obtain the final expanded low-light substation equipment defects Image collection.
优选地，利用预设数据集对风格迁移网络进行训练，得到训练好网络权重参数，包括：Preferably, the style transfer network is trained using a preset data set to obtain trained network weight parameters, including:
将MS COCO 2017数据集打包为批次大小固定的图像数据集合，并对图像进行大小调整、随机裁剪和归一化预处理，得到训练数据；Pack the MS COCO 2017 data set into a fixed batch size image data set, and perform size adjustment, random cropping and normalization preprocessing on the images to obtain training data;
将所述训练数据输入所述风格迁移网络中进行多轮次的训练，并逐步微调所述风格迁移网络的编解码块的权重参数，得到训练好网络权重参数；编解码块包括编码网络和解码网络；每阶段的所述编解码块的训练流程为：输入特征经过所述编码网络得到编码特征，并通过所述解码网络还原输入特征，而后计算所述输入特征与输出特征间的内容重建损失、图像平滑损失、内容感知损失及解码块反演损失，并完成损失迭代、梯度反传、参数更新直至达到设定的训练轮次，而后保存编解码块的权重参数。The training data is input into the style transfer network for multiple rounds of training, and the weight parameters of the encoding and decoding blocks of the style transfer network are gradually fine-tuned to obtain well-trained network weight parameters; the encoding and decoding blocks include an encoding network and a decoding network. Network; the training process of the encoding and decoding blocks at each stage is: input features are obtained through the encoding network to obtain encoding features, and the input features are restored through the decoding network, and then the content reconstruction loss between the input features and the output features is calculated. , image smoothing loss, content-aware loss and decoding block inversion loss, and complete loss iteration, gradient backpropagation, parameter update until the set training round is reached, and then save the weight parameters of the encoding and decoding block.
优选地，遍历原始变电设备缺陷图像和弱光场景图像，并利用加载所述网络权重参数的所述风格迁移网络，根据所述原始变电设备缺陷图像和弱光场景图像对缺陷图像数据进行扩充，得到低光照变电设备缺陷图像数据，并验证风格迁移效果，包括：Preferably, the original substation equipment defect images and low-light scene images are traversed, and the style migration network loaded with the network weight parameters is used to perform the defect image data according to the original substation equipment defect images and low-light scene images. Expansion, obtain low-light substation equipment defect image data, and verify the style transfer effect, including:
从变电设备缺陷图像集合中随机选择一张原始的内容图像，从低光照场景图中集合S中随机选择一张风格图像，利用训练好的编码网络完成内容图像和风格图像的特征提取，得到内容编码特征与风格编码特征集合，同时利用高频跳跃连接模块实现内容图像编码特征的高低频频率分量分解，得到高频残差分量；Randomly select an original content image from the substation equipment defect image collection, randomly select a style image from the set S in the low-light scene image, and use the trained coding network to complete the feature extraction of the content image and style image, and we get A collection of content coding features and style coding features, and a high-frequency skip connection module is used to decompose the high- and low-frequency frequency components of the content image coding features to obtain high-frequency residual components;
前三阶段的风格迁移内容编码特征与风格编码特征通过通道相关-自适应实例归一化实现内容风格特征在通道分布的对齐，并通过解码网络进行通道维度压缩和空间维度扩张操作，在维度变换后与高频残差分量融合以补充空间信息，得到下阶段风格迁移的内容特征输入；第四阶段的风格迁移以风格编码特征为输入，采用空间相关-自适应实例归一化实现内容风格特征在空间分布上的对齐，并利用解码网络进还原输入图像的通道、空间维度，得到风格化后的内容图像；In the first three stages of style migration, the content coding features and style coding features are aligned through channel correlation-adaptive instance normalization to achieve the alignment of content style features in channel distribution, and the channel dimension compression and spatial dimension expansion operations are performed through the decoding network, and the dimension transformation Finally, it is fused with the high-frequency residual component to supplement the spatial information and obtain the content feature input for the next stage of style transfer. The fourth stage of style transfer uses style coding features as input and uses spatial correlation-adaptive instance normalization to achieve content style features. Align the spatial distribution, and use the decoding network to restore the channels and spatial dimensions of the input image to obtain the stylized content image;
计算原始的内容图像与风格化后的内容图像之间的结构相似性、风格化后内容图像的自然图像质量评价值及单张图像的平均风格迁移时间，并求取指标的平均值为最终的风格迁移评价指标。Calculate the structural similarity between the original content image and the stylized content image, the natural image quality evaluation value of the stylized content image, and the average style transfer time of a single image, and calculate the average value of the indicators as the final Style transfer evaluation index.
优选地，滤除所述低光照变电设备缺陷图像数据中的低质量生成图像，并依托对抗验证方法利用分类模型筛选出与真实图像集合样本分布近似的生成图像，得到最终扩充的低光照变电设备缺陷图像集合，包括：Preferably, low-quality generated images in the low-light substation equipment defect image data are filtered out, and a classification model is used to screen out generated images that are similar to the sample distribution of the real image collection by relying on the adversarial verification method to obtain the final expanded low-light transformer. Collection of electrical equipment defect images, including:
设定结构相似性及自然图像质量评价值的阈值，将低于阈值的所述低光照变电设备缺陷图像数据的生成图像进行删除处理，并通过计算生成图像的感知哈希值，对于感知哈希值相同的生成图像进行删除处理，得到初步扩充图像；Set a threshold for structural similarity and natural image quality evaluation value, delete the generated image of the low-light substation equipment defect image data that is lower than the threshold, and calculate the perceptual hash value of the generated image, for the perceptual hash value Generated images with the same hash value are deleted and a preliminary expanded image is obtained;
从初步扩充图像的集合随机抽取与真实图像集合相同数量的图像，并制作分类标签，与真实图像混合后随机划分训练集与验证集，并完成对抗验证模型的训练；Randomly extract the same number of images as the real image set from the preliminary expanded image set, make classification labels, mix them with the real images and randomly divide the training set and verification set, and complete the training of the adversarial verification model;
以初步扩充图像的集合为测试集，利用训练好的对抗验证模型完成生成图像集合的分类，筛选出预测标签为1的图像直至遍历完生成图像集合，并设定评判阈值，滤除低于阈值的生成图像，得到符合真实图像样本分布的最终扩充的低光照变电设备缺陷图像集合。Use the initially expanded image set as the test set, use the trained adversarial verification model to complete the classification of the generated image set, filter out the images with a predicted label of 1 until the traversal is completed to generate the image set, and set the judgment threshold to filter out the images below the threshold Generate images to obtain the final expanded low-light substation equipment defect image collection that conforms to the real image sample distribution.
根据本发明提供的具体实施例，本发明公开了以下技术效果：According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:
本发明提供了一种变电设备缺陷图像数据扩充与数据清洗方法，包括：利用预设数据集对风格迁移网络进行训练，得到训练好的网络权重参数；遍历原始变电设备缺陷图像和弱光场景图像，并利用加载所述网络权重参数的所述风格迁移网络，根据所述原始变电设备缺陷图像和弱光场景图像对缺陷图像数据进行扩充，得到低光照变电设备缺陷图像数据，并验证风格迁移效果；滤除所述低光照变电设备缺陷图像数据中的低质量生成图像，并依托对抗验证方法利用分类模型筛选出与真实图像集合样本分布近似的生成图像，得到最终扩充的低光照变电设备缺陷图像集合。本发明有助于解决深度学习模型在弱光场景泛化性弱的问题。The invention provides a method for data expansion and data cleaning of substation equipment defect images, which includes: using a preset data set to train a style transfer network to obtain trained network weight parameters; traversing the original substation equipment defect images and low light scene image, and use the style migration network loaded with the network weight parameters to expand the defect image data according to the original substation equipment defect image and low-light scene image to obtain low-light substation equipment defect image data, and Verify the style transfer effect; filter out the low-quality generated images in the low-light substation equipment defect image data, and rely on the adversarial verification method to use the classification model to screen out the generated images that are similar to the sample distribution of the real image collection, and obtain the final expanded low-quality generated images. Collection of images of lighting substation equipment defects. The present invention helps solve the problem of weak generalization of deep learning models in low-light scenes.
附图说明Description of the drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the drawings of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.
图1为本发明实施例提供的方法流程图；Figure 1 is a method flow chart provided by an embodiment of the present invention;
图2为本发明实施例提供的流程框图。Figure 2 is a flow chart provided by an embodiment of the present invention.
图3为本发明实施例提供的多阶段训练策略示意图。Figure 3 is a schematic diagram of a multi-stage training strategy provided by an embodiment of the present invention.
图4为本发明实施例提供的数据扩充示意图。Figure 4 is a schematic diagram of data expansion provided by an embodiment of the present invention.
图5为本发明实施例提供的联合空间-通道相关性自适应实例归一化结构示意图。Figure 5 is a schematic diagram of the normalized structure of an example of joint spatial-channel correlation adaptation provided by an embodiment of the present invention.
图6为本发明实施例提供的本方法与基线风格迁移方法的效果对比图。Figure 6 is a comparison diagram of the effects of this method and the baseline style transfer method provided by the embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.
本发明的目的是提供一种变电设备缺陷图像数据扩充与数据清洗方法，有助于解决深度学习模型在弱光场景泛化性弱的问题。The purpose of the present invention is to provide a method for data expansion and data cleaning of substation equipment defect images, which helps to solve the problem of weak generalization of deep learning models in low-light scenes.
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。In order to make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.
图1为本发明实施例提供的方法流程图，如图1所示，本发明提供了一种变电设备缺陷图像数据扩充与数据清洗方法，包括：Figure 1 is a method flow chart provided by an embodiment of the present invention. As shown in Figure 1, the present invention provides a method for data expansion and data cleaning of substation equipment defect images, including:
步骤100：利用预设数据集对风格迁移网络进行训练，得到训练好网络权重参数；Step 100: Use the preset data set to train the style transfer network and obtain the trained network weight parameters;
步骤200：遍历原始变电设备缺陷图像和弱光场景图像，并利用加载所述权重参数的风格迁移网络，根据所述原始变电设备缺陷图像和弱光场景图像对缺陷图像数据进行扩充，得到低光照变电设备缺陷图像数据，并验证风格迁移效果；Step 200: Traverse the original substation equipment defect images and low-light scene images, and use the style migration network loaded with the weight parameters to expand the defect image data according to the original substation equipment defect images and low-light scene images to obtain Low-light substation equipment defect image data and verify the style transfer effect;
步骤300：滤除所述低光照变电设备缺陷图像数据中的低质量生成图像，并依托对抗验证方法利用分类模型筛选出与真实图像集合样本分布近似的生成图像，得到最终扩充的低光照变电设备缺陷图像集合。Step 300: Filter out the low-quality generated images in the low-light substation equipment defect image data, and rely on the adversarial verification method to use the classification model to screen out the generated images that are similar to the sample distribution of the real image set, and obtain the final expanded low-light transform. Collection of electrical equipment defects images.
可选地，本实施例中利用风格迁移网络从清晨、黄昏、黑夜等弱光场景和沙尘、暴雨、大雪、浓雾等恶劣天气场景中的变电站图像中提场景特征，并通过联合空间-通道特征分布补偿机制的自适应实例归一化方法将低光照场景迁移到变电设备缺陷图像中，生成低光照缺陷图像并通过对抗验证模型清洗生成图像集合，保证与原始图像样本分布的一致性，从而实现高质量的低光照变电设备缺陷图像数据扩充。Optionally, in this embodiment, the style transfer network is used to extract scene features from substation images in low-light scenes such as early morning, dusk, and night, and severe weather scenes such as sand, heavy rain, heavy snow, and dense fog, and through the joint space- The adaptive instance normalization method of the channel feature distribution compensation mechanism migrates low-light scenes to substation equipment defect images, generates low-light defect images, and generates image sets through adversarial verification model cleaning to ensure consistency with the original image sample distribution. , thereby achieving high-quality low-light substation equipment defect image data expansion.
变电设备缺陷图像数据扩充与数据清洗方法的整个流程可以用图2表示。The entire process of the substation equipment defect image data expansion and data cleaning method can be represented in Figure 2.
步骤1：风格迁移网络训练；利用MS COCO 2017数据集完成风格迁移网络前向传播、损失计算及反向传播，并保存训练好的模型权重参数。具体而言，风格迁移网络采用对称的编解码结构，编码网络以最大池化层为界限将VGG16划分为4个独立的编码块，解码网络为VGG16的翻转对称结构，同样按最大池化层为界限被划分4个独立的解码块，需要说明的是，编解码块中的卷积层均采用反射填充，以避免结构伪影的生成。风格迁移网络采用多阶段训练策略，每个阶段针对单个编解码块块进行感知和重建方面的加强训练，其示意图如图3所示，训练步骤如下：Step 1: Style transfer network training; use the MS COCO 2017 data set to complete the forward propagation, loss calculation and back propagation of the style transfer network, and save the trained model weight parameters. Specifically, the style transfer network adopts a symmetric encoding and decoding structure. The encoding network divides VGG16 into 4 independent encoding blocks with the maximum pooling layer as the boundary. The decoding network is the flip symmetric structure of VGG16, and the maximum pooling layer is also used. The boundary is divided into four independent decoding blocks. It should be noted that the convolutional layers in the encoding and decoding blocks all use reflection filling to avoid the generation of structural artifacts. The style transfer network adopts a multi-stage training strategy. Each stage performs enhanced training on perception and reconstruction for a single encoding and decoding block. The schematic diagram is shown in Figure 3. The training steps are as follows:
S1：随机选取图像；从MS COCO 2017数据集中随机选取一个批次的图像，调整此批次图像大小为512×512，并随机裁剪出256×256的图像经归一化后转换为输入特征在本实施例中，批次大小N为16，通道数C为3，宽度H和高度W为256。S1: Randomly select images; randomly select a batch of images from the MS COCO 2017 data set, adjust the image size of this batch to 512×512, and randomly crop out 256×256 images and convert them into input features after normalization In this example, the batch size N is 16, the number of channels C is 3, and the width H and height W are 256.
S2：多阶段风格迁移网络训练；第1阶段编解码网络训练时，输入特征I经过编码网络φ1(·)得到编码特征φ1(I)，并通过解码网络还原输入特征/>而后计算输入与输出特征间的内容重建损失和图像平滑损失，并进行损失迭代、梯度反传、参数更新直至达到设定的训练轮次，而后保存编解码网络的权重参数。在本实例中，训练轮次设定为10，编码网络加载在ImageNet预训练的权重，解码网络不加载权重。总损失/>内容重建损失Lcc计算式如下：S2: Multi-stage style transfer network training; in the first stage of encoding and decoding network training, the input feature I passes through the encoding network φ1 (·) to obtain the encoding feature φ1 (I), and passes through the decoding network Restore input features/> Then calculate the content reconstruction loss and image smoothing loss between the input and output features, and perform loss iteration, gradient backpropagation, and parameter update until the set training round is reached, and then save the weight parameters of the encoding and decoding network. In this example, the training round is set to 10, the encoding network loads the weights pre-trained on ImageNet, and the decoding network does not load the weights. Total loss/> The content reconstruction loss Lcc is calculated as follows:
式(1)中，与/>表示内容重建损失和图像平滑损失，均采用均方误差损失。式(2)中，/>I′ij分别为输入特征I和输出特征I′第l张特征图、(x,y)位置上的元素值。In formula (1), with/> Represents content reconstruction loss and image smoothing loss, both using mean square error loss. In formula (2),/> I′ij are the element values at the (x, y) position of the l-th feature map and (x, y) position of the input feature I and output feature I′ respectively.
而后进行第n∈{2,3,4}阶段编解码网络训练；输入特征I经过第n个编码网络φn(·)得到编码特征φn(I)，并通过第n个解码网络还原输入特征/>而后计算输入与输出特征间的内容重建损失、图像平滑损失、内容感知损失及解码块反演损失，并进行损失迭代、梯度反传、参数更新直至达到设定的训练轮次，而后保存编解码网络的权重参数。在本实例中，训练轮次设定为10，编解网络块加载N-1次训练保存的权重参数。总损失内容感知损失/>解码反演损失/>计算式如下：Then the n ∈ {2, 3, 4} stage encoding and decoding network training is performed; the input feature I passes through the nth encoding network φn (·) to obtain the encoding feature φn (I), and passes through the nth decoding network Restore input features/> Then calculate the content reconstruction loss, image smoothing loss, content awareness loss and decoding block inversion loss between the input and output features, and perform loss iteration, gradient backpropagation, and parameter update until the set training round is reached, and then save the encoding and decoding The weight parameters of the network. In this example, the training rounds are set to 10, and the decoding network block loads the weight parameters saved by N-1 trainings. total loss Content awareness loss/> Decoding inversion loss/> The calculation formula is as follows:
式(4)中，内容重建损失图像平滑损失/>内容感知损失/>和解码反演损失均采用均方误差损失。式(4)中，/>为输入特征经第n个编码网络φn(I)的输出特征，为输出特征/>经第n个编码网络φn(I)的输出特征。式(5)中，/>为第n-1个编码网络φn-1(I)的输出特征，/>为第n个解码网络的输出特征。In equation (4), the content reconstruction loss Image smoothing loss/> Content awareness loss/> and decoding inversion loss Mean square error loss is used. In formula (4),/> is the output feature of the input feature through the nth coding network φn (I), is the output feature/> The output features of the nth encoding network φn (I). In formula (5),/> is the output feature of the n-1 coding network φn-1 (I),/> is the output feature of the nth decoding network.
步骤2：低光照变电设备缺陷图像数据扩充；遍历原始变电设备缺陷图像和变电站弱光场景图像，利用加载权重的风格迁移网络完成缺陷图像数据扩充，并验证风格迁移效果。具体而言，首先挑选变电设备缺陷图像和变电站弱光照场景图分别构建内容图像集合C和风格图像集合S，在本实例中，所选变电设备缺陷为渗漏油，变电站场景图选取清晨、黄昏、黑夜三类弱光场景和沙尘、暴雨、大雪、浓雾四类恶劣天气场景。图像扩充示意图如图4所示，包含以下步骤：Step 2: Expand low-light substation equipment defect image data; traverse the original substation equipment defect images and substation low-light scene images, use the weighted style transfer network to complete the defect image data expansion, and verify the style transfer effect. Specifically, we first select substation equipment defect images and substation low-light scene images to construct content image set C and style image set S respectively. In this example, the selected substation equipment defect is oil leakage, and the substation scene image is early morning. There are three types of low-light scenes, dusk and night, and four types of severe weather scenes including sand and dust, heavy rain, heavy snow and thick fog. The schematic diagram of image expansion is shown in Figure 4, which includes the following steps:
S3：随机选取内容风格图像并提取内容风格特征；从集合C中随机选择一张内容图像c，从集合S中随机选择一张风格图像s，利用训练好的编码网络φ4(·)完成内容图像和风格图像的特征提取，得到内容编码特征与风格编码特征集合其中/>表示第4个编码网络的第n层编码块，而后利用高频跳跃连接(High-frequency Skip Connection，HSC)模块实现内容图像编码特征的高低频频率分量分解，得到高频残差分量/>第n个编码块的高频残差分量/>的计算方法如下：S3: Randomly select content style images and extract content style features; randomly select a content image c from the set C, randomly select a style image s from the set S, and use the trained coding network φ4 (·) to complete the content Feature extraction of image and style images to obtain content encoding features Collection of features encoded with style Among them/> Represents the nth layer coding block of the fourth coding network, and then uses the High-frequency Skip Connection (HSC) module to implement the content image coding features Decompose the high- and low-frequency frequency components to obtain the high-frequency residual component/> High-frequency residual component of the nth coding block/> The calculation method is as follows:
式(6)中，(·)up表示由双线性插值实现的上采样操作，Avgpool(·)表示平均池化操作。In formula (6), (·)up represents the upsampling operation implemented by bilinear interpolation, and Avgpool (·) represents the average pooling operation.
S4：多阶段风格迁移；前m∈{1,2,3}阶段风格迁移内容编码特征与风格编码特征/>通过通道相关-自适应实例归一化(Channel correlation-AdaptiveInstance Normalization,C-AdaIN)实现内容风格特征在通道分布的对齐，而后通过解码块/>进行通道维度压缩和空间维度扩张操作，通道压缩操作由3×3卷积实现，空间维度扩张操作由双线性插值实现，在维度变换后与高频残差分量/>融合以补充空间信息，从而得到第m+1阶段风格迁移的内容特征输入/>S4: Multi-stage style transfer; first m∈{1,2,3} stage style transfer content encoding features and style encoding features/> The alignment of content style features in channel distribution is achieved through channel correlation-adaptive instance normalization (C-AdaIN), and then through the decoding block/> Perform channel dimension compression and spatial dimension expansion operations. The channel compression operation is implemented by 3×3 convolution, and the spatial dimension expansion operation is implemented by bilinear interpolation. After dimension transformation, it is combined with the high-frequency residual component/> Fusion to supplement spatial information to obtain the content feature input of the m+1 stage style transfer/>
第四阶段风格迁移以风格编码特征φ4-1(s)为输入，采用空间相关-自适应实例归一化(Spatial correlation-Adaptive Instance Normalization,S-AdaIN)实现内容风格特征在空间分布上的对齐，而后利用解码块/>还原输入图像的通道、空间维度，得到风格化后的内容图像。C-AdaIN与S-AdaIN结构示意图如图5所示，其中C-AdaIN对内容编码特征φ(c)与风格编码特征φ(s)的转换流程如下式：The fourth stage of style transfer starts with The style encoding feature φ4-1 (s) is the input, and spatial correlation-Adaptive Instance Normalization (S-AdaIN) is used to align the content style features in the spatial distribution, and then the decoding block is used /> Restore the channels and spatial dimensions of the input image to obtain the stylized content image. The schematic structural diagram of C-AdaIN and S-AdaIN is shown in Figure 5. The conversion process of C-AdaIN between content coding feature φ(c) and style coding feature φ(s) is as follows:
cs＝ccor×AdaIN(φ(c),φ(s)) (7)cs =ccor ×AdaIN(φ(c),φ(s)) (7)
ccor＝catt×sigmoid(catt×(satt)T) (8)ccor =catt ×sigmoid(catt ×(satt )T ) (8)
式(7)中，cs表示风格迁移后输出特征图，ccor表示内容图像编码特征与风格图像编码特征间通道特征分布的互相关矩阵，其计算式如式(8)所示，其中AdaIN(·)表示自适应实例归一化操作，其计算式如式(9)所示。式(8)中，catt与satt分别表示内容图像编码特征与风格图像特征间通道权重，通过沿通道方向的全局自适应池化得出。sigmoid(·)表示利用sigmoid函数完成归一化操作。(·)T表示维度转置操作。式(9)中，Norm()表示沿通道方向的归一化操作，σ(·)表示方差计算操作，u(·)表示均值计算操作。C-AdaIN在内容图像和风格图像统计特征分布对齐的同时对齐了两者通道维度上的相关关系，以缩小特征传递误差。In Equation (7), cs represents the output feature map after style transfer, and ccor represents the cross-correlation matrix of channel feature distribution between content image encoding features and style image encoding features. Its calculation formula is as shown in Equation (8), where AdaIN (·) represents the adaptive instance normalization operation, and its calculation formula is shown in Equation (9). In Equation (8), catt and satt respectively represent the channel weights between content image coding features and style image features, which are obtained through global adaptive pooling along the channel direction. sigmoid(·) means using the sigmoid function to complete the normalization operation. (·)T represents the dimension transpose operation. In formula (9), Norm() represents the normalization operation along the channel direction, σ(·) represents the variance calculation operation, and u(·) represents the mean calculation operation. C-AdaIN not only aligns the statistical feature distribution of the content image and the style image, but also aligns the correlation relationship in the channel dimension of the two to reduce the feature transfer error.
其中S-AdaIN对内容编码特征φ(c)与风格编码特征φ(s)的转换流程如下式：The conversion process of S-AdaIN between content coding features φ(c) and style coding features φ(s) is as follows:
S-AdaIN(x,y)＝σ(scor×σ(φ(s)))×Norm(φ(c))+u(scor×u(φ(s)))(10)S-AdaIN(x,y)＝σ(scor ×σ(φ(s)))×Norm(φ(c))+u(scor ×u(φ(s)))(10)
scor＝Sigmoid(Norm(φ(s))×(Norm(φ(c)))T) (11)scor =Sigmoid(Norm(φ(s))×(Norm(φ(c)))T ) (11)
式(10)中，σ(·)表示方差计算操作，u(·)表示均值计算操作。Norm()表示沿通道方向的归一化操作，Scor表示内容图像编码特征与风格图像编码特征间空间特征分布的互相关矩阵。其计算如式(11)所示，其中sigmoid(·)表示利用sigmoid函数完成归一化操作。(·)T表示维度转置操作。In formula (10), σ(·) represents the variance calculation operation, and u(·) represents the mean calculation operation. Norm() represents the normalization operation along the channel direction, and Scor represents the cross-correlation matrix of spatial feature distribution between content image encoding features and style image encoding features. The calculation is shown in Equation (11), where sigmoid(·) represents the use of the sigmoid function to complete the normalization operation. (·)T represents the dimension transpose operation.
S5：样本生成与指标计算；遍历完所有的风格图像集合，并计算内容图像与风格化后的内容图像之间的结构相似性(Structural Similarity,SSIM)、风格化后内容图像的自然图像质量评价值(Natural Image Quality Evaluator,NIQE)及单张图像的平均风格迁移时间，并求取指标的平均值为最终的风格迁移评价指标。其中，平均SSIM越大，迁移图像与内容图像结构越接近，平均NIQE值越小越接近自然真实图像，在本实施例中，随机选取10张变电设备渗漏油图像及8张变电站场景图(2张黑夜场景图像)，共生成低光照光照场景下的变电设备渗漏油图像80张。图6给出了本专利方法与基线方法PhotoWCT2的效果对比图，PhotoWCT2的核心组件采用白化与着色变换(Whiten-Color Transform,WCT)，而本专利方法采用联合空间-通道特征分布补偿机制的自适应实例归一化为核心组件(jointSpatial-Channel feature distributions Adaptive Instance Normalization，CS-AdaIN)，表1给出不同风格迁移网络图像迁移质量和图像迁移速度量化指标的对比结果，其中第一行为基线方法，第二行为本专利方法，其表明本专利方法在不同分辨率情况下图像迁移速度超过基线方法，图像迁移质量优于基线方法。S5: Sample generation and index calculation; traverse all style image collections, and calculate the structural similarity (SSIM) between the content image and the stylized content image, and the natural image quality evaluation of the stylized content image. Value (Natural Image Quality Evaluator, NIQE) and the average style transfer time of a single image, and the average value of the indicators is calculated as the final style transfer evaluation indicator. Among them, the larger the average SSIM, the closer the migration image is to the content image structure, and the smaller the average NIQE value is, the closer it is to the natural real image. In this embodiment, 10 oil leakage images of substation equipment and 8 substation scene images are randomly selected. (2 dark night scene images), a total of 80 images of oil leakage from substation equipment under low-light lighting scenes were generated. Figure 6 shows the comparison of the effects of this patented method and the baseline method PhotoWCT2. The core component of PhotoWCT2 uses whiten-color transform (WCT), while the patented method uses the automatic joint space-channel feature distribution compensation mechanism. Adaptive instances are normalized into core components (jointSpatial-Channel feature distributions Adaptive Instance Normalization, CS-AdaIN). Table 1 gives the comparison results of image transfer quality and image transfer speed quantification indicators of different style transfer networks. The first line is the baseline method. , the second line is the patented method, which shows that the image migration speed of the patented method exceeds the baseline method under different resolutions, and the image migration quality is better than the baseline method.
表1Table 1
步骤三：低光照变电设备缺陷图像数据清洗；具体而言，首先滤除低指标图像、高度相似图像后得到初步扩充的低光照变电设备缺陷图像集合。而后针对真实变电设备缺陷图像集合及生成的低光照变电设备缺陷图像集合制作分类标签，并从生成的低光照变电设备缺陷图像集合依次筛选与真实变电设备缺陷图像集合相同数量的样本，划分训练集与验证集并完成真实图像与生成图像分类模型的训练，而后将生成的低光照变电设备图像集合作为测试集，筛选出与真实图像集合样本分布近似的生成图像直至遍历完整个生成图像集合，统计并筛选在不同模型测试情况下与真实图像样本集合样本分布近似的生成图像，得到最终扩充的低光照变电设备缺陷图像集合。Step 3: Low-light substation equipment defect image data cleaning; specifically, low-index images and highly similar images are first filtered out to obtain a preliminary expanded set of low-light substation equipment defect images. Then, classification labels are made for the real substation equipment defect image collection and the generated low-light substation equipment defect image collection, and the same number of samples as the real substation equipment defect image collection are filtered from the generated low-light substation equipment defect image collection. , divide the training set and the verification set and complete the training of the real image and generated image classification models, and then use the generated low-light substation equipment image set as the test set to filter out the generated images that are similar to the sample distribution of the real image set until the entire image is traversed Generate an image set, count and filter the generated images that are similar to the sample distribution of the real image sample set under different model testing conditions, and obtain the final expanded low-light substation equipment defect image set.
S6：低质量图像过滤；对于生成的低光照变电设备缺陷图像集合，设定SSIM及NIQE阈值，将低于阈值的生成图像进行删除处理，并哈感知希函数通过计算生成图像的感知哈希值，对于感知哈希值相同的生成图像进行删除处理。在本实例中，以渗漏油图像数据清洗为例，将10张真实变电设备缺陷图像的NIQE平均值作为阈值，SSIM阈值设定为0.75，通过滤除低质量生成图像后，得到初步扩充的渗漏油图像。S6: Low-quality image filtering; for the generated low-light substation equipment defect image set, set the SSIM and NIQE thresholds, delete the generated images below the threshold, and calculate the perceptual hash of the generated image through the hashing function. value, delete the generated images with the same perceptual hash value. In this example, taking oil leakage image data cleaning as an example, the NIQE average of 10 real substation equipment defect images is used as the threshold, and the SSIM threshold is set to 0.75. After filtering out low-quality generated images, preliminary expansion is obtained image of oil leak.
S7:对抗验证模型训练；从生成图像集合随机抽取与真实图像集合相同数量的图像，并制作分类标签，与真实图像混合后随机划分训练集与验证集，并完成对抗验证模型的训练。在本实施例中，从生成的渗漏油图像集合中依次抽取10张，标签赋值为0，与10张标签赋值为1的真实图像渗漏油集合混合后划分训练集与验证集，其比例为7：3。对抗验证模型选用分类网络ResNet50。S7: Adversarial verification model training: Randomly extract the same number of images as the real image set from the generated image set, make classification labels, mix them with real images and randomly divide the training set and verification set, and complete the training of the adversarial verification model. In this embodiment, 10 images with a label assigned a value of 0 are extracted from the generated oil leakage image set, mixed with 10 real oil leakage images with a label assigned a value of 1, and then divided into a training set and a verification set. The ratio It’s 7:3. The classification network ResNet50 is selected as the adversarial verification model.
S8:样本筛选；以生成图像集合为测试集，利用训练好的对抗验证模型完成生成图像集合的分类，筛选出预测标签为1的图像直至遍历完生成图像集合，即与真实图像样本分布近似的生成图像，统计每个生成图像被不同样本训练出的分类预测为真实图像样本的次数，设定评判阈值，并滤除低于阈值的生成图像，从而得到符合真实图像样本分布的生成图像。在本实施例中，以不同生成图像组合共训练对抗模型6次，评判阈值设定为3，并由此得到最终扩充的低光照变电设备渗漏油图像。S8: Sample screening; use the generated image set as the test set, use the trained adversarial verification model to complete the classification of the generated image set, and filter out the images with the predicted label 1 until the traversal is completed to generate the image set, which is similar to the real image sample distribution Generate images, count the number of times each generated image is predicted to be a real image sample by classification trained by different samples, set a judgment threshold, and filter out generated images below the threshold, thereby obtaining a generated image that conforms to the distribution of real image samples. In this embodiment, the adversarial model was trained 6 times with different generated image combinations, and the evaluation threshold was set to 3, and the final expanded low-light substation equipment leakage oil image was obtained.
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的系统而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. As for the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For relevant details, please refer to the description in the method section.
本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处。综上所述，本说明书内容不应理解为对本发明的限制。This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and the core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the present invention There will be changes in the specific implementation methods and application scope of the ideas. In summary, the contents of this description should not be construed as limitations of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311105621.1ACN117079078A (en) | 2023-08-30 | 2023-08-30 | Defect image data expansion and data cleaning method for power transformation equipment |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311105621.1ACN117079078A (en) | 2023-08-30 | 2023-08-30 | Defect image data expansion and data cleaning method for power transformation equipment |
| Publication Number | Publication Date |
|---|---|
| CN117079078Atrue CN117079078A (en) | 2023-11-17 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311105621.1APendingCN117079078A (en) | 2023-08-30 | 2023-08-30 | Defect image data expansion and data cleaning method for power transformation equipment |
| Country | Link |
|---|---|
| CN (1) | CN117079078A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119810394A (en)* | 2025-03-13 | 2025-04-11 | 成都创世汉沙科技有限公司 | A data processing method and system based on image acquisition device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119810394A (en)* | 2025-03-13 | 2025-04-11 | 成都创世汉沙科技有限公司 | A data processing method and system based on image acquisition device |
| CN119810394B (en)* | 2025-03-13 | 2025-07-04 | 成都创世汉沙科技有限公司 | Data processing method and system based on image acquisition equipment |
| Publication | Publication Date | Title |
|---|---|---|
| CN107784676B (en) | Compressed sensing measurement matrix optimization method and system based on autoencoder network | |
| CN111462013B (en) | Single-image rain removing method based on structured residual learning | |
| CN108986050A (en) | A kind of image and video enhancement method based on multiple-limb convolutional neural networks | |
| CN113780389B (en) | Deep learning semi-supervised dense matching method and system based on consistency constraint | |
| CN116523875B (en) | Insulator defect detection method based on FPGA preprocessing and improvement YOLOv5 | |
| CN111787187A (en) | Method, system and terminal for video restoration using deep convolutional neural network | |
| CN108875596A (en) | A kind of railway scene image, semantic dividing method based on DSSNN neural network | |
| CN111696033B (en) | Real image super-resolution model and method based on corner-guided cascaded hourglass network structure learning | |
| CN114612803B (en) | Improved CENTERNET transmission line insulator defect detection method | |
| CN116523985B (en) | Structure and texture feature guided double-encoder image restoration method | |
| CN116416237A (en) | Power transmission line defect detection method based on improved YOLOv5 and fuzzy image enhancement | |
| CN117079078A (en) | Defect image data expansion and data cleaning method for power transformation equipment | |
| CN113721859A (en) | Image repeating data deleting method based on artificial intelligence | |
| CN115082798A (en) | A method for detecting pin defects in transmission lines based on dynamic receptive field | |
| CN116468625A (en) | Single image defogging method and system based on pyramid efficient channel attention mechanism | |
| CN114155171A (en) | An image inpainting method and system based on dense multi-scale fusion | |
| CN118195899A (en) | A lightweight hybrid attention distillation network based image super-resolution model | |
| CN119579891A (en) | Remote sensing image semantic segmentation method based on dual-branch multi-scale fusion network | |
| CN117197530A (en) | Insulator defect identification method based on improved YOLOv8 model and cosine annealing learning rate decay method | |
| CN116523742A (en) | Satellite remote sensing image super-resolution reconstruction technology based on two-way countermeasure generation network | |
| Wang et al. | Target detection algorithm based on super-resolution color remote sensing image reconstruction | |
| CN114022356B (en) | Super-resolution method and system for river flow and water level remote sensing images based on wavelet domain | |
| CN118632027B (en) | A point cloud compression method based on graph convolutional network | |
| CN112734732B (en) | Detection method of leaky cable fixture in railway tunnel based on improved SSD algorithm | |
| CN114639020A (en) | Segmentation network, segmentation system and segmentation device for target object of image |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |