CN114972721A - A method for identifying and locating insulator strings in transmission lines based on deep learning - Google Patents

Abstract

The invention provides a power transmission line insulator string recognition and positioning method based on deep learning, which can effectively solve the problems of insulator string recognition and defect positioning by combining a Faster R-CNN network model and a YOLOv3 network model. Meanwhile, the problem of insufficient insulator string data sets can be effectively solved by a data enhancement method and quantitative analysis of the method. The insulator string recognition and positioning task is divided into two stages, the insulator string recognition is carried out by using an Faster R-CNN network model in the first stage, the defect positioning is carried out by using a YOLOv3 network model in the second stage, various factors in the task are comprehensively considered, and an insulator string recognition network based on the Faster R-CNN network model and an insulator string defect positioning network based on the YOLOv3 network model are designed. The scheme has the advantages of higher accuracy and higher recall rate at present.

Description

Translated fromChinese

一种基于深度学习的输电线路绝缘子串识别定位方法A method for identifying and locating insulator strings in transmission lines based on deep learning

技术领域technical field

本发明涉及工业物联网与边缘计算领域，具体涉及一种输电线路绝缘子串识别定位方法。The invention relates to the field of industrial internet of things and edge computing, in particular to a method for identifying and locating an insulator string of a transmission line.

背景技术Background technique

近年来，在利用无人机对输电线路巡检的过程中，会产生大量的图片数据，目前大多是利用人工对绝缘子串图片进行判读，来完成绝缘子串的目标标注以及缺陷的定位。使用这种方法效率低，而且需要经验丰富的工作人员进行判断。所以，提出一种自动化的图像识别方式来进行绝缘子串的识别和缺陷定位具有很高的应用价值。近年来，利用计算机视觉技术对输电线路上电气元件的检测成为近几年的研究热点。In recent years, in the process of using drones to inspect transmission lines, a large amount of picture data will be generated. At present, most of the pictures of insulator strings are manually interpreted to complete the target labeling and defect positioning of insulator strings. Using this method is inefficient and requires the judgment of experienced staff. Therefore, it is of high application value to propose an automatic image recognition method for insulator string recognition and defect location. In recent years, the detection of electrical components on transmission lines using computer vision technology has become a research hotspot in recent years.

绝缘子串作为输电线路中的物理介质，具有电气绝缘和支撑线路的功能，对电力系统的正常运行具有十分关键的作用。一旦输电线路上的绝缘子串发生故障，就会导致输电系统的停滞或崩溃。在拍摄图像中，绝缘子串的缺陷大小相对于整幅图片而言很小，所以直接在原图中定位，会使得准确率降低。并且，采用将识别定位任务分为两阶段的方法会带来模型准确率不高的问题。As the physical medium in the transmission line, the insulator string has the functions of electrical insulation and supporting the line, and plays a very critical role in the normal operation of the power system. Once the insulator string on the transmission line fails, it will cause the stagnation or collapse of the transmission system. In the captured image, the defect size of the insulator string is very small compared to the whole picture, so positioning directly in the original image will reduce the accuracy. Moreover, the method of dividing the identification and localization task into two stages will bring about the problem of low model accuracy.

发明内容SUMMARY OF THE INVENTION

本发明的目的在于提供一种更精确的基于深度学习的输电线路绝缘子串识别定位方法，能够准确的识别出图像中的绝缘子串，并对缺陷的部分进行定位，提出了一种新的绝缘子串识别定位的方案。The purpose of the present invention is to provide a more accurate identification and positioning method of transmission line insulator string based on deep learning, which can accurately identify the insulator string in the image and locate the defective part, and proposes a new insulator string Identify the positioning scheme.

本发明为实现上述目的所采用的技术方案是：一种基于深度学习的输电线路绝缘子串识别定位方法，包括以下步骤：The technical solution adopted by the present invention to achieve the above object is: a deep learning-based method for identifying and locating an insulator string of a transmission line, comprising the following steps:

1)采用Faster R-CNN网络模型，对输入的绝缘子串图像进行识别；1) Using the Faster R-CNN network model to identify the input insulator string image;

2)采用YOLOv3模型，对绝缘子串的缺陷进行定位。2) Using the YOLOv3 model to locate the defects of the insulator strings.

所述步骤1)中，采用基于VGG16的Faster R-CNN网络模型对图像进行识别，其中，所述Faster R-CNN网络模型的损失函数由RPN部分和分类器部分各自的分类损失和边界框回归损失组成；分类损失函数均采用多类别的交叉熵损失函数，边界框回归损失函数均采用平方差损失函数。In the step 1), the image is identified by the Faster R-CNN network model based on VGG16, wherein the loss function of the Faster R-CNN network model is returned by the respective classification loss and bounding box regression of the RPN part and the classifier part. Loss composition; the classification loss function adopts the multi-category cross entropy loss function, and the bounding box regression loss function adopts the square difference loss function.

步骤1)中，采用基于VGG16的Faster R-CNN网络模型对图像进行识别，其中，所述Faster R-CNN网络模型的边界框回归损失函数实现方式包括以下步骤：In step 1), the VGG16-based Faster R-CNN network model is used to identify the image, wherein the implementation of the bounding box regression loss function of the Faster R-CNN network model includes the following steps:

首先，将图像分成w×h个网格，并对网格大小进行归一化；First, divide the image into w×h grids and normalize the grid size;

然后，在处理后的图像中进行回归，回归公式如下：Then, regression is performed on the processed image, and the regression formula is as follows:

G″_x＝σ(d'_x(A))+c_xG″_x =σ(d'_x (A))+c_x

G″_y＝σ(d'_y(A))+c_yG″_y =σ(d'_y (A))+c_y

G″_w＝A'_wexp(d'_w(A))G″_w =A'_w exp(d'_w (A))

G″_h＝A'_hexp(d_h'(A))G″_h =A'_h exp(d_h '(A))

其中，其中，G″_x,G″_y是先验框坐标，G″_w,G″_h是先验框的宽和高，σ()函数是sigmoid函数，c_x和c_y是预测框所在网格的左上角到图像左上角的相对距离，A'_w和A'_h是预测框的相对大小，d'_x(A),d'_y(A),d'_w(A),d'_h(A)是四个需要学习的参数。Among them, G″_x , G″_y are the coordinates of the prior frame, G″_w , G″_h are the width and height of the prior frame, the σ() function is the sigmoid function, and c_x and c_y are where the prediction frame is located The relative distance from the upper left corner of the grid to the upper left corner of the image, A'_w and A'_h are the relative sizes of the prediction boxes, d'_x (A),d'_y (A),d'_w (A),d'_h (A) are four parameters that need to be learned.

在计算边界框回归损失时，对先验框的高和宽以及预测框的高和宽分别进行开方处理后，再计算平方误差损失。When calculating the regression loss of the bounding box, the height and width of the prior box and the height and width of the prediction box are respectively squared, and then the squared error loss is calculated.

所述YOLOv3模型中的检测器部分包括顺次的卷积块集合、3×3卷积块、1×1卷积层，所述1×1卷积层生成结果信息，以表示预测缺陷所处的位置以及是否为缺陷的置信度；The detector part in the YOLOv3 model includes a sequential set of convolution blocks, a 3×3 convolution block, and a 1×1 convolution layer. The 1×1 convolution layer generates result information to indicate where the prediction defect is located. the location and confidence of whether it is a defect;

所述的卷积块集合包括多个依次连接的卷积块，依次为1×1卷积块、3×3卷积块、1×1卷积块、3×3卷积块、1×1卷积块；The convolution block set includes a plurality of convolution blocks connected in sequence, which are 1×1 convolution block, 3×3 convolution block, 1×1 convolution block, 3×3 convolution block, and 1×1 convolution block. convolution block;

所述卷积块包含依次的一个卷积层、一个BN层、一个LeakyReLU激活函数。The convolution block includes a convolution layer, a BN layer, and a LeakyReLU activation function in sequence.

Faster R-CNN中的分类损失采用交叉熵损失函数，所述边界框回归损失采用平方差损失函数；所述YOLOv3模型中，回归损失中的中心点坐标、置信度损失和分类损失采用交叉熵损失函数，回归损失中的宽和高采用平方差损失函数。The classification loss in Faster R-CNN adopts the cross entropy loss function, and the bounding box regression loss adopts the square difference loss function; in the YOLOv3 model, the center point coordinates, confidence loss and classification loss in the regression loss adopt the cross entropy loss. function, the width and height in the regression loss use the squared difference loss function.

所述YOLOv3模型，采用k-means算法对标签框进行聚类，生成3个anchorbox，使原图像下采样到原来的八分之一，且不需要另外两个尺度的特征图。The YOLOv3 model uses the k-means algorithm to cluster the label boxes, and generates 3 anchor boxes, so that the original image is down-sampled to one-eighth of the original image, and the feature maps of the other two scales are not required.

一种基于深度学习的输电线路绝缘子串识别定位系统，包括：A deep learning-based identification and positioning system for transmission line insulator strings, comprising:

绝缘子串识别模块，用于采用Faster R-CNN网络模型，对输入的绝缘子串图像进行识别；The insulator string identification module is used to identify the input insulator string image using the Faster R-CNN network model;

绝缘子串缺陷定位模块，用于采用YOLOv3模型，对绝缘子串的缺陷进行定位。The insulator string defect location module is used to locate the defects of the insulator string using the YOLOv3 model.

一种基于深度学习的输电线路绝缘子串识别定位装置，包括存储器和处理器；所述存储器，用于存储计算机程序；所述处理器，用于当执行所述计算机程序时，实现一种基于深度学习的输电线路绝缘子串识别定位方法。A deep learning-based transmission line insulator string identification and positioning device includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement a depth-based insulator string when executing the computer program. Learning the identification and positioning method of insulator strings in transmission lines.

一种计算机可读存储介质，所述存储介质上存储有计算机程序，当所述计算机程序被处理器执行时，实现一种基于深度学习的输电线路绝缘子串识别定位方法。A computer-readable storage medium having a computer program stored on the storage medium, when the computer program is executed by a processor, implements a deep learning-based method for identifying and locating an insulator string of a transmission line.

本发明具有以下有益效果及优点：The present invention has the following beneficial effects and advantages:

本发明提出一种基于深度学习的输电线路绝缘子串识别定位方法，不仅可以解决绝缘子串的识别问题，还可以很好地解决绝缘子串的缺陷定位问题，本发明与其他方法相比，具有更高的准确率和召回率。The present invention proposes a method for identifying and locating insulator strings of transmission lines based on deep learning, which can not only solve the problem of identifying insulator strings, but also solve the problem of locating defects of insulator strings well. Compared with other methods, the present invention has higher advantages. precision and recall.

附图说明Description of drawings

图1为本发明的绝缘子串识别定位流程图；Fig. 1 is the insulator string identification and positioning flow chart of the present invention;

图2为本发明的绝缘子串识别网络架构图；Fig. 2 is an insulator string identification network architecture diagram of the present invention;

图3为本发明的绝缘子串定位网络架构图；FIG. 3 is an architecture diagram of an insulator string positioning network of the present invention;

图4a为本发明提出的识别定位算法流程结果图的具体实施样例一；Fig. 4a is the specific implementation example 1 of the result diagram of the identification and positioning algorithm flow chart proposed by the present invention;

图4b为本发明提出的识别定位算法流程结果图的具体实施样例二。FIG. 4b is a specific implementation example 2 of the result diagram of the identification and positioning algorithm flow chart proposed by the present invention.

具体实施方式Detailed ways

下面结合附图及实施例对本发明做进一步的详细说明。The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图对本发明的具体实施方式做详细的说明。在下面的描述中阐述了很多具体细节以便于充分理解本发明。但本发明能够以很多不同于在此描述的其他方式来实施,本领域技术人员可以在不违背发明内涵的情况下做类似改进，因此本发明不受下面公开的具体实施的限制。In order to make the above objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the invention. Therefore, the present invention is not limited by the specific implementation disclosed below.

本发明提供了一种深度学习的输电线路绝缘子串识别定位方法，通过结合FasterR-CNN网络模型和YOLOv3网络模型，能够有效解决绝缘子串识别以及缺陷定位问题。同时，通过数据增强方法并对方法进行定量分析，能够有效解决绝缘子串数据集不足问题。将绝缘子串识别定位任务分为两个阶段，第一阶段利用Faster R-CNN网络模型进行绝缘子串识别，第二阶段利用YOLOv3网络模型进行缺陷定位，全面考虑任务中的多种因素并设计了基于Faster R-CNN网络模型的绝缘子串识别网络以及基于YOLOv3网络模型的绝缘子串缺陷定位网络。使方案具有目前更高正确率和更高召回率的优点。The invention provides a deep learning method for identifying and locating insulator strings of transmission lines. By combining the FasterR-CNN network model and the YOLOv3 network model, the problems of identifying and locating insulator strings and defects can be effectively solved. At the same time, through the data enhancement method and quantitative analysis of the method, the problem of insufficient insulator string data set can be effectively solved. The task of insulator string identification and positioning is divided into two stages. The first stage uses the Faster R-CNN network model for insulator string identification, and the second stage uses the YOLOv3 network model for defect localization. The insulator string identification network of the Faster R-CNN network model and the insulator string defect location network based on the YOLOv3 network model. Make the scheme have the advantages of higher accuracy and higher recall at present.

一种基于深度学习的输电线路绝缘子串识别定位方法，包括以下步骤：A method for identifying and locating insulator strings of transmission lines based on deep learning, comprising the following steps:

1)采用Faster R-CNN网络模型，对输入的绝缘子串图像进行识别，该网络模型需要根据绝缘子串的特征进行修改；1) Using the Faster R-CNN network model to identify the input insulator string image, the network model needs to be modified according to the characteristics of the insulator string;

2)采用YOLOv3模型，对绝缘子串的缺陷进行定位，该网络需要根据绝缘子串的特征进行修改。2) Using the YOLOv3 model to locate the defects of the insulator strings, the network needs to be modified according to the characteristics of the insulator strings.

步骤1)将Faster R-CNN网络模型根据绝缘子串的特征进行修改，具体为：Step 1) Modify the Faster R-CNN network model according to the characteristics of the insulator strings, specifically:

在进行边界框回归损失时，由于原来的变换公式中，如下所示：When performing bounding box regression loss, since the original transformation formula is as follows:

G'_x＝A_w×d_x(A)+A_xG'_x =A_w ×d_x (A)+A_x

G'_y＝A_h×d_y(A)+A_yG'_y =A_h ×_dy (A)+A_y

G'_w＝A_w×exp[d_w(A)]G'_w =A_w ×exp[d_w (A)]

G'_h＝A_h×exp[d_h(A)]G'_h =A_h ×exp[d_h (A)]

并没有对中心点的变换进行约束，所以预测边框的中心可能出现在任何位置，早期训练容易不稳定，所以需要将原图像水平进行分成w×h个网格，并对网格大小进行归一化，令一个网格的宽＝1，高＝1，再进行变换，其公式为：There is no constraint on the transformation of the center point, so the center of the predicted frame may appear in any position, and the early training is easy to be unstable, so it is necessary to divide the original image horizontally into w×h grids, and normalize the grid size , let the width of a grid = 1, the height = 1, and then transform, the formula is:

G″_x＝σ(d'_x(A))+c_xG″_x =σ(d'_x (A))+c_x

G″_y＝σ(d'_y(A))+c_yG″_y =σ(d'_y (A))+c_y

G″_w＝A'_wexp(d'_w(A))G″_w =A'_w exp(d'_w (A))

G″_h＝A'_hexp(d′_h(A))G″_h =A'_h exp(d'_h (A))

其中，d'_x(A),d'_y(A),d'_w(A),d'_h(A)是四个需要学习的四种变换，σ()函数就是sigmoid函数，能够将因变量约束在(0，1)的区间内，c_x和c_y是anchorbox网格左上角到图像左上角的距离，A'_w和A'_h是anchorbox在该体系下的宽和高。计算损失时，可以再将其映射回原图像中。由于绝缘子串的长宽差别较大，所以需要对预测后的长宽进行开方处理，而后计算平方误差损失。Among them, d'_x (A), d'_y (A), d'_w (A), d'_h (A) are the four transformations that need to be learned, and the σ() function is the sigmoid function, which can convert the The variables are constrained in the interval (0, 1), c_x and c_y are the distance from the upper left corner of the anchorbox grid to the upper left corner of the image, and A'_w and A'_h are the width and height of the anchorbox under this system. When calculating the loss, it can then be mapped back to the original image. Since the length and width of the insulator strings are quite different, it is necessary to square the predicted length and width, and then calculate the squared error loss.

步骤2)中，将YOLOv3网络模型根据绝缘子串的特征进行修改，具体为：In step 2), the YOLOv3 network model is modified according to the characteristics of the insulator string, specifically:

k-means聚类后，只需要生成3个anchor box，原图像下采样到原来的八分之一，且不需要另外两个尺度的特征图。After k-means clustering, only 3 anchor boxes need to be generated, the original image is downsampled to one-eighth of the original, and the feature maps of the other two scales are not required.

如图1所示，此流程图主要包括2个步骤，As shown in Figure 1, this flowchart mainly includes 2 steps,

步骤1：使用基于的VGG16的FasterR-CNN网络模型对绝缘子串进行识别；Step 1: Use the VGG16-based FasterR-CNN network model to identify the insulator strings;

步骤2：使用YOLOv3网络模型对识别结果进行缺陷定位。Step 2: Use the YOLOv3 network model to locate defects on the recognition results.

如图2所示，为本发明使用的绝缘子串识别网络架构图，对于绝缘子串的定位，本发明采用基于VGG16的Faster R-CNN网络模型对图像进行识别，并对该网络模型的损失函数进行了修改，使其更适合绝缘子串的定位。FasterR-CNN网络架构由特征提取部分、RPN部分、RoIPooling部分和分类器部分组成。其特征提取部分使用VGG16网络架构，VGG16网络架构由13个步长为1的3×3的卷积层、13个ReLU激活函数，以及4个步长为2的2×2最大池化层组成，每次池化操作后，特征图的长宽都变为原来的

通道数变为原来的2倍，直到最后变为原图像的

并达到512个通道。假设featuremap的大小为w×h，其上的每一个点都对应k个anchorbox，设置k＝9。RPN部分由卷积层、softmax函数以及proposal层组成。剩余部分可以分为两条线，左边一条线，通过1×1大小的卷积层，将特征图的通道数变为18，这是因为每个anchorbox都有属于前景的概率和背景的概率，采用softmax函数进行分类。右边一条线，通过1×1大小的卷积层，将特征图的通道数变为36，这是因为每个anchorbox的位置信息包含四个数值。最终，在proposal层中，对anchorbox进行边框回归。然后，按照softmax函数分类时得到的属于前景的概率，由大到小排序anchors，提取前N个anchors作为positiveanchor，一般N＝6000。限定超出原图像的positive anchors的边界为图像边界，剔除尺寸非常小的positive anchors，最后对剩余的positiveanchor进行NMS回归，最终获得resignproposal。RoI Pooling部分的输入是特征提取部分的featuremap和RPN部分的resignproposal，在原图像的featuremap中，将resignproposal对应的部分，水平分成7×7大小的网格，对每个网格进行maxpooling处理后，送入分类器部分。分类器部分包括全连接层、ReLU激活函数和softmax函数。该部分利用全连接层与softmax函数对输入进行分类，并计算属于该类的概率。同时，再次利用边框回归获得更加精确的目标检测框。As shown in FIG. 2, it is the network architecture diagram of the insulator string identification used in the present invention. For the positioning of the insulator string, the present invention adopts the Faster R-CNN network model based on VGG16 to identify the image, and the loss function of the network model is used. Modifications have been made to make it more suitable for the positioning of insulator strings. The FasterR-CNN network architecture consists of feature extraction part, RPN part, RoIPooling part and classifier part. The feature extraction part uses the VGG16 network architecture, which consists of 13 3×3 convolutional layers with stride 1, 13 ReLU activation functions, and 4 2×2 max pooling layers with stride 2. , after each pooling operation, the length and width of the feature map become the original

The number of channels becomes 2 times the original, until it finally becomes the size of the original image

And reach 512 channels. Assuming that the size of the featuremap is w×h, each point on it corresponds to k anchorboxes, and k=9 is set. The RPN part consists of a convolutional layer, a softmax function and a proposal layer. The remaining part can be divided into two lines, one line on the left, through a 1×1 convolutional layer, the number of channels of the feature map is changed to 18, this is because each anchorbox has a probability of belonging to the foreground and a probability of the background, The softmax function is used for classification. A line on the right, through a 1×1 convolutional layer, changes the number of channels of the feature map to 36, because the position information of each anchorbox contains four values. Finally, in the proposal layer, bounding box regression is performed on the anchorbox. Then, according to the probability of belonging to the foreground obtained when the softmax function is classified, the anchors are sorted from large to small, and the first N anchors are extracted as positive anchors, generally N=6000. The boundary of the positive anchors beyond the original image is defined as the image boundary, the positive anchors with very small size are removed, and finally the remaining positive anchors are subjected to NMS regression, and finally the resignproposal is obtained. The input of the RoI Pooling part is the featuremap of the feature extraction part and the resignproposal of the RPN part. In the featuremap of the original image, the part corresponding to the resignproposal is horizontally divided into 7×7 grids, and each grid is processed by maxpooling. into the classifier section. The classifier part includes fully connected layer, ReLU activation function and softmax function. This part uses a fully connected layer with a softmax function to classify the input and calculate the probability of belonging to that class. At the same time, the bounding box regression is used again to obtain a more accurate target detection frame.

本发明中涉及到的实验均在ubuntu18.04，python3.6，rtx2080ti上进行，绝缘子串分割的标签使用LabelMe软件进行标注，在训练阶段，初始学习率设为0.0001，权重衰减系数设为0.0005，动量值设为0.9，优化算法使用Adam。The experiments involved in the present invention are all carried out on ubuntu18.04, python3.6, rtx2080ti, the labels of insulator string segmentation are marked with LabelMe software, in the training stage, the initial learning rate is set to 0.0001, and the weight attenuation coefficient is set to 0.0005, The momentum value is set to 0.9, and the optimization algorithm uses Adam.

损失函数由RPN部分和分类器部分各自的分类损失和边界框回归损失组成，分类损失函数均采用二值交叉熵损失函数，边界框回归损失函数均采用平方差损失函数。在进行边界框回归损失时，由于原来的变换公式中，如下所示：The loss function consists of the classification loss of the RPN part and the classifier part and the bounding box regression loss. The classification loss function adopts the binary cross-entropy loss function, and the bounding box regression loss function adopts the squared difference loss function. When performing bounding box regression loss, since the original transformation formula is as follows:

G'_x＝A_w×d_x(A)+A_xG'_x =A_w ×d_x (A)+A_x

G'_y＝A_h×d_y(A)+A_yG'_y =A_h ×_dy (A)+A_y

G'_w＝A_w×exp[d_w(A)]G'_w =A_w ×exp[d_w (A)]

G'_h＝A_h×exp[d_h(A)]G'_h =A_h ×exp[d_h (A)]

其中，G'_x,G'_y,G'_w,G'_h是先验框的中心点坐标以及宽高，A_x和A_y是预测框中心点的坐标，A_w和A_h是预测框的宽高，d_x(A),d_y(A),d_w(A),d_h(A)是四个需要学习的参数。Among them, G'_x , G'_y , G'_w , G'_h are the coordinates of the center point and width and height of the prior frame, A_x and A_y are the coordinates of the center point of the prediction frame, and A_w and A_h are the prediction frame. The width and height of d_x (A), d_y (A), d_w (A), and d_h (A) are four parameters that need to be learned.

原变换公式中，并没有对中心点的变换进行约束，所以预测边框的中心可能出现在任何位置，早期训练容易不稳定，所以需要将原图像水平进行分成w×h个网格，并对网格大小进行归一化，令一个网格的宽＝1，高＝1，再进行变换，其公式为：In the original transformation formula, there is no restriction on the transformation of the center point, so the center of the predicted frame may appear in any position, and the early training is easy to be unstable, so it is necessary to divide the original image horizontally into w × h grids, and analyze the grid. The grid size is normalized, so that the width of a grid = 1, the height = 1, and then transformed, the formula is:

G″_x＝σ(d'_x(A))+c_xG″_x =σ(d'_x (A))+c_x

G″_y＝σ(d'_y(A))+c_yG″_y =σ(d'_y (A))+c_y

G″_w＝A'_wexp(d'_w(A))G″_w =A'_w exp(d'_w (A))

G″_h＝A'_hexp(d′_h(A))G″_h =A'_h exp(d'_h (A))

其中，d'_x(A),d'_y(A),d'_w(A),d'_h(A)是四个需要学习的四种变换，σ()函数就是sigmoid函数，能够变量约束在(0，1)的区间内，c_x和c_y是anchorbox网格左上角到图像左上角的距离，A'_w和A'_h是anchorbox在该体系下的宽和高。计算损失时，可以再将其映射回原图像中。由于绝缘子串的长宽差别较大，所以需要对预测后的长宽进行开方处理，而后计算平方误差损失。Among them, d'_x (A), d'_y (A), d'_w (A), d'_h (A) are the four transformations that need to be learned, and the σ() function is the sigmoid function, which can constrain the variables In the interval (0, 1), c_x and c_y are the distances from the upper left corner of the anchorbox grid to the upper left corner of the image, and A'_w and A'_h are the width and height of the anchorbox under this system. When calculating the loss, it can then be mapped back to the original image. Since the length and width of the insulator strings are quite different, it is necessary to square the predicted length and width, and then calculate the squared error loss.

如图3所示，为本发明使用的绝缘子串定位网络架构图，对于缺陷的定位，本发明采用修改后的YOLOv3网络对图像进行目标检测。首先采用k-means算法对标签框进行分析，由于绝缘子串缺陷大小相差不大，可以选择生成较少数量的anchorbox。YOLOv3网络架构由修改之后的Darknet53部分与检测器部分组成。其Darknet部分由步幅为1和步幅为2的3×3卷积层、步幅为1的1×1卷积层、BN层、LeakyReLU激活函数和残差连接组成。每一个卷积层都需要在后面连接BN层与LeakyReLU层。每次使用步幅为2的3×3卷积层后，特征图尺寸就变为原来的二分之一，直到变为最初的八分之一。残差连接在该部分共使用了11次。其检测器部分由步幅为1的1×1卷积层和3×3卷积层、BN层、LeakyReLU层和sigmoid函数组成。该部分在最后采用步幅为1的1×1卷积层，生成结果信息。损失函数包括置信度损失、分类损失和位置损失。置信度损失和类别损失均采用二值交叉熵损失函数，分类损失采用平方差损失函数。As shown in FIG. 3 , it is an architecture diagram of the insulator string location network used in the present invention. For the location of defects, the present invention adopts the modified YOLOv3 network to perform target detection on the image. First, the k-means algorithm is used to analyze the label box. Since the defect size of the insulator string is not much different, a smaller number of anchor boxes can be generated. The YOLOv3 network architecture consists of the modified Darknet53 part and the detector part. Its Darknet part consists of 3×3 convolutional layers with stride 1 and stride 2, 1×1 convolutional layers with stride 1, BN layer, LeakyReLU activation function, and residual connections. Each convolutional layer needs to connect the BN layer and the LeakyReLU layer later. After each use of a 3×3 convolutional layer with a stride of 2, the feature map size is reduced to one-half until it becomes one-eighth. Residual connections are used a total of 11 times in this part. Its detector part consists of 1×1 convolutional layers and 3×3 convolutional layers with stride 1, BN layer, LeakyReLU layer and sigmoid function. This part uses a 1×1 convolutional layer with stride 1 at the end to generate the result information. Loss functions include confidence loss, classification loss, and location loss. Both confidence loss and class loss use binary cross entropy loss function, and classification loss uses square difference loss function.

如图4a、图4b所示，是使用本发明提出的绝缘子串识别定位算法对绝缘子串的具体实施样例，从上到下依次是：图4a是绝缘子串的识别结果；图4b绝缘子串的缺陷定位结果。As shown in Figure 4a and Figure 4b, it is a specific implementation example of the insulator string using the insulator string identification and positioning algorithm proposed by the present invention, from top to bottom: Figure 4a is the identification result of the insulator string; Figure 4b Defect location results.

Claims (10)

Translated fromChinese

1.一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，包括以下步骤：1. a transmission line insulator string identification and positioning method based on deep learning, is characterized in that, comprises the following steps:1)采用Faster R-CNN网络模型，对输入的绝缘子串图像进行识别；1) Using the Faster R-CNN network model to identify the input insulator string image;2)采用YOLOv3模型，对绝缘子串的缺陷进行定位。2) Using the YOLOv3 model to locate the defects of the insulator strings.2.根据权利要求1所述的一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，所述步骤1)中，采用基于VGG16的Faster R-CNN网络模型对图像进行识别，其中，所述Faster R-CNN网络模型的损失函数由RPN部分和分类器部分各自的分类损失和边界框回归损失组成；分类损失函数均采用多类别的交叉熵损失函数，边界框回归损失函数均采用平方差损失函数。2. a kind of transmission line insulator string identification and positioning method based on deep learning according to claim 1, is characterized in that, in described step 1), adopts the Faster R-CNN network model based on VGG16 to identify the image, wherein , the loss function of the Faster R-CNN network model is composed of the respective classification loss of the RPN part and the classifier part and the bounding box regression loss; the classification loss function adopts the multi-category cross entropy loss function, and the bounding box regression loss function adopts the Squared difference loss function.3.根据权利要求1或2所述的一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，步骤1)中，采用基于VGG16的Faster R-CNN网络模型对图像进行识别，其中，所述Faster R-CNN网络模型的边界框回归损失函数实现方式包括以下步骤：3. a kind of transmission line insulator string identification and positioning method based on deep learning according to claim 1 and 2, is characterized in that, in step 1), adopt the Faster R-CNN network model based on VGG16 to identify the image, wherein , the implementation of the bounding box regression loss function of the Faster R-CNN network model includes the following steps:首先，将图像分成w×h个网格，并对网格大小进行归一化；First, divide the image into w×h grids and normalize the grid size;然后，在处理后的图像中进行回归，回归公式如下：Then, regression is performed on the processed image, and the regression formula is as follows:G″_x＝σ(d′_x(A))+c_xG″_x =σ(d′_x (A))+c_xG″_y＝σ(d′_y(A))+c_yG″_y =σ(d′_y (A))+c_yG″_w＝A′_wexp(d′_w(A))G″_w =A′_w exp(d′_w (A))G″_h＝A′_hexp(d′_h(A))G″_h =A′_h exp(d′_h (A))其中，其中，G″_x,G″_y是先验框坐标，G″_w,G″_h是先验框的宽和高，σ()函数是sigmoid函数，c_x和c_y是预测框所在网格的左上角到图像左上角的相对距离，A′_w和A′_h是预测框的相对大小，d′_x(A),d′_y(A),d′_w(A),d′_h(A)是四个需要学习的参数。Among them, G″_x , G″_y are the coordinates of the prior frame, G″_w , G″_h are the width and height of the prior frame, the σ() function is the sigmoid function, and c_x and c_y are where the prediction frame is located The relative distance from the upper left corner of the grid to the upper left corner of the image, A′_w and A′_h are the relative sizes of the prediction box, d′_x (A), d′_y (A), d′_w (A), d′_h (A) are four parameters that need to be learned.4.根据权利要求3所述的一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，在计算边界框回归损失时，对先验框的高和宽以及预测框的高和宽分别进行开方处理后，再计算平方误差损失。4. A deep learning-based transmission line insulator string identification and positioning method according to claim 3, characterized in that, when calculating the bounding box regression loss, the height and width of the prior frame and the height and width of the prediction frame are calculated. After the square root processing is performed separately, the squared error loss is calculated.5.根据权利要求1所述的一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，所述YOLOv3模型中的检测器部分包括顺次的卷积块集合、3×3卷积块、1×1卷积层，所述1×1卷积层生成结果信息，以表示预测缺陷所处的位置以及是否为缺陷的置信度；5. A deep learning-based method for identifying and locating transmission line insulator strings according to claim 1, wherein the detector part in the YOLOv3 model comprises sequential convolution block sets, 3×3 convolution blocks block, a 1×1 convolutional layer that generates result information to indicate where the predicted defect is and the confidence of whether it is a defect;所述的卷积块集合包括多个依次连接的卷积块，依次为1×1卷积块、3×3卷积块、1×1卷积块、3×3卷积块、1×1卷积块；The convolution block set includes a plurality of convolution blocks connected in sequence, which are 1×1 convolution block, 3×3 convolution block, 1×1 convolution block, 3×3 convolution block, and 1×1 convolution block. convolution block;所述卷积块包含依次的一个卷积层、一个BN层、一个LeakyReLU激活函数。The convolution block includes a convolution layer, a BN layer, and a LeakyReLU activation function in sequence.6.根据权利要求1所述的一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，Faster R-CNN中的分类损失采用交叉熵损失函数，所述边界框回归损失采用平方差损失函数；所述YOLOv3模型中，回归损失中的中心点坐标、置信度损失和分类损失采用交叉熵损失函数，回归损失中的宽和高采用平方差损失函数。6. A deep learning-based method for identifying and locating transmission line insulator strings according to claim 1, wherein the classification loss in Faster R-CNN adopts a cross-entropy loss function, and the bounding box regression loss adopts a square difference Loss function: In the YOLOv3 model, the center point coordinates, confidence loss and classification loss in the regression loss use the cross entropy loss function, and the width and height in the regression loss use the square difference loss function.7.根据权利要求1所述的一种基于深度学习的输电线路绝缘子串识别定位方法，其特征在于，所述YOLOv3模型，采用k-means算法对标签框进行聚类，生成3个anchorbox，使原图像下采样到原来的八分之一，且不需要另外两个尺度的特征图。7. a kind of transmission line insulator string identification and positioning method based on deep learning according to claim 1, is characterized in that, described YOLOv3 model, adopts k-means algorithm to carry out clustering to label frame, generates 3 anchorbox, makes 3 anchorboxes. The original image is downsampled to one-eighth of the original size, and no feature maps of the other two scales are required.8.一种基于深度学习的输电线路绝缘子串识别定位系统，其特征在于，包括：8. A transmission line insulator string identification and positioning system based on deep learning, characterized in that, comprising:绝缘子串识别模块，用于采用Faster R-CNN网络模型，对输入的绝缘子串图像进行识别；The insulator string identification module is used to identify the input insulator string image using the Faster R-CNN network model;绝缘子串缺陷定位模块，用于采用YOLOv3模型，对绝缘子串的缺陷进行定位。The insulator string defect location module is used to locate the defects of the insulator string using the YOLOv3 model.9.一种基于深度学习的输电线路绝缘子串识别定位装置，其特征在于，包括存储器和处理器；所述存储器，用于存储计算机程序；所述处理器，用于当执行所述计算机程序时，实现如权利要求1-7任一项所述的一种基于深度学习的输电线路绝缘子串识别定位方法。9. A transmission line insulator string identification and positioning device based on deep learning, characterized in that it comprises a memory and a processor; the memory is used for storing a computer program; the processor is used when executing the computer program , to realize a deep learning-based method for identifying and locating insulator strings of transmission lines according to any one of claims 1-7.10.一种计算机可读存储介质，其特征在于，所述存储介质上存储有计算机程序，当所述计算机程序被处理器执行时，实现如权利要求1-7任一项所述的一种基于深度学习的输电线路绝缘子串识别定位方法。10. A computer-readable storage medium, characterized in that, a computer program is stored on the storage medium, and when the computer program is executed by a processor, the one described in any one of claims 1-7 is implemented A deep learning-based identification and location method for insulator strings in transmission lines.

Images