CN115410087A - Transmission line foreign matter detection method based on improved YOLOv4 - Google Patents
Transmission line foreign matter detection method based on improved YOLOv4Download PDFInfo
- Publication number
- CN115410087A CN115410087ACN202211052529.9ACN202211052529ACN115410087ACN 115410087 ACN115410087 ACN 115410087ACN 202211052529 ACN202211052529 ACN 202211052529ACN 115410087 ACN115410087 ACN 115410087A
- Authority
- CN
- China
- Prior art keywords
- transmission line
- yolov4
- network model
- foreign
- foreign object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及目标检测技术，特别是涉及一种基于改进YOLOv4的输电线路异物检测方法。The invention relates to target detection technology, in particular to an improved YOLOv4-based foreign object detection method for transmission lines.
背景技术Background technique
在电能传输过程中保障输电线路安全稳定高效运行具有显著意义。这些年来，因为放风筝、气球等人为活动导致产生输电线路异物，进而影响电力系统稳定运行的事件络绎不绝，严重的甚至造成了跳闸事故，部分片区电力系统陷入瘫痪。输电线路上悬挂的异物主要有风筝、气球、塑料薄膜类的垃圾，以及在杆塔上鸟类搭建的巢穴。如何能及时发现这些异物成为了输电线智能巡检任务的重要课题。It is of great significance to ensure the safe, stable and efficient operation of transmission lines in the process of power transmission. Over the years, due to human activities such as kite flying and balloons, there have been an endless stream of incidents that have caused foreign objects on transmission lines, which have affected the stable operation of the power system. In severe cases, tripping accidents have even been caused, and the power system in some areas has been paralyzed. The foreign objects hanging on the power transmission line mainly include kites, balloons, plastic film garbage, and nests built by birds on poles and towers. How to detect these foreign objects in time has become an important issue in the intelligent inspection task of transmission lines.
输电线路异物检测的方法分为两类：传统的人工特征提取方法和基于深度学习的方法。传统方法通过检测输电导线外形轮廓判断异物存在与否，检测效果受到复杂背景、噪声等因素影响，精度较低。近年来，深度学习成为了脍炙人口的热点技术，摆脱了传统机器学习带来的一些弊端。神经网络不易受检测目标的几何变换、形变、光照等因素影响，降低了异物附着在输电线上产生的形变带来的识别难度。且它能自动生成检测目标所对应的特征，避免了人工设计特征的繁琐，相较于传统目标检测算法具有显著优势。基于深度学习的目标检测算法分为一阶段和二阶段两类，二阶段算法需要生成大量候选框，计算量极为庞大，检测速度很慢，在实际工业应用上完全无法达到实时检测的程度，这类算法的代表是Faster R-CNN。而一阶段算法则不产生候选框，直接回归目标的类别置信度和位置坐标，因此可获得较快速度，这类算法的代表是SSD和YOLO系列算法。YOLOv4是目前的主流目标检测算法，具有较好的检测效果。然而YOLOv4的模型体积仍然很大，为了获得高精度，模型参数量巨大，难以实际部署在存储空间有限的嵌入式设备上。The methods of foreign object detection on transmission lines are divided into two categories: traditional manual feature extraction methods and deep learning-based methods. The traditional method judges the presence or absence of foreign matter by detecting the outline of the transmission wire. The detection effect is affected by factors such as complex background and noise, and the accuracy is low. In recent years, deep learning has become a popular and popular technology, which has got rid of some disadvantages brought about by traditional machine learning. The neural network is not easily affected by the geometric transformation, deformation, illumination and other factors of the detection target, which reduces the difficulty of recognition caused by the deformation caused by foreign objects attached to the transmission line. And it can automatically generate the features corresponding to the detection target, avoiding the tediousness of manually designing features, and has significant advantages over traditional target detection algorithms. The target detection algorithm based on deep learning is divided into two types: one-stage and two-stage. The two-stage algorithm needs to generate a large number of candidate frames, which requires a huge amount of calculation and a slow detection speed. In practical industrial applications, it is completely impossible to achieve real-time detection. The representative of the class algorithm is Faster R-CNN. The one-stage algorithm does not generate candidate boxes, but directly returns the category confidence and position coordinates of the target, so it can obtain faster speed. The representatives of this type of algorithm are SSD and YOLO series algorithms. YOLOv4 is the current mainstream target detection algorithm and has a good detection effect. However, the model size of YOLOv4 is still very large. In order to obtain high precision, the model parameters are huge, and it is difficult to actually deploy it on embedded devices with limited storage space.
发明内容Contents of the invention
发明目的：本发明的目的是提供一种基于改进YOLOv4的输电线路异物检测方法，旨在减小模型的参数量，提高模型的鲁棒性，获取更高的精度和速度，实现性能的提升和实时的检测，更利于搭载在移动端等存储空间和计算能力有限的嵌入式设备上。Purpose of the invention: The purpose of the invention is to provide a transmission line foreign object detection method based on improved YOLOv4, which aims to reduce the parameter amount of the model, improve the robustness of the model, obtain higher accuracy and speed, and realize performance improvement and Real-time detection is more conducive to carrying on embedded devices with limited storage space and computing power such as mobile terminals.
技术方案：本发明的一种基于改进YOLOv4的输电线路异物检测方法，包括以下步骤：Technical solution: A foreign object detection method for transmission lines based on improved YOLOv4 of the present invention includes the following steps:
采集输电线路巡检视频，并对其进行分帧处理，将符合输电线路附有异物的场景图片进行数据清理；Collect the transmission line inspection video, and process it into frames, and clean up the data of the scene pictures that match the transmission line with foreign objects;
对数据清理后的符合输电线路附有异物的场景图片进行标签化处理，基于标签化处理后的符合输电线路附有异物的场景图片，构建输电线路场景下异物检测数据集，并划分训练集、验证集和测试集；Label the scene pictures with foreign objects attached to the transmission line after data cleaning, and construct a foreign object detection data set under the transmission line scene based on the labeled scene pictures with foreign objects attached to the transmission line, and divide the training set, validation set and test set;
构建改进YOLOv4网络模型；Construct and improve the YOLOv4 network model;
基于训练集对改进YOLOv4网络模型进行训练，基于验证集对训练好的改进YOLOv4网络模型进行验证，并保存在验证集上检测精度最高的权重和超参数；Train the improved YOLOv4 network model based on the training set, verify the trained improved YOLOv4 network model based on the verification set, and save the weights and hyperparameters with the highest detection accuracy on the verification set;
利用保存好的权重对测试集图片进行检测，获得输电线路异物图像的检测结果。The saved weights are used to detect the pictures in the test set, and the detection results of the foreign object images of the transmission line are obtained.
进一步的，将符合输电线路附有异物的场景图片进行数据清理的方法为：Further, the method for data cleaning of the scene pictures with foreign matter attached to the transmission line is as follows:
将符合输电线路附有异物的场景图片全部转化为.jpg或.png格式；Convert all the scene pictures that meet the transmission line with foreign objects into .jpg or .png format;
将格式转化后的符合输电线路附有异物的场景图片进行数据增强扩充数据集，包括水平翻转、色域变换、尺寸缩放以及mosaic数据增强方法，得到异物图像；Perform data enhancement and expand the data set on the scene pictures with foreign objects attached to the transmission line after format conversion, including horizontal flip, color gamut transformation, size scaling and mosaic data enhancement methods, to obtain foreign object images;
将异物图像嵌入到输电线路背景图像中，且异物图像要比输电线路背景图像小。Embed the image of the foreign object into the background image of the transmission line, and the image of the foreign object is smaller than the background image of the transmission line.
进一步的，对数据清理后的符合输电线路附有异物的场景图片标签化处理的方法为：Further, the method of labeling the scene pictures with foreign matter attached to the transmission line after the data cleaning is as follows:
使用labelimg标注工具对每张图片进行标注，形成相应的xml标签文件，该xml标签文件格式为PASCAL VOC，包含图片中矩形目标的两个对角坐标以及给定的类别。Use the labelimg labeling tool to label each picture to form a corresponding xml label file. The format of the xml label file is PASCAL VOC, which contains the two diagonal coordinates of the rectangular target in the picture and the given category.
进一步的，改进YOLOv4网络模型的构建方法为：Further, the construction method of the improved YOLOv4 network model is:
以YOLOv4网络模型为基础模型；Based on the YOLOv4 network model;
将YOLOv4网络模型的骨干网络CSPDarkNet53替换为轻量化的GhostNet进行特征提取；Replace the backbone network CSPDarkNet53 of the YOLOv4 network model with the lightweight GhostNet for feature extraction;
将YOLOv4网络模型的特征金字塔池化SPP模块改进为SPPF模块；Improve the feature pyramid pooling SPP module of the YOLOv4 network model to an SPPF module;
将YOLOv4网络模型的三层卷积块和五层卷积块以及下采样层和预测层中的卷积核大小为3×3的卷积层替换为深度可分离卷积层；Replace the three-layer convolution block and five-layer convolution block of the YOLOv4 network model, as well as the convolution layer with a convolution kernel size of 3×3 in the downsampling layer and prediction layer, with a depth-separable convolution layer;
在YOLOv4网络模型的上采样和下采样层后插入ECA模块；Insert the ECA module after the upsampling and downsampling layers of the YOLOv4 network model;
应用YOLOv4原始算法的路径聚合网络和预测层部分，将普通卷积中的Leaky ReLU激活函数替换为SiLU激活函数；Apply the path aggregation network and prediction layer part of the YOLOv4 original algorithm, and replace the Leaky ReLU activation function in the ordinary convolution with the SiLU activation function;
应用k-means聚类算法，生成适配输电线路场景下异物检测数据集的中小尺寸的六个锚点框大小，再结合默认的三个大尺寸的锚点框，构成最终九个异物锚点框大小；Apply the k-means clustering algorithm to generate six small and medium-sized anchor frame sizes suitable for the foreign object detection data set in the transmission line scene, and then combine the default three large-size anchor frames to form the final nine foreign object anchor points frame size;
最后获得改进YOLOv4网络模型。Finally, the improved YOLOv4 network model is obtained.
本发明的一种基于改进YOLOv4的输电线路异物检测系统，包括：A transmission line foreign object detection system based on improved YOLOv4 of the present invention, comprising:
数据采集模块，用于采集输电线路巡检视频；The data collection module is used to collect the transmission line inspection video;
数据处理模块，用于对采集的输电线路航拍视频进行分帧处理，将符合输电线路附有异物的场景图片进行数据清理；The data processing module is used to divide the collected aerial video of the transmission line into frames, and clean up the data of the scene pictures with foreign objects attached to the transmission line;
标注模块，用于对数据清理后的符合输电线路附有异物的场景图片进行标签化处理，基于标签化处理后的符合输电线路附有异物的场景图片，构建输电线路场景下异物检测数据集，并划分训练集、验证集和测试集；The labeling module is used to label the scene pictures with foreign objects attached to the transmission line after data cleaning, and construct a foreign object detection data set under the transmission line scene based on the scene pictures with foreign objects attached to the transmission line after the labeling processing. And divide the training set, verification set and test set;
模型构建模块，用于构建改进YOLOv4网络模型；Model building block, used to build and improve the YOLOv4 network model;
模型训练及验证模块，用于基于训练集对改进YOLOv4网络模型进行训练，基于验证集对训练好的改进YOLOv4网络模型进行验证，并保存在验证集上检测精度最高的权重和超参数；The model training and verification module is used to train the improved YOLOv4 network model based on the training set, verify the trained improved YOLOv4 network model based on the verification set, and save the weights and hyperparameters with the highest detection accuracy on the verification set;
测试模块，用于利用保存好的权重对测试集图片进行检测，获得输电线路异物图像的检测结果。The test module is used to detect the pictures in the test set by using the saved weights, and obtain the detection result of the foreign object image of the transmission line.
优选的，将符合输电线路附有异物的场景图片进行数据清理的过程为：Preferably, the process of cleaning up the data of the scene picture with foreign matter attached to the transmission line is as follows:
将符合输电线路附有异物的场景图片全部转化为.jpg或.png格式；Convert all the scene pictures that meet the transmission line with foreign objects into .jpg or .png format;
将格式转化后的符合输电线路附有异物的场景图片进行数据增强扩充数据集，包括水平翻转、色域变换、尺寸缩放以及mosaic数据增强方法，获得异物图像；Carry out data enhancement and expand the data set of the scene pictures with foreign objects attached to the transmission line after format conversion, including horizontal flip, color gamut transformation, size scaling and mosaic data enhancement methods, to obtain foreign object images;
将异物图像嵌入到输电线路背景图像中，且异物图像要比输电线路背景图像小。Embed the image of the foreign object into the background image of the transmission line, and the image of the foreign object is smaller than the background image of the transmission line.
优选的，对数据清理后的符合输电线路附有异物的场景图片标签化处理的过程为：Preferably, the process of labeling the scene pictures with foreign matter attached to the transmission line after the data cleaning is as follows:
使用labelimg标注工具对每张图片进行标注，形成相应的xml标签文件，该xml标签文件格式为PASCAL VOC，包含图片中矩形目标的两个对角坐标以及给定的类别。Use the labelimg labeling tool to label each picture to form a corresponding xml label file. The format of the xml label file is PASCAL VOC, which contains the two diagonal coordinates of the rectangular target in the picture and the given category.
优选的，改进YOLOv4网络模型的构建过程为：Preferably, the construction process of improving the YOLOv4 network model is:
以YOLOv4网络模型为基础模型；Based on the YOLOv4 network model;
将YOLOv4网络模型的骨干网络CSPDarkNet53替换为轻量化的GhostNet进行特征提取；Replace the backbone network CSPDarkNet53 of the YOLOv4 network model with the lightweight GhostNet for feature extraction;
将YOLOv4网络模型的特征金字塔池化SPP模块改进为SPPF模块；Improve the feature pyramid pooling SPP module of the YOLOv4 network model to an SPPF module;
将YOLOv4网络模型的三层卷积块和五层卷积块以及下采样层和预测层中的卷积核大小为3×3的卷积层替换为深度可分离卷积层；Replace the three-layer convolution block and five-layer convolution block of the YOLOv4 network model, as well as the convolution layer with a convolution kernel size of 3×3 in the downsampling layer and prediction layer, with a depth-separable convolution layer;
在YOLOv4网络模型的上采样和下采样层后插入ECA模块；Insert the ECA module after the upsampling and downsampling layers of the YOLOv4 network model;
应用YOLOv4原始算法的路径聚合网络和预测层部分，将普通卷积中的Leaky ReLU激活函数替换为SiLU激活函数；Apply the path aggregation network and prediction layer part of the YOLOv4 original algorithm, and replace the Leaky ReLU activation function in the ordinary convolution with the SiLU activation function;
应用k-means聚类算法，生成适配自制数据集的中小尺寸的六个锚点框大小，再结合默认的三个大尺寸的锚点框，构成最终九个异物锚点框大小；Apply the k-means clustering algorithm to generate six anchor frame sizes of small and medium sizes adapted to the self-made data set, and then combine the default three large-size anchor frame sizes to form the final nine foreign object anchor frame sizes;
最后获得改进YOLOv4网络模型。Finally, the improved YOLOv4 network model is obtained.
本发明的一种装置设备，包括存储器和处理器，其中：An apparatus of the present invention, comprising a memory and a processor, wherein:
存储器，用于存储能够在处理器上运行的计算机程序；memory for storing computer programs capable of running on the processor;
处理器，用于在运行所述计算机程序时，执行如上述一种基于改进YOLOv4的输电线路异物检测方法的步骤。The processor is configured to execute the steps of the above-mentioned improved YOLOv4-based foreign object detection method for transmission lines when running the computer program.
本发明的一种存储介质，所述存储介质上存储有计算机程序，所述计算机程序被至少一个处理器执行时实现如上述一种基于改进YOLOv4的输电线路异物检测方法的步骤。A storage medium of the present invention, wherein a computer program is stored on the storage medium, and when the computer program is executed by at least one processor, the steps of the above-mentioned improved YOLOv4-based transmission line foreign object detection method are implemented.
有益效果：与现有技术相比，本发明的显著技术效果为：(1)本发明中针对YOLOv4网络模型进行改进，提高了对目标，特别是小目标的检测精度，模型参数量缩减到了原模型的17％，检测速度也得到了一定提升，能更高效的实现搭载于嵌入式设备上，完成智能巡检清障任务。(2)本发明采用的GhostNet轻量化骨干网络具有更少的参数量，但特征提取能力仍然很强，足以满足输电线路异物检测任务的需求。(3)本发明采用的SPPF模块相较于原模块拥有更高的效率，可以提高运行速度，加快模型收敛。(4)本发明采用的深度可分离卷积可以将参数量缩小到普通卷积的八分之一，但检测精度却只是略微下降，更利于工业级应用。(5)本发明引入的ECA通道注意力机制是一种轻量化模块，能提高网络的特征表征能力，提高检测精度，增加的参数量完全可以忽略不计。(6)本发明采用的SiLU激活函数更平滑，能提高性能，效果相较Leaky ReLU函数更好，特别是在深层网络中优势更加明显。(7)本发明采用k-means聚类生成适配锚点框，针对数据集锚点框的选取方式，对于其他目标检测领域对应目标锚点框的选取有一定参考意义。(8)本发明的最终模型同样适用于其他目标检测领域，可拓展到其他领域的工业端部署。Beneficial effects: Compared with the prior art, the remarkable technical effect of the present invention is: (1) improve the YOLOv4 network model in the present invention, have improved the detection accuracy to target, especially small target, and model parameter amount has been reduced to original 17% of the model, and the detection speed has also been improved to a certain extent, which can be more efficiently implemented on embedded devices to complete intelligent inspection and obstacle removal tasks. (2) The GhostNet lightweight backbone network adopted in the present invention has fewer parameters, but the feature extraction capability is still strong enough to meet the requirements of the transmission line foreign object detection task. (3) Compared with the original module, the SPPF module used in the present invention has higher efficiency, can improve the running speed, and accelerate the model convergence. (4) The depth-separable convolution adopted in the present invention can reduce the number of parameters to one-eighth of the ordinary convolution, but the detection accuracy is only slightly reduced, which is more conducive to industrial-grade applications. (5) The ECA channel attention mechanism introduced in the present invention is a lightweight module, which can improve the feature representation ability of the network and improve the detection accuracy, and the increased parameter amount can be completely ignored. (6) The SiLU activation function adopted in the present invention is smoother and can improve performance, and the effect is better than that of the Leaky ReLU function, especially in deep networks. (7) The present invention adopts k-means clustering to generate an adaptive anchor point frame, and the selection method of the anchor point frame of the data set has certain reference significance for the selection of the corresponding target anchor point frame in other target detection fields. (8) The final model of the present invention is also applicable to other target detection fields, and can be extended to industrial deployment in other fields.
附图说明Description of drawings
图1为YOLO4网络模型结构图；Figure 1 is a structural diagram of the YOLO4 network model;
图2为改进YOLOv4网络模型结构图；Figure 2 is a structural diagram of the improved YOLOv4 network model;
图3为本发明方法流程图；Fig. 3 is a flow chart of the method of the present invention;
图4为特征提取网络GhostNet中的Ghost模块；Figure 4 is the Ghost module in the feature extraction network GhostNet;
图5为特征提取网络GhostNet中的Ghost瓶颈结构；Figure 5 is the Ghost bottleneck structure in the feature extraction network GhostNet;
图6为SPPF模块替代原SPP模块结构图；Figure 6 is a structural diagram of the SPPF module replacing the original SPP module;
图7为普通卷积结构图；Figure 7 is a general convolution structure diagram;
图8为深度可分离卷积结构图；Figure 8 is a depth separable convolution structure diagram;
图9为ECA(Efficient Channel Attention)注意力机制模块结构图Figure 9 is a block diagram of the ECA (Efficient Channel Attention) attention mechanism
图10为SiLU(Sigmoid Weighted Linear Unit)激活函数曲线图；Fig. 10 is a SiLU (Sigmoid Weighted Linear Unit) activation function curve;
图11为部分实验样本示例图，其中(a)和(b)为正常样本，(c)和(d)为输电线路附着风筝的样本，(e)和(f)为输电线路附着气球的样本，(g)和(h)为输电线路附着垃圾的样本，(i)和(j)为输电线路搭挂鸟巢的样本；Figure 11 is an illustration of some experimental samples, where (a) and (b) are normal samples, (c) and (d) are samples with kites attached to transmission lines, and (e) and (f) are samples with balloons attached to transmission lines , (g) and (h) are samples of garbage attached to transmission lines, (i) and (j) are samples of bird’s nests hung on transmission lines;
图12为不同模型检测效果对比图，其中，(a)～(c)分别为含气球的输电线异物场景下YOLOv4网络的检测效果、YOLOv5网络的检测效果和改进YOLOv4网络的检测效果图；(d)～(f)分别为风筝的输电线异物场景下YOLOv4网络的检测效果、YOLOv5网络的检测效果和改进YOLOv4网络的检测效果图；(g)～(i)分别为垃圾的输电线异物场景下YOLOv4网络的检测效果、YOLOv5网络的检测效果和改进YOLOv4网络的检测效果图；(j)～(l)分别为鸟巢的输电线异物场景下YOLOv4网络的检测效果、YOLOv5网络的检测效果和改进YOLOv4网络的检测效果图；(m)～(o)分别为补充的小目标数据集的输电线路场景下YOLOv4网络的检测效果、YOLOv5网络的检测效果和改进YOLOv4网络的检测效果图。Figure 12 is a comparison of the detection effects of different models, where (a) to (c) are the detection effect of the YOLOv4 network, the detection effect of the YOLOv5 network, and the detection effect of the improved YOLOv4 network in the scene of foreign objects in the transmission line containing balloons; ( d)～(f) are the detection effect of the YOLOv4 network, the detection effect of the YOLOv5 network and the detection effect of the improved YOLOv4 network in the scene of the foreign object on the power line of the kite; (g)～(i) are the scene of the foreign object on the power line of the garbage The following is the detection effect of YOLOv4 network, the detection effect of YOLOv5 network and the detection effect diagram of improved YOLOv4 network; (j)~(l) are the detection effect of YOLOv4 network, the detection effect and improvement of YOLOv5 network in the scene of foreign objects in the power line of the Bird's Nest respectively The detection effect diagram of the YOLOv4 network; (m)~(o) are the detection effect diagram of the YOLOv4 network, the detection effect of the YOLOv5 network and the detection effect diagram of the improved YOLOv4 network in the transmission line scene of the supplementary small target data set, respectively.
具体实施方式Detailed ways
下面结合附图对本发明的技术方案进一步说明。The technical solution of the present invention will be further described below in conjunction with the accompanying drawings.
一种基于改进YOLOv4的输电线路异物检测方法，包括数据集的准备，数据预处理，以及网络的训练和预测。现有技术下的YOLOv4网络模型结构图如图1所示，本发明所述的改进YOLOv4网络模型结构图如图2所示，可分为骨干网络的特征提取、特征金字塔和路径聚合网络的特征融合、探测头的预测三个部分。输入的416×416大小的图片进入骨干网络(GhostNet)进行层层特征提取，当输出通道数第一次达到40,116,160时，即到达GhostNet的第4，10，12层时，将该三层特征图输出至特征融合网络，特征图大小对应为(52,52,40)，(26,26,112)，(13,13,160)，分别对应小、中、大三种尺寸。之后通过特征金字塔(SPPF模块)结构和路径聚合网络进行上下层的特征融合，输出进入三种对应尺寸的探测头，分别做分类回归和坐标回归，输出原图中的目标种类和目标坐标。A foreign object detection method for transmission lines based on improved YOLOv4, including data set preparation, data preprocessing, and network training and prediction. The structure diagram of the YOLOv4 network model under the prior art is shown in Figure 1, and the structure diagram of the improved YOLOv4 network model described in the present invention is shown in Figure 2, which can be divided into feature extraction, feature pyramid and path aggregation network features of the backbone network There are three parts of fusion and detection head prediction. The input 416×416 size picture enters the backbone network (GhostNet) for layer-by-layer feature extraction. When the number of output channels reaches 40, 116, 160 for the first time, that is, when it reaches the 4th, 10th, and 12th layers of GhostNet, the three-layer feature map Output to the feature fusion network, the size of the feature map corresponds to (52, 52, 40), (26, 26, 112), (13, 13, 160), corresponding to three sizes of small, medium and large. Afterwards, the features of the upper and lower layers are fused through the feature pyramid (SPPF module) structure and the path aggregation network, and the output enters three types of detection heads of corresponding sizes, which perform classification regression and coordinate regression respectively, and output the target type and target coordinates in the original image.
如图3所示，本发明的一种基于改进YOLOv4的输电线路异物检测方法，具体步骤如下：As shown in Figure 3, a foreign object detection method for transmission lines based on improved YOLOv4 of the present invention, the specific steps are as follows:
S1、通过无人机采集输电线路巡检视频，并对采集的输电线路航拍视频进行分帧处理，将符合输电线路附有异物的场景图片进行数据清理；S1. Use drones to collect inspection videos of transmission lines, and perform frame processing on the collected aerial videos of transmission lines, and clean up the data of scene pictures that match transmission lines with foreign objects;
将无人机拍摄到的输电线路巡检视频逐帧分解，筛选出符合输电线路附有异物的场景图片；Decompose the transmission line inspection video captured by the drone frame by frame, and screen out the scene pictures that match the transmission line with foreign objects;
将符合输电线路附有异物的场景图片全部转化为.jpg或.png格式，并将图片尺寸调整为416×416像素大小；Convert all the scene pictures that meet the transmission line with foreign objects into .jpg or .png format, and adjust the picture size to 416×416 pixels;
将尺寸调整后的符合输电线路附有异物的场景图片进行数据增强扩充数据集，包括水平翻转、色域变换、尺寸缩放以及mosaic数据增强方法，得到异物图像；Perform data enhancement on the resized scene pictures with foreign objects attached to transmission lines, including horizontal flip, color gamut transformation, size scaling and mosaic data enhancement methods, to obtain foreign object images;
为了提高泛化性和对小目标的检测性能，采用PS技术将异物图像嵌入到输电线路背景图像中，进一步丰富数据集，加强模型对小目标的训练能力，其中要确保输电线路背景图像明显比异物图像尺寸大。In order to improve the generalization and the detection performance of small targets, PS technology is used to embed the foreign object image into the background image of the transmission line, further enrich the data set, and strengthen the training ability of the model for small targets. The image size of the foreign object is large.
本实施例中，异物图像是250×250像素，输电线路背景图像是5472×3078像素，因此将250×250像素的异物图像嵌入到5472×3078像素的输电线路背景图像中；具体为将日常活动中的气球、风筝等进行一定程度的扭曲然后以不同尺寸附着在输电线路的不同位置，模拟搭挂在输电线路上的产生一定形变的异物，背景均为真实的输电线场景。In this embodiment, the image of the foreign object is 250×250 pixels, and the background image of the power transmission line is 5472×3078 pixels, so the foreign object image of 250×250 pixels is embedded into the background image of the power transmission line of 5472×3078 pixels; specifically, daily activities The balloons, kites, etc. in the game are distorted to a certain extent and then attached to different positions of the transmission line in different sizes, simulating foreign objects hanging on the transmission line and causing a certain deformation. The background is the real transmission line scene.
S2、对数据清理后的符合输电线路附有异物的场景图片进行标签化处理，基于标签化处理后的符合输电线路附有异物的场景图片，构建输电线路场景下异物检测数据集，并划分训练集、验证集和测试集；S2. Label the scene pictures with foreign objects attached to the transmission line after data cleaning, and construct a foreign object detection data set under the transmission line scene based on the labeled scene pictures with foreign objects attached to the transmission line, and divide them into training set, validation set and test set;
使用labelimg标注工具对每张图片进行标注，形成相应的xml标签文件，该文件格式为PASCAL VOC，包含图片中矩形目标的两个对角坐标以及给定的类别；Use the labelimg labeling tool to label each picture to form a corresponding xml label file. The file format is PASCAL VOC, which contains the two diagonal coordinates of the rectangular target in the picture and the given category;
基于构建的输电线路场景下异物检测数据集，以7:2:1的比例构建训练集、验证集和测试集。Based on the constructed foreign object detection data set in the transmission line scene, the training set, verification set and test set are constructed in a ratio of 7:2:1.
S3、构建改进YOLOv4网络模型；S3. Build and improve the YOLOv4 network model;
以YOLOv4网络模型为基础模型；Based on the YOLOv4 network model;
将YOLOv4网络模型的骨干网络CSPDarkNet53替换为轻量化的GhostNet进行特征提取；Replace the backbone network CSPDarkNet53 of the YOLOv4 network model with the lightweight GhostNet for feature extraction;
本发明采用的GhostNet轻量化骨干网络具有更少的参数量，但特征提取能力仍然很强，足以满足输电线路异物检测任务的需求。该网络包含大量的Ghost模块和步长为1和2的两种Ghost瓶颈结构，如图4和图5所示。Ghost模块由两个操作构成，第一是输入通过普通卷积生成不含冗余的特征图1，第二是通过identity(恒等变换)和廉价的线性运算Φ生成完整的特征图。通过廉价的线性运算花费低成本获得了高效益。步长为1和2的两种ghost瓶颈结构都包含两个ghost模块，其中第一个模块用于增加通道数目，第二个模块用于减少通道数目，最终使其与输入通道数相匹配。中间放置一个SE注意力机制模块用于增强特征提取能力。Shortcut为捷径分支，用于保留输入特征层，方便之后将两条支路特征图进行特征融合获得最终输出。具体的网络配置如表1所示。The GhostNet lightweight backbone network adopted by the present invention has fewer parameters, but the feature extraction capability is still strong enough to meet the requirements of the transmission line foreign object detection task. The network contains a large number of Ghost modules and two Ghost bottleneck structures with a step size of 1 and 2, as shown in Figure 4 and Figure 5. The Ghost module consists of two operations. The first is to generate a
表1 GhostNet的网络配置表Table 1 Network configuration table of GhostNet
应用YOLOv4原始算法的特征金字塔池化(SPP)模块，在原算法的基础上采用更快的特征金字塔池化(SPPF)模块，替换原模块SPP，如图6所示。原始的SPP模块采用5×5,9×9,13×13大小的最大池化核，改进的SPPF模块则通过堆叠不同数量的5×5最大池化核，同样能达到相同的的效果。输入通过特征金字塔池化模块可以提取不同尺寸的空间特征信息，提升模型对于空间布局和物体变性的鲁棒性，同时保证不同尺寸的输入都能得到统一固定的输出。Apply the feature pyramid pooling (SPP) module of the YOLOv4 original algorithm, and use the faster feature pyramid pooling (SPPF) module on the basis of the original algorithm to replace the original module SPP, as shown in Figure 6. The original SPP module uses 5×5, 9×9, 13×13 maximum pooling kernels, and the improved SPPF module can achieve the same effect by stacking different numbers of 5×5 maximum pooling kernels. The input can extract spatial feature information of different sizes through the feature pyramid pooling module, which improves the robustness of the model to spatial layout and object variability, and at the same time ensures that inputs of different sizes can get uniform and fixed outputs.
SPPF模块设置了一条主线通道，将输入串行通过多个5×5最大池化核。第一条支路直接连接到输出，等效于一个1×1的最大池化核。第二条支路经过了一个5×5的卷积核，再连接到输出，等效于一个5×5的最大池化核。第三条支路通过两个5×5的池化核，再连接到输出，等效于一个9×9的最大池化核。最后一条支路通过三个5×5的池化核，再连接到输出，等效于一个13×13的最大池化核。最终将四条支路的输出堆叠起来。The SPPF module sets up a mainline channel, which serially passes the input through multiple 5×5 maximum pooling cores. The first branch is directly connected to the output, which is equivalent to a 1×1 max pooling kernel. The second branch passes through a 5×5 convolution kernel and is connected to the output, which is equivalent to a 5×5 maximum pooling kernel. The third branch passes through two 5×5 pooling kernels, and then connects to the output, which is equivalent to a 9×9 maximum pooling kernel. The last branch passes through three 5×5 pooling kernels, and then connects to the output, which is equivalent to a 13×13 max pooling kernel. Finally, the outputs of the four branches are stacked.
将YOLOv4网络模型的三层卷积块和五层卷积块以及下采样层和预测层中的卷积核大小为3×3的卷积层替换为深度可分离卷积层，普通卷积如图7所示，深度可分离卷积如图8所示，它由逐通道卷积和逐点卷积两部分构成，逐通道卷积具体为一个卷积核负责一个通道，一个通道只被一个卷积核卷积，这个过程产生的特征图通道数和输入的通道数完全一样。将输入层的每个通道独立进行卷积运算，没有有效利用不同通道间在相同空间位置上的特征信息，因此需要再通过逐点卷积将这些特征图进行组合生成新的特征图。在相同输入得到相同输入的前提下，采用深度可分离卷积的参数量只有常规卷积的1/3，因此采用深度可分离卷积可以减小模型参数量，让模型尺寸更小。Replace the three-layer convolution block and five-layer convolution block of the YOLOv4 network model, as well as the convolution layer with a convolution kernel size of 3×3 in the downsampling layer and prediction layer, with a depth-separable convolution layer. Ordinary convolution such as As shown in Figure 7, the depth-separable convolution is shown in Figure 8. It consists of two parts: channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution is specifically a convolution kernel responsible for one channel, and one channel is only used by one Convolution kernel convolution, the number of feature map channels generated by this process is exactly the same as the number of input channels. Convolving each channel of the input layer independently does not effectively utilize the feature information of different channels at the same spatial position. Therefore, it is necessary to combine these feature maps by point-by-point convolution to generate a new feature map. Under the premise that the same input gets the same input, the parameter amount of the depth-separable convolution is only 1/3 of the conventional convolution, so the depth-separable convolution can reduce the model parameter amount and make the model size smaller.
考虑到模型的参数量大部分来自于卷积层，因此尝试采用轻量化网络最常用到且效果较好的深度可分离卷积，进一步减小模型参数量。通过将三层卷积块，五层卷积块，以及下采样层和预测层中的全部3×3卷积层替换为深度可分离卷积层，共计替换了15层3×3卷积层。Considering that most of the parameters of the model come from the convolutional layer, we try to use the depthwise separable convolution, which is most commonly used in lightweight networks and has a better effect, to further reduce the amount of model parameters. By replacing three-layer convolutional blocks, five-layer convolutional blocks, and all 3×3 convolutional layers in the downsampling layer and prediction layer with depthwise separable convolutional layers, a total of 15 layers of 3×3 convolutional layers are replaced .
在YOLOv4网络模型的上采样和下采样层后插入ECA(efficient channelattention)模块；Insert the ECA (efficient channelattention) module after the upsampling and downsampling layers of the YOLOv4 network model;
在网络模型的两个上采样层和下采样层之后插入ECA模块，采样后会通过该注意力机制模块聚焦于信息更有效的通道，从而能过滤掉部分无用信息的干扰，使信息融合的效果更好。由于还有另一条特征融合的支路与它堆叠，所以不用担心有效信息的丢失。ECA模块结构如图9所示，考虑到降维操作会对通道注意力的预测产生负面影响，且获取依赖关系效率低且不必要；基于此，提出了一种针对CNN的高效通道注意力(ECA)模块，避免了降维，有效地实现了跨通道交互。通过大小为k的快速一维卷积实现，其中核大小k表示局部跨通道交互的覆盖范围，即有多大范围参与了一个通道的注意预测。Insert the ECA module after the two upsampling layers and downsampling layers of the network model. After sampling, the attention mechanism module will focus on the more effective channel of information, so that the interference of some useless information can be filtered out, and the effect of information fusion can be improved. better. Since there is another branch of feature fusion stacked with it, there is no need to worry about the loss of effective information. The ECA module structure is shown in Figure 9. Considering that the dimensionality reduction operation will have a negative impact on the prediction of channel attention, and the acquisition of dependencies is inefficient and unnecessary; based on this, an efficient channel attention for CNN is proposed ( ECA) module, which avoids dimensionality reduction and effectively realizes cross-channel interaction. It is implemented by fast one-dimensional convolution with size k, where the kernel size k represents the coverage of local cross-channel interaction, that is, how much range participates in the attention prediction of a channel.
应用YOLOv4网络模型的路径聚合网络和预测层部分，采用SiLU(SigmoidWeighted Linear Unit)激活函数替代原算法的Leaky ReLU激活函数，函数曲线如图10所示。The path aggregation network and prediction layer part of the YOLOv4 network model is applied, and the SiLU (SigmoidWeighted Linear Unit) activation function is used to replace the Leaky ReLU activation function of the original algorithm. The function curve is shown in Figure 10.
输入特征图进入路径聚合网络，再将进入预测层，过程中激活函数采用SiLU，公式如下：The input feature map enters the path aggregation network, and then enters the prediction layer. During the process, the activation function uses SiLU, and the formula is as follows:
f(x)＝x·sigmoid(x)f(x)=x sigmoid(x)
应用k-means聚类算法，生成适配自制数据集的九个锚点框尺寸大小；Apply the k-means clustering algorithm to generate nine anchor frame sizes adapted to the self-made dataset;
采用k-means聚类算法生成九个锚点框，分别对应大、中、小三种尺度；通过表2的对比实验分析采用原始锚点框和k-means聚类后生成的锚点框，提出保留k-means聚类生成的小、中尺寸的六个锚点框，针对大尺寸锚点框仍采用原始锚点框中的大尺寸锚点框值。The k-means clustering algorithm is used to generate nine anchor boxes, which correspond to three scales: large, medium, and small; through the comparison experiment in Table 2, the original anchor box and the anchor box generated by k-means clustering are analyzed, and the proposed The six small and medium-sized anchor boxes generated by k-means clustering are retained, and the large-sized anchor box value in the original anchor box is still used for the large-sized anchor box.
表2初始锚点框的对比实验Table 2 Comparison experiment of initial anchor box
最终获得改进YOLOv4网络模型，改进YOLOv4网络模型结构如图2所示。Finally, the improved YOLOv4 network model is obtained, and the structure of the improved YOLOv4 network model is shown in Figure 2.
S4、基于数据集对改进YOLOv4网络模型进行训练，保存训练好的权重；基于训练集对改进YOLOv4网络模型进行训练，验证集用于模型选择以及调参，保存最终训练好的权重；S4. Train the improved YOLOv4 network model based on the data set, and save the trained weights; train the improved YOLOv4 network model based on the training set, and use the verification set for model selection and parameter adjustment, and save the final trained weights;
(5)利用保存好的权重对测试集图片进行检测，获得输电线路异物图像的检测结果。(5) Use the saved weights to detect the pictures in the test set, and obtain the detection results of the transmission line foreign object images.
本发明的一种基于改进YOLOv4的输电线路异物检测系统，包括：A transmission line foreign object detection system based on improved YOLOv4 of the present invention, comprising:
数据采集模块，用于采集输电线路巡检视频；The data collection module is used to collect the transmission line inspection video;
数据处理模块，用于对采集的输电线路航拍视频进行分帧处理，将符合输电线路附有异物的场景图片进行数据清理；The data processing module is used to divide the collected aerial video of the transmission line into frames, and clean up the data of the scene pictures with foreign objects attached to the transmission line;
标注模块，用于对数据清理后的符合输电线路附有异物的场景图片进行标签化处理，基于标签化处理后的符合输电线路附有异物的场景图片，构建输电线路场景下异物检测数据集，并划分训练集、验证集和测试集；The labeling module is used to label the scene pictures with foreign objects attached to the transmission line after data cleaning, and construct a foreign object detection data set under the transmission line scene based on the scene pictures with foreign objects attached to the transmission line after the labeling processing. And divide the training set, verification set and test set;
模型构建模块，用于构建改进YOLOv4网络模型；Model building block, used to build and improve the YOLOv4 network model;
模型训练及验证模块，用于基于训练集对改进YOLOv4网络模型进行训练，基于验证集对训练好的改进YOLOv4网络模型进行验证，并保存在验证集上检测精度最高的权重和超参数；The model training and verification module is used to train the improved YOLOv4 network model based on the training set, verify the trained improved YOLOv4 network model based on the verification set, and save the weights and hyperparameters with the highest detection accuracy on the verification set;
测试模块，用于利用保存好的权重对测试集图片进行检测，获得输电线路异物图像的检测结果。The test module is used to detect the pictures in the test set by using the saved weights, and obtain the detection result of the foreign object image of the transmission line.
本发明的一种装置设备，包括存储器和处理器，其中：An apparatus of the present invention, comprising a memory and a processor, wherein:
存储器，用于存储能够在处理器上运行的计算机程序；memory for storing computer programs capable of running on the processor;
处理器，用于在运行所述计算机程序时，执行如上述一种基于改进YOLOv4的输电线路异物检测方法的步骤，并能达到与上述方法一致的技术效果。The processor is configured to execute the steps of the above-mentioned improved YOLOv4-based transmission line foreign object detection method when running the computer program, and can achieve the same technical effect as the above-mentioned method.
本发明的一种存储介质，所述存储介质上存储有计算机程序，所述计算机程序被至少一个处理器执行时实现如上述一种基于改进YOLOv4的输电线路异物检测方法的步骤，并能达到与上述方法一致的技术效果。A storage medium of the present invention, the storage medium is stored with a computer program, and when the computer program is executed by at least one processor, the steps of the above-mentioned method for detecting foreign matter in transmission lines based on improved YOLOv4 can be achieved, and can achieve the same The above-mentioned method has the same technical effect.
实施例：Example:
本实施例的输电线路场景下异物检测数据集共包含4496张样本数据，分为四类，其中nest(鸟巢)2541张、balloon(气球)704张、kite(风筝)691张、trash(垃圾)643张。训练集、验证集和测试集按照数据集7:2:1的比例划分，图11为部分实验样例，(a),(b)为制作数据集时PS技术采用的大尺寸输电线场景，(c)，(d)为输电线上搭挂风筝，(e)，(f)为输电线上搭挂气球，(g)，(h)为输电线上搭挂垃圾，(i)，(j)为输电线杆塔上附着鸟巢。The foreign object detection data set in the transmission line scene of this embodiment contains a total of 4496 sample data, which are divided into four categories, including nest (bird's nest) 2541 pieces, balloon (balloon) 704 pieces, kite (kite) 691 pieces, trash (garbage) 643 sheets. The training set, verification set, and test set are divided according to the ratio of 7:2:1 in the data set. Figure 11 shows some experimental samples. (a) and (b) are the large-scale transmission line scenarios used by PS technology when making the data set. (c), (d) are hanging kites on the transmission line, (e), (f) are hanging balloons on the transmission line, (g), (h) are hanging garbage on the transmission line, (i), ( j) The bird's nest is attached to the transmission line tower.
本实施例中，模型的搭建、训练和结果的测试均在Pytorch框架下完成，使用CUDA并行计算架构，同时将cudnn加速库集成到Pytorch框架下加速计算机的计算能力。In this embodiment, model building, training, and result testing are all completed under the Pytorch framework, using the CUDA parallel computing architecture, and at the same time integrating the cudnn acceleration library into the Pytorch framework to accelerate the computing power of the computer.
(1)预测结果性能评价指标；(1) Prediction result performance evaluation index;
AP(Average Precision)是计算某一类P-R曲线下的面积，指曲线横纵坐标分别为准确率和召回率所围成的面积大小。mAP(Mean Average Precision)则是计算所有类别P-R曲线下面积的平均值，也就是所有类别对应AP的平均值，计算公式如下，其中C为总类别数。AP (Average Precision) is to calculate the area under a certain type of P-R curve, which refers to the size of the area surrounded by the accuracy rate and recall rate on the horizontal and vertical coordinates of the curve. mAP (Mean Average Precision) is to calculate the average value of the area under the P-R curve of all categories, that is, the average value of AP corresponding to all categories. The calculation formula is as follows, where C is the total number of categories.
其中，APi表示第i类目标的AP值，mAP就是所有类别各自的AP相加的总的值/总类别数，就是他们的平均AP，FPS(Frame Per Second)是用来评估目标检测的速度，即每秒内可以处理的图片数量或者处理一张图片所需时间来评估检测速度，时间越短，速度越快。Among them, APi represents the AP value of the i-th type of target, mAP is the total value of the sum of the respective APs of all categories/the total number of categories, which is their average AP, and FPS (Frame Per Second) is used to evaluate target detection Speed, that is, the number of pictures that can be processed per second or the time required to process a picture to evaluate the detection speed, the shorter the time, the faster the speed.
(2)改进模型的消融实验；(2) The ablation experiment of the improved model;
在改进YOLOv4网络模型训练中，模型结构的损失函数loss值越小越好，期望值为0。在使用各种方式改进网络的过程中，通过mAP、FPS、参数量、模型体积等性能评价指标来反映模型性能提升。输入图片的尺寸调整为416×416，批处理大小为8，优化器采用Adam，采用余弦退火衰减算法阶段性改变当前学习率，初始学习率设置为0.001，最小学习率设置为0.00001，余弦周期设置为5。平滑标签大小设置为0.005，权重衰减大小设置为0.0005。迭代训练100个epoch(世代)。将验证集图片输入进网络，每10个epoch计算一次模型当前训练性能，得到模型当前检测精度(mAP)以及每一类目标的AP值。In the improved YOLOv4 network model training, the smaller the loss value of the model structure, the better, and the expected value is 0. In the process of using various methods to improve the network, the performance improvement of the model is reflected by performance evaluation indicators such as mAP, FPS, parameter quantity, and model volume. The size of the input image is adjusted to 416×416, the batch size is 8, the optimizer uses Adam, and the cosine annealing decay algorithm is used to change the current learning rate in stages. The initial learning rate is set to 0.001, the minimum learning rate is set to 0.00001, and the cosine period is set. for 5. The smooth label size is set to 0.005 and the weight decay size is set to 0.0005. Iteratively train for 100 epochs (generations). Input the verification set pictures into the network, calculate the current training performance of the model every 10 epochs, and obtain the current detection accuracy (mAP) of the model and the AP value of each type of target.
针对YOLOv4原模型，本实施例通过调参将其中的一些小技巧调整到了最佳情况，之后以原模型作为基准，加入了不同的改动以期望模型获得更好的性能。最终结果如表3所示。本实施例选择目前主流的目标检测模型YOLOv4作为初始模型A，采用轻量化骨干网络GhostNet来替换原始的骨干网络得到模型(B)，其mAP为97.07％，相较于原YOLOv4模型(A)提升了1.37％，证明了在相对简单的检测任务中轻量化网络比更深层的复杂网络能够更快收敛，更容易拟合。同时参数量远少于原YOLOv4模型，推理速度也得到了一定提升，内存占用减少到了原模型的61.5％。采用更快的特征金字塔池化模块SPPF替换原模块SPP，来加快模型的运行速度和收敛速度。在使用了该模块后，模型(C)推理速度进一步提升，模型的mAP也略微提升，参数量和内存占用不变。采用轻量化网络最常用到且效果较好的深度可分离卷积，进一步减小模型参数量。模型(D)的参数量减少到了原模型(C)的四分之一，检测速度也略微提升了，但模型的mAP也减少了1.17％。在改进的YOLOv4的两个上采样层和下采样层之后插入ECA模块，那么采样后会通过该注意力机制模块聚焦于信息更有效的通道，从而能过滤掉部分无用信息的干扰，使信息融合的效果更好。由于还有另一条特征融合的支路与它堆叠，所以不用担心有效信息的丢失。除此之外，将Leaky ReLU替换为SiLU激活函数。模型(E)的检测速度略微下降，但mAP增加了0.74％。针对该特定背景环境下的目标进行检测，采用k-means聚类算法，在综合考量数据集中样本尺寸后，为了增强网络对小目标的检测能力，保留了聚类生成的中小尺寸锚点框，大尺寸仍沿用原始的由COCO数据集采用的锚点框大小，最终本发明网络(F)的mAP增加了0.39％并且没有任何性能损失。For the original model of YOLOv4, this embodiment adjusts some of the tricks to the best situation through parameter tuning, and then uses the original model as a benchmark to add different changes to expect the model to achieve better performance. The final results are shown in Table 3. In this embodiment, the current mainstream target detection model YOLOv4 is selected as the initial model A, and the lightweight backbone network GhostNet is used to replace the original backbone network to obtain the model (B), whose mAP is 97.07%, which is improved compared with the original YOLOv4 model (A) 1.37%, proving that the lightweight network can converge faster and be easier to fit than the deeper complex network in relatively simple detection tasks. At the same time, the number of parameters is far less than that of the original YOLOv4 model, and the inference speed has also been improved to a certain extent, and the memory usage has been reduced to 61.5% of the original model. The faster feature pyramid pooling module SPPF is used to replace the original module SPP to speed up the running speed and convergence speed of the model. After using this module, the inference speed of the model (C) is further improved, and the mAP of the model is also slightly improved, and the parameter amount and memory usage remain unchanged. Depth separable convolution, which is most commonly used in lightweight networks and has a good effect, is used to further reduce the amount of model parameters. The number of parameters of the model (D) is reduced to a quarter of the original model (C), and the detection speed is also slightly improved, but the mAP of the model is also reduced by 1.17%. Insert the ECA module after the two upsampling layers and downsampling layers of the improved YOLOv4, then after sampling, the attention mechanism module will focus on the channel with more effective information, so that the interference of some useless information can be filtered out, and the information can be fused The effect is better. Since there is another branch of feature fusion stacked with it, there is no need to worry about the loss of effective information. Besides that, replace Leaky ReLU with SiLU activation function. The detection speed of model (E) decreases slightly, but the mAP increases by 0.74%. For the detection of targets in this specific background environment, the k-means clustering algorithm is adopted. After comprehensive consideration of the sample size in the data set, in order to enhance the network’s ability to detect small targets, the small and medium-sized anchor boxes generated by clustering are retained. The large size still uses the original anchor box size adopted by the COCO dataset, and finally the mAP of the network (F) of the present invention increases by 0.39% without any performance loss.
表3改进YOLOv4的消融实验Table 3 Improved YOLOv4 ablation experiments
(3)改进模型与不同模型的比较；(3) Comparison of the improved model with different models;
为验证此方法就有更好效果，在同一环境和相同数据集的前提下，与其它主流算法做对比实验：YOLOv4-tiny、YOLOv3、SSD、Faster R-CNN、YOLOv5。其中SSD和YOLO系列算法为一阶段网络，Faster R-CNN为二阶段网络。具体实验结果如表4所示。由表4结果可知，本发明方法能够有效提高对输电线路四种异物的检测精度。本发明方法在四类异物的平均准确率均为最高，同时IOU阈值为0.5和0.75下的mAP分别为97.30％和64.56％，同样为最高值。FPS达到52.4，满足实时检测的需求。同时参数量只有原模型YOLOv4的17％，更利于搭载于内存有限的嵌入式设备。In order to verify that this method has a better effect, under the premise of the same environment and the same data set, a comparative experiment with other mainstream algorithms: YOLOv4-tiny, YOLOv3, SSD, Faster R-CNN, YOLOv5. Among them, the SSD and YOLO series algorithms are one-stage networks, and Faster R-CNN is a two-stage network. The specific experimental results are shown in Table 4. From the results in Table 4, it can be seen that the method of the present invention can effectively improve the detection accuracy of the four kinds of foreign objects on the transmission line. The method of the present invention has the highest average accuracy rate for the four types of foreign bodies, and the mAPs under the IOU thresholds of 0.5 and 0.75 are respectively 97.30% and 64.56%, which are also the highest values. The FPS reaches 52.4, which meets the needs of real-time detection. At the same time, the number of parameters is only 17% of the original model YOLOv4, which is more conducive to carrying embedded devices with limited memory.
表4不同算法检测结果对比Table 4 Comparison of detection results of different algorithms
(4)检测效果对比；(4) Comparison of detection effects;
为了比较该算法的实际检测效果，与YOLOv4和YOLOv5两款精度最高的模型进行对比。检测效果如图12所示。第一、二、三、四排分别是含气球、风筝、垃圾、鸟巢的输电线异物场景，即图(a)～(c)是含气球的输电线异物场景，图(d)～(f)是风筝的输电线异物场景，图(g)～(i)是垃圾的输电线异物场景，图(j)～(l)是鸟巢的输电线异物场景，第五排(即图(m)～(o))是补充的小目标数据集的输电线路场景。左边一列(即图(a)、(d)、(g)、(j)、(m))为YOLOv4网络的检测效果，中间一列(即图(b)、(e)、(h)、(k)、(n))为YOLOv5网络的检测效果，右边一列(即图(c)、(f)、(i)、(l)、(o))为改进YOLOv4网络的检测效果。在对第一行输电线路悬挂着的气球的检测中，YOLOv4没有检测出来，YOLOv5和本发明提出的改进YOLOv4网络均检测到了，且改进YOLOv4网络检测的置信度更高。第二行风筝的检测与第一行的检测结果类似，改进YOLOv4网络仍然获得了最佳的检测结果。在对第三行输电线路上悬挂的垃圾的检测中，YOLOv4未检出，YOLOv5存在误检，将输电线认成了垃圾，同时另外两处垃圾的检测框并未很好地将垃圾的全貌框选进去，而改进的YOLOv4精确地框出了两处垃圾且置信度极高。第四行为对挂载在杆塔上的鸟巢检测场景，YOLOv5未检出，YOLOv4和改进的YOLOv4均检出并获得了很高的置信度，且改进的YOLOv4对鸟巢全貌框选更精准，YOLOv4框的过于狭小，实际并未将整个鸟巢包含进预测框内。最后一行是特别针对训练小目标检测能力补充的数据集，在将250×250像素的异物图像嵌入到5472×3078像素的背景图像中并进行一定的扭曲翻转，可以使被检测目标以极小的尺寸出现在图像中，极大提高网络在输电线路巡检过程中对小目标的检测能力。同样最终结果表明改进的YOLOv4取得了最高的置信度。综上所述，改进YOLOv4网络模型相较于主流高性能探测器，总体上在检测中小目标中获得了最佳的性能，且在参数量上具有显著优势，检测速度也能够满足实际工业场景的需求，具备应用于输电线路场景异物检测的高效能力。In order to compare the actual detection effect of the algorithm, it is compared with the two most accurate models of YOLOv4 and YOLOv5. The detection effect is shown in Figure 12. The first, second, third, and fourth rows are foreign object scenes on power lines containing balloons, kites, garbage, and bird’s nests, that is, pictures (a) to (c) are scenes of foreign objects on power lines containing balloons, and pictures (d) to (f ) is the scene of foreign matter in the power line of the kite, Figures (g)~(i) are the scene of foreign matter in the power line of garbage, Figures (j)~(l) are the scene of foreign matter in the power line of the bird's nest, the fifth row (that is, Figure (m) ~(o)) is the transmission line scene of the supplementary small object dataset. The left column (that is, pictures (a), (d), (g), (j), (m)) is the detection effect of the YOLOv4 network, and the middle column (that is, pictures (b), (e), (h), ( k), (n)) is the detection effect of the YOLOv5 network, and the right column (that is, the figure (c), (f), (i), (l), (o)) is the detection effect of the improved YOLOv4 network. In the detection of the balloon hanging on the first transmission line, YOLOv4 did not detect it, but both YOLOv5 and the improved YOLOv4 network proposed by the present invention detected it, and the improved YOLOv4 network detection has higher confidence. The detection results of the second row of kites are similar to those of the first row, and the improved YOLOv4 network still obtains the best detection results. In the detection of the garbage hanging on the third row of transmission lines, YOLOv4 did not detect it, and YOLOv5 had false detections, and recognized the transmission line as garbage. At the same time, the detection frames of the other two garbage did not capture the whole picture of the garbage. The frame is selected, and the improved YOLOv4 accurately frames two garbage with a high degree of confidence. The fourth behavior is for the detection scene of the bird’s nest mounted on the tower. YOLOv5 did not detect it, but both YOLOv4 and the improved YOLOv4 detected it and obtained a high degree of confidence. is too narrow to actually include the entire bird's nest in the prediction frame. The last line is a supplementary data set specially aimed at training small target detection capabilities. After embedding a 250×250 pixel foreign object image into a 5472×3078 pixel background image and performing a certain twist and flip, the detected target can be detected with a very small The size appears in the image, which greatly improves the network's ability to detect small targets during the transmission line inspection process. The same final result shows that the improved YOLOv4 has achieved the highest confidence. In summary, compared with mainstream high-performance detectors, the improved YOLOv4 network model has generally achieved the best performance in detecting small and medium-sized targets, and has significant advantages in the number of parameters, and the detection speed can also meet the requirements of actual industrial scenarios. It has the high-efficiency ability to detect foreign objects in transmission line scenarios.
Claims (10)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211052529.9ACN115410087A (en) | 2022-08-30 | 2022-08-30 | Transmission line foreign matter detection method based on improved YOLOv4 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211052529.9ACN115410087A (en) | 2022-08-30 | 2022-08-30 | Transmission line foreign matter detection method based on improved YOLOv4 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115410087Atrue CN115410087A (en) | 2022-11-29 |
Family
ID=84162991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211052529.9APendingCN115410087A (en) | 2022-08-30 | 2022-08-30 | Transmission line foreign matter detection method based on improved YOLOv4 |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115410087A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116310895A (en)* | 2023-02-23 | 2023-06-23 | 淮阴工学院 | Sheep flock counting method based on improved YOLOv5 algorithm |
| CN116385954A (en)* | 2023-02-15 | 2023-07-04 | 江苏方天电力技术有限公司 | High-precision power transmission line abnormality detection method and system |
| CN116403303A (en)* | 2023-03-24 | 2023-07-07 | 西南交通大学 | Intelligent substation inspection method based on YOLOv5 |
| CN116665227A (en)* | 2023-06-15 | 2023-08-29 | 西北工业大学 | A method and system for reading recognition of liquid crystal digital display instruments based on deep learning |
| CN117541890A (en)* | 2023-11-08 | 2024-02-09 | 北京天易数聚科技有限公司 | Power grid equipment target detection method, system, equipment and medium |
- 2022
- 2022-08-30CNCN202211052529.9Apatent/CN115410087A/enactivePending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116385954A (en)* | 2023-02-15 | 2023-07-04 | 江苏方天电力技术有限公司 | High-precision power transmission line abnormality detection method and system |
| CN116310895A (en)* | 2023-02-23 | 2023-06-23 | 淮阴工学院 | Sheep flock counting method based on improved YOLOv5 algorithm |
| CN116403303A (en)* | 2023-03-24 | 2023-07-07 | 西南交通大学 | Intelligent substation inspection method based on YOLOv5 |
| CN116665227A (en)* | 2023-06-15 | 2023-08-29 | 西北工业大学 | A method and system for reading recognition of liquid crystal digital display instruments based on deep learning |
| CN117541890A (en)* | 2023-11-08 | 2024-02-09 | 北京天易数聚科技有限公司 | Power grid equipment target detection method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114241422B (en) | Student classroom behavior detection method based on ESRGAN and improved YOLOv s | |
| CN115410087A (en) | Transmission line foreign matter detection method based on improved YOLOv4 | |
| CN110298266B (en) | Object detection method based on deep neural network based on multi-scale receptive field feature fusion | |
| CN110348376B (en) | Pedestrian real-time detection method based on neural network | |
| CN110321874A (en) | A kind of light-weighted convolutional neural networks pedestrian recognition method | |
| CN110298387A (en) | Incorporate the deep neural network object detection method of Pixel-level attention mechanism | |
| CN116597224A (en) | Potato defect detection method based on improved YOLO V8 network model | |
| CN115205667A (en) | A Dense Object Detection Method Based on YOLOv5s | |
| CN114708231A (en) | Sugarcane aphid target detection method based on light-weight YOLO v5 | |
| CN112784756B (en) | Human body identification tracking method | |
| CN111178177A (en) | Cucumber disease identification method based on convolutional neural network | |
| CN114049554B (en) | Lawn obstacle detection method based on lightweight YOLOv5s model | |
| CN115578624A (en) | Agricultural pest model construction method, detection method and device | |
| CN114972208A (en) | YOLOv 4-based lightweight wheat scab detection method | |
| CN115439744A (en) | Lightweight detection method of tea buds in complex environment based on target detection | |
| CN114332942A (en) | Night infrared pedestrian detection method and system based on improved YOLOv3 | |
| CN111881803A (en) | Livestock face recognition method based on improved YOLOv3 | |
| CN118537802A (en) | Deep learning-based chicken and egg target detection method for chicken farm | |
| CN113902901A (en) | A method and system for object separation based on lightweight detection | |
| Qiu et al. | Computer vision technology based on deep learning | |
| CN116402769A (en) | A high-precision textile defect intelligent detection method considering both large and small objects | |
| CN116682106A (en) | Deep learning-based intelligent detection method and device for diaphorina citri | |
| CN119296174A (en) | A classroom behavior recognition method based on deformable multi-scale adaptive detection network | |
| CN117710841A (en) | Small target detection method and device for aerial image of unmanned aerial vehicle | |
| CN115240118B (en) | Light smoking detection method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |