Disclosure of Invention
In view of the above, the invention aims to provide a method and a related device for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data, which realize high-efficiency detection of hidden danger of power distribution network equipment based on large-view-field video data by comprehensively utilizing multi-view-angle video data collected by an unmanned aerial vehicle and an advanced video processing technology, and improve the detection accuracy and the operation convenience.
In order to achieve the above purpose, the technical scheme provided by the invention is as follows:
In a first aspect, the invention provides a method for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data, which comprises the following steps:
Acquiring video data of multi-path distribution network equipment with synchronous time stamps under different visual angles;
According to the visual angle difference of the video data of the multi-path distribution network equipment, the space and time parameters of the video data of the multi-path distribution network equipment are aligned by combining the synchronous time stamp;
Based on the aligned video data of the multi-path power distribution network equipment, extracting the characteristics of the multi-path video frame images and matching the characteristics to obtain a splicing relationship among the multi-path video frame images;
determining a splicing seam between two video frame images to be spliced based on a splicing relationship;
based on the splicing relation and the splicing seam of the multiple paths of video frame images, carrying out fusion splicing on the multiple paths of video frame images to obtain fusion video data;
and detecting the fused video data by using a pre-trained hidden danger detection model to obtain a hidden danger detection result of the power distribution network equipment.
Further, extracting features of the multiple paths of video frame images and matching, including:
Extracting multi-level features of multiple paths of video frame images by using a neural network to obtain a coarse feature image meeting the small-size requirement and a fine feature image meeting the large-size requirement of each path of video frame image;
Adding a leavable position code to the rough feature map of each path, converting the rough feature map into a one-dimensional vector, and converting the one-dimensional vector into a feature representation easy to match through a feature matching module based on a transducer to obtain a feature vector;
Calculating a similarity matrix between video frame images of other paths and video frame images of a main path by taking any path as the main path and using a pixel-by-pixel vector inner product mode, calculating optimal matching by using a dual-Softmax method, filtering partial outlier matching pairs by a mutual neighbor algorithm to obtain rough matching point pairs which enable multi-path video frame images to be matched pairwise, wherein the similarity matrix is used for representing similarity between feature vectors of two video frame images;
And mapping the rough matching point pairs into corresponding fine feature graphs, inputting a mapping region into a feature matching module based on a Transformer, calculating the matching probability of the central feature of the video frame image of the main path and all the features of the matched video frame images of other paths, taking the feature position with the highest matching probability as the matching point position with sub-pixel precision, wherein the matching point position with sub-pixel precision represents the splicing relationship between every two matched video frame images.
Further, the stitching seam is an optimal stitching seam, the optimal stitching seam is determined by a dynamic programming method with the minimum image difference as a target, and the determining process comprises the following steps:
determining an energy summation function of two video frame images which are spliced pixel by pixel, wherein the energy summation function is used for quantifying the image difference degree of the overlapping area of the two video frame images;
according to the minimum energy sum, iterating, traversing each row in turn to obtain the position of the source of the minimum energy and the path, and simultaneously recording the minimum energy and the vector reaching each row and each column of each row and the position of the source of each column of each row;
And backtracking the splice joints, backtracking the splice joints from the last row through a position matrix of a recorded source, and selecting the splice joint with the minimum energy sum as the optimal splice joint.
Further, fusing and splicing the multiple paths of video frame images, including:
wavelet transformation is carried out on the video frame images to be spliced, and a Mallat algorithm is adopted for decomposition to obtain a low-frequency component and a high-frequency component;
a fusion method of weighted average is adopted for the low-frequency components, so that fused low-frequency components are obtained;
fusing the high-frequency components by adopting a window coefficient absolute value-based method to obtain fused high-frequency components;
and carrying out wavelet inverse transformation on the fused low-frequency component and high-frequency component to obtain a fused image.
Further, according to the visual angle difference of the video data of the multi-path power distribution network equipment, the spatial and time parameters of the video data of the multi-path power distribution network equipment are aligned by combining the synchronous time stamp, and the method comprises the following steps:
time alignment, selecting a main path, acquiring video frame data of the main path at a set time, indexing time and video frame data of frames of other paths before and after the set time, and acquiring the video frame data of the other paths at the set time by a linear interpolation method;
the space alignment is carried out, geometric correction is carried out on each path of video data according to camera calibration parameters of different visual angles, and distortion of different degrees, which occurs when each camera images, is eliminated;
Brightness alignment, after each path of video data frame image is converted from RGB format to HSV format, the brightness of the video frame image at the same time in each path of video data is unified;
And (3) aligning coordinate systems, taking an image reference system acquired by a main road as a base reference system, calculating projection transformation models of video frame images of other roads according to camera external parameters between the other roads and the main road, and projecting multi-path video frame data to the base reference system.
Further, the hidden danger detection model is formed by adopting YOLOv network training, and detects the fused video data by utilizing the pre-trained hidden danger detection model, and the method comprises the following steps:
segmenting the fusion video data into a plurality of sub-images according to the input image size of YOLOv networks, and stacking all the sub-images into a batch;
Inputting data into a hidden danger detection model according to batches, outputting hidden danger target detection results of power distribution network equipment of all sub-images in the batches, combining all detection results, and converting the position coordinates of the targets in the sub-images into the position coordinates of the original images;
And performing non-maximum value inhibition treatment on all the converted detection results to obtain a final hidden danger detection result.
Further, obtaining video data of the multi-path distribution network device with the synchronous time stamp under different view angles includes:
arranging the multiple visual angles according to the rule that adjacent visual angles overlap to set degrees, and constructing a multi-visual angle shooting system, wherein video shooting parameters of each visual angle in the multi-visual angle shooting system are the same;
and placing the multi-view shooting system in a scene of the power distribution network equipment to be detected, and setting a video shooting mode to continuously acquire data.
In a second aspect, the present invention provides a power distribution network equipment hidden trouble detection device based on large-view-field spliced video data, including:
The data acquisition module is used for acquiring video data of the multi-path power distribution network equipment with the synchronous time stamp under different visual angles;
the preprocessing module is used for aligning the space and time parameters of the video data of the multi-path distribution network equipment according to the visual angle difference of the video data of the multi-path distribution network equipment and combining the synchronous time stamp;
The video fusion splicing module is used for extracting characteristics of multiple paths of video frame images based on the aligned multiple paths of video data of the power distribution network equipment and matching the characteristics to obtain a splicing relationship between the multiple paths of video frame images, determining a splicing seam between two video frame images to be spliced based on the splicing relationship, and fusing and splicing the multiple paths of video frame images based on the splicing relationship and the splicing seam of the multiple paths of video frame images to obtain fused video data;
And the hidden danger detection module is used for detecting the fused video data by utilizing a pre-trained hidden danger detection model to obtain a hidden danger detection result of the power distribution network equipment.
Accordingly, the present invention provides a computer device comprising a processor and a memory:
the memory is used for storing the computer program and sending the instructions of the computer program to the processor;
the processor executes the power distribution network equipment hidden danger detection method based on the large-view-field spliced video data according to the instruction of the computer program.
Accordingly, the invention provides a computer readable storage medium, which is characterized in that a computer program is stored on the computer readable storage medium, and the computer program realizes the method for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data according to the first aspect when being executed by a processor.
In summary, the invention provides a method for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data and a related device. The spatial and time parameters of the multi-channel video data are precisely aligned by combining the synchronous time stamps, so that the continuity and consistency of the spliced video in time and space are ensured, false detection omission caused by time dislocation or visual angle difference is avoided, and the reliability and the effectiveness of a detection result are improved. By accurately extracting and matching the image characteristics of the multiple paths of video frames, the method can efficiently determine the splicing relation and the optimal splicing seam between the video frames, reduce visual distortion and seam trace in the splicing process, generate high-quality fusion video data and provide clear and continuous visual basis for subsequent hidden danger detection. The fusion video data is automatically detected by utilizing the pre-trained hidden danger detection model, the videos do not need to be checked one by one manually, the detection speed and the working efficiency are greatly improved, meanwhile, errors caused by human factors are reduced, and the objectivity and the standardization of the detection process are enhanced. The invention solves the problem that a single power distribution network device needs to carry out multiple fine shooting by efficiently integrating multiple paths of video resources for unified detection.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is apparent that the embodiments described below are only some embodiments of the present invention, not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the embodiment provides a method for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data, which includes the following steps:
and S1, acquiring video data of the multi-path distribution network equipment with the synchronous time stamp under different visual angles.
It should be noted that, in this step, a plurality of cameras or video capturing devices may be disposed at different positions and angles of the power distribution network during implementation, so as to ensure that they record video data simultaneously. The key is that all devices are equipped with a time synchronization mechanism (e.g., GPS clock synchronization) so that each frame of video has a common time stamp even though it is taken from a different perspective. This provides a temporal frame of reference for subsequent processing, ensuring consistency of the data.
S2, aligning the space and time parameters of the video data of the multi-path distribution network equipment according to the visual angle difference of the video data of the multi-path distribution network equipment and combining with the synchronous time stamp,
It should be noted that, because the positions and directions of the cameras are different, the captured video is not uniform in space. This step uses the time stamps in the video data and the known camera position information to computationally adjust the temporal and spatial parameters of each video stream so that the videos from different perspectives are aligned in time and space in preparation for subsequent stitching.
And S3, extracting characteristics of the multiple paths of video frame images based on the aligned multiple paths of video data of the power distribution network equipment and matching to obtain a splicing relationship among the multiple paths of video frame images.
It should be noted that, matching feature points, such as feature descriptors of SIFT, ORB, etc., are found among the video frames, and how to connect different video frames with each other is determined by the corresponding relationship of the feature points. This process involves image processing techniques in order to find pairs of frames that can be seamlessly connected to determine the order and manner in which video frame images are stitched.
And S4, determining a splicing seam between two video frame images to be spliced based on the splicing relation.
It should be noted that after the splicing relationship of the video frames is clarified, the optimal overlapping area (i.e., the splice seam) between each pair of adjacent frames is further accurately calculated, so as to minimize the splice trace on the premise of keeping the scene continuity, and smooth transition is usually achieved through an image fusion algorithm.
And S5, based on the splicing relation and the splicing seam of the multiple paths of video frame images, carrying out fusion splicing on the multiple paths of video frame images to obtain fusion video data.
It should be noted that, by using the information obtained in the previous step, multiple video frames are fused into a single large-field video according to the correct sequence and the overlapping area by using the image stitching technology, and this process may include color and brightness adjustment to ensure the visual consistency and naturalness of the final video.
And S6, detecting the fused video data by using a pre-trained hidden danger detection model to obtain a hidden danger detection result of the power distribution network equipment.
It should be noted that, in this step, the hidden danger detection model of machine learning or deep learning is applied to analyze the spliced video data. The models learn the mode of identifying common hidden danger of power distribution network equipment, such as insulation breakage, component loosening and the like, in a training stage. By means of model reasoning, the system can automatically mark out suspected hidden danger areas, provide accurate inspection clues for maintenance personnel, and accelerate hidden danger investigation and repair.
The embodiment provides a hidden danger detection method for power distribution network equipment based on large-view-field spliced video data, which remarkably expands the detected view field range by combining video data under different view angles and solves the problem of repeated shooting of a plurality of power distribution network equipment at the same position through the large view field. The spatial and time parameters of the multi-channel video data are precisely aligned by combining the synchronous time stamps, so that the continuity and consistency of the spliced video in time and space are ensured, false detection omission caused by time dislocation or visual angle difference is avoided, and the reliability and the effectiveness of a detection result are improved. By accurately extracting and matching the image characteristics of the multiple paths of video frames, the method can efficiently determine the splicing relation and the optimal splicing seam between the video frames, reduce visual distortion and seam trace in the splicing process, generate high-quality fusion video data and provide clear and continuous visual basis for subsequent hidden danger detection. The fusion video data is automatically detected by utilizing the pre-trained hidden danger detection model, the videos do not need to be checked one by one manually, the detection speed and the working efficiency are greatly improved, meanwhile, errors caused by human factors are reduced, and the objectivity and the standardization of the detection process are enhanced. The invention solves the problem that a single power distribution network device needs to carry out multiple fine shooting by efficiently integrating multiple paths of video resources for unified detection. The novel intelligent inspection technology is provided for detecting hidden danger of power distribution network equipment in power inspection, inspection efficiency is improved, and intelligent development of power inspection is promoted.
Referring to fig. 2, fig. 2 shows a technical roadmap of a method for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data. The invention is further described below in connection with fig. 2.
In a preferred embodiment of the present invention, for step S1, obtaining video data of multiple paths of distribution network devices with synchronous time stamps under different viewing angles includes:
Arranging the multiple visual angles according to the rule that adjacent visual angles overlap to set degrees, constructing a multi-visual angle shooting system, wherein video shooting parameters of each visual angle in the multi-visual angle shooting system are the same, arranging the multi-visual angle shooting system in a scene of the power distribution network equipment to be detected, and setting a video shooting mode to continuously acquire data.
In some specific implementations of this embodiment, the unmanned aerial vehicle may use a cradle head to mount a multi-view camera, where the multi-view camera is composed of a plurality of color cameras with the same parameters, and the angles of view of the cameras are set to overlap by 5 °. The multi-view camera is used for acquiring images of the power distribution network equipment at different view angles.
The multi-view camera acquires video data and has the advantages that a large-view video is acquired after multi-view video is spliced, the video data can comprise a plurality of power distribution network devices and large-size power distribution network devices in a single scene, compared with image data, the video data has stronger information redundancy, high-quality and multi-angle image data do not need to be shot deliberately, technical requirements of patrol personnel are effectively reduced, and data acquisition efficiency of the power distribution network devices in the power patrol process is improved.
In a preferred embodiment of the present invention, for step S2, aligning spatial and temporal parameters of video data of multiple distribution network devices according to a viewing angle difference of the video data of the multiple distribution network devices in combination with a synchronization timestamp includes:
S21, time alignment, selecting a main path, acquiring video frame data of the main path at a set time, indexing time and video frame data of frames of other paths before and after the set time, and acquiring video frame data of other paths at the set time by a linear interpolation method.
In some embodiments, time-aligning video data includes selecting a primary camera and acquiring the primary cameraAt the position ofTime of day data indexing other camerasAt the position ofTime of frame before and after time of day、And obtaining data of other cameras by a linear interpolation methodTime of day data:
Wherein,、Respectively representing a main camera and other cameras;、 Representing other cameras respectivelyAt the position of、Video frame data collected at the moment; representing other camerasAt the position ofTime aligned video frame data.
S22, performing space alignment, performing geometric correction on each path of video data according to camera calibration parameters of different visual angles, and eliminating distortion of different degrees of each camera during imaging.
In some embodiments, spatially aligning the video data includes geometrically correcting each path of video data according to calibration parameters of each camera to eliminate distortion of each camera to varying degrees during imaging.
S23, brightness alignment, namely converting each path of video data frame image from an RGB format to an HSV format, and unifying the brightness of the video frame image at the same time in each path of video data.
In some embodiments, the brightness alignment of the video data includes adaptively adjusting and unifying brightness of the video frame image at the same time in each path of video data by using an illumination gain compensation coefficient after converting the frame image from the RGB format to the HSV format:
Wherein,、Respectively representing brightness values of the images before and after illumination gain compensation; Representing the illumination gain compensation coefficient.
Solving the illumination gain compensation coefficient of each camera image through the extreme point of the illumination gain error function between the main camera image and other camera images:
Wherein,Indicating the error in the gain compensation,、Gain coefficients respectively representing the main camera image and other camera images;、 respectively representing brightness values of the main camera and other camera images;、 Respectively representing pixel coordinates of the main camera and other camera images;、 Respectively representing the overlapping areas of the main camera and the other camera images.
In order to simplify the calculation and improve the robustness of the gain, the empirical formula of the illumination gain error function is:
Wherein,、Average brightness values of overlapping areas of the main camera image and other camera images are respectively represented;、 standard deviation of error and gain are respectively expressed, and in general casesThe number of the holes in the substrate is approximately 0.7,The number of the holes in the film is approximately 0.1,Representing the number of pixels in the overlap.
To solve for the gain factor, a closed form solution can be obtained by having a derivative of the function of 0. Let the pair of、The derivative is derived as a linear system of equations:
ObtainingIs a number of 1, and is not limited by the specification,Is that。
And S24, aligning a coordinate system, taking an image reference system acquired by a main road as a reference system, calculating projection transformation models of video frame images of other roads according to external parameters of cameras between the other roads and the main road, and projecting multi-path video frame data to the reference system.
In some embodiments, performing coordinate system alignment on video data includes taking an image reference system acquired by a main camera as a reference frame, calculating a projection transformation model of video frame images of other cameras according to external parameters of angle differences, displacement differences and the like between the other cameras and the main camera, projecting multiple paths of video frame data to a unified coordinate system, and reducing parallax among the paths of video data.
The method comprises the steps of projecting multiple paths of video frame images to a unified coordinate system in a cylindrical projection mode, and projecting a plane image to a curved surface of a cylinder:
Wherein,Representing pixel coordinates of the image after cylindrical projection;、 Respectively representing the width and height of the image.
In this embodiment, the generalized normalization operation is performed on multiple paths of video data, and the multiple paths of video data are preprocessed in four aspects of time, space, brightness and coordinate system. The time alignment is used for unifying video frame images of all paths of videos at the same moment, the space alignment is used for eliminating the same-parameter camera data acquisition difference caused by camera distortion, the brightness alignment is used for unifying the brightness of the video frame images of all paths of videos at the same moment, and the coordinate system alignment is used for eliminating the space coordinates of video data of different visual angles.
The generalized normalization of the multipath video data has the advantages that differences among the paths of video data caused by objective factors such as environment, equipment and the like can be reduced or eliminated, the video data fusion difficulty is reduced, the quality of the fused video data is improved, and a high-quality data basis is provided for the subsequent hidden danger detection of the power distribution network equipment.
In a preferred embodiment of the present invention, extracting and matching features of multiple video frame images includes:
and S31, extracting multi-level features of the multi-path video frame images by using a neural network to obtain a coarse feature image of each path of video frame image meeting the small-size requirement and a fine feature image of each path of video frame image meeting the large-size requirement.
In some embodiments, feature extraction is performed on a video frame image, and a ResNet depth neural network loaded with an ImageNet pre-training weight is used to extract multi-level features of the video frame image, including an original imageSmall-sized coarse feature map of size and original imageLarge-sized thin feature patterns of dimensions.
S32, adding a leachable position code to the rough feature map of each path, converting the rough feature map into a one-dimensional vector, and converting the one-dimensional vector into a feature representation easy to match through a feature matching module based on a transducer to obtain a feature vector.
It should be noted that, a learner position code is added to the rough feature map of the multi-path video frame image pixel by pixel, and the rough feature map is converted into a one-dimensional vector, and the information between the multi-path video frame images is fused by a feature matching module based on a transducer, and the rough feature map is converted into a feature representation easy to match.
And S33, calculating a similarity matrix between video frame images of other paths and video frame images of the main path by taking any path as the main path and using a pixel-by-pixel vector inner product mode, calculating optimal matching by using a dual-Softmax method, and filtering partial outlier matching pairs by a mutual neighbor algorithm to obtain rough matching point pairs which enable multi-path video frame images to be matched pairwise, wherein the similarity matrix is used for representing similarity between feature vectors of two video frame images.
It should be noted that the calculation formula of dual-Softmax is as follows:
Wherein,Representing a matching probability matrix;、 representing video frame image characteristics of the main camera and other cameras respectively;、 pixel coordinates representing image features of video frames of the main camera and the other cameras, respectively.
And S34, mapping the rough matching point pairs into corresponding fine feature graphs, inputting the mapping areas into a feature matching module based on a transform, calculating the matching probabilities of the central features of the video frame images of the main path and all the features of the matched video frame images of other paths, taking the feature position with the highest matching probability as the matching point position with sub-pixel precision, wherein the matching point position with sub-pixel precision represents the splicing relationship between every two matched video frame images.
It should be noted that the point pairs in the rough matching、Mapping to corresponding fine feature map, inputting the mapping region to a feature matching module based on a transducer, further fine matching features, calculatingCenter featureThe matching probability of all the features in the model is obtained to obtain the corresponding probability distributionMatching point positions of sub-pixel precision in (a).
In this embodiment, a ResNet neural network that has been pre-trained based on ImageNet is used to perform feature extraction on the video frame image, so as to obtain a small-size coarse feature map and a large-size fine feature map. And realizing coarse feature matching and fine feature matching through a similarity measurement and LoFTR feature matching module among the feature images of the multipath video frame, and obtaining the matching point positions of the sub-pixel level through calculating probability distribution among the features. And searching the optimal seam between the multiple paths of video frame images by adopting a dynamic programming method, and realizing the splicing of the video frame images by utilizing an image fusion algorithm based on wavelet transformation. And carrying out post-processing on the spliced video frame images to obtain high-quality large-view-field spliced video.
In a preferred embodiment of the present invention, the stitching seam is an optimal stitching seam, and the optimal stitching seam is determined by a dynamic programming method with the aim of minimizing image differences, and the determining process includes:
and S41, determining an energy sum function of two spliced video frame images pixel by pixel, wherein the energy sum function is used for quantifying the image difference degree of the overlapping area of the two video frame images.
In some embodiments, a pixel-by-pixel energy summation function for multiple video frame images is constructed using a concept based on minimizing differencesThe structural and color differences between the images at the seams are evaluated, and the energy function is calculated as follows:
Wherein,Representing all pixel points through which the splice joint passes; representing weights used to balance the color difference term and the structure difference term; Representing the color difference of the pixels at the same position in the overlapped area; representing the structural differences of the pixels at the same position in the overlapping area.
Wherein,、Respectively represent、A Sobel operator in the direction;、 Representing video frame images of the main camera and the other cameras, respectively.
And S42, initializing a splicing seam path, taking the first row of the overlapped area as the minimum energy and path of the first step, iterating according to the minimum energy sum, traversing each row in turn to obtain the position of the source of the minimum energy and path, and recording the minimum energy and vector reaching each column of each row and the position of the source of each column of each row.
It should be noted that, the optimal splice seam is found in combination with the dynamic programming method. And sequentially traversing each row according to the minimum energy sum to obtain the positions of the minimum energy and path sources, namely the upper left, the upper top and the upper right. The minimum energy and vector reaching each row and each column, and the location of the source of each path reaching each row and each column need to be recorded for backtracking.
And S43, backtracking the splice joint from the last row through a position matrix of a recorded source, and selecting the splice joint with the minimum energy sum as the optimal splice joint.
It should be noted that, the splice seam is traced back, and the splice seam is traced back from the last line through the position matrix of the recording source. At the position ofIn the direction, 1 is subtracted if it is the upper left, 1 is added if it is the upper right, and it is unchanged if it is the top,The direction is decremented by 1. All the splice seams are compared and the seam with the smallest sum of energy is selected as the optimal seam.
In a preferred embodiment of the present invention, the fusion splicing of multiple video frame images includes:
and S51, carrying out wavelet transformation on the video frame images to be spliced, and decomposing by adopting a Mallat algorithm to obtain a low-frequency component and a high-frequency component.
S52, a fusion method of weighted average is adopted for the low-frequency components, and the fused low-frequency components are obtained.
In some embodiments, a fusion method of weighted average is used for the low frequency components, and the fusion algorithm is as follows:
Wherein,、Low frequency coefficients respectively representing video frame images of the main camera and other cameras;、 the weight values corresponding to the low frequency coefficients are respectively represented, and the sum is 1.
And S53, fusing the high-frequency components by adopting a window coefficient absolute value based method to obtain fused high-frequency components.
In some specific embodiments, the high-frequency components are fused by adopting a window coefficient absolute value based method, and the fusion algorithm is as follows:
Wherein,、Representing the high frequency coefficients of the video frame images of the main camera and the other cameras, respectively.
S54, carrying out wavelet inverse transformation on the fused low-frequency component and high-frequency component to obtain a fused image.
And S55, large-view-field spliced video post-processing.
The details and edges of the spliced image are enhanced by sharpening filtering processing and a local contrast enhancement method. And secondly, reducing the noise level of the spliced image by an image denoising technology. And finally, storing the spliced video frame images as a video sequence, thereby outputting large-field video data.
In a preferred embodiment of the present invention, the hidden danger detection model is trained by YOLOv networks, and the training steps include:
and S61, constructing a power distribution network equipment target detection data set according to the large-view-field spliced video data.
Firstly, data cleaning is carried out to remove video frame images with poor imaging effect, such as over-exposure of a camera, under-exposure of the camera, virtual focus of the camera, dynamic blurring, internal reference calibration errors and the like.
And secondly, marking and screening hidden dangers of power distribution network equipment in video data frame by frame, and dividing the data into a training set and a verification set to form a standard data set which can be learned by the neural network. The data set comprises the hidden danger images, the category and the position information of the power distribution network equipment.
And S62, training YOLOV the target detection neural network by using the hidden danger detection training data set.
Firstly, using the difference between a minimized predicted value and a true value as a neural network optimization criterion, adopting a focal loss as a classification loss function of a potential risk target detection network of power distribution network equipment, reducing the influence of unbalance of the data volume and identification complexity of potential risks of different power distribution network equipment on the stability of the network, and adopting a cross-correlation loss as a positioning loss function of a potential risk target frame of the power distribution network equipment:
Wherein,Representing a hidden danger classification loss function of power distribution network equipment; representing a predicted class of equipment hazards of a power distribution network the proximity to the true class is such that,A closer to 1 indicates more accurate prediction results; represents an adjustable factor, greater than 0; representing potential equipment hazard target frame positioning loss of the power distribution network;、 And the prediction area and the real area of the hidden danger target frame of the power distribution network equipment are represented.
Secondly, updating network parameters by adopting a gradient descent method, and using a small batch random gradient descent method as a parameter optimization method:
Wherein,、、Lr represents the learning rate, representing the step size of each optimization; An exponential moving average representing the gradient; An exponential moving average representing the square of the gradient; Representing a very small non-zero number, avoiding network optimization failure caused by divisor 0 in the optimization process, wherein the divisor is defaulted to be。
And S63, evaluating the trained network, and evaluating the hidden danger classification and positioning effects of the power distribution network equipment by adopting mAP and IoU to the YOLOV target detection neural network. The method comprises the steps of calculating average Precision AP of each category, taking average, constructing PR curves of Precision and Recall rate (Recall) for the average Precision AP of each category, calculating the area between the PR curves and a coordinate axis to obtain the average Precision AP of each category, calculating common area and all areas by IoU by adopting a calculation mode of the intersection ratio of each prediction target and the real target, and dividing the common area by all areas to obtain a final result.
Calculating mAP:
Calculation IoU:
Wherein P (r) represents a PR curve function; Representing a recall maximum; Representing a prediction target region; Representing the real target area.
In a further embodiment of the present invention, detecting the fused video data using a pre-trained hidden danger detection model includes:
And S71, segmenting the fusion video data into a plurality of sub-images according to the input image size of YOLOv networks, and stacking all the sub-images into one batch.
In some embodiments, the sliding window size is set according to the input image size of YOLOV networks, the hidden danger video frame image of the large-field distribution network is uniformly segmented into a plurality of sub-images according to the overlapping area of 10%, and all the sub-images are stacked into a batch.
S71, inputting data into a hidden danger detection model according to batches, outputting hidden danger target detection results of power distribution network equipment of all sub-images in the batches, combining all detection results, and converting the position coordinates of the target in the sub-images into the position coordinates of the original image.
In some embodiments, according to the batch input data to the YOLOV network after training, outputting the detection results of the hidden danger targets of the distribution network of all the sub-images in the batch, combining all the detection results, and converting the position coordinates of the targets in the sub-images into the position coordinates of the original images:
Wherein,、Respectively representing the left upper corner coordinate and the right lower corner coordinate of the target frame prediction area in the original image;、 respectively representing the left upper corner coordinate and the right lower corner coordinate of the target frame prediction area in the original image; representing the position coordinates of the upper left corner of the subgraph in the original graph.
And S71, performing non-maximum value inhibition treatment on all the converted detection results to obtain a final hidden danger detection result.
And performing non-maximum value inhibition treatment on all converted detection results to obtain a final hidden danger detection result, removing hidden danger of repeated detection to obtain a final hidden danger detection result, and finishing the detection of hidden danger of the large-view-field power distribution network equipment with high pixels.
According to the embodiment, the potential-hazard detection data set of the power distribution network equipment is constructed by utilizing video data, the target detection network is trained and verified by utilizing the data set, and the potential-hazard detection of the power distribution network equipment, which maintains a high-pixel large-field-of-view image, is realized by combining a sliding block detection method.
Based on the same inventive concept, the embodiment of the application also provides a power distribution network equipment hidden danger detection device based on the large-view-field spliced video data, which is used for realizing the power distribution network equipment hidden danger detection method based on the large-view-field spliced video data. The implementation scheme of the system for solving the problems is similar to the implementation scheme recorded in the method, so the specific limitation in the embodiment of the power distribution network equipment hidden danger detection device based on the large-view-field spliced video data provided below can be referred to the limitation of the power distribution network equipment hidden danger detection method based on the large-view-field spliced video data, and the description is omitted here.
Referring to fig. 3, this embodiment provides a power distribution network equipment hidden trouble detection device based on large-view-field spliced video data, including:
The data acquisition module is used for acquiring video data of the multi-path power distribution network equipment with the synchronous time stamp under different visual angles;
the preprocessing module is used for aligning the space and time parameters of the video data of the multi-path distribution network equipment according to the visual angle difference of the video data of the multi-path distribution network equipment and combining the synchronous time stamp;
The video fusion splicing module is used for extracting characteristics of multiple paths of video frame images based on the aligned multiple paths of video data of the power distribution network equipment and matching the characteristics to obtain a splicing relationship between the multiple paths of video frame images, determining a splicing seam between two video frame images to be spliced based on the splicing relationship, and fusing and splicing the multiple paths of video frame images based on the splicing relationship and the splicing seam of the multiple paths of video frame images to obtain fused video data;
And the hidden danger detection module is used for detecting the fused video data by utilizing a pre-trained hidden danger detection model to obtain a hidden danger detection result of the power distribution network equipment.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the system is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
Referring to fig. 4, the embodiment of the invention further provides a computer device 40, which comprises a memory 402, a processor 401 and a computer program 403 stored on the memory 402, wherein the computer program 403 implements the method for detecting hidden danger of the power distribution network device based on the large-field spliced video data according to any one of the above methods when being executed on the processor 401.
The computer device 40 may be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device 40 may include, but is not limited to, a processor 401, a memory 402. It will be appreciated by those skilled in the art that fig. 4 is merely an example of computer device 40 and is not intended to limit computer device 40, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.
The Processor 401 may be a central processing unit (Central Processing Unit, CPU), but the Processor 401 may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), off-the-shelf Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 402 may in some embodiments be an internal storage unit of the computer device 40, such as a hard disk or a memory of the computer device 40. The memory 402 may also be an external storage device of the computer device 40 in other embodiments, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. that are provided on the computer device 40. Further, the memory 402 may also include both internal and external storage units of the computer device 40. The memory 402 is used to store an operating system, application programs, boot loader (BootLoader), data, and other programs, such as program code for the computer program. The memory 402 may also be used to temporarily store data that has been output or is to be output.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when being run by a processor, realizes the method for detecting hidden danger of power distribution network equipment based on large-view-field spliced video data according to any one of the methods.
In this embodiment, the integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least any entity or device capable of carrying computer program code to a camera device/terminal equipment, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the disclosed embodiments of the application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The foregoing embodiments are merely for illustrating the technical solution of the present invention, but not for limiting the same, and although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that modifications may be made to the technical solution described in the foregoing embodiments or equivalents may be substituted for parts of the technical features thereof, and that such modifications or substitutions do not depart from the spirit and scope of the technical solution of the embodiments of the present invention in essence.