CN115480586A - Method and system for swarm control of unmanned aerial vehicles for embankment monitoring - Google Patents

Abstract

Translated fromChinese

本发明公开了一种堤坝监测无人机群控制方法及系统，方法包括：根据无人机的性能，对无人机群中的无人机进行分级，将无人机分为一级无人机和其它无人机；利用一级无人机采集堤坝视频数据；使用基于深度学习的目标监测方法分析堤坝视频数据，得到堤坝故障点位置和类型；一级无人机飞行至堤坝故障点，并以堤坝故障点为中心，获取堤坝故障点周围预设范围内的其它点坐标，以此将其它点坐标发送至其它无人机；其它无人机到达指定坐标点，获取堤坝故障点多角度图像，并进行边缘监测，得到堤坝故障点多角度边缘，同时将其传输至一级无人机；一级无人机合成堤坝故障点多角度边缘，进行堤坝险情评估。本发明有益效果是：提高堤坝监测效率和精度。

The invention discloses a control method and system for a group of unmanned aerial vehicles for embankment monitoring. The method includes: according to the performance of the unmanned aerial vehicles, classifying the unmanned aerial vehicles in the unmanned aerial vehicle group, and dividing the unmanned aerial vehicles into first-level unmanned aerial vehicles and unmanned aerial vehicles. Other unmanned aerial vehicles; use the first-level UAV to collect video data of the dam; use the target monitoring method based on deep learning to analyze the video data of the dam, and obtain the location and type of the fault point of the dam; the first-level UAV flies to the fault point of the dam, and uses The fault point of the dam is the center, and the coordinates of other points within the preset range around the fault point of the dam are obtained, so as to send the coordinates of other points to other drones; other drones arrive at the specified coordinate point, and obtain multi-angle images of the fault point of the dam, And carry out edge monitoring, get the multi-angle edge of the dam fault point, and transmit it to the first-level UAV; the first-level UAV synthesizes the multi-angle edge of the dam fault point, and evaluates the danger of the dam. The invention has the beneficial effects of improving the monitoring efficiency and precision of dykes and dams.

Description

Translated fromChinese

一种堤坝监测无人机群控制方法及系统Method and system for swarm control of unmanned aerial vehicles for embankment monitoring

技术领域technical field

本发明涉及无人机控制领域，尤其涉及一种堤坝监测无人机群控制方法及系统。The invention relates to the field of unmanned aerial vehicle control, in particular to a method and system for controlling an unmanned aerial vehicle group for embankment monitoring.

背景技术Background technique

传统的堤坝监测是通过人工巡逻的方式来实现，不仅需要大量的人力投入，还存在漏查的风险，且巡逻人员的人身安全也难以保障。采用无人机代替人力是一种可行的解决方案，而使用无人机集群进行巡逻能够对堤坝情况进行更好地分析识别。同时，在实际应用中，考虑到不同无人机的算力等情况，常使用边缘计算的方式对数据进行处理。Traditional embankment monitoring is achieved through manual patrolling, which not only requires a large amount of manpower input, but also has the risk of missing inspections, and the personal safety of patrolling personnel is also difficult to guarantee. The use of drones to replace manpower is a feasible solution, and the use of drone swarms for patrols can better analyze and identify the condition of embankments. At the same time, in practical applications, considering the computing power of different drones, edge computing is often used to process data.

堤坝的险情监测可以被看作目标检测问题，而随着深度学习技术的迅速发展，基于卷积神经网络的目标检测方法目前已趋于成熟，和无人机结合应用可以有效地提升堤坝险情监测的性能。The danger monitoring of dams can be regarded as a target detection problem. With the rapid development of deep learning technology, the target detection method based on convolutional neural network has become mature, and the combination application with UAV can effectively improve the danger monitoring of dams. performance.

发明内容Contents of the invention

为了针对现有技术中堤坝险情监测采用无人机群的控制问题，本发明提供一种堤坝监测无人机群控制方法及系统。方法包括以下步骤：In order to solve the problem of using unmanned aerial vehicle swarm control for dam danger monitoring in the prior art, the present invention provides a control method and system for levee monitoring unmanned aerial vehicle swarm. The method includes the following steps:

S1、根据无人机的性能，对无人机群中的无人机进行分级，将无人机分为一级无人机和其它无人机；S1. Classify the drones in the drone group according to the performance of the drones, and divide the drones into first-class drones and other drones;

S2、利用一级无人机采集堤坝视频数据；S2, using a first-level UAV to collect video data of the embankment;

S3、使用基于深度学习的目标监测方法分析堤坝视频数据，得到堤坝故障点位置和类型；S3. Use the target monitoring method based on deep learning to analyze the video data of the dam to obtain the location and type of the fault point of the dam;

S4、一级无人机飞行至堤坝故障点，并以堤坝故障点为中心，获取堤坝故障点周围预设范围内的其它点坐标，以此将其它点坐标发送至其它无人机；S4. The first-level drone flies to the fault point of the embankment, and takes the fault point of the embankment as the center to obtain the coordinates of other points within the preset range around the fault point of the embankment, so as to send the coordinates of other points to other drones;

S5、其它无人机到达指定坐标点，获取堤坝故障点多角度图像，并进行边缘监测，得到堤坝故障点多角度边缘，同时将其传输至一级无人机；S5. Other UAVs arrive at the designated coordinate point, obtain multi-angle images of the dam fault point, and perform edge monitoring, obtain the multi-angle edge of the dam fault point, and transmit it to the first-level UAV at the same time;

S6、一级无人机合成堤坝故障点多角度边缘，进行堤坝险情评估。S6. The first-level UAV synthesizes the multi-angle edge of the fault point of the embankment and evaluates the danger of the embankment.

本发明提供的有益效果是：通过使用无人机集群对堤坝进行巡逻解放了人力，更加的省时省力，同时利用目标检测算法识别故障点保证了监测的性能，边缘计算的方式也对算力进行了合理的利用，更加适应实际应用，使得堤坝监测能够有效地进行。The beneficial effects provided by the present invention are: the manpower is liberated by using the unmanned aerial vehicle cluster to patrol the embankment, which saves more time and effort, and at the same time, the use of the target detection algorithm to identify the fault point ensures the monitoring performance, and the edge computing method also reduces the computing power. Reasonable utilization is carried out, and it is more suitable for practical application, so that dyke monitoring can be carried out effectively.

附图说明Description of drawings

图1是本发明方法流程示意图。Fig. 1 is a schematic flow chart of the method of the present invention.

具体实施方式detailed description

为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地描述。In order to make the purpose, technical solution and advantages of the present invention clearer, the embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

请参考图1，图1是本发明方法流程示意图；一种堤坝监测无人机群控制方法，包括以下：Please refer to Fig. 1, Fig. 1 is a schematic flow chart of the method of the present invention; a dam monitoring UAV group control method, including the following:

S1、根据无人机的性能，对无人机群中的无人机进行分级，将无人机分为一级无人机和其它无人机；S1. Classify the drones in the drone group according to the performance of the drones, and divide the drones into first-class drones and other drones;

需要说明的是，步骤S1中，无人机分级规则为：对无人机的属性影响因子进行综合评定得到无人机的优先级。It should be noted that in step S1, the UAV classification rule is: the priority of the UAV is obtained by comprehensively evaluating the attribute influencing factors of the UAV.

所述综合评定过程如下：The comprehensive evaluation process is as follows:

1架无人机的属性影响因子共N个；作为一种实施例，属性影响因子包括无人机的姿态稳定性、算力、剩余电量等因素；There are a total of N attribute influencing factors of an unmanned aerial vehicle; as an example, the attribute influencing factors include factors such as the attitude stability, computing power, and remaining power of the unmanned aerial vehicle;

对每个属性影响因子按照预设值进行定量打分；作为一种实施例，比如将姿态稳定性分为高、中、低三个档次，并依次赋予3分、2分、1分等，对于其它影响因子，如算力和剩余电量等，可按同理进行；Quantitatively score each attribute impact factor according to the preset value; as an example, for example, attitude stability is divided into three grades of high, medium and low, and 3 points, 2 points, 1 point, etc. are assigned in turn, for Other influencing factors, such as computing power and remaining power, can be carried out in the same way;

对属性影响因子进行加权求和，得到总分数；需要说明的是，越重要的影响因子所对应的权重越大，影响因子的重要程度取决于实际情况，比如在需要长时间任务时，最重要的影响因子为剩余电量，而在需要精确监测时，最重要的影响因子为算力；关于权重的分配也可根据实际情况进行分配。The weighted sum of the attribute impact factors is used to obtain the total score; it should be noted that the more important the impact factor, the greater the weight, and the importance of the impact factor depends on the actual situation. For example, when a long-term task is required, the most important The influencing factor is the remaining power, and when accurate monitoring is required, the most important influencing factor is computing power; the distribution of weights can also be made according to the actual situation.

根据总分数的高低排序，排在第一位的为一级无人机，其余为其它无人机。According to the ranking of the total score, the first-ranked UAV is the first-class UAV, and the rest are other UAVs.

若第一无人机有多个，则依次比较第一无人机的属性影响因子的重要程度，重要程度高的作为第一无人机，剩余的分类为其它无人机；其中属性影响因子的重要程度根据实际情况设定。比如对粗排序得到的优先级最高的几架无人机的排序进行调整：先比较不同无人机的算力，算力高的优先级更高；若算力相同，再比较姿态稳定性，姿态稳定性越好的优先级更高；若姿态稳定性系统，再比较剩余电量，剩余电量越多的优先级更高，若全部相同，则优先级相同。最后选择出一架优先级最高的无人机为一级无人机，若所有优先级最高的无人机不止一架则随机选择一架作为一级无人机。If there are multiple first UAVs, then compare the importance of the attribute influencing factors of the first UAV in turn, and the one with the highest importance is regarded as the first UAV, and the rest are classified as other UAVs; the attribute influencing factors The importance of is set according to the actual situation. For example, adjust the ranking of several UAVs with the highest priority obtained by rough sorting: first compare the computing power of different UAVs, and the priority of higher computing power is higher; if the computing power is the same, then compare the attitude stability, The better the attitude stability, the higher the priority; if the attitude stability system, then compare the remaining power, the more the remaining power, the higher the priority, if they are all the same, the priority is the same. Finally, a UAV with the highest priority is selected as the first-level UAV. If there are more than one UAV with the highest priority, one is randomly selected as the first-level UAV.

S2、利用一级无人机采集堤坝视频数据；S2, using a first-level UAV to collect video data of the embankment;

S3、使用基于深度学习的目标监测方法分析堤坝视频数据，得到堤坝故障点位置和类型；S3. Use the target monitoring method based on deep learning to analyze the video data of the dam to obtain the location and type of the fault point of the dam;

作为一种实施例，由一级无人机使用目标检测算法对采集的视频进行实时分析，判断前方是否有故障点，若无故障点则继续飞行，接着进行目标检测。使用的目标检测算法为在YOLOv5框架上的改进方案，利用搜集的堤坝故障点数据集对模型进行微调训练。首先对原始图片帧进行自适应缩放的处理，之后采用卷积网络对输入的图像进行特征提取和融合，且为了适应堤坝监测任务，将focus层优化成卷积层以提升推理速度，最终预测出堤坝故障点位置和类型。As an embodiment, the first-level UAV uses the target detection algorithm to analyze the collected video in real time to determine whether there is a fault point ahead, and if there is no fault point, continue to fly, and then perform target detection. The target detection algorithm used is an improved solution based on the YOLOv5 framework, and the model is fine-tuned and trained using the collected data set of dam fault points. First, the original picture frame is processed with adaptive scaling, and then the convolutional network is used to extract and fuse the input image features. In order to adapt to the task of embankment monitoring, the focus layer is optimized into a convolutional layer to improve the reasoning speed, and finally predict the Location and type of dam failure.

S4、一级无人机飞行至堤坝故障点，并以堤坝故障点为中心，获取堤坝故障点周围预设范围内的其它点坐标，以此将其它点坐标发送至其它无人机；S4. The first-level drone flies to the fault point of the embankment, and takes the fault point of the embankment as the center to obtain the coordinates of other points within the preset range around the fault point of the embankment, so as to send the coordinates of other points to other drones;

作为一种实施例，当一级无人机发现故障点时，选择距离最近的故障点位置作为目标位置；一级无人机指挥无人机集群到达指定位置：具体步骤如下：As an embodiment, when the first-level drone finds a fault point, select the nearest fault point position as the target location; the first-level drone directs the drone cluster to reach the designated location: the specific steps are as follows:

一级无人机先飞行至目标位置正上方，之后拍摄一张图像I对其进行自适应分析，从而对无人机集群的位置进行调整，让无人机集群所在位置不仅能够保证拍摄图像的清晰度，而且恰好能观测到故障点的全貌。The first-level drone flies directly above the target position, and then takes an image I for adaptive analysis, thereby adjusting the position of the drone cluster, so that the location of the drone cluster can not only ensure the accuracy of the captured image Clarity, and just can observe the whole picture of the fault point.

自适应分析算法为基于目标检测的自适应分析方法；一级无人机先自行调整至位置x处，之后通过通信协议发送指令给二级无人机，使其飞行至距离故障点的垂直高度为h，以x点为圆心，r为半径的圆周上，呈等距离排列。The adaptive analysis algorithm is an adaptive analysis method based on target detection; the first-level UAV first adjusts itself to the position x, and then sends instructions to the second-level UAV through the communication protocol to make it fly to the vertical height from the fault point is h, on the circle with point x as the center and r as the radius, they are arranged equidistantly.

S5、其它无人机到达指定坐标点，获取堤坝故障点多角度图像，并进行边缘监测，得到堤坝故障点多角度边缘，同时将其传输至一级无人机；S5. Other UAVs arrive at the designated coordinate point, obtain multi-angle images of the dam fault point, and perform edge monitoring, obtain the multi-angle edge of the dam fault point, and transmit it to the first-level UAV at the same time;

所有无人机在指定位置对故障点进行监测，各自对指定角度下的故障点图像进行边缘检测，之后通过通信协议将多角度数据发送给一级无人机；All UAVs monitor the fault point at the designated location, each performs edge detection on the fault point image at the designated angle, and then sends the multi-angle data to the first-level UAV through the communication protocol;

S6、一级无人机合成堤坝故障点多角度边缘，进行堤坝险情评估。S6. The first-level UAV synthesizes the multi-angle edge of the fault point of the embankment and evaluates the danger of the embankment.

作为一种实施例，一级无人机对来自无人机集群的多个角度的数据进行处理，根据多角度数据对故障点的类别和危险级别进行评估，通过对不同角度下的故障类别和严重程度进行对比和综合分析，得到故障类别，危险级别等；As an example, the first-level unmanned aerial vehicle processes the data from multiple angles of the unmanned aerial vehicle cluster, and evaluates the category and danger level of the fault point according to the multi-angle data. Severity is compared and comprehensively analyzed to obtain fault category, danger level, etc.;

将多角度的观测结果融合成一张全局图，用于提高故障处理的效率和准确率，共有n个角度的数据，最终合成一张图像作为故障观测图像。The multi-angle observation results are fused into a global map to improve the efficiency and accuracy of fault handling. There are n angles of data in total, and finally an image is synthesized as the fault observation image.

一种堤坝监测无人机群控制系统，包括优先级分析模块、目标检测模块、通信模块、边缘计算模块和危险评估模块。其中优先级分析模块用于对无人机集群的优先级进行分析，确定一级无人机和二级无人机；目标检测模块用于一级无人机对收集的巡逻视频进行理解分析，确定视频中是否包含故障点以及故障点的具体位置；通信模块用于一级和二级无人机间的通信，包括一级无人机向二级无人机发送指令，二级无人机向一级无人机传送数据；边缘计算模块用于二级无人机对收集到的特定角度的监测图像进行边缘检测，并且将原图像和边缘检测数据通过通信模块发送给一级无人机；危险评估模块用于一级无人机对收集到的多角度监测数据进行处理，合成多角度图像，判定故障类别并对危险程度进行评估。各模块的实现过程如下：An unmanned aerial vehicle group control system for embankment monitoring, including a priority analysis module, a target detection module, a communication module, an edge computing module and a risk assessment module. Among them, the priority analysis module is used to analyze the priority of the UAV cluster to determine the first-level UAV and the second-level UAV; the target detection module is used for the first-level UAV to understand and analyze the collected patrol video, Determine whether the video contains the fault point and the specific location of the fault point; the communication module is used for communication between the first-level and second-level UAVs, including the first-level UAV sending instructions to the second-level UAV, and the second-level UAV Send data to the first-level UAV; the edge computing module is used by the second-level UAV to detect the edge of the monitoring image collected at a specific angle, and send the original image and edge detection data to the first-level UAV through the communication module ; The hazard assessment module is used by the first-level UAV to process the collected multi-angle monitoring data, synthesize multi-angle images, determine the fault category and evaluate the degree of danger. The implementation process of each module is as follows:

优先级分析模块：根据无人机的性能，对无人机群中的无人机进行分级，将无人机分为一级无人机和其它无人机；Priority analysis module: according to the performance of drones, the drones in the drone group are classified, and the drones are divided into first-class drones and other drones;

目标检测模块：利用一级无人机采集堤坝视频数据；使用基于深度学习的目标监测方法分析堤坝视频数据，得到堤坝故障点位置和类型；Target detection module: use the first-level UAV to collect video data of the dam; use the target monitoring method based on deep learning to analyze the video data of the dam, and obtain the location and type of the fault point of the dam;

通信模块：一级无人机飞行至堤坝故障点，并以堤坝故障点为中心，获取堤坝故障点周围预设范围内的其它点坐标，以此将其它点坐标发送至其它无人机；Communication module: the first-level UAV flies to the fault point of the embankment, and takes the fault point of the embankment as the center to obtain the coordinates of other points within the preset range around the fault point of the embankment, so as to send the coordinates of other points to other UAVs;

边缘计算模块：其它无人机到达指定坐标点，获取堤坝故障点多角度图像，并进行边缘监测，得到堤坝故障点多角度边缘，同时将其传输至一级无人机；Edge computing module: other drones arrive at the designated coordinate point, obtain multi-angle images of dam fault points, and perform edge monitoring, obtain multi-angle edges of dam fault points, and transmit them to the first-level UAV at the same time;

危险评估模块：一级无人机合成堤坝故障点多角度边缘，进行堤坝险情评估。Hazard assessment module: Level-1 UAV synthesizes multi-angle edges of dam fault points to assess dam danger.

本发明的有益效果是：通过使用无人机集群对堤坝进行巡逻解放了人力，更加的省时省力，同时利用目标检测算法识别故障点保证了监测的性能，边缘计算的方式也对算力进行了合理的利用，更加适应实际应用，使得堤坝监测能够有效地进行。The beneficial effects of the present invention are: the use of UAV clusters to patrol the embankment liberates manpower, saves time and effort, and at the same time uses the target detection algorithm to identify fault points to ensure the performance of monitoring, and the edge computing method also improves computing power. In order to make reasonable use, it is more suitable for practical application, so that the monitoring of dykes and dams can be carried out effectively.

以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (8)

1. A dam monitoring unmanned aerial vehicle group control method is characterized in that: the method comprises the following steps:

s1, classifying unmanned aerial vehicles in an unmanned aerial vehicle cluster according to the performance of the unmanned aerial vehicles, and dividing the unmanned aerial vehicles into primary unmanned aerial vehicles and other unmanned aerial vehicles;

s2, acquiring dam video data by using a primary unmanned aerial vehicle;

s3, analyzing dam video data by using a target monitoring method based on deep learning to obtain the position and type of a dam fault point;

s4, the primary unmanned aerial vehicle flies to a dam fault point, and coordinates of other points in a preset range around the dam fault point are obtained by taking the dam fault point as a center, so that the coordinates of the other points are sent to other unmanned aerial vehicles;

s5, when other unmanned aerial vehicles reach the designated coordinate point, acquiring a multi-angle image of the dam fault point, carrying out edge monitoring to obtain a multi-angle edge of the dam fault point, and transmitting the multi-angle edge to the primary unmanned aerial vehicle;

s6, synthesizing multi-angle edges of dam fault points by the primary unmanned aerial vehicle, and carrying out dam dangerous case assessment.

2. A method as claimed in claim 1, wherein the method comprises the steps of: in step S1, the classification rule of the unmanned aerial vehicle is: and comprehensively evaluating the attribute influence factors of the unmanned aerial vehicle to obtain the priority of the unmanned aerial vehicle.

3. A method as claimed in claim 2, wherein the method comprises: the comprehensive evaluation process is as follows:

the number of attribute influence factors of 1 unmanned aerial vehicle is N;

quantitatively scoring each attribute influence factor according to a preset value;

weighting and summing the attribute influence factors to obtain a total score;

according to the ranking of the total scores, the first unmanned plane is ranked, and the rest are other unmanned planes.

4. A bank monitoring drone group control method according to claim 3, characterized by: if the number of the first unmanned machines is multiple, sequentially comparing the importance degrees of the attribute influence factors of the first unmanned machines, taking the first unmanned machines with high importance degrees as the first unmanned machines, and classifying the rest unmanned machines as other unmanned machines; wherein the importance degree of the attribute influence factor is set according to the actual situation.

5. A dam monitoring drone group control method according to claim 1, characterized by: and step S3, analyzing the dam video data by adopting a YoloV5 network.

6. A method as claimed in claim 1, wherein the method comprises the steps of: the step S4 specifically comprises the following steps:

s41, when the primary unmanned aerial vehicle finds a fault point, selecting the position of the fault point as a target position;

s42: the first-level unmanned aerial vehicle commands the second-level unmanned aerial vehicle cluster to reach the designated position, and the specific steps are as follows:

one-level unmanned aerial vehicle flies directly over to the target location, later shoots an image I and carries out self-adaptation analysis to it, adjusts the position of unmanned aerial vehicle cluster, acquires other point coordinates of predetermineeing the within range around the dykes and dams trouble point, makes other unmanned aerial vehicles can observe dykes and dams trouble general appearance.

7. A method as claimed in claim 6, wherein the method comprises: the self-adaptive analysis method comprises the following steps: a target detection-based adaptive analysis method; the first-level unmanned aerial vehicle is adjusted to a position x, then an instruction is sent to other unmanned aerial vehicles through a communication protocol, so that the unmanned aerial vehicles fly to a position h away from a fault point, and are arranged equidistantly on a circle with the x point as the circle center and r as the radius.

8. The utility model provides a dykes and dams monitoring unmanned aerial vehicle crowd control system which characterized in that: the system comprises:

the system comprises a priority analysis module, a target detection module, a communication module, an edge calculation module and a danger evaluation module;

a priority analysis module: classifying the unmanned planes in the unmanned plane cluster according to the performance of the unmanned planes, and dividing the unmanned planes into a first-level unmanned plane and other unmanned planes;

a target detection module: acquiring dam video data by using a primary unmanned aerial vehicle; analyzing dam video data by using a target monitoring method based on deep learning to obtain the position and the type of a dam fault point;

a communication module: the primary unmanned aerial vehicle flies to a dam fault point, and obtains coordinates of other points in a preset range around the dam fault point by taking the dam fault point as a center, so that the coordinates of the other points are sent to other unmanned aerial vehicles;

an edge calculation module: when other unmanned aerial vehicles reach the designated coordinate point, acquiring a multi-angle image of the dam fault point, carrying out edge monitoring to obtain a multi-angle edge of the dam fault point, and transmitting the multi-angle edge to the primary unmanned aerial vehicle;

a risk assessment module: and synthesizing multi-angle edges of dam fault points by the primary unmanned aerial vehicle, and evaluating dam dangerous cases.

Images