技术领域technical field
本发明涉及计算机控制技术领域，尤其涉及一种基于飞行器的设施检测方法及控制设备。The invention relates to the technical field of computer control, in particular to an aircraft-based facility detection method and control equipment.
背景技术Background technique
某些设施，需要用户定期对这些设施进行巡检、维护，以便于确认这些设施的安全状态。例如，对于电塔、大桥、高楼等设施，需要定期巡检来确保这些设施的安全及正常运行。Some facilities require users to regularly inspect and maintain these facilities in order to confirm the safety status of these facilities. For example, for facilities such as electric towers, bridges, and high-rise buildings, regular inspections are required to ensure the safety and normal operation of these facilities.
然而，对于一些处于特殊位置的设施，特别是一些处于险峻位置处的电塔、大桥等设施，进行周期性的巡检变得十分困难。并且，此类设施通常数目较多，周期性的巡检需要耗费大量的人力。However, for some facilities in special locations, especially some facilities such as power towers and bridges in dangerous locations, it becomes very difficult to conduct periodic inspections. Moreover, there are usually a large number of such facilities, and periodic inspections require a lot of manpower.
发明内容Contents of the invention
本发明实施例提供了一种基于飞行器的设施检测方法及控制设备，通过对飞行器的控制来实现对目标设施的巡检。Embodiments of the present invention provide an aircraft-based facility detection method and control equipment, which realize patrol inspection of target facilities by controlling the aircraft.
一方面，本发明实施例提供了一种基于飞行器的设施检测方法，包括：On the one hand, an embodiment of the present invention provides an aircraft-based facility detection method, including:
当飞行器位于针对目标设施的检测位置时，获取包括所述目标设施的环境图像；When the aircraft is located at a detection position for a target facility, acquiring an environment image including the target facility;
从所述环境图像中确定出所述目标设施所属的图像区域，并对该图像区域进行图像分割，得到关于所述目标设施的检测对象；Determining the image area to which the target facility belongs from the environment image, and performing image segmentation on the image area to obtain detection objects about the target facility;
根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的飞行规则；Acquiring flight rules about the detection object according to the image position of the detection object in the environment image and the detection position;
根据所述飞行规则控制所述飞行器飞行，以便于完成对所述目标设施的检测。The flight of the aircraft is controlled according to the flight rules, so as to complete the detection of the target facility.
另一方面，本发明实施例还提供了一种控制设备，包括：处理器和数据接口；On the other hand, the embodiment of the present invention also provides a control device, including: a processor and a data interface;
所述数据接口，用于与飞行器交互数据；The data interface is used for exchanging data with the aircraft;
所述处理器，用于当飞行器位于针对目标设施的检测位置时，获取包括所述目标设施的环境图像；从所述环境图像中确定出所述目标设施所属的图像区域，并对该图像区域进行图像分割，得到关于所述目标设施的检测对象；根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的飞行规则；根据所述飞行规则生成控制指令，并通过所述数据接口发送给所述飞行器以控制所述飞行器飞行，以便于完成对所述目标设施的检测。The processor is configured to acquire an environment image including the target facility when the aircraft is at a detection position for the target facility; determine the image area to which the target facility belongs from the environment image, and determine the image area for the image area Carry out image segmentation to obtain the detection object of the target facility; obtain flight rules for the detection object according to the image position of the detection object in the environment image and the detection position; generate control instructions according to the flight rules , and send it to the aircraft through the data interface to control the flight of the aircraft, so as to complete the detection of the target facility.
本发明实施例在对某些设施进行巡检时，特别是对较高设施或者处于不易到达的地区的设施，能够基于图像识别和自动控制飞行的方式对设施中的一个或者多个需要检测的对象进行巡检，降低了巡检的人力成本以及安全性，提高了巡检的效率。In the embodiments of the present invention, when inspecting certain facilities, especially for facilities that are relatively tall or located in areas that are difficult to reach, one or more of the facilities that need to be detected can be detected based on image recognition and automatic flight control. Objects are inspected, which reduces the labor cost and safety of inspections, and improves the efficiency of inspections.
附图说明Description of drawings
图1是本发明实施例的一种对目标设施进行检测的方法流程示意图；Fig. 1 is a schematic flow chart of a method for detecting a target facility according to an embodiment of the present invention;
图2是本发明实施例的一种基于飞行器的设施检测方法的流程示意图；2 is a schematic flow chart of an aircraft-based facility detection method according to an embodiment of the present invention;
图3是本发明实施例的另一种基于飞行器的设施检测方法的流程示意图；3 is a schematic flowchart of another aircraft-based facility detection method according to an embodiment of the present invention;
图4是本发明实施例的一种确定飞行规则的方法流程示意图；Fig. 4 is a schematic flow chart of a method for determining flight rules according to an embodiment of the present invention;
图5是本发明实施例的一种设施检测装置的结构示意图；5 is a schematic structural diagram of a facility detection device according to an embodiment of the present invention;
图6是本发明实施例的一种控制设备的结构示意图。Fig. 6 is a schematic structural diagram of a control device according to an embodiment of the present invention.
具体实施方式Detailed ways
本发明实施例通过使用视觉技术，采集图像，可以在远处检测、识别出需要检测的目标设施，自动飞行到待检测的目标设施的附近。并通过图像分割识别技术，将目标设施中的各个部分分割区分开来，得到该目标设施的一个或者多个检测对象，有针对地使用携带的传感器(如相机，热成像仪等)对识别出的各个检测对象进行检测与记录。检测记录结束后，无人机等飞行器将自动返航或者在电量充足的情况下去临近的下一个需要检测的目标设施进行检测。In the embodiment of the present invention, by using vision technology to collect images, the target facility to be detected can be detected and identified at a distance, and the target facility to be detected can be automatically flew to the vicinity of the target facility to be detected. And through the image segmentation and recognition technology, each part of the target facility is segmented and distinguished, and one or more detection objects of the target facility are obtained, and the carried sensors (such as cameras, thermal imagers, etc.) are used in a targeted manner to identify Each detection object is detected and recorded. After the detection record is completed, the UAV and other aircraft will automatically return to the voyage or go to the next target facility that needs to be detected for detection when the battery is sufficient.
如图1所示，是本发明实施例的一种对目标设施进行检测的方法流程示意图。本发明实施例的设施检测方法可以由一个控制设备来执行，该控制设备可以配置在飞行器上。并且，在本发明实施例中，以无人机来作为飞行器对所述方法进行说明，该无人机上挂载有用于对目标设施进行检测的传感器。该控制设备所执行的主要步骤如下。As shown in FIG. 1 , it is a schematic flowchart of a method for detecting a target facility according to an embodiment of the present invention. The facility detection method in the embodiment of the present invention can be executed by a control device, and the control device can be configured on an aircraft. Moreover, in the embodiment of the present invention, the method is described by taking an unmanned aerial vehicle as an aircraft, and the unmanned aerial vehicle is mounted with sensors for detecting target facilities. The main steps performed by the control device are as follows.
S101：确定目标设施的位置。可以利用GPS(Global Positioning System，全球定位系统)信息等位置信息，大概定义目标设施的位置，控制无人机自主向该目标设施飞行，以至目标设施出现在无人机的观察范围之内，该观察范围主要是指无人机挂载的传感器的探测范围，例如相机的拍摄范围。用户可以预先在对控制设备进行配置的用户界面上，输入一个或多个目标设施的位置信息，例如，在一个显示有地图的界面中，通过触摸点击的方式在该包括地图的界面上指定一个或多个位置点，控制设备可以将这些位置点记录为目标设施的位置点。控制设备可以在无人机开启了巡检模式时，控制无人机基于各个目标设施的位置点飞行，以便于向对应的一个或者多个目标设施飞行，以监控一个目标设施，或者监控某条路线上的多个目标设施，例如，相连的多个电塔。S101: Determine the location of the target facility. The location information such as GPS (Global Positioning System, Global Positioning System) information can be used to roughly define the location of the target facility, and the UAV is controlled to fly autonomously to the target facility, so that the target facility appears within the observation range of the UAV. The observation range mainly refers to the detection range of the sensors mounted on the UAV, such as the shooting range of the camera. The user can pre-input the location information of one or more target facilities on the user interface for configuring the control device, for example, in an interface displaying a map, specify a location on the interface including the map by touching and clicking. or a plurality of location points, which can be recorded by the control device as location points of the target facility. The control device can control the drone to fly based on the location points of each target facility when the drone is in the inspection mode, so as to fly to the corresponding one or more target facilities to monitor a target facility, or to monitor a certain Multiple target facilities on a route, for example, multiple connected power towers.
S102：基于图像分割识别以及深度图进行避障处理。在向目标设施飞行的过程中，可以基于图像分割识别以及获取的深度图进行主动避障，以便于安全飞行到可以检测目标设施的区域。S102: Perform obstacle avoidance processing based on image segmentation recognition and a depth map. During the flight to the target facility, active obstacle avoidance can be performed based on image segmentation and recognition and the obtained depth map, so as to safely fly to the area where the target facility can be detected.
无人机自主飞行过程中需要检测飞行路径中的各种障碍物。避障的首要任务是检测出飞行方向上的障碍物。深度图可以基于双目视觉探测的方式探测并计算得到，从而定位飞行路径上的障碍。深度图的获取可以使用双目计算匹配得到，也可以用基于结构光或者红外的设备计算得到。基于结构光或者红外的设备可得到相对质量更高的深度图。During the autonomous flight of UAVs, it is necessary to detect various obstacles in the flight path. The primary task of obstacle avoidance is to detect obstacles in the direction of flight. The depth map can be detected and calculated based on binocular vision detection, so as to locate obstacles on the flight path. The acquisition of the depth map can be obtained by binocular calculation and matching, or by calculation based on structured light or infrared equipment. Devices based on structured light or infrared can obtain relatively higher-quality depth maps.
为了进一步提高深度图的精度，使得在纹理不丰富并且目标物过小时能够避免发生漏检与误检的情况，在本发明实施例中可以进一步结合图像分割技术，来进行障碍物的确定和飞行避障。由于图像分割不需要做匹配，对于纹理不丰富的区域也能有较好的识别效果，因此可以配合深度图一起使用，一方面可以得到更好的深度图，另一方面，也可以给深度图中的每一个点赋予实际意义，有助于无人机进行路径规划，确定无人机飞行的飞行规则。In order to further improve the accuracy of the depth map, so that missed detection and false detection can be avoided when the texture is not rich and the target is too small, in the embodiment of the present invention, image segmentation technology can be further combined to determine obstacles and fly Avoidance. Since image segmentation does not need to be matched, it can also have a better recognition effect on areas with less texture, so it can be used together with the depth map. On the one hand, a better depth map can be obtained. On the other hand, the depth map can also be given Each point in is endowed with practical meaning, which is helpful for the path planning of the UAV and determines the flight rules of the UAV flight.
在飞行的过程中，无人机不断估计、修正与目标设施的相对位置。在一个实施例中，无人机可以使用视觉跟踪算法，将目标设施锁定在图像可观测的范围内，通过目标设施在图像中大小的变化和当前的飞行速度，估计与目标设施的相对距离。无人机也可以按照特定轨迹飞行来大致获取场景中物体的深度信息，为距离避障提供参考。During the flight, the UAV constantly estimates and corrects the relative position of the target facility. In one embodiment, the UAV can use a visual tracking algorithm to lock the target facility within the observable range of the image, and estimate the relative distance to the target facility through the size change of the target facility in the image and the current flight speed. The UAV can also fly according to a specific trajectory to roughly obtain the depth information of objects in the scene, and provide a reference for distance obstacle avoidance.
在一个实施例中，可以基于拍摄到的图像识别确定出本次检测的巡检场景，巡检场景具体可以根据本次检测的目标设施来进行分类的，例如包括：巡检电塔的场景、巡检大桥的场景等，确定出的巡检场景可以为飞行避障、规划飞行路线提供参考信息。例如，在巡检场景为巡检电塔的场景时，由于在这些场景下，相邻的电塔之间一般有电力线相连接，连接的部位相对比较固定，无人机在设定路线时，可选择绕开电力线密集的区域。In one embodiment, the inspection scene detected this time can be determined based on the captured image recognition, and the inspection scene can be classified according to the target facility detected this time, for example, including: the scene of inspection electric tower, The inspection scene of the bridge, etc., the determined inspection scene can provide reference information for flight obstacle avoidance and flight route planning. For example, when the inspection scene is the inspection scene of electric towers, since in these scenes, there are usually power lines connecting adjacent electric towers, and the connected parts are relatively fixed. When setting the route of the UAV, Optionally bypass areas with dense power lines.
针对特定的巡检场景，无人机可以配备相应的传感器以进一步提高避障的可靠性。比如，针对电力系统的设备巡检，热成像仪可以用于检测电线的存在，对于需要靠近电力线进行巡检提供更鲁棒的避障。For specific inspection scenarios, drones can be equipped with corresponding sensors to further improve the reliability of obstacle avoidance. For example, for equipment inspections in power systems, thermal imaging cameras can be used to detect the presence of wires, and provide more robust obstacle avoidance for inspections that need to be close to power lines.
S103：当检测到目标设施进入无人机的观察范围之后，检测目标设施的具体位置，并向目标设施飞行。本发明实施例可以基于人工特征的视觉检测方法来检测图像范围内的目标设施。也可以基于深层神经网络的识别算法通过对电塔等目标设施的海量图像数据学习，可以从数据中学到更加稳定可靠的图像特征，从而得到更精确的识别结果。S103: After detecting that the target facility has entered the observation range of the drone, detect the specific location of the target facility, and fly to the target facility. In the embodiment of the present invention, a target facility within an image range can be detected based on a visual detection method of artificial features. The recognition algorithm based on deep neural network can also learn more stable and reliable image features from the data by learning massive image data of target facilities such as electric towers, so as to obtain more accurate recognition results.
检测算法运行在无人机观察到的图像上，在图像中检测、定位目标设施的位置。一旦在图像中发现待检测的目标设施，在图像中将其锁定，并逐渐飞向待检测的该目标设施。此过程中，可以使用跟踪算法锁定检测到的目标设施，并利用检测的结果作为参考来确定无人机的飞行路线。The detection algorithm runs on the image observed by the UAV to detect and locate the location of the target facility in the image. Once the target facility to be detected is found in the image, it is locked in the image and gradually flies to the target facility to be detected. In this process, a tracking algorithm can be used to lock the detected target facilities, and the detection results can be used as a reference to determine the flight path of the UAV.
为了定位目标设施在图像中的位置，可以在目标设施中选择多个特征点。基于全图的分割识别可以使得无人机能够根据目标设施的类别选择更加稳定的图像特征点，这些图像特征点一般需要一直稳定地存在于目标设施上，不移动，容易被检测发现。比如，选择在电塔上的特征点比水面的特征点要更加稳定，基于这些图像特征点，可以提高为无人机计算SLAM(Simultaneous localization and mapping，同时定位与建图)的精度。因此，无人机可以更加灵活的调整姿态与路线。比如，为了使飞行路线更加安全，可以选择在局部路径中，待检测的目标设施不出现在视野范围之内，通过障碍物后，根据估计的相对位置，重新将待检测的目标设施锁定在图像中。In order to locate the position of the target facility in the image, multiple feature points can be selected in the target facility. The segmentation recognition based on the whole image can enable the UAV to select more stable image feature points according to the category of the target facility. These image feature points generally need to exist stably on the target facility all the time, do not move, and are easy to be detected. For example, the feature points selected on the electric tower are more stable than the feature points on the water surface. Based on these image feature points, the accuracy of SLAM (Simultaneous localization and mapping, simultaneous positioning and mapping) calculation for UAVs can be improved. Therefore, the UAV can adjust its attitude and route more flexibly. For example, in order to make the flight route safer, you can choose that in the local path, the target facility to be detected does not appear within the field of vision. After passing through obstacles, according to the estimated relative position, the target facility to be detected is re-locked in the image. middle.
S104：识别设施的各个组成部分，并针对性的检测与记录。在到达了对目标设施的检测位置时，例如，与目标设施的距离在预设的距离范围内的区域中的某个位置时，可以进一步地基于图像从目标设施中识别出检测对象，例如，目标设施为电塔时，识别出本次需要检测的检测对象为整个塔头，或者固定电力线的部件。S104: Identify each component of the facility, and perform targeted detection and recording. When the detection position of the target facility is reached, for example, when the distance from the target facility is within a preset distance range, the detection object can be further identified from the target facility based on the image, for example, When the target facility is an electric tower, it is identified that the detection object to be detected this time is the entire tower head, or a component of a fixed power line.
图像分割算法将提供像素级别的识别与分割，提供图像中每一个像素的类别信息，这里的类别信息的作用主要在于确定对该类别信息对应的检测对象，进而确定出需要采用的巡检策略，指导无人机飞行。The image segmentation algorithm will provide pixel-level identification and segmentation, and provide category information for each pixel in the image. The role of the category information here is mainly to determine the detection object corresponding to the category information, and then determine the inspection strategy that needs to be adopted. Guide the drone to fly.
可以将待检测的目标设施从图像中分割出来，确定出仅包括目标设施的局部图像区域，再对该局部图像区域进行分析识别，对目标设施的不同的位置识别出不同的类别，得到本次需要检测的关键部分，该关键部分即为检测对象，从而可以对目标设施的关键部分进行针对性的检测与记录。为了给巡检提供更加准确的信息，在与目标设施的距离小于预设的距离阈值时，可以使用针对目标设施的特定部位进行分割、识别的对象模型。The target facility to be detected can be segmented from the image, and the local image area including only the target facility can be determined, and then the partial image area can be analyzed and identified, and different categories can be identified for different positions of the target facility, and this time The key part that needs to be detected is the detection object, so that the key part of the target facility can be targeted for detection and recording. In order to provide more accurate information for inspection, when the distance to the target facility is less than the preset distance threshold, an object model for segmentation and identification of specific parts of the target facility can be used.
这种对象模型除了能将目标设施与背景分割开之外，还可以细化分割、识别目标设施的组成部分，得到一个或多个检测对象。对于巡检过程中，可以对例如电塔等目标设施的各个组成部分进行识别，在图像中标记出各组成部分的位置。用户可以事先指定针对每个组成部分的巡检策略。在一个实施例中，该巡检策略包括但不限于：环绕拍摄，远近持续视频拍摄，以及定点使用高精相机拍照。与远程识别的图像分割模型类似，所述对象模型的输入是图像，输出像素级别的识别结果，其中表示了每个像素所属的特定类别，可以包括背景以及细化的目标设施中各种组成部件，也就是说每个像素点可以为背景类别的像素点，或者为某个检测对象类别(例如塔头)的像素点。所述对象模型可以根据实际需要巡检的目标设施来配置，例如，对于电塔，可以配置塔头、塔脚等对象模型以便于识别出目标设施在图像中的塔头和塔脚。In addition to separating the target facility from the background, this object model can also refine the segmentation, identify the components of the target facility, and obtain one or more detection objects. For the inspection process, it is possible to identify various components of target facilities such as electric towers, and mark the position of each component in the image. Users can specify the inspection strategy for each component in advance. In one embodiment, the inspection strategy includes, but is not limited to: surrounding shooting, continuous video shooting near and far, and fixed-point shooting with high-precision cameras. Similar to the image segmentation model for remote recognition, the input of the object model is an image, and the output is a pixel-level recognition result, which indicates the specific category to which each pixel belongs, and can include various components in the background and refined target facilities , that is to say, each pixel point can be a pixel point of the background category, or a pixel point of a certain detection object category (such as a tower head). The object model can be configured according to the target facility that actually needs to be inspected. For example, for an electric tower, object models such as tower head and tower foot can be configured to identify the tower head and tower foot of the target facility in the image.
根据识别结果以及用户的设定，生成具体的巡检方案。巡检方案的生成包括：轨迹的生成，每一段轨迹的时间分配等。求解轨迹的准则会考虑的信息包括：完成所有检测对象的巡检的前提下，如何更可能的减少飞行时间，如何选择更加安全的飞行路线，比如包括如何避开电力线，如何使得拍摄画面稳定可靠，无人机的特性，比如无人机的最大最小加速度，以及进行检测的检测设备的特效，例如相机的FOV，其它用于检测的传感器的最佳使用距离等。According to the recognition result and the user's settings, a specific inspection plan is generated. The generation of the inspection plan includes: the generation of the trajectory, the time allocation of each segment of the trajectory, etc. The information that will be considered in the criteria for solving the trajectory includes: under the premise of completing the inspection of all detected objects, how to reduce the flight time more likely, how to choose a safer flight route, such as how to avoid power lines, how to make the shooting picture stable and reliable , the characteristics of the UAV, such as the maximum and minimum acceleration of the UAV, and the special effects of the detection equipment for detection, such as the FOV of the camera, the optimal use distance of other sensors used for detection, etc.
当轨迹生成完成后，无人机执行该计算出的轨迹，对目标设施各个待检测部件(检测对象)进行巡检。此时，无人机仍然会不断的更新观察，实时动态的修正轨迹以保证能安全有效的进行巡检。After the trajectory generation is completed, the UAV executes the calculated trajectory to inspect each component to be detected (detected object) of the target facility. At this time, the UAV will continue to update and observe, and dynamically correct the trajectory in real time to ensure safe and effective inspections.
S105：检测完毕，自动返航。待完成检测任务，无人机可以根据开始执行巡检任务时记录的出发点信息和/或记录的飞行数据来实现自动返航。在一个实施例中，可以结合视觉里程计(visual odometry)与GPS信息指导无人机返航。视觉里程计通过图像特征匹配的方式估计无人机在执行任务中的轨迹。结合图像分割算法的视觉里程计可以选择更好的匹配特征从而实现更精准的轨迹估计。S105: After the detection is completed, it will automatically return to the voyage. After the inspection task is completed, the UAV can automatically return to home according to the starting point information and/or recorded flight data recorded when the inspection task is started. In one embodiment, visual odometry (visual odometry) and GPS information can be combined to guide the UAV to return home. Visual odometry estimates the trajectory of UAVs during missions by means of image feature matching. Visual odometry combined with image segmentation algorithms can select better matching features to achieve more accurate trajectory estimation.
可以通过分析执行任务时的飞行轨迹，计算出返回起飞点最优的返航路线。比如，执行任务中探索、尝试、重复的飞行轨迹可以绕过。其中，可以绕过的飞行轨迹可以是在实现上述的视觉里程计得到的飞行轨迹中标记出的部分轨迹。The optimal return route to the take-off point can be calculated by analyzing the flight trajectory during mission execution. For example, the flight path of exploring, trying, and repeating during missions can be bypassed. Wherein, the flight trajectory that can be bypassed may be a part of the trajectory marked in the flight trajectory obtained by implementing the above-mentioned visual odometer.
在一个实施例中，还可以结合GPS坐标，纠正为返航估计的飞行轨迹。并且可以进一步地通过定位传感器，在无人机到达起飞点附近之后，实现更加精准的返航。In one embodiment, the estimated flight track for the return flight can also be corrected in combination with the GPS coordinates. And it can further use the positioning sensor to achieve a more accurate return flight after the UAV reaches the vicinity of the take-off point.
在一个实施例中，无人机可以根据之前的记录所生成的视觉里程计确认最佳的返航路线，同时在返航的过程中开启避障功能。In one embodiment, the UAV can confirm the best return route based on the visual odometer generated by previous records, and at the same time enable the obstacle avoidance function during the return process.
另外，在实现飞行避障时，可以结合已知的2D/3D地图来确定飞行路线上的障碍物，例如确定在地图上已经标识的建筑物、山脉等障碍物，进而可以在确定飞行路径时选择绕过这些障碍物。为了更快地从目标设施中确定出检测对象，可以在目标设施上设定用于标识检测对象的标记，基于这些标记以及拍摄的图像，快速地从目标设施所在的图像区域中分割并定位出一个或多个检测对象。在获取深度图时，不仅可以基于双目视觉的方式获取，也可以使用类似激光雷达之类的装置来获取。In addition, when realizing flight obstacle avoidance, known 2D/3D maps can be combined to determine obstacles on the flight route, such as determining obstacles such as buildings and mountains that have been marked on the map, and then can be used when determining the flight path. Choose to go around these obstacles. In order to identify the detection object from the target facility more quickly, it is possible to set marks on the target facility to identify the detection object, based on these marks and the captured image, quickly segment and locate the target facility from the image area One or more detected objects. When obtaining the depth map, it can be obtained not only based on binocular vision, but also by using devices such as lidar.
本发明实施例在对某些设施进行巡检时，特别是对较高设施或者处于不易到达的地区的设施，能够基于图像识别和自动控制飞行的方式对设施中的一个或者多个需要检测的对象进行巡检，降低了巡检的人力成本以及安全性，提高了巡检的效率。In the embodiments of the present invention, when inspecting certain facilities, especially for facilities that are relatively tall or located in areas that are difficult to reach, one or more of the facilities that need to be detected can be detected based on image recognition and automatic flight control. Objects are inspected, which reduces the labor cost and safety of inspections, and improves the efficiency of inspections.
再请参见图2，是本发明实施例的一种基于飞行器的设施检测方法的流程示意图，本发明实施例的所述方法可以由一个专用的控制设备来执行，该控制设备可以配置在无人机等飞行器上。该控制设备也可以作为一个地面端设备，通过无线的方式与无人机等飞行器交互数据，进而完成对目标设施的巡检任务。Please refer to FIG. 2 again, which is a schematic flow chart of an aircraft-based facility detection method according to an embodiment of the present invention. The method described in this embodiment of the present invention can be performed by a dedicated control device, which can be configured in an unmanned Waiting for the aircraft. The control device can also be used as a ground-end device to exchange data with drones and other aircraft wirelessly, and then complete the inspection task of the target facility.
S201：当飞行器位于针对目标设施的检测位置时，获取包括所述目标设施的环境图像。该检测位置的作用主要在于：可以触发对目标设施的相关处理以便于完成对目标设施的巡检任务。S201: Acquire an environment image including the target facility when the aircraft is at a detection position for the target facility. The main function of the detection position is that it can trigger related processing on the target facility so as to complete the inspection task on the target facility.
所述检测位置可以是指位于某个位置区域中的位置点，该位置区域可以是指与目标设施的距离在一个预设的距离范围内的区域。控制设备基于检测到的所述飞行器的位置(例如GPS位置坐标)或所述飞行器上报的位置，并根据目标设施的位置，来确定飞行器是否到达针对目标设施的检测位置。The detection location may refer to a location point located in a certain location area, and the location area may refer to an area within a preset distance range from the target facility. The control device determines whether the aircraft has reached the detection position for the target facility based on the detected position of the aircraft (such as GPS position coordinates) or the position reported by the aircraft, and according to the position of the target facility.
是否到达检测位置也可以由飞行器自行判断，在一个实施例中，飞行器可以根据拍摄到的包括目标设施的图像进行分析，基于预设的目标设施的实际大小、目标设施在图像中的大小、目标设施在图像中的位置来估计飞行器与目标设施之间的距离，如果该距离在一个预设的距离范围内，则可以认为飞行器到底了对目标设施进行检测的检测位置。Whether it has reached the detection position can also be judged by the aircraft itself. In one embodiment, the aircraft can analyze the captured images including the target facility, based on the preset actual size of the target facility, the size of the target facility in the image, and the target facility. The position of the facility in the image is used to estimate the distance between the aircraft and the target facility. If the distance is within a preset distance range, it can be considered that the aircraft has reached the detection position for detecting the target facility.
该检测位置还可以是一个特定的位置，在该位置上得到的图像中包括所述目标设施，或者在该位置上能够对图像中包括的目标设施进行分割，例如，如果图像中，目标设施所占区域满足预设的条件(目标设施的像素点的个数大于预设的阈值、或者目标设施所占图像区域的面积大于预设的阈值)时，飞行器所在的位置即可以认为是检测位置。The detection position can also be a specific position, the image obtained at this position includes the target facility, or the target facility included in the image can be segmented at this position, for example, if in the image, the target facility When the occupied area meets the preset conditions (the number of pixels of the target facility is greater than the preset threshold, or the area of the image area occupied by the target facility is greater than the preset threshold), the position of the aircraft can be considered as the detection position.
无人机等飞行器上配置了摄像机等拍摄装置，到达检测位置后，触发拍摄装置采集环境图像，对于在S201中采集到的环境图像，主要用于确定出检测对象，并确定针对检测对象进行检测时所使用的飞行规则。The drone and other aerial vehicles are equipped with cameras and other shooting devices. After reaching the detection position, the shooting device is triggered to collect environmental images. The environmental images collected in S201 are mainly used to determine the detection object and determine the detection object. flight rules in use.
在一个实施例中，可以在显示地图的交互界面上配置一个或者多个待检测的设施位置点；将被选中的一个或者多个设施位置点所对应的设施确定为目标设施；根据选中的设施位置点控制飞行器向目标设施飞行，以便于飞行到针对目标设施的检测位置。如果用户在交互界面上选择了多个目标设施，可以控制飞行器由远及近或由近及远先后执行下述的步骤完成多个目标设施的巡检。或者基于飞行器的剩余能量，完成其中的一个或者其中的部分目标设施的巡检。如果控制设备为一个包括显示器的智能终端，例如智能手机、平板电脑等，则可以直接显示一个包括地图的交互界面给用户。如果控制设备挂载在飞行器上，则所述控制设备可以通过自带的无线通信接口，或者通过所述飞行器上的无线通信接口，发送指令以触发用于接收飞行器的数据的检测端显示交互界面，并接收在所述交互界面上确定的目标设施的位置，以控制飞行器飞行。In one embodiment, one or more facility location points to be detected can be configured on the interactive interface displaying the map; the facility corresponding to the selected one or more facility location points is determined as the target facility; according to the selected facility The position point controls the aircraft to fly to the target facility, so as to fly to the detection position for the target facility. If the user selects multiple target facilities on the interactive interface, the aircraft can be controlled to perform the following steps sequentially from far to near or from near to far to complete the inspection of multiple target facilities. Or based on the remaining energy of the aircraft, complete the inspection of one or part of the target facilities. If the control device is an intelligent terminal including a display, such as a smart phone, a tablet computer, etc., an interactive interface including a map can be directly displayed to the user. If the control device is mounted on the aircraft, the control device can send an instruction through its own wireless communication interface or through the wireless communication interface on the aircraft to trigger the detection end for receiving data from the aircraft to display an interactive interface , and receive the location of the target facility determined on the interaction interface, so as to control the flight of the aircraft.
S202：从所述环境图像中确定出所述目标设施所属的图像区域，并对该图像区域进行图像分割，得到关于所述目标设施的检测对象。确定目标设施所属的图像区域也可以是基于图像分割实现的。可以基于所述环境图像中像素的亮度及颜色进行图像分割，得到环境图像中所述目标设施所属的图像区域和目标设施的各个检测对象。S202: Determine the image area to which the target facility belongs from the environment image, and perform image segmentation on the image area to obtain detection objects about the target facility. Determining the image area to which the target facility belongs may also be realized based on image segmentation. Image segmentation may be performed based on the brightness and color of pixels in the environment image to obtain the image area to which the target facility belongs and each detection object of the target facility in the environment image.
所述检测对象可以是所述目标设施的全部，所述检测对象也是所述目标设施的部分部件，例如电塔的塔头、固定电力线的固定部件等等。检测对象由用户预先指定。例如，用户指定需要巡检整个电塔，那么在S202中，可以在得到电塔的图像区域后，将该图像区域中的整个电塔作为检测对象。用户也可以指定巡检电塔的塔头部分，在得到电塔的图像区域后，再分割得到塔头作为检测对象。The detection object may be the whole of the target facility, and the detection object is also a part of the target facility, such as a tower head of an electric tower, a fixed component for fixing a power line, and the like. The detection object is pre-specified by the user. For example, if the user specifies that the entire electric tower needs to be inspected, then in S202, after the image area of the electric tower is obtained, the entire electric tower in the image area can be used as a detection object. The user can also specify the tower head part of the inspection tower, and after obtaining the image area of the tower, segment the tower head as the detection object.
在对所述图像区域进行分割时，主要是基于为所述目标设施预设的对象模型来对所述图像区域进行分割，从图像区域的目标设施中确定一个或者多个检测对象。在一个实施例中，所述对该图像区域进行图像分割，得到关于所述目标设施的检测对象具体可以包括：获取为所述目标设施预设的对象模型；按照对象模型对图像区域进行图像分割，得到与所述对象模型之间的形状相似度满足相似度条件的检测对象。所述对象模型主要用于识别出目标设施上的某个组成部分，例如，预设的关于塔头的对象模型能够协助识别出电塔的塔头部分。可以预设不同角度的多个对象模型来对应一个检测对象，以便于在不同角度获取到的关于目标设施的图像时，均能够准确地分割确定出目标设施中的检测对象。进一步地，所述对象模型配置有模型标识，根据模型标识获取对应检测对象关联的巡检策略。该模型标识可以为一个名称，例如上述的关于塔头的对象模型的模型标识即为“塔头”。在基于对象模型识别出一个检测对象后，该检测对象的标识与对象模型对应，检测对象的标识可以与对象模型的模型标识相同。基于检测对象的标识，可以从预先设置的映射关系库中确定出与该检测对象的标识关联的巡检策略，这些巡检策略主要用于指示如何对检测对象进行巡检的巡检规则，包括环绕飞行环绕拍摄，远近持续视频拍摄，以及定点使用高精相机拍照等规则。在执行下述的S203时，可以进一步地基于巡检规则来获取关于所述检测对象的飞行规则。When segmenting the image area, the image area is mainly segmented based on an object model preset for the target facility, and one or more detection objects are determined from the target facility in the image area. In one embodiment, the performing image segmentation on the image area to obtain the detection object about the target facility may specifically include: acquiring an object model preset for the target facility; performing image segmentation on the image area according to the object model , to obtain the detection object whose shape similarity with the object model satisfies the similarity condition. The object model is mainly used to identify a certain component of the target facility, for example, the preset object model about the tower head can assist in identifying the tower head part of the electric tower. Multiple object models from different angles can be preset to correspond to one detection object, so that the detection object in the target facility can be accurately segmented and determined when images of the target facility are acquired at different angles. Further, the object model is configured with a model identifier, and the patrol policy associated with the corresponding detected object is acquired according to the model identifier. The model identifier may be a name, for example, the above-mentioned model identifier of the object model about the tower head is "tower head". After a detection object is identified based on the object model, the identifier of the detection object corresponds to the object model, and the identifier of the detection object may be the same as the model identifier of the object model. Based on the identification of the detection object, the inspection strategy associated with the identification of the detection object can be determined from the preset mapping relationship library. These inspection strategies are mainly used to indicate how to inspect the inspection rules for the detection object, including Surrounding flying and shooting, continuous video shooting from far and near, and fixed-point use of high-precision cameras to take pictures and other rules. When performing the following S203, flight rules about the detected object may be further obtained based on inspection rules.
S203：根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的飞行规则。图像位置可以是检测对象在图像中的像素位置，基于图像位置可以确定检测对象相对于飞行器的方位。可以以所述检测位置为起点，确定出能够从上方、下方等方位对检测对象进行检测的飞行规则，或者确定出环绕所述检测对象飞行的飞行规则。该飞行规则可以为一个飞行轨迹，例如控制无人机分飞行的飞行轨迹，基于该飞行轨迹，能够实现对S203: Acquire flight rules about the detection object according to the image position of the detection object in the environment image and the detection position. The image position may be a pixel position of the detection object in the image, and based on the image position, the orientation of the detection object relative to the aircraft may be determined. The detection position can be used as a starting point to determine the flight rules capable of detecting the detection object from above, below and other directions, or determine the flight rules for flying around the detection object. The flight rule can be a flight trajectory, such as the flight trajectory for controlling the sub-flight of the UAV. Based on the flight trajectory, it is possible to realize the
S204：根据所述飞行规则控制所述飞行器飞行，以便于完成对所述目标设施的检测。在确定出飞行规则后，控制飞行器按照该飞行规则飞行，即可完成对检测对象的巡检。S204: Control the flight of the aircraft according to the flight rules, so as to complete the detection of the target facility. After the flight rule is determined, the aircraft is controlled to fly according to the flight rule, and the inspection of the detection object can be completed.
在所述S201之前向目标设施飞行以到达检测位置的过程中、或者在S204控制所述飞行器飞行的过程中，可以实时或周期性地接收所述飞行器返回的位置信息，所述位置信息包括：由所述飞行器生成的所述飞行器相对于目标对象的距离信息和方向信息，或由所述飞行器返回的所述飞行器的位置坐标信息。根据接收的位置信息和所述目标设施的位置，在交互界面上实时显示所述飞行器和目标设施之间的相对位置。During the process of flying to the target facility to reach the detection position before S201, or during the process of controlling the flight of the aircraft in S204, the position information returned by the aircraft may be received in real time or periodically, and the position information includes: The distance information and direction information of the aircraft relative to the target object generated by the aircraft, or the position coordinate information of the aircraft returned by the aircraft. According to the received position information and the position of the target facility, the relative position between the aircraft and the target facility is displayed on the interactive interface in real time.
本发明实施例在对某些设施进行巡检时，特别是对较高设施或者处于不易到达的地区的设施，能够基于图像识别和自动控制飞行的方式对设施中的一个或者多个需要检测的对象进行巡检，降低了巡检的人力成本以及安全性，提高了巡检的效率。In the embodiments of the present invention, when inspecting certain facilities, especially for facilities that are relatively tall or located in areas that are difficult to reach, one or more of the facilities that need to be detected can be detected based on image recognition and automatic flight control. Objects are inspected, which reduces the labor cost and safety of inspections, and improves the efficiency of inspections.
再请参见图3，是本发明实施例的另一种基于飞行器的设施检测方法的流程示意图，本发明实施例的所述方法可以由一个专用的控制设备来执行，该控制设备可以配置在无人机等飞行器上。该控制设备也可以作为一个地面端设备，通过无线的方式与无人机等飞行器交互数据，进而完成对目标设施的巡检任务。Please refer to FIG. 3 again, which is a schematic flow chart of another aircraft-based facility detection method according to an embodiment of the present invention. The method described in this embodiment of the present invention can be executed by a dedicated control device, which can be configured in an On man-machines and other aircraft. The control device can also be used as a ground-end device to exchange data with drones and other aircraft wirelessly, and then complete the inspection task of the target facility.
S301：当飞行器位于针对目标设施的检测位置时，获取包括所述目标设施的环境图像。所述检测位置是在离所述目标设施在预设的距离范围内的其中一个位置。所述环境图像时所述飞行器上携带的拍摄头等传感器采集到的环境图像。S301: Acquire an environment image including the target facility when the aircraft is at a detection position for the target facility. The detection location is one of the locations within a preset distance from the target facility. The environmental image is an environmental image collected by a sensor such as a camera carried on the aircraft.
S302：从所述环境图像中确定出所述目标设施所属的图像区域，并对该图像区域进行图像分割，得到关于所述目标设施的检测对象。基于图像分割技术分割确定出所述图像区域，并基于预设的对象模型来分析确定出目标设施的检测对象。S302: Determine the image area to which the target facility belongs from the environment image, and perform image segmentation on the image area to obtain detection objects about the target facility. The image area is segmented and determined based on the image segmentation technology, and the detection object of the target facility is analyzed and determined based on the preset object model.
S303：根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的飞行规则。所述图像位置的作用在于：需要确保检测对象一直位于图像中。根据所述图像位置能够确定检测对象的相对方向，进一步地，在对该检测对象进行检测时，如果需要将当前需要检测的检测对象保持在画面中间，则在生成包括飞行轨迹的飞行规则时，考虑该图像位置。当需要检测的检测对象仅包括一个时，只需要生成针对该检测对象的飞行规则，例如生成环绕该检测对象的飞行轨迹。如果检测对象包括多个，则需要生成一个飞行规则，基于该飞行规则能够先后对多个检测对象进行检测。例如，对电塔的检测包括“塔头”、“塔身”、“塔脚”三个检测对象时，可以生成飞行规则，该飞行规则包括从检测位置开始，先沿着飞行规则中的飞行轨迹往上飞检测塔头，接着沿着该飞行轨迹往下飞检测塔身，最后沿着轨迹再到塔脚，从而在多段飞行轨迹上完成对“塔头”、“塔身”、“塔脚”三个检测对象的检测。S303: Acquire flight rules about the detection object according to the image position of the detection object in the environment image and the detection position. The effect of the image position is that it needs to ensure that the detection object is always located in the image. According to the image position, the relative direction of the detection object can be determined. Further, when the detection object is detected, if it is necessary to keep the detection object currently to be detected in the middle of the screen, when generating the flight rules including the flight trajectory, Consider this image location. When only one detection object needs to be detected, it is only necessary to generate flight rules for the detection object, for example, generate a flight trajectory around the detection object. If there are multiple detection objects, it is necessary to generate a flight rule based on which multiple detection objects can be detected successively. For example, when the detection of an electric tower includes three detection objects of "tower head", "tower body" and "tower foot", a flight rule can be generated. The flight rule includes starting from the detection position and flying along the flight rule. Fly up the trajectory to detect the tower head, then fly down along the flight trajectory to detect the tower body, and finally follow the trajectory to the tower foot, so as to complete the "tower head", "tower body" and "tower" on the multi-segment flight trajectory Foot" detection of three detection objects.
其中，所述飞行规则包括飞行轨迹，所述S303具体可以包括：获取与所述检测对象关联的巡检策略；根据所述检测对象在所述环境图像中的图像位置和所述检测位置，生成满足所述巡检策略的飞行轨迹。在检测对象仅为一个时，直接基于该检测对象对应的巡检策略即可生成一个飞行轨迹。例如，在一个简单的实施例中，检测对象在飞行器采集到的图像的中间位置，当巡检策略为远近持续视频拍摄，则可以生成以所述检测位置为起始点，生成一条从起始点到所述目标设施所在位置的一条直线轨迹，以便于飞行器能够由远及近地持续拍摄所述目标设施的检测对象。Wherein, the flight rule includes a flight trajectory, and the S303 may specifically include: obtaining an inspection policy associated with the detection object; generating The flight trajectory that satisfies the inspection strategy. When there is only one detection object, a flight trajectory can be generated directly based on the inspection strategy corresponding to the detection object. For example, in a simple embodiment, the detection object is in the middle of the image collected by the aircraft. When the inspection strategy is continuous video shooting from far and near, a line from the starting point to the A straight track of the location of the target facility, so that the aircraft can continuously photograph the detection object of the target facility from far to near.
如果得到的所述检测对象包括多个，则每一个检测对象均关联了巡检策略，所述S303具体可以包括：获取每一个检测对象的巡检策略；根据各检测对象在所述环境图像中的图像位置和所述检测位置，生成飞行规则；其中，所述飞行规则中包括满足所有巡检策略的飞行轨迹，或者包括多段飞行轨迹，每一段飞行轨迹满足部分巡检策略。If the obtained detected objects include multiple detected objects, each detected object is associated with an inspection strategy, and the S303 may specifically include: obtaining an inspection strategy for each detected object; The image position and the detection position are used to generate flight rules; wherein, the flight rules include flight trajectories that satisfy all inspection strategies, or include multiple flight trajectories, and each flight trajectory meets part of the inspection strategies.
在上述生成飞行规则时，还进一步基于预设的限制条件对飞行规则的生成进行约束。所述限制条件包括：基于飞行参数和检测参数设置的条件，所述飞行参数包括：飞行距离参数、飞行时长参数、飞行安全参数、能量损耗参数、飞行速度参数中的任意一种或多种。When the flight rules are generated above, the generation of the flight rules is further restricted based on the preset restriction conditions. The limiting conditions include: conditions set based on flight parameters and detection parameters, and the flight parameters include: any one or more of flight distance parameters, flight duration parameters, flight safety parameters, energy loss parameters, and flight speed parameters.
在一个实施例中，如果飞行距离参数被配置为1等有效数值时，表明在生成满足一个或者多个巡检策略的飞行轨迹时，进一步还要求飞行轨迹的总长度最短，使得飞行器的飞行距离最短，以节省能耗。当飞行时长参数被配置为1等有效数值时，表明在生成满足一个或者多个巡检策略的飞行轨迹时，进一步还要求飞行器按照所采用的飞行轨迹时以预设的速度飞行时，所耗费的时间最短，以提高巡检效率。当飞行安全参数被配置为1等有效数值时，表明优先选择安全的飞行轨迹，将一些可能存在障碍物的轨迹排除，例如在电塔作为目标设施时，排除掉可能会穿过电线的轨迹，以确保飞行安全。当能量损耗参数被配置为1等有效数值时，表明优先选择能耗低的轨迹作为最终的飞行轨迹。In one embodiment, if the flight distance parameter is configured as an effective value such as 1, it indicates that when generating flight trajectories satisfying one or more inspection strategies, the total length of the flight trajectories is further required to be the shortest, so that the flight distance of the aircraft The shortest to save energy. When the flight duration parameter is configured as an effective value such as 1, it indicates that when generating flight trajectories satisfying one or more inspection strategies, it is further required that the aircraft fly at a preset speed according to the adopted flight trajectories. The shortest time to improve inspection efficiency. When the flight safety parameter is configured as an effective value such as 1, it indicates that a safe flight trajectory is preferred, and some trajectories that may have obstacles are excluded. to ensure flight safety. When the energy loss parameter is configured as an effective value such as 1, it indicates that the trajectory with low energy consumption is preferred as the final flight trajectory.
S304：根据所述飞行规则控制所述飞行器飞行。在一个实施例中，还包括：生成检测参数，所述检测参数用于在控制所述飞行器飞行的过程中指示所述飞行器对检测对象进行巡检的感测参数，所述感测参数包括：用于对检测对象进行检测的传感器的拍摄角度参数、用于对检测对象进行拍摄的拍摄机的拍摄参数。例如，对于通过云台挂载拍摄机的无人机，检测参数具体可以是用于控制云台角度的参数，对拍摄机焦距、白平衡、快门等进行控制的参数。S304: Control the flight of the aircraft according to the flight rules. In one embodiment, it also includes: generating detection parameters, the detection parameters are used to instruct the aircraft to inspect the sensing parameters of the detection object during the process of controlling the flight of the aircraft, the sensing parameters include: The shooting angle parameters of the sensor used to detect the detection object, and the shooting parameters of the camera used to shoot the detection object. For example, for a UAV with a camera mounted on a gimbal, the detection parameters may specifically be parameters for controlling the angle of the gimbal, and parameters for controlling the focal length, white balance, shutter, etc. of the camera.
S305：在根据所述飞行规则控制所述飞行器飞行的过程中，获取检测得到的检测图像。根据所述飞行规则控制所述飞行器飞行的过程即为对检测对象进行巡检的过程，拍摄到图像或者根据图像生成的视频可以即时传递给巡检用户，巡检用户通过查看图像或者根据图像生成的视频来确定一个或者多个检测对象是否正常。S305: During the process of controlling the flight of the aircraft according to the flight rules, acquire a detected detection image. The process of controlling the flight of the aircraft according to the flight rules is the process of patrolling the detection object. The captured image or the video generated according to the image can be transmitted to the patrol user in real time. video to determine whether one or more detected objects are normal.
对于采集到的图像，控制设备还可以进一步地进行分析，确定当前检测的对象在检测图像中的位置。例如当前在检测电塔的塔头时，分析确定作为检测对象的塔头在检测图像中的位置。同样可以根据图像分割技术来确定当前检测的检测对象所在的图像区域，并进一步地确定该检测对象所在的像素位置。For the collected images, the control device can further analyze and determine the position of the currently detected object in the detected image. For example, when detecting the tower head of an electric tower, the analysis determines the position of the tower head as the detection object in the detection image. Similarly, the image region where the currently detected detection object is located can be determined according to the image segmentation technology, and the pixel position where the detection object is located can be further determined.
S306：根据检测图像中检测对象的位置和为检测对象设置的巡检策略，更新所述飞行规则。更新飞行规则主要为了保证能够检测到当前需要检测的检测对象，例如需要保证检测对象的位置是在检测图像的图像中心区域。在本发明实施例中，可以执行所述S306，和/或对上述提及的感测参数进行更新调整。或者先调整感测参数，如果无法满足预设的检测需求，例如无法保证检测对象的位置是在检测图像的图像中心区域，则执行所述S306，更新飞行规则(还可以进一步结合对感测参数的更新调整)，以满足预设的检测需求。S306: Update the flight rules according to the position of the detection object in the detection image and the patrol policy set for the detection object. The main purpose of updating the flight rules is to ensure that the detection object that needs to be detected can be detected, for example, it is necessary to ensure that the detection object is located in the image center area of the detection image. In the embodiment of the present invention, S306 may be executed, and/or the above-mentioned sensing parameters may be updated and adjusted. Or adjust the sensing parameters first, if the preset detection requirements cannot be met, for example, the position of the detection object cannot be guaranteed to be in the image center area of the detection image, then execute the S306 to update the flight rules (you can also further combine the sensing parameters update adjustment) to meet the preset detection requirements.
S307：根据更新后的飞行规则控制所述飞行器飞行，以便于完成对所述目标设施的检测。也就是说继续对检测对象进行检测得到对应的检测图像或者基于图像生成的视频，并返回给巡检用户查看。S307: Control the flight of the aircraft according to the updated flight rules, so as to complete the detection of the target facility. That is to say, continue to detect the detection object to obtain the corresponding detection image or video generated based on the image, and return it to the inspection user for viewing.
S308：在完成对所述目标设施的检测后，根据预设的返航轨迹控制所述飞行器返回；所述预设的返航轨迹包括：记录的在所述飞行器飞行至所述检测位置之前的飞行轨迹。S308: After the detection of the target facility is completed, control the aircraft to return according to the preset return trajectory; the preset return trajectory includes: the recorded flight trajectory of the aircraft before flying to the detection position .
本发明实施例在对某些设施进行巡检时，特别是对较高设施或者处于不易到达的地区的设施，能够基于图像识别和自动控制飞行的方式对设施中的一个或者多个需要检测的对象进行巡检，降低了巡检的人力成本以及安全性，提高了巡检的效率。并且还能够自动返航和避障，进一步地满足了巡检的自动化、智能化需求，提高了巡检的安全性。In the embodiments of the present invention, when inspecting certain facilities, especially for facilities that are relatively tall or located in areas that are difficult to reach, one or more of the facilities that need to be detected can be detected based on image recognition and automatic flight control. Objects are inspected, which reduces the labor cost and safety of inspections, and improves the efficiency of inspections. And it can also automatically return to the voyage and avoid obstacles, which further meets the automation and intelligent requirements of inspections and improves the safety of inspections.
再请参见图4，是本发明实施例的一种确定飞行规则的方法流程示意图，本发明实施例的所述方法包括如下步骤。Please refer to FIG. 4 again, which is a schematic flowchart of a method for determining flight rules in an embodiment of the present invention. The method in the embodiment of the present invention includes the following steps.
S401：根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的初始飞行规则。初始飞行规则包括一段或者多段飞行轨迹，在一个实施例中，在生成初始飞行规则时，除了考虑所述检测对象在环境图像中的图像位置和所述检测位置外，还进一步参考了上述提及的限制参数。S401: According to the image position of the detection object in the environment image and the detection position, acquire an initial flight rule about the detection object. The initial flight rules include one or more flight trajectories. In one embodiment, in addition to considering the image position of the detection object in the environment image and the detection position, the above mentioned limit parameters.
S402：获取所述飞行器的剩余能量值。所述剩余能量值包括无人机的剩余电量值等数据。S402: Obtain the remaining energy value of the aircraft. The remaining energy value includes data such as the remaining power value of the drone.
S403：根据所述剩余能量值对所述初始飞行规则进行调整，将调整后得到的规则作为关于所述检测对象飞行规则。根据剩余能量值确定能够支撑飞行器飞行的距离，如果不能够覆盖所述初始飞行规则中的飞行轨迹，则可以选择执行部分飞行轨迹，得到本次需要执行的飞行规则。在执行部分飞行轨迹对检测对象进行巡检后，自动记录所述初始飞行规则并记录已经执行的飞行轨迹，以便于下一次继续在该初始飞行规则的基础上，从已执行的部分飞行轨迹，重新确定在初始飞行规则上进行调整，生成新的包括飞行轨迹的飞行规则。S403: Adjust the initial flight rules according to the remaining energy value, and use the adjusted rules as the flight rules for the detection object. Determine the distance that can support the flight of the aircraft according to the remaining energy value. If the flight trajectory in the initial flight rules cannot be covered, you can choose to execute part of the flight trajectory to obtain the flight rules that need to be executed this time. After executing part of the flight track to inspect the detected object, automatically record the initial flight rules and record the flight track that has been executed, so that the next time you can continue on the basis of the initial flight rules, from the part of the flight track that has been executed, Re-determine adjustments to the initial flight rules to generate new flight rules including flight trajectories.
可以根据无人机的电池电量等情况来对飞行轨迹等进行智能的调整，进一步确保了巡检安全。The flight trajectory can be intelligently adjusted according to the battery power of the drone, which further ensures the safety of inspections.
本发明实施例还提供了一种计算机存储介质，该计算机存储介质中存储有程序指令，在执行这些程序指令时，实现上述图1、图2、图3或图4所对应实施例的相应方法。The embodiment of the present invention also provides a computer storage medium, the computer storage medium stores program instructions, and when these program instructions are executed, the corresponding methods of the above-mentioned embodiments corresponding to FIG. 1 , FIG. 2 , FIG. 3 or FIG. 4 are implemented. .
下面对本发明实施例的设施飞行器及控制设备进行描述。The facility aircraft and control equipment of the embodiments of the present invention are described below.
请参见图5，是本发明实施例的一种设施检测装置的结构示意图，本发明实施例的所述设施飞行器可以设置到无人机等可飞行的能够执行巡检任务的飞行器中。所述设施飞行器包括如下结构。Please refer to FIG. 5 , which is a schematic structural diagram of a facility inspection device according to an embodiment of the present invention. The facility aircraft in this embodiment of the present invention can be installed in a flying aircraft capable of performing inspection tasks such as drones. The facility aircraft includes the following structures.
获取模块501，用于当飞行器位于针对目标设施的检测位置时，获取包括所述目标设施的环境图像；确定模块502，用于从所述环境图像中确定出所述目标设施所属的图像区域，并对该图像区域进行图像分割，得到关于所述目标设施的检测对象；处理模块503，用于根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的飞行规则；控制模块504，用于根据所述飞行规则控制所述飞行器飞行，以便于完成对所述目标设施的检测。An acquisition module 501, configured to acquire an environment image including the target facility when the aircraft is at a detection position for the target facility; a determination module 502, configured to determine the image area to which the target facility belongs from the environment image, And image segmentation is performed on the image area to obtain the detection object of the target facility; the processing module 503 is configured to obtain the detection object's information about the detection object according to the image position of the detection object in the environment image and the detection position Flight rules; a control module 504, configured to control the flight of the aircraft according to the flight rules, so as to complete the detection of the target facility.
进一步可选地，所述飞行规则包括飞行轨迹，所述处理模块503，具体用于获取与所述检测对象关联的巡检策略；根据所述检测对象在所述环境图像中的图像位置和所述检测位置，生成满足所述巡检策略的飞行轨迹。Further optionally, the flight rule includes a flight trajectory, and the processing module 503 is specifically configured to obtain an inspection strategy associated with the detection object; according to the image position of the detection object in the environment image and the The detection position is used to generate a flight trajectory that satisfies the inspection strategy.
进一步可选地，得到的所述检测对象包括多个，每一个检测对象均关联了巡检策略，所述处理模块503，具体用于获取每一个检测对象的巡检策略；根据各检测对象在所述环境图像中的图像位置和所述检测位置，生成飞行规则；其中，所述飞行规则中包括满足所有巡检策略的飞行轨迹，或者包括多段飞行轨迹，每一段飞行轨迹满足部分巡检策略。Further optionally, the obtained detection objects include multiple detection objects, and each detection object is associated with an inspection strategy, and the processing module 503 is specifically used to obtain the inspection strategy of each detection object; The image position in the environment image and the detection position generate flight rules; wherein, the flight rules include flight trajectories that satisfy all inspection strategies, or include multiple flight trajectories, and each flight trajectory satisfies a part of the inspection strategy .
进一步可选地，生成的飞行规则还满足预设的限制参数；所述限制参数包括：飞行距离参数、飞行时长参数、飞行安全参数、能量损耗参数中的任意一种或多种。Further optionally, the generated flight rules also meet preset limit parameters; the limit parameters include: any one or more of flight distance parameters, flight duration parameters, flight safety parameters, and energy loss parameters.
进一步可选地，所述装置还可以包括：生成模块505，用于生成检测参数，所述检测参数用于在控制所述飞行器飞行的过程中指示所述飞行器对检测对象进行巡检的感测参数，所述感测参数包括：用于对检测对象进行检测的传感器的拍摄角度参数、用于对检测对象进行拍摄的拍摄机的拍摄参数。Further optionally, the device may further include: a generation module 505, configured to generate a detection parameter, the detection parameter is used to instruct the aircraft to inspect the detection object during the process of controlling the flight of the aircraft; The sensing parameters include: shooting angle parameters of the sensor used to detect the detection object, and shooting parameters of the camera used to shoot the detection object.
进一步可选地，所述处理模块503，还用于在根据所述飞行规则控制所述飞行器飞行的过程中，获取检测得到的检测图像；根据检测图像中检测对象的位置和为检测对象设置的巡检策略，更新所述飞行规则；根据更新后的飞行规则控制所述飞行器飞行，以便于完成对所述目标设施的检测。Further optionally, the processing module 503 is also configured to acquire a detected detection image during the process of controlling the flight of the aircraft according to the flight rules; according to the position of the detection object in the detection image and the An inspection strategy, updating the flight rules; controlling the flight of the aircraft according to the updated flight rules, so as to complete the detection of the target facility.
进一步可选地，所述处理模块503，具体用于根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的初始飞行规则；获取所述飞行器的剩余能量值；根据所述剩余能量值对所述初始飞行规则进行调整，将调整后得到的规则作为关于所述检测对象飞行规则。Further optionally, the processing module 503 is specifically configured to obtain initial flight rules about the detection object according to the image position of the detection object in the environment image and the detection position; obtain the remaining energy of the aircraft value; adjust the initial flight rules according to the remaining energy value, and use the adjusted rules as the flight rules for the detection object.
进一步可选地，所述确定模块502，具体用于获取为所述目标设施预设的对象模型；按照对象模型对图像区域进行图像分割，得到与所述对象模型之间的形状相似度满足相似度条件的检测对象。Further optionally, the determining module 502 is specifically configured to obtain an object model preset for the target facility; perform image segmentation on the image region according to the object model, and obtain a shape similarity with the object model that satisfies similarity The detection object of the degree condition.
进一步可选地，所述对象模型配置有模型标识，根据模型标识获取与检测对象关联的巡检策略。Further optionally, the object model is configured with a model identifier, and the inspection policy associated with the detected object is acquired according to the model identifier.
进一步可选地，所述装置还可以包括：设置模块506，用于在显示地图的交互界面上配置一个或者多个待检测的设施位置点；将被选中的一个或者多个设施位置点所对应的设施确定为目标设施；根据选中的设施位置点控制飞行器向目标设施飞行，以便于飞行到针对目标设施的检测位置。Further optionally, the device may further include: a setting module 506, configured to configure one or more facility location points to be detected on the interactive interface displaying the map; the one or more facility location points to be selected correspond to The facility is determined as the target facility; the aircraft is controlled to fly to the target facility according to the selected facility location point, so as to fly to the detection position for the target facility.
进一步可选地，所述装置还可以包括：接收模块507，用于接收所述飞行器返回的位置信息，所述位置信息包括：由所述飞行器生成的所述飞行器相对于目标对象的距离信息和方向信息，或由所述飞行器返回的所述飞行器的位置坐标信息。Further optionally, the device may further include: a receiving module 507, configured to receive position information returned by the aircraft, where the position information includes: distance information generated by the aircraft relative to the target object and direction information, or position coordinate information of the aircraft returned by the aircraft.
进一步可选地，所述控制模块504，还用于在飞行器向目标设施飞行过程中，控制飞行器按照指定规则向目标设施飞行，所述指定规则用于指示所述飞行器飞行到用于获取深度图的至少两个能够以不同角度拍摄的拍摄位置；基于至少两个拍摄位置来获取所述飞行器在行进方向上的深度图；根据获取到的深度图进行飞行避障处理。Further optionally, the control module 504 is further configured to control the aircraft to fly to the target facility according to specified rules during the flight process of the aircraft to the target facility, and the specified rules are used to instruct the aircraft to fly to the target facility for obtaining the depth map. At least two shooting positions that can be shot at different angles; based on the at least two shooting positions, the depth map of the aircraft in the direction of travel is obtained; and the flight obstacle avoidance process is performed according to the obtained depth map.
进一步可选地，所述控制模块504，还用于在完成对所述目标设施的检测后，根据预设的返航轨迹控制所述飞行器返回；所述预设的返航轨迹包括：记录的在所述飞行器飞行至所述检测位置之前的飞行轨迹。Further optionally, the control module 504 is further configured to control the return of the aircraft according to a preset return trajectory after the detection of the target facility is completed; the preset return trajectory includes: the recorded The flight path before the aircraft flies to the detection position.
本发明实施例在对某些设施进行巡检时，特别是对较高设施或者处于不易到达的地区的设施，能够基于图像识别和自动控制飞行的方式对设施中的一个或者多个需要检测的对象进行巡检，降低了巡检的人力成本以及安全性，提高了巡检的效率。并且还能够自动返航和避障，进一步地满足了巡检的自动化、智能化需求，提高了巡检的安全性。In the embodiments of the present invention, when inspecting certain facilities, especially for facilities that are relatively tall or located in areas that are difficult to reach, one or more of the facilities that need to be detected can be detected based on image recognition and automatic flight control. Objects are inspected, which reduces the labor cost and safety of inspections, and improves the efficiency of inspections. And it can also automatically return to the voyage and avoid obstacles, which further meets the automation and intelligent requirements of inspections and improves the safety of inspections.
再请参见图6，是本发明实施例的一种控制设备的结构示意图，本发明实施例的所述控制设备包括供电电路，该控制设备可以由一块单独的电池供电，也可以通过一个供电接口由无人机等飞行器的电池供电。所述控制设备还可包括处理器601、数据接口602以及存储器603。Please refer to Fig. 6 again, which is a schematic structural diagram of a control device according to an embodiment of the present invention. The control device according to this embodiment of the present invention includes a power supply circuit. The control device can be powered by a separate battery or through a power supply interface Powered by the battery of an aerial vehicle such as a drone. The control device may further include a processor 601 , a data interface 602 and a memory 603 .
所述数据接口602主要用于与飞行器交互数据，或者进一步地，所述数据接口602还可以与地面的用于接收并显示由飞行器检测到的图像等数据的监控设备之间交互数据。The data interface 602 is mainly used to exchange data with the aircraft, or further, the data interface 602 can also exchange data with monitoring equipment on the ground for receiving and displaying data such as images detected by the aircraft.
所述存储器603可以包括易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储器603也可以包括非易失性存储器(non-volatilememory)，例如快闪存储器(flash memory)，硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)；存储器603还可以包括上述种类的存储器的组合。The memory 603 may include a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); the memory 603 may also include a non-volatile memory (non-volatile memory), such as a flash memory ( flash memory), hard disk (hard disk drive, HDD) or solid-state disk (solid-state drive, SSD); the storage 603 may also include a combination of the above types of storage.
所述处理器601可以是中央处理器(central processing unit，CPU)。所述处理器601还可以进一步包括硬件芯片。上述硬件芯片可以是专用集成电路(application-specific integrated circuit，ASIC)，可编程逻辑器件(programmable logic device，PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logicdevice，CPLD)，现场可编程逻辑门阵列(field-programmable gate array，FPGA)，通用阵列逻辑(generic array logic，GAL)或其任意组合。The processor 601 may be a central processing unit (central processing unit, CPU). The processor 601 may further include a hardware chip. The aforementioned hardware chip may be an application-specific integrated circuit (application-specific integrated circuit, ASIC), a programmable logic device (programmable logic device, PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a general array logic (generic array logic, GAL) or any combination thereof.
可选地，所述存储器603还用于存储程序指令。所述处理器601可以调用所述程序指令，实现如本申请图1，2，3和4所对应实施例中所示的设施检测方法。Optionally, the memory 603 is also used to store program instructions. The processor 601 can invoke the program instructions to implement the facility detection method shown in the embodiments corresponding to FIGS. 1 , 2 , 3 and 4 of the present application.
在一个实施例中，所述处理器601，用于当飞行器位于针对目标设施的检测位置时，获取包括所述目标设施的环境图像；从所述环境图像中确定出所述目标设施所属的图像区域，并对该图像区域进行图像分割，得到关于所述目标设施的检测对象；根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的飞行规则；根据所述飞行规则生成控制指令，并通过所述数据接口602发送给所述飞行器以控制所述飞行器飞行，以便于完成对所述目标设施的检测。In one embodiment, the processor 601 is configured to acquire an environment image including the target facility when the aircraft is at a detection position for the target facility; determine the image to which the target facility belongs from the environment image area, and perform image segmentation on the image area to obtain the detection object of the target facility; according to the image position of the detection object in the environment image and the detection position, obtain the flight rules of the detection object; according to The flight rules generate control instructions and send them to the aircraft through the data interface 602 to control the flight of the aircraft so as to complete the detection of the target facility.
可选地，所述飞行规则包括飞行轨迹，所述处理器601，用于获取与所述检测对象关联的巡检策略；根据所述检测对象在所述环境图像中的图像位置和所述检测位置，生成满足所述巡检策略的飞行轨迹。Optionally, the flight rule includes a flight trajectory, and the processor 601 is configured to obtain an inspection policy associated with the detected object; according to the image position of the detected object in the environment image and the detected position, and generate a flight trajectory that satisfies the inspection strategy.
可选地，得到的所述检测对象包括多个，每一个检测对象均关联了巡检策略，所述处理器601，用于获取每一个检测对象的巡检策略；根据各检测对象在所述环境图像中的图像位置和所述检测位置，生成飞行规则；其中，所述飞行规则中包括满足所有巡检策略的飞行轨迹，或者包括多段飞行轨迹，每一段飞行轨迹满足部分巡检策略。Optionally, the obtained detected objects include multiple detected objects, and each detected object is associated with an inspection strategy, and the processor 601 is configured to obtain the inspection strategy of each detected object; according to each detected object in the The image position in the environment image and the detection position generate flight rules; wherein, the flight rules include flight trajectories satisfying all inspection strategies, or include multiple flight trajectories, and each flight trajectory satisfies part of the inspection strategies.
可选地，生成的飞行规则还满足预设的限制参数；所述限制参数包括：飞行距离参数、飞行时长参数、飞行安全参数、能量损耗参数中的任意一种或多种。Optionally, the generated flight rules also meet preset limit parameters; the limit parameters include: any one or more of flight distance parameters, flight duration parameters, flight safety parameters, and energy loss parameters.
可选地，所述处理器601，还用于生成检测参数，并通过所述数据接口602将所述检测参数发送给飞行器，所述检测参数用于在控制所述飞行器飞行的过程中指示所述飞行器对检测对象进行巡检的感测参数，所述感测参数包括：用于对检测对象进行检测的传感器的拍摄角度参数、用于对检测对象进行拍摄的拍摄机的拍摄参数。Optionally, the processor 601 is also configured to generate detection parameters and send the detection parameters to the aircraft through the data interface 602, the detection parameters are used to indicate the The sensing parameters for the aircraft to inspect the detection object, the sensing parameters include: the shooting angle parameters of the sensor for detecting the detection object, and the shooting parameters of the camera for shooting the detection object.
可选地，所述处理器601，还用于在根据所述飞行规则控制所述飞行器飞行的过程中，获取检测得到的检测图像；根据检测图像中检测对象的位置和为检测对象设置的巡检策略，更新所述飞行规则；根据更新后的飞行规则控制所述飞行器飞行，以便于完成对所述目标设施的检测。Optionally, the processor 601 is further configured to acquire a detected detection image during the process of controlling the flight of the aircraft according to the flight rules; The inspection strategy is updated, and the flight rules are updated; the flight of the aircraft is controlled according to the updated flight rules, so as to complete the inspection of the target facility.
可选地，所述处理器601，用于根据检测对象在所述环境图像中的图像位置和所述检测位置，获取关于所述检测对象的初始飞行规则；获取所述飞行器的剩余能量值；根据所述剩余能量值对所述初始飞行规则进行调整，将调整后得到的规则作为关于所述检测对象飞行规则。Optionally, the processor 601 is configured to obtain an initial flight rule about the detection object according to the image position of the detection object in the environment image and the detection position; obtain the remaining energy value of the aircraft; The initial flight rules are adjusted according to the remaining energy value, and the adjusted rules are used as the flight rules for the detection object.
可选地，所述处理器601，用于获取为所述目标设施预设的对象模型；按照对象模型对图像区域进行图像分割，得到与所述对象模型之间的形状相似度满足相似度条件的检测对象。Optionally, the processor 601 is configured to acquire an object model preset for the target facility; perform image segmentation on the image region according to the object model, and obtain a shape similarity with the object model that satisfies a similarity condition detection object.
可选地，所述对象模型配置有模型标识，根据模型标识获取与检测对象关联的巡检策略。Optionally, the object model is configured with a model identifier, and the patrol policy associated with the detected object is obtained according to the model identifier.
可选地，所述处理器601，还用于在显示地图的交互界面上配置一个或者多个待检测的设施位置点；将被选中的一个或者多个设施位置点所对应的设施确定为目标设施；根据选中的设施位置点控制飞行器向目标设施飞行，以便于飞行到针对目标设施的检测位置。Optionally, the processor 601 is further configured to configure one or more facility location points to be detected on the interactive interface displaying the map; and determine the facility corresponding to the selected one or more facility location points as the target Facility: Control the aircraft to fly to the target facility according to the selected facility location point, so as to fly to the detection position for the target facility.
可选地，所述处理器601，还用于接收所述飞行器返回的位置信息，所述位置信息包括：由所述飞行器生成的所述飞行器相对于目标对象的距离信息和方向信息，或由所述飞行器返回的所述飞行器的位置坐标信息。Optionally, the processor 601 is further configured to receive position information returned by the aircraft, where the position information includes: distance information and direction information of the aircraft relative to the target object generated by the aircraft, or information generated by the aircraft The position coordinate information of the aircraft returned by the aircraft.
可选地，所述处理器601，还用于在飞行器向目标设施飞行过程中，控制飞行器按照指定规则向目标设施飞行，所述指定规则用于指示所述飞行器飞行到用于获取深度图的至少两个能够以不同角度拍摄的拍摄位置；基于至少两个拍摄位置来获取所述飞行器在行进方向上的深度图；根据获取到的深度图进行飞行避障处理。Optionally, the processor 601 is further configured to control the aircraft to fly to the target facility according to a specified rule during the flight of the aircraft to the target facility, and the specified rule is used to instruct the aircraft to fly to the location for obtaining the depth map. At least two shooting positions that can be shot at different angles; based on the at least two shooting positions, a depth map of the aircraft in the direction of travel is obtained; and flight obstacle avoidance processing is performed according to the obtained depth map.
可选地，所述处理器601，还用于在完成对所述目标设施的检测后，根据预设的返航轨迹控制所述飞行器返回；所述预设的返航轨迹包括：记录的在所述飞行器飞行至所述检测位置之前的飞行轨迹。Optionally, the processor 601 is further configured to control the return of the aircraft according to a preset return trajectory after the detection of the target facility is completed; the preset return trajectory includes: The flight track before the aircraft flies to the detection position.
本发明实施例在对某些设施进行巡检时，特别是对较高设施或者处于不易到达的地区的设施，能够基于图像识别和自动控制飞行的方式对设施中的一个或者多个需要检测的对象进行巡检，降低了巡检的人力成本以及安全性，提高了巡检的效率。并且还能够自动返航和避障，进一步地满足了巡检的自动化、智能化需求，提高了巡检的安全性。In the embodiments of the present invention, when inspecting certain facilities, especially for facilities that are relatively tall or located in areas that are difficult to reach, one or more of the facilities that need to be detected can be detected based on image recognition and automatic flight control. Objects are inspected, which reduces the labor cost and safety of inspections, and improves the efficiency of inspections. And it can also automatically return to the voyage and avoid obstacles, which further meets the automation and intelligent requirements of inspections and improves the safety of inspections.
以上所揭露的仅为本发明部分实施例而已，当然不能以此来限定本发明之权利范围，因此依本发明权利要求所作的等同变化，仍属本发明所涵盖的范围。The above disclosures are only some embodiments of the present invention, which certainly cannot limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111071109.0ACN113791641B (en) | 2017-04-28 | 2017-04-28 | Facility detection method and control equipment based on aircraft |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/082501WO2018195955A1 (en) | 2017-04-28 | 2017-04-28 | Aircraft-based facility detection method and control device |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111071109.0ADivisionCN113791641B (en) | 2017-04-28 | 2017-04-28 | Facility detection method and control equipment based on aircraft |
| Publication Number | Publication Date |
|---|---|
| CN108496129Atrue CN108496129A (en) | 2018-09-04 |
| CN108496129B CN108496129B (en) | 2021-10-01 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780004504.2AActiveCN108496129B (en) | 2017-04-28 | 2017-04-28 | An aircraft-based facility detection method and control device |
| CN202111071109.0AActiveCN113791641B (en) | 2017-04-28 | 2017-04-28 | Facility detection method and control equipment based on aircraft |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111071109.0AActiveCN113791641B (en) | 2017-04-28 | 2017-04-28 | Facility detection method and control equipment based on aircraft |
| Country | Link |
|---|---|
| CN (2) | CN108496129B (en) |
| WO (1) | WO2018195955A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109358650A (en)* | 2018-12-14 | 2019-02-19 | 国网冀北电力有限公司检修分公司 | Inspection path planning method, device, unmanned aerial vehicle and computer-readable storage medium |
| CN109471447A (en)* | 2018-12-14 | 2019-03-15 | 国网冀北电力有限公司检修分公司 | UAV navigation method, device, UAV and data readable storage device |
| CN109885083A (en)* | 2019-03-06 | 2019-06-14 | 国网陕西省电力公司检修公司 | Flying platform and inspection method for fine inspection of transmission lines based on lidar |
| CN109885098A (en)* | 2019-04-11 | 2019-06-14 | 株洲时代电子技术有限公司 | A kind of bridge sidebar inspection flight course planning method |
| CN111401146A (en)* | 2020-02-26 | 2020-07-10 | 长江大学 | Unmanned aerial vehicle power inspection method, device and storage medium |
| CN111582117A (en)* | 2020-04-29 | 2020-08-25 | 长江大学 | Unmanned aerial vehicle illegal building inspection method, equipment and storage medium |
| CN112068591A (en)* | 2020-08-25 | 2020-12-11 | 中国南方电网有限责任公司超高压输电公司天生桥局 | Unmanned aerial vehicle for automatic inspection of power transmission line, control method and device and storage medium |
| CN112119361A (en)* | 2019-09-23 | 2020-12-22 | 深圳市大疆创新科技有限公司 | Returning control method and device for movable platform and movable platform |
| CN112233270A (en)* | 2020-10-30 | 2021-01-15 | 国家电网有限公司 | Unmanned aerial vehicle is intelligence around tower system of patrolling and examining independently |
| CN112236727A (en)* | 2019-07-30 | 2021-01-15 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle simulated flight method and device and recording medium |
| CN113014904A (en)* | 2021-02-24 | 2021-06-22 | 苏州臻迪智能科技有限公司 | Method, device and system for processing inspection image of unmanned aerial vehicle and storage medium |
| CN113112098A (en)* | 2021-05-12 | 2021-07-13 | 上海野戈智能科技发展有限公司 | Building defect detection method and device |
| CN113625730A (en)* | 2021-06-30 | 2021-11-09 | 南京邮电大学 | Four-rotor self-adaptive fault-tolerant control method based on overtorque sliding mode |
| CN113741413A (en)* | 2020-05-29 | 2021-12-03 | 广州极飞科技股份有限公司 | Operation method of unmanned equipment, unmanned equipment and storage medium |
| CN113872680A (en)* | 2021-12-03 | 2021-12-31 | 上海特金信息科技有限公司 | TDOA (time difference of arrival) auxiliary RID (Rich Internet protocol) signal receiving control method, device and system |
| CN114326794A (en)* | 2021-12-13 | 2022-04-12 | 广东省建设工程质量安全检测总站有限公司 | Curtain wall defect identification method, control terminal, server and readable storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114636424B (en)* | 2019-02-21 | 2024-04-19 | 国网浙江省电力有限公司平湖市供电公司 | Substation inspection path planning method based on wearable devices |
| CN110307837B (en)* | 2019-07-22 | 2023-04-18 | 湖南中图通无人机技术有限责任公司 | Unmanned aerial vehicle navigation system and method based on image recognition |
| CN112262319A (en)* | 2019-08-30 | 2021-01-22 | 深圳市大疆创新科技有限公司 | Detection method of power line, millimeter wave radar, system, and storage medium |
| WO2021223125A1 (en)* | 2020-05-06 | 2021-11-11 | 深圳市大疆创新科技有限公司 | Patrol inspection method and system, unmanned aerial vehicle, ground control platform, and storage medium |
| CN112180955B (en)* | 2020-08-26 | 2024-02-20 | 国网安徽省电力有限公司淮南供电公司 | Secondary review method and system based on visual feedback for automatic inspection drones |
| CN112229845A (en)* | 2020-10-12 | 2021-01-15 | 国网河南省电力公司濮阳供电公司 | High-precision Rao Tower intelligent inspection method for UAV based on visual navigation technology |
| CN112327920B (en)* | 2020-11-16 | 2023-07-14 | 国网新疆电力有限公司检修公司 | A UAV autonomous obstacle avoidance inspection path planning method and device |
| CN112788292B (en)* | 2020-12-28 | 2023-05-26 | 深圳市朗驰欣创科技股份有限公司 | Method and device for determining inspection observation points, inspection robot and storage medium |
| CN114265407A (en)* | 2021-07-09 | 2022-04-01 | 上海宝升实业有限公司 | Construction machinery equipment inspection service system |
| DE102021123124A1 (en) | 2021-09-07 | 2023-03-09 | Spleenlab GmbH | Method for controlling an unmanned aerial vehicle for an inspection flight to inspect an object and inspection unmanned aerial vehicle |
| CN113938609B (en)* | 2021-11-04 | 2023-08-22 | 中国联合网络通信集团有限公司 | Regional monitoring method, device and equipment |
| CN115052133B (en)* | 2022-07-06 | 2023-09-12 | 国网江苏省电力有限公司南通市通州区供电分公司 | Unmanned aerial vehicle-based power distribution rack acceptance method |
| CN115275870B (en)* | 2022-09-28 | 2022-12-06 | 合肥优晟电力科技有限公司 | Inspection system based on high-altitude line maintenance |
| CN115716540A (en)* | 2022-10-08 | 2023-02-28 | 深圳市金精博科技有限公司 | Intelligent device for testing fatigue resistance of flight rudder wing |
| CN116225062B (en)* | 2023-03-14 | 2024-01-16 | 广州天勤数字科技有限公司 | Unmanned aerial vehicle navigation method applied to bridge inspection and unmanned aerial vehicle |
| CN116878518B (en)* | 2023-09-06 | 2023-11-21 | 滨州市华亿电器设备有限公司 | Unmanned aerial vehicle inspection path planning method for urban power transmission line maintenance |
| CN116909318B (en)* | 2023-09-14 | 2023-11-24 | 众芯汉创(江苏)科技有限公司 | Unmanned aerial vehicle autonomous routing inspection route planning system based on high-precision three-dimensional point cloud |
| CN117389293B (en)* | 2023-10-31 | 2024-05-24 | 广州天海翔航空科技有限公司 | Flight control management method and system for inspection unmanned aerial vehicle |
| CN117428774B (en)* | 2023-11-23 | 2024-06-21 | 中国船舶集团有限公司第七一六研究所 | Industrial robot control method and system for ship inspection |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102589524A (en)* | 2011-01-13 | 2012-07-18 | 华北电网有限公司北京超高压公司 | A power line inspection method |
| CN102941920A (en)* | 2012-12-05 | 2013-02-27 | 南京理工大学 | High-tension transmission line inspection robot based on multi-rotor aircraft and method using robot |
| CN103196430A (en)* | 2013-04-27 | 2013-07-10 | 清华大学 | Mapping navigation method and system based on flight path and visual information of unmanned aerial vehicle |
| KR20130127822A (en)* | 2012-05-15 | 2013-11-25 | 한국전자통신연구원 | Apparatus and method of processing heterogeneous sensor fusion for classifying and positioning object on road |
| CN104298248A (en)* | 2014-10-08 | 2015-01-21 | 南京航空航天大学 | Accurate visual positioning and orienting method for rotor wing unmanned aerial vehicle |
| CN105551032A (en)* | 2015-12-09 | 2016-05-04 | 国网山东省电力公司电力科学研究院 | Pole image collection system and method based on visual servo |
| CN105678289A (en)* | 2016-03-07 | 2016-06-15 | 谭圆圆 | Control method and device of unmanned aerial vehicle |
| CN105787447A (en)* | 2016-02-26 | 2016-07-20 | 深圳市道通智能航空技术有限公司 | Method and system of unmanned plane omnibearing obstacle avoidance based on binocular vision |
| CN106054931A (en)* | 2016-07-29 | 2016-10-26 | 北方工业大学 | Unmanned aerial vehicle fixed-point flight control system based on visual positioning |
| CN106127788A (en)* | 2016-07-04 | 2016-11-16 | 触景无限科技（北京）有限公司 | A kind of vision barrier-avoiding method and device |
| CN106228862A (en)* | 2016-09-28 | 2016-12-14 | 国家电网公司 | Emulation training method patrolled and examined by a kind of power transmission line unmanned machine |
| CN205920057U (en)* | 2015-09-29 | 2017-02-01 | 柳州欧维姆机械股份有限公司 | Detect fissured many rotor unmanned aerial vehicle testing platform system in structure surface |
| CN106468918A (en)* | 2015-08-18 | 2017-03-01 | 航天图景（北京）科技有限公司 | A kind of standardized data acquisition method of line data-logging and system |
| CN106477038A (en)* | 2016-12-20 | 2017-03-08 | 北京小米移动软件有限公司 | Image capturing method and device, unmanned plane |
| CN106504362A (en)* | 2016-10-18 | 2017-03-15 | 国网湖北省电力公司检修公司 | Inspection method of power transmission and transformation system based on UAV |
| CN106595631A (en)* | 2016-10-25 | 2017-04-26 | 纳恩博（北京）科技有限公司 | Method for avoiding obstacles and electronic equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101477169B (en)* | 2009-01-16 | 2011-07-13 | 华北电力大学 | Electric power circuit detection method by polling flying robot |
| US9459889B2 (en)* | 2014-05-19 | 2016-10-04 | Qualcomm Incorporated | Systems and methods for context-aware application control |
| CN104049641B (en)* | 2014-05-29 | 2017-08-25 | 深圳市大疆创新科技有限公司 | A kind of automatic landing method, device and aircraft |
| CN104035446B (en)* | 2014-05-30 | 2017-08-25 | 深圳市大疆创新科技有限公司 | The course generation method and system of unmanned plane |
| CN105023014B (en)* | 2015-08-21 | 2018-11-23 | 马鞍山市安工大工业技术研究院有限公司 | A kind of shaft tower target extraction method in unmanned plane inspection transmission line of electricity image |
| CN105955308B (en)* | 2016-05-20 | 2018-06-29 | 腾讯科技（深圳）有限公司 | The control method and device of a kind of aircraft |
| CN106054924B (en)* | 2016-07-06 | 2019-08-30 | 北京大为远达科技发展有限公司 | A kind of unmanned plane accompanying flying method, accompanying flying device and accompanying flying system |
| CN106354156A (en)* | 2016-09-29 | 2017-01-25 | 腾讯科技（深圳）有限公司 | Method and device for tracking target object, and air vehicle |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102589524A (en)* | 2011-01-13 | 2012-07-18 | 华北电网有限公司北京超高压公司 | A power line inspection method |
| KR20130127822A (en)* | 2012-05-15 | 2013-11-25 | 한국전자통신연구원 | Apparatus and method of processing heterogeneous sensor fusion for classifying and positioning object on road |
| CN102941920A (en)* | 2012-12-05 | 2013-02-27 | 南京理工大学 | High-tension transmission line inspection robot based on multi-rotor aircraft and method using robot |
| CN103196430A (en)* | 2013-04-27 | 2013-07-10 | 清华大学 | Mapping navigation method and system based on flight path and visual information of unmanned aerial vehicle |
| CN104298248A (en)* | 2014-10-08 | 2015-01-21 | 南京航空航天大学 | Accurate visual positioning and orienting method for rotor wing unmanned aerial vehicle |
| CN106468918A (en)* | 2015-08-18 | 2017-03-01 | 航天图景（北京）科技有限公司 | A kind of standardized data acquisition method of line data-logging and system |
| CN205920057U (en)* | 2015-09-29 | 2017-02-01 | 柳州欧维姆机械股份有限公司 | Detect fissured many rotor unmanned aerial vehicle testing platform system in structure surface |
| CN105551032A (en)* | 2015-12-09 | 2016-05-04 | 国网山东省电力公司电力科学研究院 | Pole image collection system and method based on visual servo |
| CN105787447A (en)* | 2016-02-26 | 2016-07-20 | 深圳市道通智能航空技术有限公司 | Method and system of unmanned plane omnibearing obstacle avoidance based on binocular vision |
| CN105678289A (en)* | 2016-03-07 | 2016-06-15 | 谭圆圆 | Control method and device of unmanned aerial vehicle |
| CN106127788A (en)* | 2016-07-04 | 2016-11-16 | 触景无限科技（北京）有限公司 | A kind of vision barrier-avoiding method and device |
| CN106054931A (en)* | 2016-07-29 | 2016-10-26 | 北方工业大学 | Unmanned aerial vehicle fixed-point flight control system based on visual positioning |
| CN106228862A (en)* | 2016-09-28 | 2016-12-14 | 国家电网公司 | Emulation training method patrolled and examined by a kind of power transmission line unmanned machine |
| CN106504362A (en)* | 2016-10-18 | 2017-03-15 | 国网湖北省电力公司检修公司 | Inspection method of power transmission and transformation system based on UAV |
| CN106595631A (en)* | 2016-10-25 | 2017-04-26 | 纳恩博（北京）科技有限公司 | Method for avoiding obstacles and electronic equipment |
| CN106477038A (en)* | 2016-12-20 | 2017-03-08 | 北京小米移动软件有限公司 | Image capturing method and device, unmanned plane |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109471447A (en)* | 2018-12-14 | 2019-03-15 | 国网冀北电力有限公司检修分公司 | UAV navigation method, device, UAV and data readable storage device |
| CN109358650A (en)* | 2018-12-14 | 2019-02-19 | 国网冀北电力有限公司检修分公司 | Inspection path planning method, device, unmanned aerial vehicle and computer-readable storage medium |
| CN109885083A (en)* | 2019-03-06 | 2019-06-14 | 国网陕西省电力公司检修公司 | Flying platform and inspection method for fine inspection of transmission lines based on lidar |
| CN109885098A (en)* | 2019-04-11 | 2019-06-14 | 株洲时代电子技术有限公司 | A kind of bridge sidebar inspection flight course planning method |
| CN109885098B (en)* | 2019-04-11 | 2022-02-11 | 株洲时代电子技术有限公司 | Method for planning inspection route of bridge side fence |
| CN112236727A (en)* | 2019-07-30 | 2021-01-15 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle simulated flight method and device and recording medium |
| CN112119361A (en)* | 2019-09-23 | 2020-12-22 | 深圳市大疆创新科技有限公司 | Returning control method and device for movable platform and movable platform |
| WO2021056144A1 (en)* | 2019-09-23 | 2021-04-01 | 深圳市大疆创新科技有限公司 | Method and apparatus for controlling return of movable platform, and movable platform |
| CN111401146A (en)* | 2020-02-26 | 2020-07-10 | 长江大学 | Unmanned aerial vehicle power inspection method, device and storage medium |
| CN111582117A (en)* | 2020-04-29 | 2020-08-25 | 长江大学 | Unmanned aerial vehicle illegal building inspection method, equipment and storage medium |
| CN113741413A (en)* | 2020-05-29 | 2021-12-03 | 广州极飞科技股份有限公司 | Operation method of unmanned equipment, unmanned equipment and storage medium |
| CN113741413B (en)* | 2020-05-29 | 2022-11-08 | 广州极飞科技股份有限公司 | Operation method of unmanned equipment, unmanned equipment and storage medium |
| CN112068591A (en)* | 2020-08-25 | 2020-12-11 | 中国南方电网有限责任公司超高压输电公司天生桥局 | Unmanned aerial vehicle for automatic inspection of power transmission line, control method and device and storage medium |
| CN112233270A (en)* | 2020-10-30 | 2021-01-15 | 国家电网有限公司 | Unmanned aerial vehicle is intelligence around tower system of patrolling and examining independently |
| CN113014904A (en)* | 2021-02-24 | 2021-06-22 | 苏州臻迪智能科技有限公司 | Method, device and system for processing inspection image of unmanned aerial vehicle and storage medium |
| CN113112098A (en)* | 2021-05-12 | 2021-07-13 | 上海野戈智能科技发展有限公司 | Building defect detection method and device |
| CN113625730A (en)* | 2021-06-30 | 2021-11-09 | 南京邮电大学 | Four-rotor self-adaptive fault-tolerant control method based on overtorque sliding mode |
| CN113625730B (en)* | 2021-06-30 | 2023-07-14 | 南京邮电大学 | An Adaptive Fault-Tolerant Control Method for Quadrotors Based on Supertwisted Sliding Mode |
| CN113872680A (en)* | 2021-12-03 | 2021-12-31 | 上海特金信息科技有限公司 | TDOA (time difference of arrival) auxiliary RID (Rich Internet protocol) signal receiving control method, device and system |
| CN113872680B (en)* | 2021-12-03 | 2022-04-26 | 特金智能科技(上海)有限公司 | TDOA (time difference of arrival) auxiliary RID (Rich Internet protocol) signal receiving control method, device and system |
| CN114326794A (en)* | 2021-12-13 | 2022-04-12 | 广东省建设工程质量安全检测总站有限公司 | Curtain wall defect identification method, control terminal, server and readable storage medium |
| Publication number | Publication date |
|---|---|
| CN113791641A (en) | 2021-12-14 |
| WO2018195955A1 (en) | 2018-11-01 |
| CN108496129B (en) | 2021-10-01 |
| CN113791641B (en) | 2024-06-18 |
| Publication | Publication Date | Title |
|---|---|---|
| CN108496129A (en) | An aircraft-based facility detection method and control device | |
| US11794890B2 (en) | Unmanned aerial vehicle inspection system | |
| US11635775B2 (en) | Systems and methods for UAV interactive instructions and control | |
| EP4093671B1 (en) | Performing 3d reconstruction via an unmanned aerial vehicle | |
| CN112567201B (en) | Distance measuring method and device | |
| JP6918672B2 (en) | Deterioration diagnosis system | |
| CN106873627B (en) | A multi-rotor unmanned aerial vehicle and method for automatic inspection of transmission lines | |
| CN108508916B (en) | A control method, device, equipment and storage medium for unmanned aerial vehicle formation | |
| CN108628337A (en) | Coordinates measurement device, contouring system and path generating method | |
| WO2021223124A1 (en) | Position information obtaining method and device, and storage medium | |
| WO2020103110A1 (en) | Image boundary acquisition method and device based on point cloud map and aircraft | |
| CN114020002A (en) | Method, device and equipment for inspecting fan blade by unmanned aerial vehicle, unmanned aerial vehicle and medium | |
| WO2020103108A1 (en) | Semantic generation method and device, drone and storage medium | |
| CN111338382A (en) | A security situation-guided UAV path planning method | |
| US20220390940A1 (en) | Interfaces And Control Of Aerial Vehicle For Automated Multidimensional Volume Scanning | |
| CN113273173A (en) | Inspection method and device for movable platform, movable platform and storage medium | |
| US20240192705A1 (en) | Automated Unmanned Aerial Vehicle Dock Verification And Landing | |
| CN117873158A (en) | Unmanned aerial vehicle routing inspection complex route optimization method based on live-action three-dimensional model | |
| US12174629B2 (en) | Information processing apparatus, information processing method, program, and information processing system | |
| US12235639B1 (en) | Unmanned aerial vehicle flight control system | |
| CN205721829U (en) | A kind of unmanned vehicle | |
| JP7437930B2 (en) | Mobile objects and imaging systems | |
| CN117311385A (en) | Unmanned aerial vehicle protection system and method based on multidimensional detection data | |
| CN119803514B (en) | Method and system for realizing double-screen display of unmanned aerial vehicle based on embedded map | |
| CN117687424B (en) | Fully automatic low-altitude remote sensing and non-sensing inspection and monitoring system based on drones |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |