Disclosure of Invention
The invention aims to provide a laser radar-based airborne flight obstacle avoidance assisting method, device and system.
In order to solve the technical problems, the invention provides an airborne flight obstacle avoidance assisting method based on a laser radar, which comprises the following steps:
performing fault judgment on a hardware module related to flight obstacle avoidance;
data acquisition of the obstacle is carried out when no fault exists;
processing data of the obstacle; and
and carrying out obstacle avoidance early warning according to the processed obstacle data.
Further, the method for performing fault judgment on the hardware module related to the flying obstacle avoidance comprises the following steps:
and receiving a power-on instruction, carrying out power-on self-test on the hardware module related to the flying obstacle avoidance, judging the fault when the power-on self-test of any module is not passed, and otherwise judging the self-test to pass.
Further, the method for collecting data of the obstacle in the absence of faults comprises the following steps:
performing laser point cloud acquisition on the obstacle after the self-inspection is passed;
performing coordinate transformation on the point cloud data of the obstacle according to the navigation data;
and adjusting the laser point cloud acquisition direction according to the servo control instruction.
Further, the method for processing the data of the obstacle comprises the following steps:
preprocessing the point cloud data of the obstacle, namely rasterizing, downsampling, separating ground objects and carrying out point cloud matching calibration on the point cloud data of the obstacle;
acquiring attribute, appearance information and pose information of the obstacle according to the preprocessed point cloud data of the obstacle, and acquiring result-level point cloud data after the point cloud semantic segmentation is completed, namely
Carrying out accurate time synchronization on original point cloud data acquired by a laser radar and airborne navigation data, carrying out ground point separation on the point cloud data, sending the separated ground data into a ground target recognition part, and sending the separated non-ground data into a high-voltage line recognition part;
the high-voltage line identification part is used for carrying out data fusion according to non-ground data of adjacent frames to identify the high-voltage line;
the ground target recognition part extracts point clouds belonging to the ground target according to gradient change conditions of the point cloud data in the forward direction of the aircraft and the vertical direction of the forward direction, associates and communicates the isolated point clouds by using a region-based growth method to obtain point clouds belonging to the ground target, calculates an external polygon and a height value of each group of point clouds, classifies the category of the ground target by using the information, performs multi-frame smoothing on target parameters, and inhibits local shielding;
matching and aligning point clouds of adjacent frames, performing point cloud splicing based on navigation information corresponding to each frame to form a local airspace point cloud image, performing regression fitting on the high-voltage line point clouds based on a branch line detection method, detecting and identifying cable endpoints, performing transition processing on the lines, and fitting the actual trend of the high-voltage line.
Further, the method for performing obstacle avoidance early warning according to the processed obstacle data comprises the following steps:
generating alarm information according to the result level point cloud data and the judgment rule to perform obstacle avoidance early warning
Performing point cloud visualization and rendering according to the result level point cloud data;
the judging rule comprises the following steps: and predicting future flight tracks according to the current flight speed vector, generating a near-air flight channel, and generating a safe flight distance according to pilot reaction time, aircraft maneuvering time, current flight speed and flight acceleration, current wind direction and wind speed, wherein a target in the safe distance range is considered as a threat target.
In a second aspect, the present invention also provides an airborne flight obstacle avoidance assisting device based on a laser radar, including:
the system comprises a laser radar sensor, a turret servo mechanism, a signal processing module and an interface control module;
the laser radar sensor is suitable for emitting laser to collect point cloud data, and the laser radar sensor is suitable for carrying out coordinate conversion on the point cloud data according to navigation data of airborne inertial navigation;
the interface control module is suitable for controlling the turret servo mechanism to adjust the gesture according to the servo control instruction and sending the adjusted gesture to the signal processing module;
the signal processing module is suitable for acquiring result-level point cloud data according to the point cloud data and the adjusted pose;
the interface control module is suitable for generating alarm information according to the result level point cloud data to perform obstacle avoidance early warning.
Further, the airborne flight obstacle avoidance auxiliary device further comprises: a power module;
the power module is suitable for supplying power to each module so as to enable each module to perform power-on self-detection.
In a third aspect, the present invention also provides an airborne flight obstacle avoidance assistance system based on a lidar, including:
airborne flight obstacle avoidance auxiliary device and carrier;
the carrier is suitable for sending a servo control instruction and navigation data to the airborne flight obstacle avoidance auxiliary device;
the onboard flight obstacle avoidance auxiliary device is suitable for generating alarm information according to the servo control instruction and the navigation data and feeding the alarm information back to the onboard.
The method has the beneficial effects that the fault judgment is carried out on the hardware module related to the flying obstacle avoidance; data acquisition of the obstacle is carried out when no fault exists; processing data of the obstacle; and carrying out obstacle avoidance early warning according to the processed obstacle data, realizing that the laser radar-based target recognition algorithm based on the deep learning network has the characteristics of good directivity, high detection resolution and quick imaging, and being capable of realizing accurate sensing and quick imaging of tiny objects such as overhead power lines and the like in a long-distance obstacle sensing and near-air range, further providing data reference for safe flight pipeline generation and local track planning, assisting pilots in carrying out route planning, and greatly guaranteeing low-air/ultra-low-air flight safety.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
Fig. 1 is a flowchart of an airborne flight obstacle avoidance assistance method based on lidar according to the present invention.
As shown in fig. 1, embodiment 1 provides a laser radar-based airborne flight obstacle avoidance assisting method, which includes: performing fault judgment on a hardware module related to flight obstacle avoidance; data acquisition of the obstacle is carried out when no fault exists; processing data of the obstacle; and carrying out obstacle avoidance early warning according to the processed obstacle data, realizing that the laser radar-based target recognition algorithm based on the deep learning network has the characteristics of good directivity, high detection resolution and quick imaging, and being capable of realizing accurate sensing and quick imaging of tiny objects such as overhead power lines and the like in a long-distance obstacle sensing and near-air range, further providing data reference for safe flight pipeline generation and local track planning, assisting pilots in carrying out route planning, and greatly guaranteeing low-air/ultra-low-air flight safety.
Fig. 2 is a specific flowchart of an airborne flight obstacle avoidance assistance method based on a lidar according to the present invention.
As shown in fig. 2, in this embodiment, the method for performing fault determination on a hardware module involved in flying obstacle avoidance includes: receiving a power-on instruction, carrying out power-on self-test on a hardware module related to flight obstacle avoidance, judging a fault when the power-on self-test of any module (hardware module) fails, otherwise judging that the self-test passes, namely receiving the power-on instruction sent by a carrier, and powering up each hardware module by a power module; each module is electrified for self-checking, and the self-checking result is summarized and reported to the carrier by the interface control module; if the self-test is not passed, the fault state is reported, and if the self-test is passed, the system or the device provided with each hardware module is initialized.
In this embodiment, the method for collecting data of the obstacle when there is no fault includes: performing laser point cloud acquisition on the obstacle after the self-inspection is passed; performing coordinate transformation on the point cloud data of the obstacle according to the navigation data; adjusting the laser point cloud acquisition direction according to the servo control instruction; after initialization, the laser radar sensor is converted into a working mode and emits laser, and laser point cloud acquisition is started on the obstacle; the laser radar sensor receives navigation data of airborne inertial navigation, performs primary coordinate conversion on the acquired point cloud, and sends the processed laser point cloud to the signal processing module in real time in the form of an Ethernet frame; at this time, if the interface control module receives a servo control instruction sent by the carrier, the interface control module controls the turret servo mechanism to adjust the gesture, the turret servo mechanism feeds back the adjusted gesture to the interface control module, and the interface control module reports the information to the carrier and simultaneously sends the information to the signal processing module to assist the point cloud; the MEMS laser radar is adopted to realize real-time high-resolution laser point cloud acquisition of a flight view field, and can realize identification of small obstacles such as mountain bodies, isolated obstacles, overhead cables and the like.
In this embodiment, the method for processing data of an obstacle includes: preprocessing the point cloud data of the obstacle, namely rasterizing, downsampling, separating ground objects and carrying out point cloud matching calibration on the point cloud data of the obstacle; acquiring attribute, appearance information and pose information of the obstacle according to the preprocessed point cloud data of the obstacle, and acquiring result-level point cloud data after the point cloud semantic segmentation is completed; the signal processing module performs preprocessing operations such as rasterization, downsampling, ground feature separation, point cloud matching calibration and the like on an original laser point cloud transmitted by the laser radar; the signal processing module runs a target recognition algorithm to obtain and output barrier properties, appearance information, pose information and the like to the interface control module, and meanwhile, the result level point cloud data after the point cloud semantic segmentation is completed is sent to the interface control module; the multi-mode fusion target recognition algorithm based on deep learning is adopted, so that real-time high-robustness target recognition and measurement can be realized; carrying out accurate time synchronization on original point cloud data acquired by a laser radar and airborne navigation data, carrying out rough ground point separation on the point cloud data, sending the separated ground data into a ground target recognition part for further processing, sending the separated non-ground data into a high-voltage line recognition part, and carrying out data fusion on the non-ground data of adjacent frames by the high-voltage line recognition part so as to further improve the accuracy and reliability of high-voltage power line detection; the ground target detection is based on an inter-frame association method, point clouds with high probability belonging to the ground target are extracted according to gradient change conditions of point cloud data in the forward direction of an airplane and the vertical direction of the forward direction, the isolated point clouds are associated and communicated by using a region growth-based method to obtain the point clouds belonging to the ground target, then the external polygon and the height value of each group of point clouds are calculated, the ground target category is classified by using the information, and multi-frame smoothing is carried out on target parameters to inhibit the influence of bad factors such as local shielding on the detection effect; considering that the point cloud echo received from the high-voltage line in single-frame point cloud data is too sparse, the point cloud echo is easily identified as noise point to be filtered in the signal processing process, firstly, matching and aligning the point clouds of adjacent frames, and performing point cloud splicing based on navigation information corresponding to each frame to form a local airspace point cloud image, so that the high-voltage line point cloud echo is ensured to be dense, and obviously distinguished from the noise point, and the condition of missed detection is prevented. On the basis, carrying out regression fitting on the high-voltage line point cloud based on a branch line detection method, detecting and identifying the end point of a cable (high-voltage line), carrying out transition treatment on the cable, and further fitting the actual trend of the cable; the result level point cloud data may include: point cloud data for ground portions and point cloud data for non-ground portions (e.g., high voltage line portions).
In this embodiment, the method for performing obstacle avoidance early warning according to the processed obstacle data includes: generating alarm information according to the result level point cloud data and the judgment rule to perform obstacle avoidance early warning, and performing point cloud visualization and rendering according to the result level point cloud data; the interface control module receives the result level point cloud sent by the signal processing module, judges whether an alarm needs to be generated or not based on an alarm judging rule, and uploads the alarm to the airborne flight control; the interface control module performs point cloud visualization based on the result level point cloud and performs rendering on the point cloud based on a point cloud rendering rule; the judging rule comprises the following steps: predicting future flight tracks according to the current flight speed vector, generating a near-air flight channel, comprehensively considering pilot reaction time, aircraft maneuvering time, current flight speed and flight acceleration, current wind direction, wind speed and other environmental factors to generate a safe flight distance, and recognizing a target in the safe distance range as a threat target.
Example 2
Fig. 3 is a schematic block diagram of an airborne flight obstacle avoidance aid based on lidar according to the present invention.
As shown in fig. 3, on the basis of embodiment 1, embodiment 2 further provides an airborne flight obstacle avoidance auxiliary device based on a laser radar, which includes: the system comprises a laser radar sensor, a turret servo mechanism, a signal processing module and an interface control module; the laser radar sensor is suitable for emitting laser to collect point cloud data, and the laser radar sensor is suitable for carrying out coordinate conversion on the point cloud data according to navigation data of airborne inertial navigation; the interface control module is suitable for controlling the turret servo mechanism to adjust the gesture according to the servo control instruction and sending the adjusted gesture to the signal processing module; the signal processing module is suitable for acquiring result-level point cloud data according to the point cloud data and the adjusted pose; the interface control module is suitable for generating alarm information according to the result level point cloud data to perform obstacle avoidance early warning; an interface control mode of separating a data stream from a control stream is adopted, so that the instantaneity and the usability of the device under the requirement of large-bandwidth data transmission and communication are ensured
In this embodiment, the airborne flight obstacle avoidance auxiliary device further includes: a power module; the power module is suitable for supplying power to each module so as to enable each module to perform power-on self-detection.
In this embodiment, the laser radar sensor is responsible for tasks such as laser emission, laser receiving, laser point cloud clutter filtering, and the like, and based on the TOF principle, realizes scanning of objects in a field of view and obtains accurate relative position data, and is a data acquisition module of an airborne flight obstacle avoidance auxiliary device; the laser radar sensor adopts the MEMS-based hybrid solid-state laser radar, and the laser radar has the advantages that three-dimensional laser point clouds in appointed market angles can be rapidly generated, and the instantaneity and the effectiveness of system point cloud output are ensured; the MEMS laser radar is adopted to realize the real-time high-resolution laser point cloud acquisition of the flight view field, and can realize the identification of small obstacles such as mountain bodies, isolated obstacles, overhead cables and the like
In this embodiment, the turret servo mechanism mainly controls the turret posture and adjusts the orientation of the lidar sensor.
In this embodiment, the signal processing module mainly completes tasks such as laser point cloud preprocessing, obstacle recognition and tracking based on laser point cloud based on an embedded system, realizes a high-robustness obstacle target recognition function through fusion of a multi-mode target recognition algorithm, and outputs a target recognition result, target pose measurement information and result level point cloud data; in order to ensure that the real-time high-density mass laser point cloud data can be ensured, the signal processing module adopts a hardware design scheme of an FPGA+ARM structure to realize real-time processing of mass point cloud data, and utilizes the high concurrency capability and the quick logic operation capability of the FPGA to realize the preprocessing operations such as filtering, matching calibration, coordinate conversion, rasterization and the like of the point cloud data, and realizes real-time scheduling and data post-processing of the FPGA platform based on the ARM platform.
In the embodiment, the interface control module mainly realizes the data interaction control of the airborne flight obstacle avoidance auxiliary devices such as a helicopter and the like on the inside and outside based on the embedded system, and realizes the functions of point cloud visualization and rendering, alarm information generation and the like; in order to ensure that the device processing interface control module adopts PCI-eX4 to realize high-speed communication with a large bandwidth between the signal processing module, adopts gigabit Ethernet to realize input and output of a large number of original point clouds and result data, realizes issuing and state reporting of control instructions through an independent RS422 serial port, and ensures real-time and usability of device operation through isolation of control flow and data flow; the embedded system-based laser point cloud signal processing and visualization are realized, and on the premise of ensuring the system function performance, the size and the power consumption of the whole machine are optimized, so that the integrated design is facilitated.
In this embodiment, the working method of the airborne flight obstacle avoidance assisting device adopts the airborne flight obstacle avoidance assisting method based on the laser radar in embodiment 1.
Example 3
Fig. 4 is a schematic block diagram of an airborne flight obstacle avoidance assistance system based on lidar according to the present invention.
As shown in fig. 4, on the basis of embodiment 2, embodiment 3 further provides an airborne flight obstacle avoidance assisting system based on a laser radar, which includes: airborne flight obstacle avoidance auxiliary device and carrier; the carrier is suitable for sending a servo control instruction and navigation data to the airborne flight obstacle avoidance auxiliary device; the onboard flight obstacle avoidance auxiliary device is suitable for generating alarm information according to the servo control instruction and the navigation data and feeding the alarm information back to the onboard.
In this embodiment, the airborne flight obstacle avoidance assistance device may be the airborne flight obstacle avoidance assistance device in embodiment 2.
In summary, the invention performs fault judgment on the hardware module related to flying obstacle avoidance; data acquisition of the obstacle is carried out when no fault exists; processing data of the obstacle; and carrying out obstacle avoidance early warning according to the processed obstacle data, realizing that the laser radar-based target recognition algorithm based on the deep learning network has the characteristics of good directivity, high detection resolution and quick imaging, and being capable of realizing accurate sensing and quick imaging of tiny objects such as overhead power lines and the like in a long-distance obstacle sensing and near-air range, further providing data reference for safe flight pipeline generation and local track planning, assisting pilots in carrying out route planning, and greatly guaranteeing low-air/ultra-low-air flight safety.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners as well. The apparatus embodiments described above are merely illustrative, for example, of the flowcharts and block diagrams in the figures that illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present invention may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
With the above-described preferred embodiments according to the present invention as an illustration, the above-described descriptions can be used by persons skilled in the relevant art to make various changes and modifications without departing from the scope of the technical idea of the present invention. The technical scope of the present invention is not limited to the description, but must be determined according to the scope of claims.