Disclosure of Invention
The invention aims to provide an airborne flight obstacle avoidance auxiliary method, device and system based on a laser radar.
In order to solve the technical problem, the invention provides an airborne flight obstacle avoidance auxiliary method based on a laser radar, which comprises the following steps:
carrying out fault judgment on a hardware module related to the flight obstacle avoidance;
acquiring data of the barrier when no fault exists;
processing data of the obstacle; and
and carrying out obstacle avoidance early warning according to the processed obstacle data.
Further, the method for judging the fault of the hardware module related to the flight obstacle avoidance includes:
and receiving a power-on instruction, carrying out power-on self-test on hardware modules involved in the flight obstacle avoidance, judging a fault when the power-on self-test of any module fails, and judging that the self-test passes if the power-on self-test of any module fails.
Further, the method for acquiring data of the obstacle in the absence of the fault includes:
carrying out laser point cloud collection on the barrier after the self-inspection is passed;
performing coordinate conversion on the point cloud data of the barrier according to the navigation data;
and adjusting the laser point cloud acquisition direction according to the servo control instruction.
Further, the method for processing the data of the obstacle comprises the following steps:
preprocessing the point cloud data of the obstacle, namely rasterizing, downsampling, separating land and objects and performing point cloud matching calibration on the point cloud data of the obstacle;
acquiring the attribute, the appearance information and the pose information of the obstacle according to the preprocessed point cloud data of the obstacle, and acquiring result-level point cloud data after completing point cloud semantic segmentation, namely the point cloud data
Carrying out accurate time synchronization on original point cloud data acquired by a laser radar and airborne navigation data, carrying out ground point separation on the point cloud data, sending the separated ground data to a ground target identification part, and sending the separated non-ground data to a high-voltage line identification part;
performing data fusion according to non-ground data of adjacent frames by a high-voltage wire identification part to identify a high-voltage wire;
the ground target identification part extracts point clouds belonging to a ground target according to gradient change conditions of the point cloud data in the advancing direction of the airplane and the direction perpendicular to the advancing direction, associates and communicates the isolated point clouds by using a region growth-based method to obtain the point clouds belonging to the ground target, calculates external polygons and height values of each group of point clouds, classifies the ground target categories by using the information, performs multi-frame smoothing on target parameters and inhibits local shielding;
matching and aligning the point clouds of adjacent frames, performing point cloud splicing based on navigation information corresponding to each frame to form a local airspace point cloud picture, performing regression fitting on the point clouds of the high-voltage line based on a branch line detection method, detecting and identifying the end point of the cable, performing transition processing on the line section, and further fitting the actual trend of the high-voltage line.
Further, the method for performing obstacle avoidance early warning according to the processed obstacle data comprises the following steps:
generating alarm information according to the result level point cloud data and the judgment rule to carry out obstacle avoidance early warning, and
performing point cloud visualization and rendering according to the result level point cloud data;
the judgment rule includes: predicting a future flight trajectory according to the current flight speed vector to generate a near-air flight channel, and generating a safe flight distance according to pilot reaction time, airplane maneuvering time, current flight speed, flight acceleration, current wind direction and wind speed, wherein targets in a safe distance range are regarded as threat targets.
In a second aspect, the present invention further provides an airborne flight obstacle avoidance assisting device based on a laser radar, including:
the system comprises a laser radar sensor, a turret servo mechanism, a signal processing module and an interface control module;
the laser radar sensor is suitable for emitting laser to collect point cloud data, and the laser radar sensor is suitable for performing coordinate conversion on the point cloud data according to navigation data of airborne inertial navigation;
the interface control module is suitable for controlling the turret servo mechanism to adjust the posture according to the servo control instruction and sending the adjusted posture to the signal processing module;
the signal processing module is suitable for acquiring result-level point cloud data according to the point cloud data and the adjusted pose;
the interface control module is suitable for generating alarm information according to the result level point cloud data to carry out obstacle avoidance early warning.
Further, the obstacle-avoiding auxiliary device for airborne flight further comprises: a power supply module;
the power supply module is suitable for supplying power to each module so as to enable each module to carry out power-on self-test.
In a third aspect, the present invention further provides an airborne flight obstacle avoidance assistance system based on a laser radar, including:
the airborne flight obstacle avoidance auxiliary device comprises an airborne flight obstacle avoidance auxiliary device and an airborne machine;
the airborne machine is suitable for sending a servo control command and navigation data to the airborne flight obstacle avoidance auxiliary device;
the airborne flight obstacle avoidance auxiliary device is suitable for generating alarm information according to the servo control instruction and the navigation data and feeding the alarm information back to the airborne machine.
The method has the advantages that the fault judgment is carried out on the hardware module related to the flight obstacle avoidance; acquiring data of the barrier when no fault exists; processing data of the obstacle; and obstacle avoidance early warning is carried out according to the processed obstacle data, so that the characteristics of good directivity, high detection resolution and quick imaging based on a laser radar are realized, and the accurate sensing and quick imaging of remote obstacle sensing, overhead power lines and other tiny objects in a near-space range can be realized by combining a target recognition algorithm based on a deep learning network, so that data reference is provided for safe flight pipeline generation and local flight path planning, a pilot is assisted to carry out route planning, and the low-altitude/ultra-low-altitude flight safety can be greatly guaranteed.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
Fig. 1 is a flow chart of an airborne flight obstacle avoidance assisting method based on a laser radar in the invention.
As shown in fig. 1, embodiment 1 provides an airborne flight obstacle avoidance assisting method based on a laser radar, including: carrying out fault judgment on a hardware module related to the flight obstacle avoidance; acquiring data of the barrier when no fault exists; processing data of the obstacle; and obstacle avoidance early warning is carried out according to the processed obstacle data, so that the characteristics of good directivity, high detection resolution and quick imaging based on a laser radar are realized, and the accurate sensing and quick imaging of remote obstacle sensing, overhead power lines and other tiny objects in a near-space range can be realized by combining a target recognition algorithm based on a deep learning network, so that data reference is provided for safe flight pipeline generation and local flight path planning, a pilot is assisted to carry out route planning, and the low-altitude/ultra-low-altitude flight safety can be greatly guaranteed.
Fig. 2 is a specific flowchart of an airborne flight obstacle avoidance assistance method based on a laser radar according to the present invention.
As shown in fig. 2, in this embodiment, the method for determining a fault of a hardware module involved in flight obstacle avoidance includes: receiving a power-on instruction, carrying out power-on self-test on hardware modules related to flight obstacle avoidance, judging a fault when any module (hardware module) fails in power-on self-test, otherwise, judging that the self-test passes, namely receiving the power-on instruction sent by a carrier, and powering on each hardware module by a power supply module; each module is electrified and self-checked, and self-checking results are collected and reported to the carrier by the interface control module; and if the self-checking fails, reporting the fault state, and if the self-checking passes, initializing (a system or a device provided with each hardware module, and the like).
In this embodiment, the method for acquiring data of an obstacle when there is no fault includes: carrying out laser point cloud collection on the barrier after the self-inspection is passed; performing coordinate conversion on the point cloud data of the barrier according to the navigation data; adjusting the laser point cloud acquisition direction according to the servo control instruction; after initialization, the laser radar sensor is switched to a working mode and emits laser, and laser point cloud collection is carried out on the obstacle; the laser radar sensor receives navigation data of airborne inertial navigation, performs primary coordinate conversion on collected point cloud, and sends the processed laser point cloud to the signal processing module in real time in an Ethernet frame mode; at the moment, if the interface control module receives a servo control instruction sent by the carrier, the turret servo mechanism is controlled to adjust the posture, the turret servo mechanism feeds back the adjusted posture to the interface control module, and the interface control module reports the information to the carrier and simultaneously sends the information to the signal processing module to assist the point cloud; the MEMS laser radar is adopted to realize the real-time high-resolution laser point cloud collection of the flight field of view, and can realize the identification of small obstacles such as mountains, isolated obstacles, overhead cables and the like.
In this embodiment, the method for processing data of an obstacle includes: preprocessing the point cloud data of the obstacle, namely rasterizing, downsampling, separating land and objects and performing point cloud matching calibration on the point cloud data of the obstacle; acquiring the attribute, the appearance information and the pose information of the obstacle according to the preprocessed point cloud data of the obstacle, and acquiring result-level point cloud data after point cloud semantic segmentation is completed; the signal processing module carries out rasterization, down sampling, ground object separation, point cloud matching calibration and other preprocessing operations on the original laser point cloud transmitted by the laser radar; the signal processing module runs a target recognition algorithm to obtain and output barrier attributes, appearance information, pose information and the like to the interface control module, and simultaneously, the result-level point cloud data after point cloud semantic segmentation is finished is sent to the interface control module; by adopting a multi-mode fusion target identification algorithm based on deep learning, real-time high-robustness target identification and measurement can be realized; carrying out precise time synchronization on original point cloud data acquired by a laser radar and airborne navigation data, carrying out rough ground point separation on the point cloud data, sending the separated ground data to a ground target identification part for further processing, sending the separated non-ground data to a high-voltage wire identification part, and carrying out data fusion on the high-voltage wire identification part by using non-ground data of adjacent frames so as to further improve the accuracy and reliability of high-voltage power line detection; the ground target detection is based on an interframe correlation method, point clouds which probably belong to ground targets are extracted according to the gradient change conditions of point cloud data in the advancing direction of an airplane and the vertical direction of the advancing direction, the isolated point clouds are correlated and communicated by using a region growth-based method to obtain the point clouds belonging to the ground targets, then the external polygon and the height value of each group of point clouds are calculated, the information is further used for classifying the ground target categories, and multi-frame smoothing is carried out on target parameters to inhibit the influence of adverse factors such as local shielding on the detection effect; considering that point cloud echoes from a high-voltage line received in single-frame point cloud data are too sparse and are easily identified as noise points to be filtered in the signal processing process, firstly, matching and aligning adjacent frame point clouds, and performing point cloud splicing based on navigation information corresponding to each frame to form a local airspace point cloud image, so that the high-voltage line point cloud echoes are ensured to be dense and are obviously distinguished from the noise points, and the condition of missing detection is prevented. On the basis, performing regression fitting on the high-voltage line point cloud based on a branch line detection method, detecting and identifying the end point of the cable (high-voltage line), and performing transition processing on the line section so as to fit the actual trend of the cable; the result-level point cloud data may include: point cloud data of ground parts and point cloud data of non-ground parts (e.g., high-voltage line parts).
In this embodiment, the method for performing obstacle avoidance early warning according to the processed obstacle data includes: generating alarm information according to the result level point cloud data and the judgment rule to carry out obstacle avoidance early warning, and carrying out point cloud visualization and rendering according to the result level point cloud data; the interface control module receives the result-level point cloud sent by the signal processing module, judges whether an alarm needs to be generated or not based on an alarm judgment rule and uploads the alarm to airborne flight control; the interface control module performs point cloud visualization based on the result level point cloud and renders the point cloud based on a point cloud rendering rule; the judgment rule includes: predicting a future flight trajectory according to the current flight speed vector to generate a near-air flight channel, and generating a safe flight distance by comprehensively considering pilot reaction time, airplane maneuvering time, current flight speed, flight acceleration, current wind direction, wind speed and other environmental factors, wherein targets in the safe distance range are regarded as threat targets.
Example 2
Fig. 3 is a schematic block diagram of an airborne flight obstacle avoidance assisting device based on a laser radar in accordance with the present invention.
As shown in fig. 3, on the basis of embodiment 1, embodiment 2 further provides an airborne flight obstacle avoidance assisting device based on a laser radar, including: the system comprises a laser radar sensor, a turret servo mechanism, a signal processing module and an interface control module; the laser radar sensor is suitable for emitting laser to collect point cloud data, and the laser radar sensor is suitable for performing coordinate conversion on the point cloud data according to navigation data of airborne inertial navigation; the interface control module is suitable for controlling the turret servo mechanism to adjust the posture according to the servo control instruction and sending the adjusted posture to the signal processing module; the signal processing module is suitable for acquiring result-level point cloud data according to the point cloud data and the adjusted pose; the interface control module is suitable for generating alarm information according to the result level point cloud data to carry out obstacle avoidance early warning; an interface control mode of separating data flow from control flow is adopted, so that the instantaneity and the availability of the device under the requirement of large-bandwidth data transmission communication are ensured
In this embodiment, the onboard flight obstacle avoidance assistance device further includes: a power supply module; the power supply module is suitable for supplying power to each module so as to enable each module to carry out power-on self-test.
In this embodiment, the lidar sensor is responsible for tasks such as laser emission, laser reception and filtering of laser point cloud clutter, realizes scanning of objects in a field of view and obtains accurate relative position data based on a TOF principle, and is a data acquisition module of the airborne flight obstacle avoidance auxiliary device; the laser radar sensor adopts a MEMS-based hybrid solid-state laser radar, and the laser radar has the advantages that three-dimensional laser point cloud in an appointed market field angle can be quickly generated, and instantaneity and effectiveness of system point cloud output are guaranteed; the MEMS laser radar is adopted to realize the real-time high-resolution laser point cloud collection of the flight field of view, and can realize the identification of small obstacles such as mountain bodies, isolated obstacles, overhead cables and the like
In the embodiment, the turret servo mechanism mainly controls the turret posture and adjusts the orientation of the laser radar sensor.
In the embodiment, the signal processing module mainly completes tasks such as laser point cloud preprocessing, obstacle identification and tracking based on laser point cloud and the like based on an embedded system, realizes a high-robustness obstacle target identification function through multi-mode target identification algorithm fusion, and outputs a target identification result, target pose measurement information and result level point cloud data; in order to ensure that the laser point cloud data with high density and large volume can be processed in real time, the signal processing module adopts a hardware design scheme of an FPGA + ARM structure to realize real-time processing of the large-volume point cloud data, preprocessing operations such as filtering, matching calibration, coordinate conversion and rasterization of the point cloud data are realized by utilizing the high concurrency capability and the fast logic operation capability of the FPGA, and real-time scheduling and data post-processing of the FPGA platform are realized based on the ARM platform.
In this embodiment, the interface control module mainly realizes internal and external data interaction control of an airborne flight obstacle avoidance auxiliary device such as a helicopter based on an embedded system, and realizes functions such as point cloud visualization and rendering, alarm information generation and the like; in order to ensure that the device processing interface control module adopts PCI-eX4 to realize high-bandwidth and high-speed communication with the signal processing module, adopts gigabit Ethernet to realize input and output of mass original point clouds and result level data, realizes issuing of control instructions and state return through an independent RS422 serial port, and ensures real-time performance and usability of the device operation through isolation of a control stream and a data stream; the embedded system based laser point cloud signal processing and visualization are realized, the size and power consumption of the whole machine are optimized on the premise of ensuring the functional performance of the system, and the integrated design is facilitated.
In this embodiment, the working method of the airborne flight obstacle avoidance assisting device adopts the airborne flight obstacle avoidance assisting method based on the laser radar in embodiment 1.
Example 3
Fig. 4 is a schematic block diagram of an airborne flight obstacle avoidance assistance system based on a laser radar according to the present invention.
As shown in fig. 4, on the basis of embodiment 2, embodiment 3 further provides an airborne flight obstacle avoidance assistance system based on a lidar, including: the airborne flight obstacle avoidance auxiliary device comprises an airborne flight obstacle avoidance auxiliary device and an airborne machine; the airborne machine is suitable for sending a servo control command and navigation data to the airborne flight obstacle avoidance auxiliary device; the airborne flight obstacle avoidance auxiliary device is suitable for generating alarm information according to the servo control instruction and the navigation data and feeding the alarm information back to the airborne machine.
In this embodiment, the onboard flight obstacle avoidance assisting device in embodiment 2 may be adopted as the onboard flight obstacle avoidance assisting device.
In summary, the invention judges the fault of the hardware module related to the flight obstacle avoidance; acquiring data of the barrier when no fault exists; processing data of the obstacle; and obstacle avoidance early warning is carried out according to the processed obstacle data, so that the characteristics of good directivity, high detection resolution and quick imaging based on a laser radar are realized, and the accurate sensing and quick imaging of remote obstacle sensing, overhead power lines and other tiny objects in a near-space range can be realized by combining a target recognition algorithm based on a deep learning network, so that data reference is provided for safe flight pipeline generation and local flight path planning, a pilot is assisted to carry out route planning, and the low-altitude/ultra-low-altitude flight safety can be greatly guaranteed.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In light of the foregoing description of the preferred embodiment of the present invention, many modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the invention. The technical scope of the present invention is not limited to the content of the specification, and must be determined according to the scope of the claims.