技术领域technical field
本发明涉及无人飞行器领域，尤其涉及一种无人机控制方法。The invention relates to the field of unmanned aerial vehicles, in particular to a method for controlling an unmanned aerial vehicle.
背景技术Background technique
多旋翼无人机(Multi-rotor aircraft)通过无线遥控设备进行控制，能够在无人驾驶的状态下执行一些特殊的任务。其垂直起降、灵活机动和可靠性高等特点，能够携带更多的探测和处理设备。Multi-rotor UAV (Multi-rotor aircraft) is controlled by wireless remote control equipment, and can perform some special tasks in an unmanned state. Its vertical take-off and landing, flexible maneuverability and high reliability can carry more detection and processing equipment.
随着微电子技术的发展，计算机硬件和微电子传感器的小型化和多样化以及控制算法的不断优化，增强了多旋翼无人机信息处理能力和速度，从而其执行任务的类型也越来越多样。With the development of microelectronics technology, the miniaturization and diversification of computer hardware and microelectronic sensors, as well as the continuous optimization of control algorithms, the information processing capability and speed of multi-rotor UAVs have been enhanced, so that the types of tasks it performs are becoming more and more diverse.
目前的无人机广泛采用GPS和惯性导航系统进行定位和导航，然而在建筑物和峡谷等GPS信号缺失的环境中，需要对无人机进行精准控制，如何提高无人机控制精度是当前需要解决的问题。Current UAVs widely use GPS and inertial navigation systems for positioning and navigation. However, in environments where GPS signals are missing, such as buildings and canyons, UAVs need to be precisely controlled. How to improve UAV control accuracy is a current need solved problem.
发明内容Contents of the invention
本发明实施例提供一种无人机控制方法，能够提高无人机控制精度。An embodiment of the present invention provides a method for controlling a drone, which can improve the control accuracy of the drone.
本发明实施例采用如下技术方案：The embodiment of the present invention adopts following technical scheme:
一种无人机控制方法，所述方法应用于无人机控制系统，所述系统包括：视觉传感器、飞行控制板、主控计算机、无人机，所述主控计算机包括依次连接的图像采集模块、处理模块、通信接口模块、控制模块，所述方法包括：A method for controlling an unmanned aerial vehicle, the method is applied to an unmanned aerial vehicle control system, the system includes: a visual sensor, a flight control board, a main control computer, and an unmanned aerial vehicle, and the main control computer includes sequentially connected image acquisition Module, processing module, communication interface module, control module, described method comprises:
所述视觉传感器采集环境图像信息；The visual sensor collects environmental image information;
所述图像采集模块获取所述视觉传感器采集的环境图像信息，并将所述环境图像信息传输至所述处理模块；The image acquisition module acquires the environmental image information collected by the visual sensor, and transmits the environmental image information to the processing module;
所述处理模块根据所述环境图像信息，得到无人机的高度、位置和姿态估计，并将视觉位姿估计通过特定消息形式发布到主题，同时发布地图主题输出 地图信息；The processing module obtains the height, position and attitude estimation of the UAV according to the environmental image information, and releases the visual pose estimation to the topic through a specific message form, and simultaneously publishes the map topic output map information;
所述飞行控制板将无人机当前位置、姿态和速度等信息通过MAVLink协议发送给所述通信接口模块；The flight control board sends information such as the current position, attitude and speed of the drone to the communication interface module through the MAVLink protocol;
所述控制模块分析订阅得到的无人机当前各个状态，并将控制指令发布到主题，供通信接口节点订阅。The control module analyzes the current states of the drone obtained through subscription, and publishes control instructions to topics for subscription by communication interface nodes.
所述通信接口模块整合从所述图像采集模块订阅得到的视觉估计位姿信息以及从所述控制模块订阅得到的控制命令，并通过MAVLink协议发送给所述飞行控制板，所述飞行控制板对所述无人机进行控制，使得所述无人机按照控制命令结合视觉位姿估计结果完成相应的自主飞行。The communication interface module integrates the visual estimated pose information subscribed from the image acquisition module and the control command subscribed from the control module, and sends them to the flight control board through the MAVLink protocol, and the flight control board The unmanned aerial vehicle is controlled so that the unmanned aerial vehicle completes the corresponding autonomous flight according to the control command and the visual pose estimation result.
可选的，还包括：所述通信接口模块从所述处理模块订阅定位结果，根据所述定位结果进行定位。Optionally, the method further includes: the communication interface module subscribing to a positioning result from the processing module, and performing positioning according to the positioning result.
可选的，还包括：Optionally, also include:
所述通信接口模块订阅自主控制命令主题，根据所述自主控制命令主题实现对无人机进行控制。The communication interface module subscribes to the subject of the autonomous control command, and controls the UAV according to the subject of the autonomous control command.
可选的，还包括：所述通信接口模块通过所述控制板获取无人机当前局部位置。Optionally, it also includes: the communication interface module acquires the current local position of the drone through the control board.
可选的，还包括：Optionally, also include:
所述通信接口模块设置有第一接口、第二接口，所述通信接口模块通过所述第一接口获取飞控心跳包、位置数据包、姿态角数据包，所述通信接口模块通过所述第二接口获取视觉位姿估计结果和控制命令。The communication interface module is provided with a first interface and a second interface. The communication interface module acquires flight control heartbeat packets, position data packets, and attitude angle data packets through the first interface. The communication interface module obtains flight control heartbeat packets, position data packets, and attitude angle data packets through the first interface. The second interface acquires visual pose estimation results and control commands.
可选的，所述控制模块获取所述无人机当前位置，生成位置控制指令，将所述位置控制命令发布到所述通信接口模块，并通过监测所述无人机的状态调整所述位置控制指令。Optionally, the control module acquires the current position of the UAV, generates a position control command, issues the position control command to the communication interface module, and adjusts the position by monitoring the state of the UAV Control instruction.
可选的，所述处理模块对所述无人机当前的位置和姿态进行估计，所述处理模块实时向所述飞行控制板提供当前的位置和姿态；Optionally, the processing module estimates the current position and attitude of the UAV, and the processing module provides the current position and attitude to the flight control board in real time;
所述飞行控制板通过订阅主题的方式获取当前的位置估计结果(x,y,z)和 四元素(x,y,z,w)形式的姿态估计结果。The flight control board obtains the current position estimation result (x, y, z) and the attitude estimation result in the form of four elements (x, y, z, w) by subscribing to the topic.
可选的，所述处理模块对每个新的关键帧Ki完成局部地图构建，步骤包括：Optionally, the processing module completes local map construction for each new key frame Ki , and the steps include:
S1、插入关键帧；S1, insert key frame;
S2、淘汰最近地图点；S2. Eliminate the nearest map point;
S3、创建新的地图点；S3. Create a new map point;
S4、局部光束平差；S4. Local beam adjustment;
S5、局部关键帧淘汰。S5. Local key frame elimination.
可选的，所述处理模块针对局部地图构建线程中最后处理的关键帧Ki进行环路闭合检测线程，所述处理模块通过关键帧Ki的词袋向量计算其与系统中关键帧的相似性，完成候选的闭环检测，如果有多个位置的关键帧与Ki的场景外观相似，则有多个闭环候选；Optionally, the processing module performs a loop closure detection thread for the last processed key frame Ki in the local map construction thread, and the processing module calculates the similarity between the key frame Ki and the key frame in the system through the bag-of-words vector calculation of the key frame K i , to complete candidate closed-loop detection, if there are multiple positions of key frames similar to the scene appearance of Ki , then there are multiple closed-loop candidates;
所述处理模块计算当前帧Ki与闭环帧Kl之间的相似变换，对回路中的累计误差进行修正，同时相似性变换将作为闭环的几何验证；The processing module calculates the similarity transformation between the current frame Ki and the closed-loop frame Kl , and corrects the cumulative error in the loop, and the similarity transformation will be used as a geometric verification of the closed loop;
所述处理模块确定闭环后，通过计算得到的相似性变换对当前关键帧的位姿进行修正，同时这个修正将影响到与当前关键帧相邻的关键帧节点，进行位姿图优化达到全局一致性，从而每个地图点根据修正的关键帧进行变换。After the processing module determines the closed loop, the pose of the current key frame is corrected by the calculated similarity transformation, and this correction will affect the key frame nodes adjacent to the current key frame, and the pose graph is optimized to achieve global consistency , so that each map point is transformed according to the corrected keyframe.
可选的，所述无人机包括位置控制器、姿态控制器、执行电机，所述位置控制器接收飞行控制信息，进行姿态的解算后，通过所述姿态控制器控制所述执行电机，从而对所述无人机进行飞行控制。Optionally, the UAV includes a position controller, an attitude controller, and an execution motor. The position controller receives flight control information, and after calculating the attitude, controls the execution motor through the attitude controller. Thereby flight control is carried out to described unmanned aerial vehicle.
基于上述技术方案的无人机控制方法，利用视觉传感器获取环境图像数据后，传输到搭载在无人机上的主控计算机，主控计算机对接收到的图像进行处理，完成位置和姿态估计以及地图构建，之后将相对位置作为飞行参数直接送到飞行控制板，飞行控制板解算当前飞行器的位置、高度和姿态数据后，对所述无人机进行控制，使得所述无人机按照控制命令结合视觉位姿估计结果完成相应的自主飞行。Based on the UAV control method of the above technical solution, after using the visual sensor to obtain the environmental image data, it is transmitted to the main control computer mounted on the UAV, and the main control computer processes the received image to complete the position and attitude estimation and the map. After the construction, the relative position is directly sent to the flight control board as a flight parameter. After the flight control board solves the position, altitude and attitude data of the current aircraft, it controls the UAV so that the UAV follows the control command. Combined with the visual pose estimation results to complete the corresponding autonomous flight.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.
附图说明Description of drawings
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本发明的实施例，并与说明书一起用于解释本发明的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.
图1为本发明实施例提供的一种无人机控制系统的结构示意图之一；Fig. 1 is one of structural schematic diagrams of a kind of unmanned aerial vehicle control system provided by the embodiment of the present invention;
图2为本发明实施例提供的一种无人机控制系统的结构示意图之二；Fig. 2 is the second structural diagram of a UAV control system provided by the embodiment of the present invention;
图3为本发明实施例提供的一种无人机控制系统的结构示意图之三；Fig. 3 is the third structural schematic diagram of a UAV control system provided by the embodiment of the present invention;
图4为本发明实施例提供的一种无人机控制系统的结构示意图之四。Fig. 4 is the fourth structural schematic diagram of a drone control system provided by an embodiment of the present invention.
具体实施方式Detailed ways
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本发明相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本发明的一些方面相一致的装置和方法的例子。Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the invention as recited in the appended claims.
图1为本发明实施例提供的一种无人机控制系统，该系统包括：视觉传感器11、飞行控制板12、主控计算机13，无人机14。FIG. 1 is an unmanned aerial vehicle control system provided by an embodiment of the present invention. The system includes: a visual sensor 11 , a flight control board 12 , a main control computer 13 , and an unmanned aerial vehicle 14 .
本发明施例中，飞行控制板12是执行自主命令和控制飞行的枢纽，主控计算机13通过飞行控制板12获取飞行数据和状态，也是通过飞行控制板12来完成自主控制飞行。In the embodiment of the present invention, the flight control board 12 is the hub for executing autonomous commands and controlling the flight, and the main control computer 13 obtains flight data and status through the flight control board 12 , and also completes the autonomous control flight through the flight control board 12 .
本发明一个实施例中，飞行控制板12可以采用如下配置：STM32F427CortexM4168MHz微处理器，RAM为256KB，flash存储为2MB。除了主要的CPU外，还有一个STM32F103处理器，避免主处理器失效时无人机失控。为拓展自主能力，飞行控制板12可以提供多个接口来连接外部传感器。In one embodiment of the present invention, the flight control board 12 can adopt the following configuration: STM32F427CortexM4168MHz microprocessor, RAM is 256KB, and flash storage is 2MB. In addition to the main CPU, there is also an STM32F103 processor to prevent the drone from losing control when the main processor fails. In order to expand the autonomous capability, the flight control board 12 may provide multiple interfaces for connecting external sensors.
本发明一个实施例中，飞行控制板12可以具有多种飞行模式满足不同的任务需求，包括：手动模式、高度和位置稳定模式、任务模式、悬停模式和离 板模式。其中，离板模式可以获取外部的(如通过串口连接的主控计算机)设置点和控制命令，包括位置、姿态、速度和加速度等，飞行控制板的控制器将根据给定的目标值进行解算和飞行。In one embodiment of the present invention, the flight control board 12 can have multiple flight modes to meet different mission requirements, including: manual mode, altitude and position stabilization mode, mission mode, hover mode and off-board mode. Among them, the off-board mode can obtain external (such as the main control computer connected through the serial port) set points and control commands, including position, attitude, speed and acceleration, etc., and the controller of the flight control board will solve the problem according to the given target value. Count and fly.
本发明实施例中，视觉传感器11可以完成对环境图像的采集，根据视觉定位和地图构建算法的需要，可以采用能提供640*480像素分辨率的图像，并以每秒最高60帧的采集频率完成图像的获取(或者320*240分辨率下120帧每秒)，有一个75度的广角摄像模式的视觉传感器。In the embodiment of the present invention, the visual sensor 11 can complete the collection of environmental images. According to the needs of visual positioning and map construction algorithms, images with a resolution of 640*480 pixels can be used, and the collection frequency can be up to 60 frames per second. To complete image acquisition (or 120 frames per second at 320*240 resolution), there is a visual sensor with a 75-degree wide-angle camera mode.
主控计算机13主要处理视觉传感器11的数据并对整个平台进行自主控制。主控计算机11的配置硬件参数可以如表1所示，主控计算机13通过USB hub扩展连接TP-link-WN823N USB Wi-Fi模块和飞行控制板TELEM2端口，建立在板主控计算机13与飞行控制板12以及地面站计算机的可靠通信。另一个USB口则直接连接PS3Eye单目视觉传感器，在板主控计算机搭载Ubuntu 12.04LTS操作系统、Hydro版的机器人操作系统(ROS)和OpenCV图像处理库。The main control computer 13 mainly processes the data of the visual sensor 11 and controls the whole platform autonomously. The configuration hardware parameters of the main control computer 11 can be shown in Table 1. The main control computer 13 is connected to the TP-link-WN823N USB Wi-Fi module and the TELEM2 port of the flight control board through the USB hub expansion, and the main control computer 13 is established on the board. Reliable communication between the control board 12 and the ground station computer. The other USB port is directly connected to the PS3Eye monocular vision sensor, and the main control computer on the board is equipped with the Ubuntu 12.04LTS operating system, the Hydro version of the robot operating system (ROS) and the OpenCV image processing library.
表1 主控计算机13配置硬件参数Table 1 Main control computer 13 configuration hardware parameters
本发明实施例提供一种无人机控制方法，所述方法应用于图1所示的无人机控制系统，该系统包括：视觉传感器11、飞行控制板12、主控计算机13、无人机14，如图2所示，所述主控计算机13包括依次连接的图像采集模块131、处理模块132、通信接口模块133、控制模块134，所述方法包括：An embodiment of the present invention provides a method for controlling an unmanned aerial vehicle. The method is applied to the unmanned aerial vehicle control system shown in FIG. 14. As shown in FIG. 2, the main control computer 13 includes an image acquisition module 131, a processing module 132, a communication interface module 133, and a control module 134 connected in sequence, and the method includes:
所述视觉传感器11采集环境图像信息；The visual sensor 11 collects environmental image information;
所述图像采集模块131获取所述视觉传感器11采集的环境图像信息，并将所述环境图像信息传输至所述处理模块132；The image acquisition module 131 acquires the environmental image information collected by the visual sensor 11, and transmits the environmental image information to the processing module 132;
所述处理模块132根据所述环境图像信息，得到无人机的高度、位置和姿态估计，并将视觉位姿估计通过特定消息形式发布到主题，同时发布地图主题输出地图信息；The processing module 132 obtains the height, position and attitude estimation of the UAV according to the environmental image information, and publishes the visual pose estimation to the theme through a specific message form, and simultaneously publishes the map theme to output map information;
所述飞行控制板12将无人机当前位置、姿态和速度等信息通过MAVLink协议发送给所述通信接口模块133；The flight control board 12 sends information such as the current position, attitude and speed of the drone to the communication interface module 133 through the MAVLink protocol;
所述控制模块134分析订阅得到的无人机当前各个状态，并将控制指令发布到主题，供通信接口节点订阅。The control module 134 analyzes the current states of the drone obtained through subscription, and publishes control instructions to topics for subscription by communication interface nodes.
所述通信接口模块133整合从所述图像采集模块订阅得到的视觉估计位姿信息以及从所述控制模块134订阅得到的控制命令，并通过MAVLink协议发送给所述飞行控制板12，所述飞行控制板12对所述无人机进行控制，使得所述无人机14按照控制命令结合视觉位姿估计结果完成相应的自主飞行。The communication interface module 133 integrates the visual estimation pose information subscribed from the image acquisition module and the control command subscribed from the control module 134, and sends them to the flight control board 12 through the MAVLink protocol. The control board 12 controls the UAV, so that the UAV 14 completes the corresponding autonomous flight according to the control command and the visual pose estimation result.
本发明实施例的无人机控制方法，利用视觉传感器获取环境图像数据后，传输到搭载在无人机上的主控计算机，主控计算机对接收到的图像进行处理，完成位置和姿态估计以及地图构建，之后将相对位置作为飞行参数直接送到飞行控制板，飞行控制板解算当前飞行器的位置、高度和姿态数据后，对所述无人机进行控制，使得所述无人机按照控制命令结合视觉位姿估计结果完成相应 的自主飞行。In the UAV control method of the embodiment of the present invention, after using the visual sensor to obtain the environmental image data, it is transmitted to the main control computer mounted on the UAV, and the main control computer processes the received image to complete the position and attitude estimation and the map After the construction, the relative position is directly sent to the flight control board as a flight parameter. After the flight control board solves the position, altitude and attitude data of the current aircraft, it controls the UAV so that the UAV follows the control command. Combined with the visual pose estimation results to complete the corresponding autonomous flight.
本发明实施例中，视觉传感器11可以为单目视觉传感器，视觉传感器11采集环境图像信息传输给主控计算机13的图像采集模块131，完成图像预处理。In the embodiment of the present invention, the visual sensor 11 may be a monocular visual sensor, and the visual sensor 11 collects environmental image information and transmits it to the image acquisition module 131 of the main control computer 13 to complete image preprocessing.
通过处理模块132根据视觉定位与地图构建算法对图像帧进行处理，完成无人机14的高度、位置和姿态估计，并将视觉位姿估计通过特定消息形式发布到主题，同时发布地图主题输出地图信息。The image frame is processed by the processing module 132 according to the visual positioning and map construction algorithm, the height, position and attitude estimation of the UAV 14 are completed, and the visual pose estimation is published to the topic through a specific message form, and the map topic output map is released at the same time information.
飞行控制板12将无人机当前位置、姿态和速度等信息通过MAVLink协议发送给通信接口模块133，主控计算机13获取无人机14的当前飞行状态。The flight control board 12 sends information such as the current position, attitude and speed of the UAV to the communication interface module 133 through the MAVLink protocol, and the main control computer 13 obtains the current flight status of the UAV 14 .
控制模块134分析订阅得到的无人机当前各个状态，并将控制指令发布到主题，供通信接口节点订阅。The control module 134 analyzes the current states of the drone obtained through subscription, and publishes control instructions to topics for subscription by communication interface nodes.
通信接口模块133整合从图像采集模块131订阅得到的视觉估计位姿信息以及从控制模块134订阅得到的控制命令，并通过MAVLink协议发送给飞行控制板12，无人机14将按照控制命令结合视觉位姿估计结果完成相应的自主飞行。The communication interface module 133 integrates the visual estimation pose information subscribed from the image acquisition module 131 and the control command subscribed from the control module 134, and sends it to the flight control board 12 through the MAVLink protocol, and the UAV 14 will combine the visual information according to the control command. The pose estimation results complete the corresponding autonomous flight.
本发明实施例中，可选的，所述通信接口模块133从所述处理模块订阅定位结果，根据所述定位结果进行定位。In the embodiment of the present invention, optionally, the communication interface module 133 subscribes to the positioning result from the processing module, and performs positioning according to the positioning result.
可选的，所述通信接口模块133订阅自主控制命令主题，根据所述自主控制命令主题实现对无人机进行控制。Optionally, the communication interface module 133 subscribes to an autonomous control command topic, and controls the UAV according to the autonomous control command topic.
可选的，还包括：所述通信接口模块133通过所述控制板获取无人机当前局部位置。Optionally, it also includes: the communication interface module 133 acquires the current local position of the drone through the control board.
可选的，所述通信接口模块133设置有第一接口、第二接口，所述通信接口模块通过所述第一接口(/mavlink/to)获取飞控心跳包、位置数据包、姿态角数据包，所述通信接口模块通过所述第二接口(/mavlink/from)获取视觉位姿估计结果和控制命令。Optionally, the communication interface module 133 is provided with a first interface and a second interface, and the communication interface module obtains flight control heartbeat packets, position data packets, and attitude angle data through the first interface (/mavlink/to) package, the communication interface module obtains the visual pose estimation result and control commands through the second interface (/mavlink/from).
本发明实施例中，通信接口模块133可以通过Mavlink协议接收无人机14 的心跳包、姿态角数据包和位置数据包等，并按类别以特定消息格式将数据发布到不同的主题上。同时订阅了vision和position等主题，再以Mavlink协议的形式将这些主题数据发送到飞行控制板12，获取外部传感器的数据并实现自主飞行控制。In the embodiment of the present invention, the communication interface module 133 can receive the heartbeat packet, attitude angle data packet and position data packet of the UAV 14 through the Mavlink protocol, and publish the data to different topics in a specific message format by category. At the same time, subscribe to topics such as vision and position, and then send these topic data to the flight control board 12 in the form of the Mavlink protocol to obtain data from external sensors and realize autonomous flight control.
本发明实施例中，通信接口模块133可以订阅的不同主题，也可以发布不同的主题，通信接口模块133可以与飞行控制板12之间信息传输的通信协议，通信接口模块133设置有/mavlink/to接口、/mavlink/from接口，/mavlink/to是通信接口模块133获取飞控心跳包、位置数据包、姿态角数据包等信息的通道，/mavlink/from是飞行控制板12从通信接口模块133获取视觉位姿估计结果和控制命令等信息的通道。In the embodiment of the present invention, the communication interface module 133 can subscribe to different topics, and can also publish different topics. The communication interface module 133 can communicate with the communication protocol for information transmission between the flight control board 12. The communication interface module 133 is provided with /mavlink/ to interface, /mavlink/from interface, /mavlink/to is the channel for the communication interface module 133 to obtain information such as flight control heartbeat packets, position data packets, and attitude angle data packets, and /mavlink/from is the communication interface module for the flight control board 12 133 A channel for obtaining information such as visual pose estimation results and control commands.
本发明实施例中，通信接口模块133可以订阅/mavros/position/vision、/mavros/setpoint/local_position、/mavros/rc/override、/mavros/safety_area/set、/mavros/mocap/pose等主题。在订阅的主题中，/mavros/position/vision为视觉定位与地图构建方法中的定位结果，将跟踪线程中的位置估计结果主题数据发布到该主题，即飞行控制板12获取视觉传感器的当前位置，实现定位。/mavros/setpoint/local_position为自主控制命令主题，将位置控制命令发布到该主题实现对无人机的自主控制。In the embodiment of the present invention, the communication interface module 133 can subscribe to topics such as /mavros/position/vision, /mavros/setpoint/local_position, /mavros/rc/override, /mavros/safety_area/set, /mavros/mocap/pose, etc. In the subscribed topic, /mavros/position/vision is the positioning result in the visual positioning and map construction method, and the topic data of the position estimation result in the tracking thread is published to this topic, that is, the flight control board 12 obtains the current position of the visual sensor , to achieve positioning. /mavros/setpoint/local_position is the subject of the autonomous control command, and the position control command is issued to this topic to realize the autonomous control of the UAV.
本发明实施例中，通信接口模块133可以发布/mavros/position/local、/mavros/imu/data、/mavros/optical_flow、/mavros/state、/mavros/rc、/mavros/battery、/mavros/camera_image、/mavros/mission/waypoints、/mavros/imu/aum_pressure等主题。在发布的主题中，/mavros/position/local是飞行控制板12在获取/mavros/position/vision主题视觉定位结果后识别的无人机当前局部位置，/mavros/imu/data为惯性测量单元数据。其它还包括气压计、遥控数据和飞行任务等主题信息。In the embodiment of the present invention, the communication interface module 133 can publish /mavros/position/local, /mavros/imu/data, /mavros/optical_flow, /mavros/state, /mavros/rc, /mavros/battery, /mavros/camera_image , /mavros/mission/waypoints, /mavros/imu/aum_pressure and other topics. In the published topic, /mavros/position/local is the current local position of the UAV recognized by the flight control board 12 after obtaining the visual positioning result of the topic /mavros/position/vision, and /mavros/imu/data is the data of the inertial measurement unit . Other topics include barometer, remote control data, and flight missions.
可选的，所述控制模块133获取所述无人机当前位置，生成位置控制指令，将所述位置控制命令发布到所述通信接口模块，并通过监测所述无人机的状态 调整所述位置控制指令。Optionally, the control module 133 acquires the current position of the UAV, generates a position control command, issues the position control command to the communication interface module, and adjusts the UAV by monitoring the status of the UAV. position control command.
本发明实施例中，控制模块134订阅飞行器的当前位置主题，提供位置反馈，再将位置控制命令发布到通信接口模块的/mavros/setpoint/local_position主题上。通过对无人机状态的实时监测调整位置控制信息。In the embodiment of the present invention, the control module 134 subscribes to the current position topic of the aircraft, provides position feedback, and then issues the position control command to the /mavros/setpoint/local_position topic of the communication interface module. Adjust position control information through real-time monitoring of UAV status.
可选的，所述处理模块133对所述无人机当前的位置和姿态进行估计，所述处理模块实时向所述飞行控制板提供当前的位置和姿态；Optionally, the processing module 133 estimates the current position and attitude of the UAV, and the processing module provides the current position and attitude to the flight control board in real time;
所述飞行控制板12通过订阅主题的方式获取当前的位置估计结果(x,y,z)和四元素(x,y,z,w)形式的姿态估计结果。The flight control board 12 obtains the current position estimation result (x, y, z) and the attitude estimation result in the form of four elements (x, y, z, w) by subscribing to a topic.
本发明实施例中，跟踪线程是对相机获取的每一帧图像进行处理，完成对每一帧图像的位姿估计，跟踪线程框架如图3所示。In the embodiment of the present invention, the tracking thread processes each frame of image acquired by the camera, and completes the pose estimation of each frame of image. The frame of the tracking thread is shown in FIG. 3 .
本发明实施例中，跟踪线程主要对当前飞行器的位置和姿态进行估计，在主控计算机处理跟踪线程的同时，需要实时地使飞行控制板获取当前的位置和姿态。所以需要在跟踪线程中完成位姿估计结果的主题发布。飞行器的飞行控制板通过订阅主题的方式获取当前传感器的位置估计结果(x,y,z)和四元素(x,y,z,w)形式的姿态估计结果，所以跟踪线程在ROS中通过PoseStamped消息类型发布一个名为pose的主题。In the embodiment of the present invention, the tracking thread mainly estimates the current position and attitude of the aircraft. While the main control computer is processing the tracking thread, it is necessary to make the flight control board acquire the current position and attitude in real time. Therefore, the topic release of the pose estimation results needs to be completed in the tracking thread. The flight control board of the aircraft obtains the position estimation result (x, y, z) of the current sensor and the attitude estimation result in the form of four elements (x, y, z, w) by subscribing to the topic, so the tracking thread passes PoseStamped in ROS The message type publishes a topic called pose.
可选的，所述处理模块132对每个新的关键帧Ki完成局部地图构建，步骤包括：Optionally, the processing module 132 completes local map construction for each new key frame Ki , and the steps include:
S1、插入关键帧；S1, insert key frame;
S2、淘汰最近地图点；S2. Eliminate the nearest map point;
S3、创建新的地图点；S3. Create a new map point;
S4、局部光束平差；S4. Local beam adjustment;
S5、局部关键帧淘汰。S5. Local key frame elimination.
可选的，所述处理模块132针对局部地图构建线程中最后处理的关键帧Ki进行环路闭合检测线程，所述处理模块通过关键帧Ki的词袋向量计算其与系统 中关键帧的相似性，完成候选的闭环检测，如果有多个位置的关键帧与Ki的场景外观相似，则有多个闭环候选；Optionally, the processing module 132 performs a loop closure detection thread on the last processed key frame Ki in the local map construction thread, and the processing module calculates its relationship with the key frame in the system through the bag-of-words vector of the key frame Ki Similarity, to complete candidate loop closure detection, if there arekeyframes with multiple positions similar to the scene appearance of Ki, there are multiple loop closure candidates;
所述处理模块132计算当前帧Ki与闭环帧Kl之间的相似变换，对回路中的累计误差进行修正，同时相似性变换将作为闭环的几何验证；The processing module 132 calculates the similarity transformation between the current frame Ki and the closed-loop frame Kl , and corrects the cumulative error in the loop, and the similarity transformation will be used as a geometric verification of the closed loop;
所述处理模块132确定闭环后，通过计算得到的相似性变换对当前关键帧的位姿进行修正，同时这个修正将影响到与当前关键帧相邻的关键帧节点，进行位姿图优化达到全局一致性，从而每个地图点根据修正的关键帧进行变换。After the processing module 132 determines the closed loop, the pose of the current key frame is corrected by the calculated similarity transformation, and this correction will affect the key frame nodes adjacent to the current key frame, and the pose graph is optimized to achieve global Consistency, so that each map point is transformed according to the corrected keyframe.
本发明实施例中，环路闭合检测线程主要是针对局部地图构建线程中最后处理的关键帧Ki来进行。通过关键帧Ki的词袋向量计算其与系统中关键帧的相似性，完成候选的闭环检测。倘若有多个位置的关键帧与Ki的场景外观相似，则有多个闭环候选。In the embodiment of the present invention, the loop closure detection thread is mainly performed on the last processed key frame Ki in the partial map construction thread. Compute the similarity between the bag-of-words vector of the key frame Ki and the key frame in the system, and complete the candidate loop closure detection. If there arekeyframes at multiple locations with similar scene appearance to Ki, there are multiple loop closure candidates.
在无人机14的视觉定位与地图构建中，地图的偏移有7个自由度，3个旋转、3个平移和1个尺度。所以判定是否为一个闭合回路，需要计算当前帧Ki与闭环帧Kl之间的相似变换，对回路中的累计误差进行修正。同时相似性变换将作为闭环的几何验证。In the visual positioning and map construction of the UAV 14, the offset of the map has 7 degrees of freedom, 3 rotations, 3 translations and 1 scale. Therefore, to determine whether it is a closed loop, it is necessary to calculate the similarity transformation between the current frame Ki and the closed loop frame Kl , and correct the cumulative error in the loop. At the same time, the similarity transformation will serve as a geometric verification of the closed loop.
在确定了闭环后，则需要完成闭环修正，通过计算得到的相似性变换对当前关键帧的位姿进行修正，同时这个修正将影响到与当前关键帧相邻的关键帧节点。最后进行位姿图优化达到全局一致性，优化后，每个地图点根据修正的关键帧进行变换。After the closed loop is determined, it is necessary to complete the closed loop correction, and correct the pose of the current keyframe through the calculated similarity transformation, and this correction will affect the keyframe nodes adjacent to the current keyframe. Finally, the pose graph is optimized to achieve global consistency. After optimization, each map point is transformed according to the corrected key frame.
本发明实施例中，如图4所示，该系统还可以包括地面站计算机15。本发明实施例中，为了搭建地面站系统完成程序的启动和初始化，并实现对无人机飞行状态的实时监控，采用一台无线路由器组网，自动分配IP地址到主控计算机和地面站计算机，使得两者完成关联，完成主控计算机画面的无线传输。In the embodiment of the present invention, as shown in FIG. 4 , the system may further include a ground station computer 15 . In the embodiment of the present invention, in order to build the ground station system to complete the startup and initialization of the program, and realize the real-time monitoring of the flight status of the drone, a wireless router is used to form a network, and the IP address is automatically assigned to the main control computer and the ground station computer. , so that the two complete the association and complete the wireless transmission of the main control computer screen.
地面站计算机15基于虚拟网络计算机(VNC)远程控制工具完成，在板主控计算机为服务端(vncserver)，搭载Ubuntu操作系统，地面站计算机为客 户端(vncviewer)，搭载WIN7操作系统。The ground station computer 15 is completed based on the virtual network computer (VNC) remote control tool. The main control computer on the board is the server (vncserver), equipped with the Ubuntu operating system, and the ground station computer is the client (vncviewer), equipped with the WIN7 operating system.
地面站计算机15的VNC的客户端访问工具选用VNCViewer，启动工具后需要对IP地址进行设置，Server窗口为主控计算机服务端的IP地址。连接成功后可看到服务端——在板主控计算机的画面，同时在地面站客户端能对服务端的主控计算机实时进行启动、设置等操作。The VNC client access tool of the ground station computer 15 selects VNCViewer for use, and the IP address needs to be set after starting the tool, and the Server window is the IP address of the server of the main control computer. After the connection is successful, you can see the screen of the server-the main control computer on the board, and at the same time, the ground station client can start and set up the main control computer of the server in real time.
如图4所示，所述无人机14包括位置控制器141、姿态控制器142、执行电机143，所述位置控制器141接收飞行控制信息，进行姿态的解算后，通过所述姿态控制器控142制所述执行电机143，从而对所述无人机14进行飞行控制。As shown in Figure 4, the UAV 14 includes a position controller 141, an attitude controller 142, and an execution motor 143. The position controller 141 receives flight control information, and after calculating the attitude, the attitude control The controller 142 controls the executive motor 143 to control the flight of the UAV 14 .
本发明实施例的无人机控制方法，利用单目视觉传感器获取环境图像数据后，传输到搭载在无人机上的在板主控计算机，主控计算机对接收到的图像进行处理，完成位置和姿态估计以及地图构建，之后将相对位置作为飞行参数直接送到底层的飞行控制板，飞行控制板解算当前飞行器的位置、高度和姿态数据后传送回主控计算机，主控计算机将控制命令参数发送到飞行器的位置控制器，形成控制回路。位置控制器则完成姿态的解算后，通过姿态控制器对飞行器进行自主飞行控制。In the UAV control method of the embodiment of the present invention, after using the monocular vision sensor to obtain the environmental image data, it is transmitted to the on-board main control computer mounted on the UAV, and the main control computer processes the received image to complete the position and Attitude estimation and map construction, and then the relative position is sent directly to the underlying flight control board as flight parameters. The flight control board calculates the current position, altitude and attitude data of the aircraft and sends them back to the main control computer. The main control computer will control the command parameters. Send to the position controller of the aircraft to form a control loop. After the position controller completes the calculation of the attitude, the autonomous flight control of the aircraft is carried out through the attitude controller.
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。Having described various embodiments of the present invention, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明 的保护范围应以权利要求的保护范围为准。The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610269672.1ACN105974932B (en) | 2016-04-27 | 2016-04-27 | Unmanned aerial vehicle (UAV) control method |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610269672.1ACN105974932B (en) | 2016-04-27 | 2016-04-27 | Unmanned aerial vehicle (UAV) control method |
| Publication Number | Publication Date |
|---|---|
| CN105974932A CN105974932A (en) | 2016-09-28 |
| CN105974932Btrue CN105974932B (en) | 2018-11-09 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610269672.1AExpired - Fee RelatedCN105974932B (en) | 2016-04-27 | 2016-04-27 | Unmanned aerial vehicle (UAV) control method |
| Country | Link |
|---|---|
| CN (1) | CN105974932B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106873619B (en)* | 2017-01-23 | 2021-02-02 | 上海交通大学 | Processing method of flight path of unmanned aerial vehicle |
| CN111247576A (en)* | 2017-08-11 | 2020-06-05 | 联想(北京)有限公司 | Subscription information configuration |
| CN107526681B (en)* | 2017-08-11 | 2020-05-26 | 上海木木聚枞机器人科技有限公司 | Robot test system and test method |
| CN107450583A (en)* | 2017-08-23 | 2017-12-08 | 浙江工业大学 | Unmanned plane motion tracking system based on the valiant imperial processor of high pass |
| CN107831776A (en)* | 2017-09-14 | 2018-03-23 | 湖南优象科技有限公司 | Unmanned plane based on nine axle inertial sensors independently makes a return voyage method |
| CN109816769A (en)* | 2017-11-21 | 2019-05-28 | 深圳市优必选科技有限公司 | Scene map generation method, device and equipment based on depth camera |
| CN108845587A (en)* | 2018-06-08 | 2018-11-20 | 赫星科技有限公司 | Unmanned aerial vehicle real-time control system and unmanned aerial vehicle |
| CN110455285A (en)* | 2019-07-22 | 2019-11-15 | 深圳联合飞机科技有限公司 | A kind of Navigation of Pilotless Aircraft method and navigation device in satellite navigation signals failure |
| CN111178342B (en)* | 2020-04-10 | 2020-07-07 | 浙江欣奕华智能科技有限公司 | Pose graph optimization method, device, equipment and medium |
| CN111796603A (en)* | 2020-06-16 | 2020-10-20 | 五邑大学 | Smoke inspection unmanned aerial vehicle system, inspection detection method and storage medium |
| CN112802104B (en)* | 2021-02-04 | 2022-09-16 | 华南理工大学 | Loop detection method based on RGB-D camera |
| CN113409485B (en)* | 2021-08-03 | 2023-12-12 | 广东电网有限责任公司佛山供电局 | Inspection data acquisition method and device, computer equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102967297A (en)* | 2012-11-23 | 2013-03-13 | 浙江大学 | Space-movable visual sensor array system and image information fusion method |
| CN104537709A (en)* | 2014-12-15 | 2015-04-22 | 西北工业大学 | Real-time three-dimensional reconstruction key frame determination method based on position and orientation changes |
| CN104950906A (en)* | 2015-06-15 | 2015-09-30 | 中国人民解放军国防科学技术大学 | Unmanned aerial vehicle remote measuring and control system and method based on mobile communication network |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102967297A (en)* | 2012-11-23 | 2013-03-13 | 浙江大学 | Space-movable visual sensor array system and image information fusion method |
| CN104537709A (en)* | 2014-12-15 | 2015-04-22 | 西北工业大学 | Real-time three-dimensional reconstruction key frame determination method based on position and orientation changes |
| CN104950906A (en)* | 2015-06-15 | 2015-09-30 | 中国人民解放军国防科学技术大学 | Unmanned aerial vehicle remote measuring and control system and method based on mobile communication network |
| Title |
|---|
| 《一种基于计算机视觉的无人机实时三维重建方法》;张臻炜 等;《设计与研究》;20151224;第19-22页* |
| 《基于ROS的室内四旋翼飞行器SLAM研究》;商博;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》;20160315;第7-16，22-51页* |
| Publication number | Publication date |
|---|---|
| CN105974932A (en) | 2016-09-28 |
| Publication | Publication Date | Title |
|---|---|---|
| CN105974932B (en) | Unmanned aerial vehicle (UAV) control method | |
| US10732647B2 (en) | Multi-sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft micro-aerial vehicle (MAV) | |
| Grabe et al. | On-board velocity estimation and closed-loop control of a quadrotor UAV based on optical flow | |
| CN110347171B (en) | Aircraft control method and aircraft | |
| Shen et al. | Vision-based state estimation for autonomous rotorcraft MAVs in complex environments | |
| US10037028B2 (en) | Systems, devices, and methods for on-board sensing and control of micro aerial vehicles | |
| CN105022401B (en) | Vision-based collaborative SLAM method for multi-quadrotor UAVs | |
| Sa et al. | Build your own visual-inertial drone: A cost-effective and open-source autonomous drone | |
| Meier et al. | PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision | |
| Blösch et al. | Vision based MAV navigation in unknown and unstructured environments | |
| Mohta et al. | Vision-based control of a quadrotor for perching on lines | |
| CN107817820A (en) | A kind of unmanned plane autonomous flight control method and system based on deep learning | |
| WO2017020856A1 (en) | Photographing device and method using drone to automatically track and photograph moving object | |
| WO2021081774A1 (en) | Parameter optimization method and apparatus, control device, and aircraft | |
| CN205015741U (en) | Companion flies unmanned aerial vehicle system of taking photo by plane and control cabinet and unmanned aerial vehicle thereof | |
| CN111796603A (en) | Smoke inspection unmanned aerial vehicle system, inspection detection method and storage medium | |
| WO2022016534A1 (en) | Flight control method of unmanned aerial vehicle and unmanned aerial vehicle | |
| Loianno et al. | A swarm of flying smartphones | |
| Courbon et al. | Visual navigation of a quadrotor aerial vehicle | |
| Valenti et al. | An autonomous flyer photographer | |
| CN112050814A (en) | A visual navigation system and method for unmanned aerial vehicle in indoor substation | |
| CN108780321A (en) | For the method for equipment pose adjustment, equipment, system and computer readable storage medium | |
| CN109521785B (en) | A portable camera intelligent rotorcraft system | |
| Weiss et al. | Inertial optical flow for throw-and-go micro air vehicles | |
| CN119270908A (en) | UAV fixed-point delivery control system and method based on inertial guidance |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20181109 |