技术领域technical field
本发明涉及一种光电吊舱技术领域，特别是涉及一种基于自主视觉跟踪的光电吊舱的控制系统及方法。The invention relates to the technical field of photoelectric pods, in particular to a control system and method for photoelectric pods based on autonomous visual tracking.
背景技术Background technique
光电吊舱在实际各种类型的飞行器和机器人中具有非常广泛的应用，常用于监控。光电吊舱云台常用安装方法是悬挂于建筑物顶部和飞行器底部，以完成对云台下方空间得实时监控。但是，现有的光电吊舱无法自主实现对目标的跟踪。Photoelectric pods have a very wide range of applications in various types of aircraft and robots, and are often used for monitoring. The common installation method of the photoelectric pod gimbal is to hang it on the top of the building and the bottom of the aircraft to complete the real-time monitoring of the space below the gimbal. However, existing photoelectric pods cannot autonomously track targets.
发明内容Contents of the invention
(一)要解决的技术问题(1) Technical problems to be solved
为了解决现有技术的上述问题，本发明提供一种基于自主视觉跟踪的光电吊舱的控制系统及方法。In order to solve the above-mentioned problems in the prior art, the present invention provides a control system and method for a photoelectric pod based on autonomous vision tracking.
(二)技术方案(2) Technical solution
为了达到上述目的，本发明实施例中提供一种基于自主视觉跟踪的光电吊舱的控制系统，光电吊舱用于承载图像采集元件，所述控制系统包括：In order to achieve the above purpose, an embodiment of the present invention provides a control system of a photoelectric pod based on autonomous visual tracking, the photoelectric pod is used to carry image acquisition components, and the control system includes:
控制器，与所述光电吊舱的驱动机构连接，用于控制所述驱动机构驱动所述光电吊舱转动；A controller, connected to the driving mechanism of the photoelectric pod, is used to control the driving mechanism to drive the photoelectric pod to rotate;
姿态获取模块，用于实时获取光电吊舱的姿态信息，所述姿态信息包括光电吊舱的俯仰角、横滚角、偏航角；The attitude acquisition module is used to obtain the attitude information of the photoelectric pod in real time, and the attitude information includes the pitch angle, roll angle, and yaw angle of the photoelectric pod;
距离获取模块，用于实时获取待跟踪目标和光电吊舱之间的直线距离；The distance acquisition module is used to obtain the straight-line distance between the target to be tracked and the photoelectric pod in real time;
所述控制器与所述姿态获取模块和距离获取模块连接，用于根据所述姿态信息和直线距离计算待跟踪目标的位置信息；The controller is connected to the attitude acquisition module and the distance acquisition module, and is used to calculate the position information of the target to be tracked according to the attitude information and the straight-line distance;
所述控制器还用于根据所述位置信息和待跟踪目标的设定位置信息生成第一控制指令，以控制所述驱动机构驱动所述光电吊舱转动，使得待跟踪目标处于设定位置，实现对待跟踪目标的跟踪，所述第一控制指令包括跟踪指令和光电吊舱的转动角速度。The controller is further configured to generate a first control instruction according to the position information and the set position information of the target to be tracked, so as to control the driving mechanism to drive the photoelectric pod to rotate so that the target to be tracked is at a set position, To realize the tracking of the target to be tracked, the first control instruction includes the tracking instruction and the rotational angular velocity of the photoelectric pod.
可选的，所述距离获取模块为无载波通信模块，包括：Optionally, the distance acquisition module is a carrierless communication module, including:
安装在光电吊舱上的标签模块，所述控制器与所述标签模块连接；A label module installed on the photoelectric pod, the controller is connected to the label module;
安装在待跟踪目标上的基站模块，所述标签模块在自身的时间戳上通过无载波通信的方式向基站模块发送请求信号，所述基站模块在自身的时间戳上接收标签模块发送的请求信号，然后向标签模块发送反馈信号，所述标签模块在自身的时间戳上接收所述反馈信号；The base station module installed on the target to be tracked, the tag module sends a request signal to the base station module through carrierless communication on its own time stamp, and the base station module receives the request signal sent by the tag module on its own time stamp , and then send a feedback signal to the label module, and the label module receives the feedback signal at its own time stamp;
第一计算模块，利用标签模块在其时间戳上发送请求信号和接收到反馈信号的时间差，以及基站模块在其时间戳上接收到请求信号和发送反馈信号的时间差，计算待跟踪目标与光电吊舱之间的直线距离。The first calculation module uses the time difference between the tag module sending the request signal and receiving the feedback signal on its time stamp, and the time difference between the base station module receiving the request signal and sending the feedback signal on its time stamp to calculate the distance between the target to be tracked and the photoelectric crane. The straight-line distance between cabins.
可选的，所述控制系统还包括：Optionally, the control system also includes:
显示屏，所述控制器与所述显示屏连接，用于控制所述显示屏显示所述图像采集元件采集的图像，以及待跟踪目标的位置信息。A display screen, the controller is connected to the display screen, and is used to control the display screen to display the image collected by the image acquisition element and the position information of the target to be tracked.
可选的，所述控制系统还包括：Optionally, the control system also includes:
遥控装置，所述控制器与所述遥控装置连接，用于获取用户通过遥控装置发送的第二控制指令，并根据所述第二控制指令控制所述驱动机构驱动所述光电吊舱转动，以使得待跟踪目标位于图像采集元件的视角范围内，所述第二控制指令包括光电吊舱的工作模式、转动角速度；a remote control device, the controller is connected to the remote control device, and is used to obtain a second control command sent by the user through the remote control device, and control the drive mechanism to drive the photoelectric pod to rotate according to the second control command, so as to The target to be tracked is located within the viewing angle range of the image acquisition element, and the second control instruction includes the working mode and rotational angular velocity of the photoelectric pod;
所述遥控装置包括按键和遥杆；The remote control device includes buttons and a joystick;
所述控制器获取用户通过按键设定的光电吊舱的工作模式指令，并控制光电吊舱的工作模式为用户设定的工作模式；The controller obtains the working mode instruction of the photoelectric pod set by the user through the button, and controls the working mode of the photoelectric pod to be the working mode set by the user;
所述控制器获取用户通过遥杆设定的光电吊舱转动的角速度，并控制驱动机构驱动光电吊舱以用户设定的角速度转动。The controller obtains the rotational angular velocity of the photoelectric pod set by the user through the joystick, and controls the driving mechanism to drive the photoelectric pod to rotate at the angular velocity set by the user.
可选的，所述控制系统还包括第二计算模块，用于根据设定关系来计算与存储待跟踪目标和光电吊舱之间的直线距离对应的图像采集元件的焦距；Optionally, the control system further includes a second calculation module, which is used to calculate the focal length of the image acquisition element corresponding to the linear distance between the stored target to be tracked and the photoelectric pod according to the set relationship;
所述图像采集元件包括调焦模块，所述控制器与所述第二计算模块和调焦模块连接，用于根据所述焦距控制调焦模块进行调焦，以使得图像采集元件采集的图像清晰。The image acquisition component includes a focusing module, the controller is connected to the second computing module and the focusing module, and is used to control the focusing module to perform focus adjustment according to the focal length, so that the image collected by the image acquisition component is clear .
本发明实施例中还提供一种基于自主视觉跟踪的光电吊舱的控制方法，其特征在于，包括：An embodiment of the present invention also provides a control method for a photoelectric pod based on autonomous visual tracking, which is characterized in that it includes:
实时获取光电吊舱的姿态信息，所述姿态信息包括光电吊舱的俯仰角、横滚角、偏航角；Obtain the attitude information of the photoelectric pod in real time, and the attitude information includes the pitch angle, roll angle and yaw angle of the photoelectric pod;
实时获取待跟踪目标和光电吊舱之间的直线距离；Obtain the straight-line distance between the target to be tracked and the photoelectric pod in real time;
根据所述姿态信息和直线距离计算待跟踪目标的位置信息；calculating the position information of the target to be tracked according to the attitude information and the straight-line distance;
根据所述位置信息和待跟踪目标的设定位置信息生成第一控制指令，以控制驱动机构驱动光电吊舱转动，使得待跟踪目标处于设定位置，实现对待跟踪目标的跟踪，所述第一控制指令包括跟踪指令和光电吊舱的转动角速度。According to the position information and the set position information of the target to be tracked, a first control instruction is generated to control the drive mechanism to drive the photoelectric pod to rotate, so that the target to be tracked is at the set position, and the track to the target to be tracked is realized. Control instructions include tracking instructions and the rotational angular velocity of the photoelectric pod.
可选的，获取待跟踪目标和光电吊舱之间的直线距离的步骤包括：Optionally, the step of obtaining the linear distance between the target to be tracked and the photoelectric pod includes:
安装在光电吊舱上的标签模块在自身的时间戳上通过无载波通信的方式向安装在待跟踪目标上的基站模块发送请求信号；The label module installed on the photoelectric pod sends a request signal to the base station module installed on the target to be tracked by means of carrierless communication on its own time stamp;
基站模块在自身的时间戳上接收请求信号，然后向标签模块发送反馈信号；The base station module receives the request signal on its own time stamp, and then sends a feedback signal to the tag module;
利用标签模块在其时间戳上发送请求信号和接收到反馈信号的时间差，以及基站模块在其时间戳上接收到请求信号和发送反馈信号的时间差，计算待跟踪目标与光电吊舱之间的直线距离。Calculate the straight line between the target to be tracked and the photoelectric pod by using the time difference between the tag module sending the request signal and receiving the feedback signal on its time stamp, and the time difference between the base station module receiving the request signal and sending the feedback signal on its time stamp distance.
可选的，所述控制方法还包括：Optionally, the control method also includes:
显示所述图像采集元件采集的图像，以及待跟踪目标的位置信息。The image collected by the image collection component and the position information of the target to be tracked are displayed.
可选的，所述控制方法还包括：Optionally, the control method also includes:
获取用户通过遥控装置发送的第二控制指令，并根据所述第二控制指令控制所述驱动机构驱动所述光电吊舱转动，以使得待跟踪目标位于图像采集元件的视角范围内，所述第二控制指令包括光电吊舱的工作模式、转动角速度。Obtain the second control instruction sent by the user through the remote control device, and control the driving mechanism to drive the photoelectric pod to rotate according to the second control instruction, so that the target to be tracked is located within the viewing angle range of the image acquisition element, the second The second control instruction includes the working mode and rotational angular velocity of the photoelectric pod.
可选的，所述控制方法还包括：Optionally, the control method also includes:
采用数据拟合方法获取待跟踪目标和光电吊舱之间的直线距离与对应的图像采集元件的焦距之间的设定关系；A data fitting method is used to obtain the setting relationship between the linear distance between the target to be tracked and the photoelectric pod and the focal length of the corresponding image acquisition element;
根据所述设定关系计算与存储待跟踪目标和光电吊舱之间的直线距离对应的图像采集元件的焦距；Calculate the focal length of the image acquisition element corresponding to the linear distance between the target to be tracked and the photoelectric pod according to the set relationship;
根据所述焦距控制图像采集元件进行调焦，以使得图像采集元件采集的图像清晰。The image acquisition element is controlled to adjust the focus according to the focal length, so that the image acquired by the image acquisition element is clear.
(三)有益效果(3) Beneficial effects
本发明的有益效果是：本发明利用光电吊舱的姿态信息、待跟踪目标和光电吊舱之间的直线距离，实时计算待跟踪目标的位置信息，通过将待跟踪目标的位置信息与设定位置信息进行比较生成控制指令，来控制光电吊舱的转动，使得待跟踪目标处于设定位置，实现对待跟踪目标的自主视觉跟踪。The beneficial effects of the present invention are: the present invention utilizes the attitude information of the photoelectric pod, the straight-line distance between the target to be tracked and the photoelectric pod to calculate the position information of the target to be tracked in real time, by combining the position information of the target to be tracked with the set The position information is compared to generate control instructions to control the rotation of the photoelectric pod, so that the target to be tracked is at the set position, and the autonomous visual tracking of the target to be tracked is realized.
附图说明Description of drawings
图1为本发明实施例中基于自主视觉跟踪的光电吊舱的控制系统的组成框图；Fig. 1 is the composition block diagram of the control system of the photoelectric pod based on autonomous visual tracking in the embodiment of the present invention;
图2为本发明实施例中基于自主视觉跟踪的光电吊舱的控制系统的方框图；Fig. 2 is the block diagram of the control system based on the photoelectric pod of autonomous visual tracking in the embodiment of the present invention;
图3为本发明实施例中基于自主视觉跟踪的光电吊舱的控制方法的流程图。Fig. 3 is a flow chart of the control method of the photoelectric pod based on autonomous visual tracking in an embodiment of the present invention.
具体实施方式Detailed ways
为了更好的解释本发明，以便于理解，下面结合附图，通过具体实施方式，对本发明作详细描述。In order to better explain the present invention and facilitate understanding, the present invention will be described in detail below through specific embodiments in conjunction with the accompanying drawings.
光电吊舱安装于飞行器底部、建筑物顶部等，用于承载图像采集元件。光电吊舱包括驱动机构，用于驱动图像采集元件对周边空间进行扫描。The photoelectric pod is installed on the bottom of the aircraft, the top of the building, etc., and is used to carry the image acquisition components. The photoelectric pod includes a driving mechanism, which is used to drive the image acquisition element to scan the surrounding space.
结合图1和图2所示，本发明实施例中提供一种基于自主视觉跟踪的光电吊舱的控制系统，包括：As shown in Figure 1 and Figure 2, an embodiment of the present invention provides a control system for a photoelectric pod based on autonomous visual tracking, including:
控制器，与所述光电吊舱的驱动机构连接，用于控制所述驱动机构驱动所述光电吊舱转动；A controller, connected to the driving mechanism of the photoelectric pod, is used to control the driving mechanism to drive the photoelectric pod to rotate;
姿态获取模块，用于实时获取光电吊舱的姿态信息，所述姿态信息包括光电吊舱的俯仰角、横滚角、偏航角；The attitude acquisition module is used to obtain the attitude information of the photoelectric pod in real time, and the attitude information includes the pitch angle, roll angle, and yaw angle of the photoelectric pod;
距离获取模块，用于实时获取待跟踪目标和光电吊舱之间的直线距离；The distance acquisition module is used to obtain the straight-line distance between the target to be tracked and the photoelectric pod in real time;
所述控制器与所述姿态获取模块和距离获取模块连接，用于根据所述姿态信息和直线距离计算待跟踪目标的位置信息；The controller is connected to the attitude acquisition module and the distance acquisition module, and is used to calculate the position information of the target to be tracked according to the attitude information and the straight-line distance;
所述控制器还用于根据所述位置信息和待跟踪目标的设定位置生成第一控制指令，以控制所述驱动机构驱动所述光电吊舱转动，使得待跟踪目标处于设定位置，实现对待跟踪目标的跟踪，所述第一控制指令包括跟踪指令和光电吊舱的转动角速度。The controller is also used to generate a first control instruction according to the position information and the set position of the target to be tracked, so as to control the driving mechanism to drive the photoelectric pod to rotate, so that the target to be tracked is at the set position, and realize For the tracking of the target to be tracked, the first control instruction includes the tracking instruction and the rotational angular velocity of the photoelectric pod.
上述控制系统利用光电吊舱的姿态信息、待跟踪目标和光电吊舱之间的直线距离，实时计算待跟踪目标的位置信息，通过将待跟踪目标的位置信息与设定位置信息进行比较生成控制指令，来控制光电吊舱的转动，使得待跟踪目标处于设定位置，实现对待跟踪目标的自主视觉跟踪。The above control system uses the attitude information of the photoelectric pod and the straight-line distance between the target to be tracked and the photoelectric pod to calculate the position information of the target to be tracked in real time, and generates a control by comparing the position information of the target to be tracked with the set position information. Command to control the rotation of the photoelectric pod, so that the target to be tracked is at the set position, and the autonomous visual tracking of the target to be tracked is realized.
待跟踪目标的位置信息具体可以为待跟踪目标在以光电吊舱为坐标原点建立的坐标系中的坐标。待跟踪目标的设定位置可以为图像采集元件采集的图像的中心。The location information of the target to be tracked may specifically be the coordinates of the target to be tracked in a coordinate system established with the photoelectric pod as the coordinate origin. The set position of the target to be tracked may be the center of the image captured by the image capture component.
所述控制器对光电吊舱的姿态信息和待跟踪目标和光电吊舱之间的直线距离进行数据融合处理，以获取待跟踪目标的位置信息。例如，所述控制器通过三角函数公式的应用，利用光电吊舱的俯仰角、横滚角和所述直线距离，实时计算待跟踪目标相对光电吊舱的相对高度。The controller performs data fusion processing on the attitude information of the photoelectric pod and the linear distance between the target to be tracked and the photoelectric pod to obtain the position information of the target to be tracked. For example, the controller calculates in real time the relative height of the target to be tracked relative to the photoelectric pod by using the pitch angle, roll angle and the straight-line distance of the photoelectric pod through the application of a trigonometric function formula.
需要说明的是，本发明获取的待跟踪目标的位置信息不仅可以用于视觉跟踪，还可以用于其它应用。It should be noted that the location information of the target to be tracked acquired by the present invention can be used not only for visual tracking, but also for other applications.
本实施例中，设置所述控制系统还包括显示屏，所述控制器与所述显示屏连接，用于控制所述显示屏显示所述图像采集元件采集的图像，以及待跟踪目标的位置信息。通过显示图像采集元件采集的图像，以及待跟踪目标的位置信息，有利于实现人机交互。In this embodiment, the control system is set to further include a display screen, the controller is connected to the display screen, and is used to control the display screen to display the image collected by the image acquisition element and the position information of the target to be tracked . By displaying the image collected by the image acquisition component and the position information of the target to be tracked, it is beneficial to realize human-computer interaction.
为了提高图像的显示质量，本实施例中采用数据拟合方法获取待跟踪目标和光电吊舱之间的直线距离与对应的图像采集元件的焦距之间的设定关系，以根据所述设定关系计算与存储待跟踪目标和光电吊舱之间的直线距离对应的图像采集元件的焦距，并根据所述焦距控制图像采集元件进行调焦(即，使得图像采集元件的焦距为上述计算得到的焦距)，以使得图像采集元件采集的图像清晰。In order to improve the display quality of the image, the data fitting method is used in this embodiment to obtain the setting relationship between the linear distance between the target to be tracked and the photoelectric pod and the focal length of the corresponding image acquisition element, so that according to the setting Relational calculation and storing the focal length of the image acquisition element corresponding to the linear distance between the target to be tracked and the photoelectric pod, and controlling the image acquisition element to focus according to the focal length (that is, the focal length of the image acquisition element is obtained by the above calculation focal length), so that the image captured by the image capture element is clear.
相应的，所述控制系统还包括第二计算模块，用于根据设定关系来计算与存储待跟踪目标和光电吊舱之间的直线距离对应的图像采集元件的焦距。所述图像采集元件包括调焦模块，所述控制器与所述第二计算模块和调焦模块连接，用于根据所述焦距控制调焦模块进行调焦，以使得图像采集元件采集的图像清晰，通过获取的存储待跟踪目标和光电吊舱之间的直线距离，对图像采集元件进行实时调焦，使得采集图像清晰，提升跟踪目标性能。Correspondingly, the control system further includes a second calculation module, configured to calculate the focal length of the image acquisition element corresponding to the stored linear distance between the target to be tracked and the photoelectric pod according to the set relationship. The image acquisition component includes a focusing module, the controller is connected to the second computing module and the focusing module, and is used to control the focusing module to perform focus adjustment according to the focal length, so that the image collected by the image acquisition component is clear , by obtaining the straight-line distance between the stored target to be tracked and the photoelectric pod, real-time focus adjustment is performed on the image acquisition component, so that the captured image is clear and the performance of the tracked target is improved.
在一个具体的实施方式中，采用无载波通信技术来获取待跟踪目标和光电吊舱之间的直线距离，则所述距离获取模块为无载波通信模块，包括：In a specific embodiment, the carrier-free communication technology is used to obtain the straight-line distance between the target to be tracked and the photoelectric pod, then the distance acquisition module is a carrier-free communication module, comprising:
安装在光电吊舱上的标签模块，所述控制器与所述标签模块连接；A label module installed on the photoelectric pod, the controller is connected to the label module;
安装在待跟踪目标上的基站模块，所述标签模块在自身的时间戳上通过无载波通信的方式向基站模块发送请求信号，所述基站模块在自身的时间戳上接收标签模块发送的请求信号，然后向标签模块发送反馈信号，所述标签模块在自身的时间戳上接收所述反馈信号；The base station module installed on the target to be tracked, the tag module sends a request signal to the base station module through carrierless communication on its own time stamp, and the base station module receives the request signal sent by the tag module on its own time stamp , and then send a feedback signal to the label module, and the label module receives the feedback signal at its own time stamp;
第一计算模块，利用标签模块在其时间戳上发送请求信号和接收到反馈信号的时间差，以及基站模块在其时间戳上接收到请求信号和发送反馈信号的时间差，计算待跟踪目标与光电吊舱之间的直线距离。The first calculation module uses the time difference between the tag module sending the request signal and receiving the feedback signal on its time stamp, and the time difference between the base station module receiving the request signal and sending the feedback signal on its time stamp to calculate the distance between the target to be tracked and the photoelectric crane. The straight-line distance between cabins.
上述无载波通信模块利用标签模块在自身的时间戳上发送和接收信号时间差t1，以及基站模块在自身的时间戳上接收和反馈信号的时间差t2，来计算待跟踪目标与光电吊舱之间的直线距离d，计算公式为d＝v(t2-t1)，v为光速(v远大于正常工作的飞行状态下的移动速度，且采样频率足够高)。The above carrierless communication module calculates the distance between the target to be tracked and the photoelectric pod by using the time difference t1 between the tag module sending and receiving the signal on its own time stamp and the time difference t 2between the base station module receiving and feeding back the signal on its own time stamp. The straight-line distance d between , the calculation formula is d=v(t2 -t1 ), v is the speed of light (v is much larger than the moving speed in the normal working flight state, and the sampling frequency is high enough).
具体的，所述无载波通信模块为超宽带(UWB)模块，所述请求信号和反馈信号为脉冲信号。Specifically, the carrierless communication module is an ultra-wideband (UWB) module, and the request signal and feedback signal are pulse signals.
当然，也可以利用其它技术来获取待跟踪目标与光电吊舱之间的直线距离，其也属于本发明的保护范围。Of course, other techniques can also be used to obtain the linear distance between the target to be tracked and the photoelectric pod, which also belongs to the protection scope of the present invention.
在对待跟踪目标进行跟踪之前，需要首先对待跟踪目标进行搜索，使其出现在光电吊舱的视角范围内，即，图像采集元件采集的图像中包括待跟踪目标的图像。Before tracking the target to be tracked, it is first necessary to search for the target to be tracked so that it appears within the viewing angle of the photoelectric pod, that is, the image collected by the image acquisition component includes the image of the target to be tracked.
为了实现人机交互的目的，本实施例中设置所述控制系统还包括：In order to realize the purpose of human-computer interaction, setting the control system in this embodiment also includes:
遥控装置，所述控制器与所述遥控装置连接，用于获取用户通过遥控装置发送的第二控制指令，并根据所述第二控制指令控制所述驱动机构驱动所述光电吊舱转动，以使得待跟踪目标位于图像采集元件的视角范围内，所述第二控制指令包括光电吊舱的工作模式、转动角速度。a remote control device, the controller is connected to the remote control device, and is used to obtain a second control command sent by the user through the remote control device, and control the drive mechanism to drive the photoelectric pod to rotate according to the second control command, so as to The target to be tracked is located within the viewing angle range of the image acquisition element, and the second control instruction includes the working mode and rotational angular velocity of the photoelectric pod.
上述技术方案通过设置遥控装置，实现用户手动搜索待跟踪目标的目的，在待跟踪目标落入光电吊舱的视角范围内后，立刻利用本发明的自主视觉追踪技术对待跟踪目标进行跟踪。The above technical solution achieves the purpose of manually searching for the target to be tracked by the user by setting the remote control device, and immediately uses the autonomous visual tracking technology of the present invention to track the target to be tracked after the target to be tracked falls within the viewing angle range of the photoelectric pod.
具体的，所述遥控装置包括按键和遥杆。Specifically, the remote control device includes buttons and a joystick.
所述控制器获取用户通过按键设定的光电吊舱的工作模式指令，并控制光电吊舱的工作模式为用户设定的工作模式；The controller obtains the working mode instruction of the photoelectric pod set by the user through the button, and controls the working mode of the photoelectric pod to be the working mode set by the user;
所述控制器获取用户通过遥杆设定的光电吊舱转动的角速度，并控制驱动机构驱动光电吊舱以用户设定的角速度转动。The controller obtains the rotational angular velocity of the photoelectric pod set by the user through the joystick, and controls the driving mechanism to drive the photoelectric pod to rotate at the angular velocity set by the user.
上述技术方案以按键和遥杆的组合方式实现人机交互。显然，也可以通过其它人机交互的方式，例如：虚拟键盘。The above-mentioned technical solution realizes human-computer interaction by means of a combination of buttons and joysticks. Apparently, other ways of human-computer interaction can also be used, for example: a virtual keyboard.
当然，也可以采用自动搜索的方式搜索待跟踪目标，即，所述控制器控制驱动机构驱动光电吊舱转动进行搜索扫描，直至待跟踪目标落入光电吊舱的视觉范围内，立刻利用本发明的自主视觉追踪技术对待跟踪目标进行跟踪。Of course, it is also possible to search for the target to be tracked by means of an automatic search, that is, the controller controls the drive mechanism to drive the photoelectric pod to rotate to search and scan until the target to be tracked falls within the visual range of the photoelectric pod, and the present invention is immediately used The autonomous visual tracking technology to track the target to be tracked.
为了保证系统的正常工作，在硬件上还包括其它结构，如：电源、稳压设备等，在此不再一一描述。In order to ensure the normal operation of the system, there are other structures on the hardware, such as: power supply, voltage stabilizing equipment, etc., which will not be described one by one here.
所述控制器选择高性能处理器，具有较强的数据处理能力，以满足图像处理的需求，提高系统的可靠性，并缩短实时运行的周期。The controller selects a high-performance processor with strong data processing capability to meet the needs of image processing, improve system reliability, and shorten the period of real-time operation.
所述控制器通过串口与系统的其它设备(包括遥控装置、光电吊舱)进行通信。The controller communicates with other equipment of the system (including the remote control device and the photoelectric pod) through the serial port.
基于同一发明构思，如图3所示，本发明实施例中还提供一种基于自主视觉跟踪的光电吊舱的控制方法，包括：Based on the same inventive concept, as shown in FIG. 3 , an embodiment of the present invention also provides a control method for a photoelectric pod based on autonomous visual tracking, including:
实时获取光电吊舱的姿态信息，所述姿态信息包括光电吊舱的俯仰角、横滚角、偏航角；Obtain the attitude information of the photoelectric pod in real time, and the attitude information includes the pitch angle, roll angle and yaw angle of the photoelectric pod;
实时获取待跟踪目标和光电吊舱之间的直线距离；Obtain the straight-line distance between the target to be tracked and the photoelectric pod in real time;
根据所述姿态信息和直线距离计算待跟踪目标的位置信息；calculating the position information of the target to be tracked according to the attitude information and the straight-line distance;
根据所述位置信息和待跟踪目标的设定位置信息生成第一控制指令，以控制驱动机构驱动光电吊舱转动，使得待跟踪目标处于设定位置，所述第一控制指令包括跟踪指令和光电吊舱的转动角速度。According to the position information and the set position information of the target to be tracked, a first control command is generated to control the drive mechanism to drive the photoelectric pod to rotate so that the target to be tracked is at the set position. The first control command includes a tracking command and a photoelectric pod. The rotational angular velocity of the pod.
上述控制方法利用光电吊舱的姿态信息、待跟踪目标和光电吊舱之间的直线距离，实时计算待跟踪目标的位置信息，通过将待跟踪目标的位置信息与设定位置信息进行比较生成控制指令，来控制光电吊舱的转动，使得待跟踪目标处于设定位置，实现对待跟踪目标的自主视觉跟踪。The above control method uses the attitude information of the photoelectric pod and the straight-line distance between the target to be tracked and the photoelectric pod to calculate the position information of the target to be tracked in real time, and generates a control by comparing the position information of the target to be tracked with the set position information. Command to control the rotation of the photoelectric pod, so that the target to be tracked is at the set position, and the autonomous visual tracking of the target to be tracked is realized.
所述设定位置可以为图像采集元件采集的图像的中心。The set position may be the center of the image captured by the image capture element.
本实施例中，所述控制方法还包括：In this embodiment, the control method further includes:
显示所述图像采集元件采集的图像，以及待跟踪目标的位置信息。The image collected by the image collection component and the position information of the target to be tracked are displayed.
通过显示图像采集元件采集的图像，以及待跟踪目标的位置信息，有利于实现人机交互。By displaying the image collected by the image acquisition component and the position information of the target to be tracked, it is beneficial to realize human-computer interaction.
为了提高显示图像的质量，所述控制方法还包括：In order to improve the quality of the displayed image, the control method also includes:
采用数据拟合方法获取待跟踪目标和光电吊舱之间的直线距离与对应的图像采集元件的焦距之间的设定关系；A data fitting method is used to obtain the setting relationship between the linear distance between the target to be tracked and the photoelectric pod and the focal length of the corresponding image acquisition element;
根据所述设定关系计算与存储待跟踪目标和光电吊舱之间的直线距离对应的图像采集元件的焦距；Calculate the focal length of the image acquisition element corresponding to the linear distance between the target to be tracked and the photoelectric pod according to the set relationship;
根据所述焦距控制图像采集元件进行调焦，以使得图像采集元件采集的图像清晰。The image acquisition element is controlled to adjust the focus according to the focal length, so that the image acquired by the image acquisition element is clear.
上述步骤采用数据拟合方法获取待跟踪目标和光电吊舱之间的直线距离与对应的图像采集元件的焦距之间的设定关系，并利用获取的存储待跟踪目标和光电吊舱之间的直线距离，对图像采集元件进行实时调焦，使得采集图像清晰，提升跟踪目标性能。The above steps use the data fitting method to obtain the setting relationship between the linear distance between the target to be tracked and the photoelectric pod and the focal length of the corresponding image acquisition element, and use the obtained stored distance between the target to be tracked and the photoelectric pod Straight-line distance, real-time focus adjustment of image acquisition components, making the acquired image clear and improving the performance of tracking targets.
在一个具体的实施方式中，采用无载波通信技术来获取待跟踪目标和光电吊舱之间的直线距离，则，获取待跟踪目标和光电吊舱之间的直线距离的步骤包括：In a specific embodiment, the carrierless communication technology is used to obtain the straight-line distance between the target to be tracked and the photoelectric pod, then, the step of obtaining the straight-line distance between the target to be tracked and the photoelectric pod includes:
安装在光电吊舱上的标签模块在自身的时间戳上通过无载波通信的方式向安装在待跟踪目标上的基站模块发送请求信号；The label module installed on the photoelectric pod sends a request signal to the base station module installed on the target to be tracked by means of carrierless communication on its own time stamp;
基站模块在自身的时间戳上接收请求信号，然后向标签模块发送反馈信号；The base station module receives the request signal on its own time stamp, and then sends a feedback signal to the tag module;
利用标签模块在其时间戳上发送请求信号和接收到反馈信号的时间差，以及基站模块在其时间戳上接收到请求信号和发送反馈信号的时间差，计算待跟踪目标与光电吊舱之间的直线距离。Calculate the straight line between the target to be tracked and the photoelectric pod by using the time difference between the tag module sending the request signal and receiving the feedback signal on its time stamp, and the time difference between the base station module receiving the request signal and sending the feedback signal on its time stamp distance.
上述步骤利用标签模块在自身的时间戳上发送和接收信号时间差t1，以及基站模块在自身的时间戳上接收和反馈信号的时间差t2，来计算待跟踪目标与光电吊舱之间的直线距离d，计算公式为d＝v(t2-t1)，v为光速(v远大于正常工作的飞行状态下的移动速度，且采样频率足够高)。The above steps use the time difference t1 between the tag module sending and receiving signals on its own time stamp, and the time difference t2 between the base station module receiving and feeding back signals on its own time stamp to calculate the straight line between the target to be tracked and the photoelectric pod The calculation formula for the distance d is d=v(t2 -t1 ), where v is the speed of light (v is much greater than the moving speed in a normal flight state, and the sampling frequency is sufficiently high).
具体的，所述请求信号和反馈信号为脉冲信号，采用超宽带无载波通信技术来获取待跟踪目标和光电吊舱之间的直线距离。Specifically, the request signal and the feedback signal are pulse signals, and the ultra-wideband carrierless communication technology is used to obtain the straight-line distance between the target to be tracked and the photoelectric pod.
为了实现人机交互的目的，本实施例中所述控制方法还包括：In order to achieve the purpose of human-computer interaction, the control method described in this embodiment further includes:
获取用户通过遥控装置发送的第二控制指令，并根据所述第二控制指令控制所述驱动机构驱动所述光电吊舱转动，以使得待跟踪目标位于图像采集元件的视角范围内，所述第二控制指令包括光电吊舱的工作模式、转动角速度。Obtain the second control instruction sent by the user through the remote control device, and control the driving mechanism to drive the photoelectric pod to rotate according to the second control instruction, so that the target to be tracked is located within the viewing angle range of the image acquisition element, the second The second control instruction includes the working mode and rotational angular velocity of the photoelectric pod.
上述技术方案通过设置遥控装置，实现用户手动搜索待跟踪目标的目的，在待跟踪目标落入光电吊舱的视角范围内后，立刻利用本发明的自主视觉追踪技术对待跟踪目标进行跟踪。The above technical solution achieves the purpose of manually searching for the target to be tracked by the user by setting the remote control device, and immediately uses the autonomous visual tracking technology of the present invention to track the target to be tracked after the target to be tracked falls within the viewing angle range of the photoelectric pod.
具体的，用户可以通过按键设定光电吊舱的工作模式，通过遥杆设定光电吊舱转动的角速度。Specifically, the user can set the working mode of the photoelectric pod through the buttons, and set the rotational angular velocity of the photoelectric pod through the joystick.
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明技术原理的前提下，还可以做出若干改进和替换，这些改进和替换也应视为本发明的保护范围。The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the technical principle of the present invention, some improvements and replacements can also be made, these improvements and replacements It should also be regarded as the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910775191.1ACN110580054B (en) | 2019-08-21 | 2019-08-21 | Control system and method of photoelectric pod based on autonomous visual tracking |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910775191.1ACN110580054B (en) | 2019-08-21 | 2019-08-21 | Control system and method of photoelectric pod based on autonomous visual tracking |
| Publication Number | Publication Date |
|---|---|
| CN110580054Atrue CN110580054A (en) | 2019-12-17 |
| CN110580054B CN110580054B (en) | 2022-06-14 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910775191.1AActiveCN110580054B (en) | 2019-08-21 | 2019-08-21 | Control system and method of photoelectric pod based on autonomous visual tracking |
| Country | Link |
|---|---|
| CN (1) | CN110580054B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111554080A (en)* | 2020-07-13 | 2020-08-18 | 成都纵横自动化技术股份有限公司 | Photoelectric pod data acquisition system |
| CN112241183A (en)* | 2020-09-11 | 2021-01-19 | 中国航空工业集团公司成都飞机设计研究所 | Ground combined control method for photoelectric pod of unmanned aerial vehicle system |
| CN112764012A (en)* | 2020-12-23 | 2021-05-07 | 武汉高德红外股份有限公司 | Photoelectric pod tracking simulation test and system |
| CN113415433A (en)* | 2021-07-30 | 2021-09-21 | 成都纵横大鹏无人机科技有限公司 | Pod attitude correction method and device based on three-dimensional scene model and unmanned aerial vehicle |
| CN113804187A (en)* | 2021-09-01 | 2021-12-17 | 河北汉光重工有限责任公司 | An integrated system for target positioning of optoelectronic pods |
| CN114859960A (en)* | 2022-03-28 | 2022-08-05 | 北京航天控制仪器研究所 | Method for continuously tracking and reconnaissance fixed-wing unmanned aerial vehicle photoelectric pod to fixed-point target |
| CN117055599A (en)* | 2023-08-31 | 2023-11-14 | 北京航翊科技有限公司 | Unmanned aerial vehicle flight control method and device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034247A (en)* | 2012-12-04 | 2013-04-10 | 浙江天地人科技有限公司 | Controlling method and controlling device for remote monitoring system |
| US20180065242A1 (en)* | 2016-09-06 | 2018-03-08 | Sharp Kabushiki Kaisha | Moving vehicle |
| CN107869989A (en)* | 2017-11-06 | 2018-04-03 | 东北大学 | A positioning method and system based on visual inertial navigation information fusion |
| CN108733066A (en)* | 2018-05-07 | 2018-11-02 | 中国人民解放军国防科技大学 | A Target Tracking Control Method Based on Pod Attitude Feedback |
| CN109377512A (en)* | 2018-09-07 | 2019-02-22 | 深圳市易成自动驾驶技术有限公司 | The method, apparatus and storage medium of target following |
| CN109458529A (en)* | 2014-04-30 | 2019-03-12 | 深圳市大疆灵眸科技有限公司 | The method of the holder and control holder of control device and its application |
| CN109466766A (en)* | 2018-11-28 | 2019-03-15 | 中电科（德阳广汉）特种飞机系统工程有限公司 | A kind of autogyro and Reconnaissance system |
| CN109548141A (en)* | 2018-11-01 | 2019-03-29 | 上海理工大学 | Calibration method of indoor environment base station coordinate position based on Kalman filter algorithm |
| CN109683130A (en)* | 2019-02-19 | 2019-04-26 | 珠海格力电器股份有限公司 | Positioning method, positioning device, storage medium and processor |
| CN109753076A (en)* | 2017-11-03 | 2019-05-14 | 南京奇蛙智能科技有限公司 | A kind of unmanned plane vision tracing implementing method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103034247A (en)* | 2012-12-04 | 2013-04-10 | 浙江天地人科技有限公司 | Controlling method and controlling device for remote monitoring system |
| CN109458529A (en)* | 2014-04-30 | 2019-03-12 | 深圳市大疆灵眸科技有限公司 | The method of the holder and control holder of control device and its application |
| US20180065242A1 (en)* | 2016-09-06 | 2018-03-08 | Sharp Kabushiki Kaisha | Moving vehicle |
| CN109753076A (en)* | 2017-11-03 | 2019-05-14 | 南京奇蛙智能科技有限公司 | A kind of unmanned plane vision tracing implementing method |
| CN107869989A (en)* | 2017-11-06 | 2018-04-03 | 东北大学 | A positioning method and system based on visual inertial navigation information fusion |
| CN108733066A (en)* | 2018-05-07 | 2018-11-02 | 中国人民解放军国防科技大学 | A Target Tracking Control Method Based on Pod Attitude Feedback |
| CN109377512A (en)* | 2018-09-07 | 2019-02-22 | 深圳市易成自动驾驶技术有限公司 | The method, apparatus and storage medium of target following |
| CN109548141A (en)* | 2018-11-01 | 2019-03-29 | 上海理工大学 | Calibration method of indoor environment base station coordinate position based on Kalman filter algorithm |
| CN109466766A (en)* | 2018-11-28 | 2019-03-15 | 中电科（德阳广汉）特种飞机系统工程有限公司 | A kind of autogyro and Reconnaissance system |
| CN109683130A (en)* | 2019-02-19 | 2019-04-26 | 珠海格力电器股份有限公司 | Positioning method, positioning device, storage medium and processor |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111554080A (en)* | 2020-07-13 | 2020-08-18 | 成都纵横自动化技术股份有限公司 | Photoelectric pod data acquisition system |
| CN112241183A (en)* | 2020-09-11 | 2021-01-19 | 中国航空工业集团公司成都飞机设计研究所 | Ground combined control method for photoelectric pod of unmanned aerial vehicle system |
| CN112241183B (en)* | 2020-09-11 | 2023-03-24 | 中国航空工业集团公司成都飞机设计研究所 | Ground combined control method for photoelectric pod of unmanned aerial vehicle system |
| CN112764012A (en)* | 2020-12-23 | 2021-05-07 | 武汉高德红外股份有限公司 | Photoelectric pod tracking simulation test and system |
| CN112764012B (en)* | 2020-12-23 | 2024-04-26 | 武汉高德红外股份有限公司 | Photoelectric pod tracking simulation test device and system |
| CN113415433A (en)* | 2021-07-30 | 2021-09-21 | 成都纵横大鹏无人机科技有限公司 | Pod attitude correction method and device based on three-dimensional scene model and unmanned aerial vehicle |
| CN113415433B (en)* | 2021-07-30 | 2022-11-29 | 成都纵横大鹏无人机科技有限公司 | Pod attitude correction method and device based on three-dimensional scene model and unmanned aerial vehicle |
| CN113804187A (en)* | 2021-09-01 | 2021-12-17 | 河北汉光重工有限责任公司 | An integrated system for target positioning of optoelectronic pods |
| CN114859960A (en)* | 2022-03-28 | 2022-08-05 | 北京航天控制仪器研究所 | Method for continuously tracking and reconnaissance fixed-wing unmanned aerial vehicle photoelectric pod to fixed-point target |
| CN117055599A (en)* | 2023-08-31 | 2023-11-14 | 北京航翊科技有限公司 | Unmanned aerial vehicle flight control method and device, electronic equipment and storage medium |
| CN117055599B (en)* | 2023-08-31 | 2024-05-14 | 北京航翊科技有限公司 | Unmanned aerial vehicle flight control method and device, electronic equipment and storage medium |
| Publication number | Publication date |
|---|---|
| CN110580054B (en) | 2022-06-14 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110580054B (en) | Control system and method of photoelectric pod based on autonomous visual tracking | |
| US11914370B2 (en) | System and method for providing easy-to-use release and auto-positioning for drone applications | |
| US12125229B2 (en) | UAV control method, device and UAV | |
| JP7260269B2 (en) | Positioning system for aeronautical non-destructive inspection | |
| CN108257145A (en) | A kind of UAV Intelligent based on AR technologies scouts processing system and method | |
| CN104793637B (en) | The real-time tracking system and tracking of a kind of mobile device | |
| CN112394382B (en) | Low-speed small target tracking device and method capable of resisting long-term shielding | |
| CN105959625A (en) | Method and device of controlling unmanned plane tracking shooting | |
| CN107968915A (en) | Underwater robot camera pan-tilt real-time control system and its method | |
| CN110498039B (en) | An intelligent monitoring system based on bionic flapping-wing aircraft | |
| CN109358656A (en) | A kind of target acquistion method suitable for airborne lidar for fluorescence | |
| KR101657086B1 (en) | Vision-based guidance of UAV equipped with a fixed camera control system | |
| CN108897342A (en) | For the positioning and tracing method and system of the civilian multi-rotor unmanned aerial vehicle fast moved | |
| CN105259929A (en) | Automatic tracking observation system and method for realizing automatic tracking observation | |
| CN110554380A (en) | Target height fixing system for radar countercheck system | |
| CN105744234A (en) | Linkage method for tracking moving object based on pixel differential ratio and system thereof | |
| CN107861127A (en) | High-precision and high-stability photoelectricity real-time tracking system and method | |
| CN209070376U (en) | A kind of holder tracing system based on image recognition | |
| CN109002059A (en) | A kind of multi-rotor unmanned aerial vehicle object real-time tracking camera system and method | |
| US12422846B2 (en) | Information processing device and information processing method | |
| CN105701789A (en) | Airplane near-field flight trajectory tracking method | |
| CN102034244A (en) | Insulator real-time target recognizing and tracking device based on motion background | |
| CN108008661A (en) | A kind of ship photoelectricity water cannon integral control system and method | |
| CN112181151A (en) | Unmanned aerial vehicle control method based on gesture recognition and gesture recognition remote control unmanned aerial vehicle | |
| KR20240147812A (en) | Following autonomous driving robot and application interworking system using sensor fusion |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |