Target positioning method for carrying multiple PTZ cameras on intelligent ground unmanned platformTechnical Field
The invention belongs to the technical field of vehicle-mounted sensing systems, and particularly relates to a target positioning method for carrying a plurality of PTZ cameras on a ground intelligent unmanned platform.
Background
With the rapid development of information technology, the capacity of an intelligent unmanned system for carrying out specific tasks is greatly leaped, the development of the intelligent unmanned system has great significance in both civil and military aspects, the intelligent unmanned system in the civil aspect can reduce the workload of people, has more advantages than people in some fields, and can improve the production work efficiency; in the military aspect, the ground intelligent unmanned platform can assist or even replace soldiers to complete battlefield maneuvering patrol, strange scene investigation, logistics material supply, combat fire attack, hardness and other tasks, so that the burden of fighters is reduced, and casualties are reduced.
The intelligent unmanned system mainly comprises four modules of environment perception, positioning navigation, planning decision and intelligent control, wherein the perception module is a main information input source of the intelligent unmanned system, the output result is the basis of the planning decision of the intelligent unmanned system, the tracking and positioning of the target are one of the main functions of the perception module, and the tracking and positioning of the target based on the image is a hotspot of research in the field of computer vision.
With the development of sensing technology, target tracking and positioning can be realized by various means such as laser radar, camera, millimeter wave radar and the like, and various sensors have the following characteristics in the aspect of ranging: (1) the laser radar is an active distance measuring sensor, can acquire centimeter-level three-dimensional information of each laser irradiation point, and can realize centimeter-level accurate positioning on a detected and tracked target through processing; the laser radar has small interference by illumination, can normally work day and night, but has high price, limited laser scanning points and insufficient information density, and particularly, as the target distance is farther and farther, the laser scanning point cloud is more and more sparse, the resolution ratio is lower and lower, the texture characteristic is seriously lost, and the perception effect is sharply reduced; (2) monocular cameras are low in cost and can provide rich texture information, and are the main means of current target identification. The monocular camera is limited by the fact that the target depth information cannot be directly obtained through a small hole imaging principle, in actual use, the depth of the target is usually estimated through a depth learning method based on the prior information of the target, but the method is large in error. The camera is used as a passive sensor and is easily influenced by ambient environments such as illumination and the like; (3) the common binocular camera can realize stereoscopic vision through stereoscopic matching, but the positioning accuracy is limited by the length of a camera base line, and high-accuracy positioning and ranging can be obtained only in a smaller range; because the pose of the common binocular camera cannot be changed and the focal length of the lens cannot be adjusted, the sensing range cannot be changed, and the high-precision distance measurement range is very small; (4) the millimeter wave radar is an active sensor and has strong detection capability on azimuth angle, distance and speed of objects within the range of 200 meters. It features strong penetrability to smoke, dust, rain and snow, and can adapt to various bad weather without being influenced by illumination variation factor. However, millimeter wave radar has many data noise points and low angular resolution and ranging accuracy, and cannot meet the high-accuracy target positioning requirement required by an intelligent unmanned system. In summary, no sensor can simultaneously meet the requirements of low cost, high precision and large range in target detection, tracking and positioning equipment adopted by the current intelligent unmanned system.
The PTZ camera is also called a pan-tilt camera, and can realize the omnibearing rotation of the camera and the zooming of a lens. The PTZ camera can acquire image information of different areas through rotation, can acquire space image information of different depths through zooming, does not reduce image resolution, can keep high-precision resolution in a large range, and adapts to the requirements of an intelligent unmanned system. The double PTZ cameras simulate the chameleon vision system, the left eye and the right eye can rotate independently, when the surrounding environment needs to be sensed, the left camera and the right camera respectively rotate to realize large-angle reconnaissance, when the target needs to be positioned, the left camera and the right camera point to the same direction to form stereoscopic vision, and the three-dimensional positioning of the target is realized. Based on the characteristics, the double PTZ cameras can form stereoscopic vision at any angle and focal length, and high-precision three-dimensional positioning of a remote target and high-quality three-dimensional reconstruction of a scene are realized. Compared with a laser radar, the PTZ camera is low in price, and large-range and high-precision target detection, tracking and positioning can be achieved through focal length adjustment and pose adjustment of the camera. Therefore, the PTZ camera can be applied to a ground intelligent unmanned platform to replace expensive photoelectric equipment, and tracking of long-distance military targets, battlefield environment perception and three-dimensional positioning of important targets are achieved. Meanwhile, the positioning and mapping of the whole battlefield area can be realized by combining the visual SLAM and the three-dimensional reconstruction technology, and the method has important military value.
Disclosure of Invention
The invention aims to overcome the defects of the traditional sensing technology and provide a target positioning method for carrying a plurality of PTZ cameras on a ground intelligent unmanned platform.
In order to achieve the purpose, the invention adopts the following technical scheme: a target positioning method for carrying a plurality of PTZ cameras on a ground intelligent unmanned platform is characterized in that a sensing system is formed by the PTZ cameras and a vehicle-mounted computing module, wherein the PTZ cameras and the vehicle-mounted computing module are installed on the ground intelligent unmanned platform, the sensing system sends images acquired by the PTZ cameras and self pose and focal length information to the vehicle-mounted computing module, meanwhile, the vehicle-mounted computing module can send instructions to the cameras to control the cameras to rotate and zoom, and an internal parameter matrix K of each PTZ camera is calculatediAnd its rotation matrix R relative to the vehicle body coordinate systemiTranslation vector TiCalculating a transformation matrix H for transforming the target to be positioned from the world coordinate system to the pixel coordinate system of each cameraiSolving the three-dimensional position of the target in the vehicle body coordinate system by using a least square method; the method comprises the following specific steps:
firstly, the position of each PTZ camera in a vehicle body coordinate system is calibrated through measurement, namely a translation vector T of the camera relative to the vehicle body coordinate systemi=(a,b,c)T;
Step two, calculating a camera coordinate system O of each PTZ cameracXyz relative to the vehicle body coordinate system Ov-a rotation matrix R of xyzi;
Step three, estimating the internal parameter matrix K of each camera under the current zoom multiplei;
Step four, calculating a vehicle body coordinate system O of the positioned targetv-conversion of coordinates in xyz to the respective camera pixel coordinate system oi-transformation matrix H of coordinates under uvi;
Step five, a target on-vehicle body coordinate system O is solved by using a least square methodv-three-dimensional position P (x) at xyzv,yv,zv)。
Further, the rotation matrix R of each camera coordinate system in the two steps relative to the vehicle body coordinate systemiThe calculating method comprises the following steps:
1) setting the direction of the initial position of the camera coordinate system to be consistent with the direction of the vehicle body coordinate system;
2) acquiring a horizontal rotation angle alpha and a pitching rotation angle beta of each PTZ camera relative to an initial position;
3) setting a direction vector of horizontal rotation of the camera at an initial position under a vehicle body coordinate system as v, and setting a direction vector of pitching rotation as h, and obtaining a rotation matrix:
wherein,
a rotation matrix representing a rotation of an angle alpha around a rotation vector v,
the meaning of the compound is the same as that of the compound,
can be obtained from the formula according to rodriegers:
can be calculated by the same method
Where v, h can be idealized to be consistent with the coordinate axis directions of the respective directions, and I is a third order identity matrix.
Further, step three is that the internal reference matrix K of each cameraiThe estimation method comprises the steps of establishing a functional relation K (f) (z) between a zoom value z and an internal parameter matrix through an early-stage discrete calibration result, and estimating the internal parameter matrix K of the camera according to the current zoom valuei。
Further, step four is that the transformation matrix HiThe calculation method comprises the following steps:
furthermore, step five, the target on-vehicle body coordinate system O is obtained by using a least square methodv-three-dimensional position P (x) at xyzv,yv,zv) The method comprises the following steps:
(1) acquiring coordinates p of a target located in the pixel coordinate system of each camera imagei(ui,vi) Coordinates P (x) of the object to be positioned in the coordinates of the vehicle bodyv,yv,zv) And pi(ui,vi) The conversion relationship between the two is as follows:
(ui,vi,1)T=Hi(xv,yv,zv,1)T (4)
(2) expanding the transformation relation of each group of the formula (4) to obtain 2i groups of equations, and writing the simultaneous equations into a matrix form to obtain:
A(2i)×3(xw,yw,zw)T=B(2i)×1 (5)
solving the formula (5) by using a least square method to obtain P (x)v,yv,zv) Three-dimensional coordinates of (a).
Further, each PTZ camera has two rotational degrees of freedom in pan and tilt and one zoom degree of freedom.
Furthermore, each PTZ camera can independently rotate to zoom and sense the surrounding environment of the vehicle body, and can also form a stereoscopic vision system in pairs or form a multi-view stereoscopic vision system to realize the three-dimensional positioning of the target.
Has the advantages that: compared with the prior art, the invention aims at the requirements of reconnaissance and accurate positioning of targets with different focal lengths and different distances under various environments by using an intelligent unmanned system, overcomes the defects of the traditional intelligent unmanned system sensor in the aspects of target positioning distance, accuracy and texture information, carries a plurality of PTZ cameras, and performs accurate three-dimensional positioning on the targets by combining the positions of the positioning targets in the images of the cameras according to the positions and postures of the cameras.
Drawings
Fig. 1 is a plan view of four variable-posture zoom cameras mounted.
In the figure: t is a unit ofiFor the position of each camera in the coordinate system of the vehicle body
Detailed Description
So that the manner in which the above recited objects, features and advantages of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to the embodiments thereof which are illustrated in the appended drawings. In addition, the embodiments and features of the embodiments of the present application may be combined with each other without conflict. In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention, and the described embodiments are merely a subset of the embodiments of the present invention, rather than a complete embodiment. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
In various embodiments of the present invention, for convenience in description and not in limitation, the term "coupled" as used in the specification and claims of the present application is not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships are changed accordingly.
Referring to the attached drawings in detail, the embodiment provides a target positioning method for carrying a plurality of PTZ cameras on a ground intelligent unmanned platform, the PTZ cameras and a vehicle-mounted computing module which are installed on the ground intelligent unmanned platform form a sensing system, the sensing system sends images acquired by the PTZ cameras and self pose and focal length information to the vehicle-mounted computing module, meanwhile, the vehicle-mounted computing module can send instructions to the cameras to control the cameras to rotate and zoom, and an internal reference matrix K of each PTZ camera is calculatediAnd its rotation matrix R relative to the vehicle body coordinate systemiTranslation vector TiCalculating a transformation matrix H for transforming the target to be positioned from the world coordinate system to the pixel coordinate system of each cameraiSolving the three-dimensional position of the target in the vehicle body coordinate system by using a least square method; the method comprises the following specific steps:
firstly, calibrating the position of each PTZ camera in a vehicle body coordinate system through measurement, namely, a translation vector of the camera relative to the vehicle body coordinate system is measured, namely, a vector T of each PTZ camera relative to an origin position under the vehicle body coordinate system is measuredi=(a,b,c)T;
Step two, calculating a camera coordinate system O of each PTZ cameracXyz relative to the vehicle body coordinate system Ov-a rotation matrix R of xyzi;
Step three, estimating the internal parameter matrix K of each camera under the current zoom multiplei;
Step four, calculating a vehicle body coordinate system O of the positioned targetv-conversion of coordinates in xyz to the respective camera pixel coordinate system oi-transformation matrix H of coordinates under uvi;
Step five, solving the target sitting on the vehicle body by using a least square methodSystem of symbols Ov-three-dimensional position P (x) at xyzv,yv,zv)。
In a preferred embodiment of the present invention, the rotation matrix R of each camera coordinate system in the two steps relative to the vehicle body coordinate systemiThe calculation method comprises the following steps:
1) coordinate system O of camerac-direction of xyz starting position in relation to the vehicle body coordinate system Ov-the directions of the xyz axes are set to coincide;
2) acquiring a horizontal rotation angle alpha and a pitching rotation angle beta of each PTZ camera relative to an initial position through a built-in SDK of the camera;
3) when an initial position is set, a direction vector of horizontal rotation of the camera under a vehicle body coordinate system is v, a direction vector of pitching rotation is h, and a rotation matrix can be obtained:
wherein,
a rotation matrix representing a rotation by an angle alpha around a rotation vector v,
the meaning of the compound is the same as that of the compound,
can be obtained from the formula according to rodriegers:
where v, h can be idealized to be consistent with the coordinate axis directions of the respective directions, and I is a third order identity matrix.
The preferable scheme of this embodiment is that, in step three, the internal reference matrix K of each camera isiThe estimation method comprises the steps of establishing a functional relation K (f) (z) between a zoom value z and an internal parameter matrix through an early stage discrete calibration result,estimating an internal reference matrix K of a camera according to a current zoom valueiZ is available from the camera SDK.
The preferred solution of this embodiment is that, in step four, the transformation matrix HiThe calculation method comprises the following steps:
preferably, in the present embodiment, the target on-vehicle body coordinate system O is obtained by using the least square method in the step fivev-three-dimensional position P (x) at xyzv,yv,zv) The method comprises the following steps:
(1) acquiring the coordinates p of the positioned target in the pixel coordinate system of each camera imagei(ui,vi) Coordinates P (x) of the object to be positioned in the coordinates of the vehicle bodyv,yv,zv) And pi(ui,vi) The conversion relationship between the two is as follows:
(ui,vi,1)T=Hi(xv,yv,zv,1)T (4)
(2) and (3) expanding the transformation relation of each group of the formula (4) to obtain 2i groups of formula:
and (3) combining the simultaneous equations and writing the simultaneous equations into a matrix form:
obtaining:
namely:
A(2i)×3(xw,yw,zw)T=B(2i)×1 (7)
solving the formula (7) by using a least square method to obtain P (x)v,yv,zv) Three-dimensional coordinates of (a).
The preferred solution of this embodiment is that each PTZ camera has two rotational degrees of freedom in pan and tilt and one zoom degree of freedom.
The optimal scheme of the embodiment is that each PTZ camera can independently rotate to zoom and sense the environment around the vehicle body, and can also form a stereo vision system in pairs or form a multi-view stereo vision system to realize three-dimensional positioning of the target.
Examples
The method comprises the steps of installing a plurality of PTZ cameras on a ground intelligent unmanned platform, positioning a target, verifying the target positioning method, performing positioning experiments on the target by using two front cameras, positioning the target under different focal lengths and angles of the cameras at a distance of 0-100m and at an interval of 5 m, and comparing a positioning value with a true value. The positioning results are shown in Table 1
TABLE 1 Long-distance target location experimental data
Within the range of 100m, the positioning distance error of the targets at different distance positions is less than 2 percent.
And positioning the targets under different angles within the range of 10m, wherein the error of the Euclidean distance between the three-dimensional position of the experimental result and the real result is less than 2.8%.
TABLE 2 results of wide-angle target positioning experiments
The above detailed description of the target positioning method for multiple PTZ cameras mounted on the intelligent unmanned ground platform with reference to the embodiments is illustrative and not restrictive, and several embodiments may be enumerated according to the limited scope, so that changes and modifications without departing from the general concept of the present invention shall fall within the protection scope of the present invention.