Disclosure of Invention
The application provides a linkage monitoring device, a linkage monitoring method and a storage medium, which can solve the problem of low calibration efficiency caused by the fact that the acquisition operation needs to be performed manually for many times in the related technology. The technical scheme is as follows:
in one aspect, a device for linkage monitoring is provided, the device comprising:
the device comprises a radar, a dome camera, a height measurement module and a central processing module, wherein the radar is fixedly connected with the dome camera relatively;
the height measurement module is fixedly connected with the ball machine and used for detecting a first height, wherein the first height is the height from the ball machine to the horizontal ground;
the central processing module is respectively connected with the radar, the dome camera and the height measurement module and is used for determining a mapping relation between radar coordinates and dome camera coordinates based on the first height, the second height and the view field included angle so as to jointly calibrate the radar and the dome camera;
the second height is the relative height of the radar and the dome camera in the vertical direction, and the view field included angle is the included angle between the zero-degree view angle of the radar and the zero-degree view angle of the dome camera.
In one possible implementation manner of the present application, the apparatus further includes: the ball machine gesture detection module is connected with the ball machine and is used for detecting the inclination angle of the ball machine relative to the horizontal ground;
correspondingly, the central processing module is also used for controlling the ball machine to be zeroed based on the inclination angle;
the height measurement module is used for detecting the height between the zeroed ball machine and the horizontal ground to obtain the first height.
In one possible implementation of the present application, the central processing module is configured to control the horizontal angle of the ball machine to be constant, and the pitch angle to be offset by the magnitude of the tilt angle, so as to correct the pitch angle to zero degrees.
In one possible implementation of the present application,
the ball machine gesture detection module is arranged in the ball machine; or,
the ball machine gesture detection module is integrally arranged with the ball machine.
In one possible implementation manner of the present application, the height measurement module is disposed at the bottom of the ball machine.
In one possible implementation manner of the present application, the central processing module is configured to:
the method comprises the steps of obtaining radar coordinate information of a target detected by a radar, wherein the radar coordinate information at least comprises a target distance and a radar detection angle, and the target distance is the distance between the target and the radar;
converting the radar coordinate information into spherical machine coordinate information based on the mapping relation;
and transmitting the coordinate information of the ball machine to the ball machine.
In one possible implementation manner of the present application, the apparatus further includes an auxiliary focusing module, where the auxiliary focusing module is configured to perform lens focusing of the dome camera based on the target distance.
In one possible implementation manner of the present application, the ball machine is used for:
adjusting the current state to the state corresponding to the coordinate information of the spherical machine;
focusing a lens of the dome camera based on the target distance through the auxiliary focusing module;
and capturing the target, and rechecking the detection result of the radar according to the captured image.
In one possible implementation manner of the present application, the mapping relationship is represented by the following formulas (1) and (2):
wherein,the p is the horizontal angle of the dome camera, the T is the pitch angle of the dome camera, the r is the target distance between the radar detected target and the radar, the alpha is the radar detection angle, and the hc For the first height, the hr And the beta is the included angle of the field of view, and the sum of the first height and the second height is the included angle of the field of view.
In one possible implementation of the application, the device further comprises a mounting bracket, and the radar and the ball machine are fixed on the mounting bracket.
In another aspect, a method of linkage monitoring is provided, the method comprising:
the central processing module acquires radar coordinate information of a target detected by the radar, wherein the radar coordinate information at least comprises a target distance and a radar detection angle, and the target distance is the distance between the target and the radar;
the central processing module converts the radar coordinate information into spherical machine coordinate information based on a mapping relation, wherein the mapping relation is the relation between radar coordinates and spherical machine coordinates, the mapping relation is determined based on a first height, a second height and a view field included angle, the first height is the height from the spherical machine to the horizontal ground, which is obtained by detection of the height measurement module, the second height is the relative height between the radar and the spherical machine in the vertical direction, and the view field included angle is the included angle between the zero-degree view field angle of the radar and the zero-degree view field angle of the spherical machine;
and the central processing module is used for capturing the target through the spherical camera based on the coordinate information of the spherical camera and rechecking the detection result of the radar based on the captured image.
In one possible implementation manner of the present application, the mapping relationship is represented by the following formulas (1) and (2):
wherein p is the horizontal angle of the dome camera, T is the pitch angle of the dome camera, r is the target distance between the radar detected target and the radar, alpha is the radar detection angle, and hc For the first height, the hr And the beta is the included angle of the field of view, and the sum of the first height and the second height is the included angle of the field of view.
In one possible implementation manner of the present application, the apparatus further includes an auxiliary focusing module, the central processing module performs snapshot on the target through the ball machine based on the coordinate information of the ball machine, and performs recheck on a detection result of the radar based on the snapshot image, including:
adjusting the current state to the state corresponding to the coordinate information of the spherical machine;
focusing a lens of the dome camera based on the target distance through the auxiliary focusing module;
and the ball machine is used for capturing the target, and the ball machine rechecks the detection result of the radar according to the captured image.
In another aspect, there is provided an apparatus comprising a processor, a communication interface, a memory and a communication bus, the processor, the communication interface and the memory completing communication with each other via the communication bus, the memory storing a computer program, the processor being configured to execute the program stored on the memory to implement the steps of the method of the other aspect.
In another aspect, a computer-readable storage medium is provided, in which a computer program is stored which, when executed by a processor, carries out the steps of the method according to the above another aspect.
In another aspect, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the method of the above another aspect.
The technical scheme that this application provided can bring following beneficial effect at least:
the height measurement module detects the height from the dome camera to the horizontal ground, namely detects the first height, and the central processing module determines the mapping relation between the radar coordinates and the dome camera coordinates based on the first height, the second height and the view field included angle. The second height is the relative height of the radar and the dome camera in the vertical direction, and the field angle is the angle between the zero-degree field angle of the radar and the zero-degree field angle of the dome camera. Namely, the radar and the ball machine are automatically calibrated in a combined mode, the fact that data are acquired manually for many times to perform combined calibration is avoided, and calibration efficiency is improved.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
First, a simple description will be given of terms related to the embodiments of the present application.
And (3) radar: an apparatus for detecting targets by electromagnetic waves has the characteristic of all-weather all-day operation. The target is irradiated by transmitting electromagnetic waves and echoes thereof are received, thereby obtaining information such as the distance from the target to the electromagnetic wave transmitting point, the distance change rate (radial velocity), the azimuth, the altitude, and the like.
Ball machine: the PTZ-variable large-range monitoring camera has the mechanical structures such as a holder and the like, and can give consideration to panoramic information and detail information of a monitoring scene. Where P represents the horizontal angle of the machine, T represents the pitch angle of the machine, and Z represents the scaling factor of the machine. By means of the cradle head control structure, the horizontal direction which can be covered by the spherical machine is 360 degrees, and the vertical direction is more than 90 degrees.
Lei Qiu linkage: through the radar and the dome camera combined monitoring, when the radar detects a suspected moving target, the dome camera can be driven to rotate and track the suspected target in real time, so that secondary identification is performed.
And (3) joint calibration: the method is to determine the conversion relation between the radar and the dome camera under different coordinate systems, or determine the conversion relation between the radar and the dome camera parameters so as to achieve the consistency description of the targets.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an apparatus for linkage monitoring according to an exemplary embodiment. The apparatus may include: radar 101, dome camera 102, altimeter module 103 and central processing module 104,
the radar 101 and the dome camera 102 are relatively fixedly connected, and as an example, the radar 101 and the dome camera 102 may be integrally designed. Further, the device also includes a mounting bracket, such that the radar 101 and the dome camera 102 are secured to the mounting bracket. The relative height of the radar 101 and the dome camera 102 in the vertical direction, which may be expressed as Δh by way of example, is fixed and known and may be pre-stored in the means for coordinated monitoring.
The radar 101 may be a millimeter wave radar, a laser radar, or the like, and the number of radars may be one or more. When the number of the radars is plural, each radar corresponds to a different detection azimuth, detection distance, wherein fig. 1 shows only a schematic structural view of a single radar. In addition, the radar may be configured with a radar antenna array so that targets are detected by the radar antenna array.
The dome camera 102 may be a high-speed spherical camera, for example, a light-sensitive camera, a thermal imaging camera, or an infrared camera, and in implementation, the dome camera may be rotated by means of a pan/tilt control structure, which is not limited in the embodiment of the present application.
The height measurement module 103 is fixedly connected with the ball machine 102, and is used for detecting a first height, which is a height from the ball machine to the horizontal ground, that is, the first height refers to Δh. As an example, the altimeter module 103 may detect the first altitude using a functional sensor such as a laser ranging, and as an example, the altimeter module 103 may be located at the bottom of the ball machine.
Next, a specific implementation of the height measurement module 103 detecting the first height will be described. As an example, the apparatus further comprises a ball machine attitude detection module coupled to the ball machine 102 for detecting an inclination of the ball machine 102 with respect to a level ground surface. In this case, the central processing module performs zeroing on the ball machine based on the inclination angle, and the height measurement module 103 is configured to detect the height between the zeroed ball machine and the horizontal ground, so as to obtain the first height.
As an example, the ball machine gesture detection module is disposed inside the ball machine 102, or the ball machine gesture detection module is disposed integrally with the ball machine 102 so as to obtain an inclination angle of the ball machine with respect to the horizontal ground. As an example, the ball machine gesture detection module may employ a gravitational acceleration or other sensor with similar functions, which is not limited in this embodiment of the present application.
Since the device is typically mounted at an inclination angle, such as y in fig. 2, the first height is usually obtained by zeroing the ball machine, where the central processing module 104 can zero the ball machine 102 based on the inclination angle y.
As an example, a specific implementation of the central processing module 104 zeroing the spherical machine 102 based on the tilt angle γ may include: the central processing module 104 controls the horizontal angle of the ball machine 102 to remain unchanged and the pitch angle to be offset by the magnitude of the tilt angle so that the pitch angle is corrected to zero degrees.
It will be appreciated that when the tilt angle γ is present, the pitch angle of the machine is not zero, i.e., the machine 102 is not at zero degrees. If the ball machine 102 is not at zero degrees, the measured height between the ball machine 102 and the level ground may be greater than the first height actually measured, so in order to accurately measure the height of the ball machine 102 relative to the level ground, the horizontal angle of the ball machine 102 may be held constant and the pitch angle offset by the amount of the tilt angle so that the pitch angle is corrected to zero degrees, thereby placing the ball machine 102 at zero degrees.
After zeroing the dome camera 102 is completed, the altimeter module 103 may detect the height between the dome camera 102 and the level ground at that time, where it may detect using optical principles, to determine the installed height of the dome camera 102 relative to the level ground, resulting in the first height.
The central processing module 104 is respectively connected with the radar 101, the dome camera 102 and the height measurement module 103, and is configured to determine a mapping relationship between a radar coordinate and a dome camera coordinate based on the first altitude, the second altitude and a field angle, so as to perform joint calibration on the radar 101 and the dome camera 102. The second height is a relative height of the radar 101 and the spherical machine 102 in a vertical direction, and the field angle is an angle between a zero angle of view of the radar 101 and a zero angle of view of the spherical machine 102.
The angle of the field of view may be known in advance, for example, the angle of the field of view may be preset when the device leaves the factory, and may be represented as β in this case, as shown in fig. 3, where the coordinate system represented by the solid line is the coordinate system corresponding to the spherical camera, and the coordinate system represented by the dashed line is the coordinate system corresponding to the radar. If the Y-axis direction is the right front of the installation of the device, namely the zero-degree view angle direction of the radar, the zero-degree view angle direction of the dome camera is-beta. In some embodiments, the β may be zero, i.e., the radar and dome camera fields of view are the same, and the zero degree angles of view overlap, with no included angle.
When the number of radars is plural, the number of the included angles of view correspondingly includes plural, that is, includes the included angle between the zero angle of view of each radar and the zero angle of view of the dome camera.
The central processing module 104 serves as a core of the linkage monitoring device, and can control the coordinated operation of the modules. As an example, the central processing module 104 may be disposed in a module where the radar is located, or may be disposed in a module where the dome camera is located, or may be separately formed into a module, for example, may be disposed in a device, and communicate with other modules through a wired or wireless manner.
In implementation, the central processing module 104 obtains the first height measured by the height measurement module 103, and obtains the second height and the included angle of the field of view, which are stored in advance, and then can determine the mapping relationship between the radar coordinates and the coordinates of the spherical machine according to the first height, the second height and the included angle of the field of view, so as to realize the joint calibration of the radar and the spherical machine.
As an example, the mapping relationship between radar coordinates and spherical coordinates can be expressed by the following formula (1) and formula (2):
wherein p is the horizontal angle of the ball machine, T is the pitch angle of the ball machine, ther is the target distance between the target detected by the radar and the radar, alpha is the radar detection angle, and hc For the first height, hr Is the sum of the first height and the second height, and beta is the included angle of the field of view.
Therefore, under the condition that the first height, the second height and the included angle of the field of view are known, the relation between the radar coordinate and the spherical machine coordinate can be converted according to the mapping relation shown in the formulas (1) and (2), so that the joint calibration of the radar sitting and the spherical machine is realized.
As another example, the above formula (1) and formula (2) may also be expressed as shown in the following formula (3) and formula (4):
next, the derivation process of the above formula will be described.
1. And (3) relation between the coordinate of the spherical machine and the real coordinate of the target.
It should be noted that the target involved in the derivation process may be any point on the horizontal ground, and there is no need to have a real target exist.
As shown in fig. 4, fig. 4 is a schematic diagram illustrating a space geometrical model of a ball machine detection target according to an exemplary embodiment. A rectangular coordinate system is established on a plane where the target is located, the front-back direction is taken as an X axis, the left-right direction is taken as a Y axis direction, assuming that the horizontal angle of the ball machine detection target is P, the pitch angle is T, the ball machine is located at a point A, and the target is located at a point B, it is easy to understand that the coordinates of the target in the X axis direction and the Y axis direction are respectively determined by the following formulas (5) and (6):
2. relation of radar coordinates to real coordinates of the target.
It should be noted that, in determining the relationship between the radar coordinates and the true coordinates of the target, the selected target is the same as the target in determining the relationship between the spherical coordinates and the true coordinates of the target, or the selected position point is the same.
Referring to fig. 5, fig. 5 is a schematic diagram illustrating a spatial geometric model of a radar detection target according to an exemplary embodiment. The direction perpendicular to the plane where the X axis and the Y axis are located is taken as the Z axis direction, the origin of the coordinate system is taken as O, the M point is assumed to be the installation position where the radar is located, and the target is located at the point B.
For ease of understanding, the planar relationship within this space will be briefly described herein. The Y-axis is the zero-degree view angle direction of the radar, the installation height of the radar is OM, and h is used herer Representation, then the hr =om; let the true coordinates of the target be (x, Y), make the vertical line of the Y axis through the B point, cross the Y axis to the S point, i.e., sb=x, os=y, connect MS and OB. Since BS is parallel to the X-axis, which is the normal vector to plane ZOX, BS is perpendicular to plane ZOX, i.e., the angle MSB is 90 degrees, where the symbol "<" denotes an angle.
According to the radar measurement principle, MB is the distance between the radar and the target, namely r, the radar detection angle is alpha, and the following formulas (7), (8) and (9) can be obtained according to the graph:
x＝rsinα (9)
the following formula (10) can be obtained from the above formulas (7), (8) and (9):
according to the relation between the spherical coordinates and the real coordinates of the target and the relation between the radar coordinates and the real coordinates of the target, the above formula (1) and formula (2) can be derived, namely, the mapping relation between the radar coordinates and the spherical coordinates can be derived.
Further, after determining the mapping relation between the radar coordinates and the spherical machine coordinates, the central processing module can control the spherical machine and the radar to carry out linkage monitoring by utilizing the mapping relation.
In one possible implementation manner, the specific implementation of controlling the ball machine and the radar to perform linkage monitoring by using the mapping relation by the central processing module may include: and acquiring radar coordinate information of a target detected by the radar, wherein the radar coordinate information at least comprises a target distance and a radar detection angle, the target distance is the distance between the target and the radar, the radar coordinate information is converted into spherical machine coordinate information based on the mapping relation, and the spherical machine coordinate information is transmitted to the spherical machine.
That is, the central processing module may convert the radar coordinate information into the spherical coordinate information by using the mapping relationship, for example, the target distance r and the radar detection angle α in the radar coordinate information may be brought into the above formulas (1) and (2), and the horizontal angle P and the pitch angle T corresponding to the spherical coordinate information may be obtained.
As an example, the apparatus may further comprise an auxiliary focusing module for performing lens focusing of the dome camera based on the target distance. The auxiliary focusing module may be located inside the ball machine or may be located outside the ball machine.
As an example, for a ball machine, after receiving the coordinate information of the ball machine, the following operations are performed: and adjusting the current state to a state corresponding to the coordinate information of the spherical machine, focusing a lens of the spherical machine based on the target distance through an auxiliary focusing module, capturing the target, and rechecking the detection result of the radar according to the captured image.
That is, for the ball machine, the horizontal angle and the pitch angle of the ball machine are adjusted according to the coordinate information of the ball machine, and the ball machine focuses the lens of the ball machine according to the target distance through the auxiliary focusing module, so that the ball machine works in a state corresponding to the coordinate information of the ball machine. After the state is adjusted, the goal is captured by the dome camera to obtain a captured image, the dome camera performs image processing according to the captured image to obtain a processing result, and then the processing result can be utilized to recheck the detection result of the radar so as to determine whether a real goal exists or not, and decision is made to output a rechecking result.
Further, the device can further comprise an alarm, and when the real target is determined to exist after rechecking, the central processing unit can further control the alarm to carry out alarm prompt.
Further, the device also comprises a protective cover which can protect the ball machine and the radar so as to prevent the ball machine and the radar from being worn, thus prolonging the service lives of the radar and the ball machine. In addition, the protective cover is generally transparent so as not to affect the normal shooting of the ball machine.
In the embodiment of the application, the height measuring module detects the height from the dome camera to the horizontal ground, namely detects the first height, and the central processing module determines the mapping relation between the radar coordinates and the dome camera coordinates based on the first height, the second height and the view field included angle. The second height is the relative height of the radar and the dome camera in the vertical direction, and the field angle is the angle between the zero-degree field angle of the radar and the zero-degree field angle of the dome camera. Namely, the radar and the ball machine are automatically calibrated in a combined mode, the fact that data are acquired manually for many times to perform combined calibration is avoided, and calibration efficiency is improved.
Referring to fig. 6, fig. 6 is a flowchart illustrating a method of linkage monitoring according to an exemplary embodiment, where the method may be implemented by the above device, and may specifically include the following steps:
step 601: the central processing module acquires radar coordinate information of a target detected by the radar, wherein the radar coordinate information at least comprises a target distance and a radar detection angle, and the target distance is the distance between the target and the radar.
The specific implementation of the method may refer to the description of the specific implementation of the apparatus, and the description is not repeated here.
Step 602: the central processing module converts the radar coordinate information into spherical machine coordinate information based on a mapping relation, wherein the mapping relation refers to a relation between radar coordinates and spherical machine coordinates.
Wherein the mapping relation is determined based on a first height, a second height and a view field included angle, the first height is the height from the dome camera to the horizontal ground, which is detected by the height measurement module, the second height is the relative height between the radar and the dome camera in the vertical direction, and the view field included angle is the included angle between the zero degree view field angle of the radar and the zero degree view field angle of the dome camera
As an example, the tilt angle of the ball machine relative to the horizontal ground may be detected by the ball machine gesture detection module, and accordingly, the central processing module controls the ball machine to zero based on the tilt angle, and then the height measurement module is configured to detect the height between the zeroed ball machine and the horizontal ground, to obtain the first height.
As an example, the central processing module controls the horizontal angle of the ball machine to remain unchanged and the pitch angle to be offset by the magnitude of the tilt angle so that the pitch angle is corrected to zero degrees.
The second height is the relative height of the radar and the dome camera in the vertical direction, and the field angle is the angle between the zero-degree field angle of the radar and the zero-degree field angle of the dome camera.
Illustratively, the mapping relationship is represented by the following formulas (1) and (2):
wherein p is the horizontal angle of the dome camera, T is the pitch angle of the dome camera, r is the target distance between the radar detected target and the radar, alpha is the radar detection angle, and hc For the first height, hr Is the sum of the first height and the second height, and beta is the included angle of the field of view.
Step 603: the central processing module is used for capturing the target through the dome camera based on the coordinate information of the dome camera, and rechecking the detection result of the radar based on the captured image.
After the mapping relation is determined, the central processing module acquires radar coordinate information of a target detected by the radar, wherein the radar coordinate information at least comprises a target distance and a radar detection angle, and the target distance is the distance between the target and the radar. Based on the mapping relation, the radar coordinate information is converted into spherical machine coordinate information, and the spherical machine coordinate information is transmitted to the spherical machine.
As an example, the spherical camera adjusts the current state to the state corresponding to the coordinate information of the spherical camera, focuses the lens of the spherical camera based on the target distance through the auxiliary focusing module, captures the target, and rechecks the detection result of the radar according to the captured image.
In the embodiment of the application, the height measuring module detects the height from the dome camera to the horizontal ground, namely detects the first height, and the central processing module determines the mapping relation between the radar coordinates and the dome camera coordinates based on the first height, the second height and the view field included angle. The second height is the relative height of the radar and the dome camera in the vertical direction, and the field angle is the angle between the zero-degree field angle of the radar and the zero-degree field angle of the dome camera. Namely, the radar and the ball machine are automatically calibrated in a combined mode, the fact that data are acquired manually for many times to perform combined calibration is avoided, and calibration efficiency is improved.
Referring to fig. 7, a schematic structural diagram of an apparatus provided in an exemplary embodiment of the present application is shown, where a central processing module may be configured in the apparatus. The apparatus includes: a processor 701, a receiver 702, a transmitter 703, a memory 704 and a bus 705.
The processor 701 includes one or more processing cores, and the processor 701 executes various functional applications and information processing by running software programs and modules.
The receiver 702 and the transmitter 703 may be implemented as one communication component, which may be a communication chip.
The memory 704 is connected to the processor 701 through the bus 705.
The memory 704 may be used for storing at least one instruction, and the processor 701 is configured to execute the at least one instruction to implement the steps performed by the apparatus in the method embodiments described above.
Further, memory 704 may be implemented by any type of volatile or nonvolatile storage device or combination thereof, including but not limited to: magnetic or optical disks, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), static Random Access Memory (SRAM), read-only memory (ROM), magnetic memory, flash memory, programmable read-only memory (PROM).
The present application provides a computer readable storage medium having stored therein at least one instruction that is loaded and executed by the processor to implement the methods provided by the above embodiments.
The present application also provides a computer program product which, when run on a computer, causes the computer to perform the method provided by the above embodiments.
In some embodiments, there is also provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of coordinated monitoring of the above embodiments. For example, the computer readable storage medium may be a ROM (Read-Only Memory), a RAM (Random-Access Memory), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
It is noted that the computer readable storage medium mentioned in the present application may be a non-volatile storage medium, in other words, may be a non-transitory storage medium.
It should be understood that all or part of the steps to implement the above-described embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions may be stored in the computer-readable storage medium described above.
That is, in some embodiments, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the method of coordinated monitoring described above.
The above embodiments are provided for the purpose of not limiting the present application, but rather, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are intended to be included within the scope of the present application.