Disclosure of Invention
The invention provides a multi-echo judging method and device for a laser radar, which are used for solving the problem of range ambiguity caused by incapability of corresponding return signals of laser radar pulses and transmitted pulse signals in the range finding process of the laser radar.
The technical scheme for solving the technical problems is as follows:
in a first aspect, the present invention provides a laser radar multi-echo discrimination method, including:
Acquiring images Ii and Ii+1 shot by cameras at the moments Ti and Ti+1, and acquiring laser radar echo pulses received in at least one period after the moment Ti, wherein I is a positive integer, the laser radar emits pulses for the first time at the moment T1, and the period is a pulse emission period of the laser radar;
According to the laser radar echo pulse and the laser radar emission pulse at the moment Ti, calculating the position coordinate P of a target point corresponding to the echo pulse;
calculating an image coordinate Mi projected onto Ii and an image coordinate Mi+1 projected onto Ii+1 according to the position coordinate P of the target point by using a camera imaging model;
And calculating the similarity of the image coordinates Mi and the image coordinates Mi+1, if the similarity is higher than a preset threshold, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti, otherwise, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti-1.
Further, when the cycle number of the obtained laser radar echo pulse is greater than one cycle, the matched laser radar echo pulse is deleted from the laser radar echo pulse obtained in the next iteration process.
Further, the upper limit of the cycle number for acquiring the laser radar echo pulse is determined according to the hardware parameters of the laser radar system.
Further, the calculating the similarity between the image coordinate Mi and the image coordinate Mi+1 includes:
acquiring a neighborhood pixel of an image coordinate Mi on an image Ii, and acquiring a neighborhood pixel of an image coordinate Mi+1 on an image Ii+1;
Calculating the corresponding local texture feature Fi according to the neighborhood pixels of the image coordinate Mi, and calculating the corresponding local texture feature Fi+1 according to the neighborhood pixels of the image coordinate Mi+1;
The similarity of the local texture features Fi and Fi+1 is measured using the correlation coefficient.
In a second aspect, the present invention provides a laser radar multi-echo discriminating apparatus, including:
The data acquisition module acquires images Ii and Ii+1 shot by cameras at the moments Ti and Ti+1, and acquires laser radar echo pulses received in at least one period after the moment Ti, wherein I is a positive integer, the laser radar transmits pulses for the first time at the moment T1, and the period is a pulse transmitting period of the laser radar;
The coordinate resolving module is used for resolving the position coordinate P of the target point corresponding to the echo pulse according to the echo pulse of the laser radar and the laser radar emission pulse at the moment Ti;
the projection calculation module calculates an image coordinate Mi projected onto Ii and an image coordinate Mi+1 projected onto Ii+1 according to the position coordinate P of the target point by using the camera imaging model;
The matching module calculates the similarity between the image coordinate Mi and the image coordinate Mi+1, if the similarity is higher than a preset threshold, the radar echo corresponding to the target point is matched with the laser radar emission pulse at the moment Ti, otherwise, the radar echo corresponding to the target point is matched with the laser radar emission pulse at the moment Ti-1.
Further, when the cycle number of the obtained laser radar echo pulse is greater than one cycle, the matched laser radar echo pulse is deleted from the laser radar echo pulse obtained in the next iteration process.
Further, the upper limit of the cycle number for acquiring the laser radar echo pulse is determined according to the hardware parameters of the laser radar system.
Further, the matching module comprises a similarity calculation module, wherein the similarity calculation module is specifically used for:
acquiring a neighborhood pixel of an image coordinate Mi on an image Ii, and acquiring a neighborhood pixel of an image coordinate Mi+1 on an image Ii+1;
Calculating the corresponding local texture feature Fi according to the neighborhood pixels of the image coordinate Mi, and calculating the corresponding local texture feature Fi+1 according to the neighborhood pixels of the image coordinate Mi+1;
The similarity of the local texture features Fi and Fi+1 is measured using the correlation coefficient.
In a third aspect, the present invention provides an electronic device comprising:
a memory for storing a computer software program;
and the processor is used for reading and executing the computer software program so as to realize the multi-echo judging method of the laser radar.
In a fourth aspect, the present invention provides a non-transitory computer readable storage medium, in which a computer software program for implementing a laser radar multi-echo discrimination method according to the first aspect of the present invention is stored.
The invention has the beneficial effects that by means of the camera module, the corresponding relation between the laser radar pulse return signal and the transmitting signal can be effectively solved, and the problem of ranging ambiguity is solved.
The method can solve the problem of radar pulse ranging ambiguity only by carrying the camera on the basis of the LiDAR system, has high economy and wider application range, and the camera system does not need to be calibrated with the LiDAR system, only needs to calibrate the camera, and has simple system and high stability.
Detailed Description
The principles and features of the present invention are described below with reference to the drawings, the examples are illustrated for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
The laser radar can obtain a series of measuring point distances according to the light flight parameters by periodically transmitting laser and receiving echoes, and can calculate the positions of the reflecting points by combining the positions and the postures of the laser. Typically, the laser transmission and reception are performed sequentially, i.e. transmit-receive-transmit-receive, with only one laser pulse in the air, and the reception and transmission are matched one to the other. If the object to be measured is far away, a plurality of pulses may occur in the air, the sequence of the plurality of pulses reaching the receiver is no longer consistent with the sequence of the pulse emission, and the receiver cannot calculate the Time of flight of the pulses correctly, so that the distance measurement of the object, namely the MTA (Multi-Time-Around) problem, cannot be obtained correctly, and the number of transceiving cycles over which the reflected signal may cross is generally called an MTA interval. As fig. 1 shows that the transmitted pulse and the received echo are matched one by one, there is no range ambiguity, and fig. 2 shows that the transmitted pulse and the received echo no longer correspond, there is a range ambiguity.
The embodiment of the invention provides a multi-echo judging method of a laser radar, which is characterized in that image information of laser radar echo time is obtained through camera equipment synchronized by a LiDAR system, and the corresponding relation between the laser radar echo time and laser pulse transmitting time is unknown, but the image information at the assumed distance measuring value is necessarily stored in the image information synchronously obtained at the transmitting pulse time by assuming that each echo corresponds to the transmitting pulse in a specified period. When the carrier moves a certain distance, the camera can acquire an image at another position, and the corresponding relation between the laser echo and the emission pulse is determined by comparing the similarity of two points of the two images.
Specifically, as shown in fig. 3, the method for determining multiple echoes of a lidar provided by the embodiment of the invention includes the following steps:
Acquiring images Ii and Ii+1 shot by cameras at the moments Ti and Ti+1, and acquiring laser radar echo pulses received in at least one period after the moment Ti, wherein I is a positive integer, the laser radar emits pulses for the first time at the moment T1, and the period is a pulse emission period of the laser radar;
According to the laser radar echo pulse and the laser radar emission pulse at the moment Ti, calculating the position coordinate P of a target point corresponding to the echo pulse;
calculating an image coordinate Mi projected onto Ii and an image coordinate Mi+1 projected onto Ii+1 according to the position coordinate P of the target point by using a camera imaging model;
And calculating the similarity of the image coordinates Mi and the image coordinates Mi+1, if the similarity is higher than a preset threshold, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti, otherwise, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti-1.
The modern laser radar system is provided with GPS, IMU and the like to acquire POS data, if the position and direction information contained in the POS (Position and Orientation System, positioning and attitude determination system) data of each laser pulse can be utilized to reconstruct a three-dimensional space scene during laser transmission, a transmitted laser pulse mathematical model can be established under a specified direction and speed, an object at a laser echo is projected into an image by combining with a camera imaging model, and the similarity between the images at the echo is calculated in the images shot by cameras at different positions so as to determine the corresponding relation between the echo pulse and the transmitted pulse, thereby solving the problem of ranging ambiguity.
Assuming that the laser radar emits laser pulses at the moment T1 and triggers the camera to shoot at the same time, the acquired image is I1, the laser pulses generate three echoes when encountering objects with different layers on the ground, and the received echo moments are T11,t12,t13 in sequence. After the laser radar emits laser pulses at the pulse interval delta T at the moment T2, the camera obtains an image I2, the laser pulses also generate three echoes after encountering objects with different layers on the ground, and the received echoes are sequentially T21,t22,t23. After the pulse interval Δt, the lidar transmits pulse T3 while acquiring the image as I3. The lidar time is shown in fig. 4.
LiDAR systems can pulse the distance to radar by a time difference Jie Suanchu between the return time of the laser pulse and the emission time of the laser pulse. As shown in fig. 4, there are three radar echoes in the radar pulse period from the time T2 to the time T3, where the radar transmit pulse corresponding to the echo T13 is at the time T1. It is first assumed that the echoes T13,t21,t22 received in the period T2 each correspond to a transmit pulse T2, from which the object position P13(x13,y13,z13),P21(x21,y21,z21),P22(x22,y22,z22) at the reflection corresponding to the echo T13,t21,t22 is resolved by the radar system as shown in fig. 5.
According to the camera imaging model:
s·M=A[R|t]P
Where s is a proportionality constant, P represents coordinates of a spatial point, M represents coordinates of a pixel point projected on an image plane, a represents a camera internal reference matrix, and r|t represents a camera rotation translation matrix.
Laser pulse return position coordinates P13(x13,y13,z13),P21(x21,y21,z21),P22(x22,y22,z22) can be projected onto image I2 image coordinates
When the camera moves to the next moment along with the carrier, such as T3, the position coordinate P13(x13,y13,z13),P21(x21,y21,z21),P22(x22,y22,z22) calculated according to the echo at the moment T2 can be converted to the image coordinate of the image I3 by the camera imaging model
According to the image coordinates on the image I2And image I3 coordinatesThe local texture LBP feature is computed for the neighborhood pixels, and two corresponding image coordinates are measured by pearson correlation coefficients (e.g.,And (3) withAnd (3) withAnd (3) with) And if the similarity of the neighborhood LBP characteristics is high, the laser echo analysis at the position P is considered to be correct. It should be understood that, in this solution, besides measuring the similarity of two image coordinates by using the LBP feature and the pearson correlation coefficient, other common image similarity determining methods may be adopted, such as cosine distance, histogram, euclidean distance, and the like.
If the degree of similarity is low, such as the coordinate P13(x13,y13,z13), the corresponding points on the images of I2 and I3AndSince the laser pulse actually matched with the echo pulse corresponding to P13(x13,y13,z13) is T1, the position obtained by analysis in the laser pulse period corresponding to T2 is a virtual position, and in the process of projection onto the camera image plane, the object point corresponding to the virtual point is actually imaged. All the resolving positions of the ambiguous laser pulse echoes are projected to different image planes at different moments, the image features between the points have larger differences, the laser pulse echoes correspond to the transmitted laser pulses, and the ranging resolving of each echo is accurate.
As shown in fig. 6, an embodiment of the present invention further provides a laser radar multi-echo discriminating apparatus, including:
The data acquisition module acquires images Ii and Ii+1 shot by cameras at the moments Ti and Ti+1, and acquires laser radar echo pulses received in at least one period after the moment Ti, wherein I is a positive integer, the laser radar transmits pulses for the first time at the moment T1, and the period is a pulse transmitting period of the laser radar;
The coordinate resolving module is used for resolving the position coordinate P of the target point corresponding to the echo pulse according to the echo pulse of the laser radar and the laser radar emission pulse at the moment Ti;
the projection calculation module calculates an image coordinate Mi projected onto Ii and an image coordinate Mi+1 projected onto Ii+1 according to the position coordinate P of the target point by using the camera imaging model;
The matching module calculates the similarity between the image coordinate Mi and the image coordinate Mi+1, if the similarity is higher than a preset threshold, the radar echo corresponding to the target point is matched with the laser radar emission pulse at the moment Ti, otherwise, the radar echo corresponding to the target point is matched with the laser radar emission pulse at the moment Ti-1.
The matching module comprises a similarity calculation module, wherein the similarity calculation module is specifically used for:
acquiring a neighborhood pixel of an image coordinate Mi on an image Ii, and acquiring a neighborhood pixel of an image coordinate Mi+1 on an image Ii+1;
calculating a corresponding local texture LBP characteristic Fi according to the neighborhood pixels of the image coordinate Mi;
The similarity of local texture LBP features Fi and Fi+1 is measured using pearson correlation coefficients.
Referring to fig. 7, fig. 7 is a schematic diagram of an embodiment of an electronic device according to an embodiment of the invention. As shown in fig. 7, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored on the memory 520 and executable on the processor 520, wherein the processor 520 executes the computer program 511 to implement the following steps:
Acquiring images Ii and Ii+1 shot by cameras at the moments Ti and Ti+1, and acquiring laser radar echo pulses received in at least one period after the moment Ti, wherein I is a positive integer, the laser radar emits pulses for the first time at the moment T1, and the period is a pulse emission period of the laser radar;
According to the laser radar echo pulse and the laser radar emission pulse at the moment Ti, calculating the position coordinate P of a target point corresponding to the echo pulse;
calculating an image coordinate Mi projected onto Ii and an image coordinate Mi+1 projected onto Ii+1 according to the position coordinate P of the target point by using a camera imaging model;
And calculating the similarity of the image coordinates Mi and the image coordinates Mi+1, if the similarity is higher than a preset threshold, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti, otherwise, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti-1.
Referring to fig. 8, fig. 8 is a schematic diagram of an embodiment of a computer readable storage medium according to an embodiment of the invention. As shown in fig. 8, the present embodiment provides a computer-readable storage medium 600 having stored thereon a computer program 611, which computer program 611 when executed by a processor implements the steps of:
Acquiring images Ii and Ii+1 shot by cameras at the moments Ti and Ti+1, and acquiring laser radar echo pulses received in at least one period after the moment Ti, wherein I is a positive integer, the laser radar emits pulses for the first time at the moment T1, and the period is a pulse emission period of the laser radar;
According to the laser radar echo pulse and the laser radar emission pulse at the moment Ti, calculating the position coordinate P of a target point corresponding to the echo pulse;
calculating an image coordinate Mi projected onto Ii and an image coordinate Mi+1 projected onto Ii+1 according to the position coordinate P of the target point by using a camera imaging model;
And calculating the similarity of the image coordinates Mi and the image coordinates Mi+1, if the similarity is higher than a preset threshold, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti, otherwise, matching the radar echo corresponding to the target point with the laser radar emission pulse at the moment Ti-1.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.