Automatic parking obstacle sensing method based on millimeter wave radarTechnical Field
The invention relates to the technical field of automatic parking of automobiles, in particular to an automatic parking obstacle sensing method based on millimeter wave radar.
Background
In order to avoid safety accidents, the intelligent driving system must pay attention to the surrounding environment of the vehicle body at all times in the automatic parking process. Millimeter wave radar, one of the sensing "organs" of intelligent driving systems, must have the ability to accurately sense the environment. At present, the millimeter wave radar is mostly applied in the driving process, and the ultrasonic radar is mostly adopted for the parking to perform environment sensing. The vehicle millimeter wave radar adopts a frequency modulation continuous wave system, the sampling starting distance is zero, and the theoretical distance blind area is extremely small, so that the vehicle millimeter wave radar has a great deal of field in the aspect of short-range target detection. In practice, due to the influence of direct current signal noise, the detection performance and the ranging performance of the near-distance target are both influenced, and the detection capability of the near-distance target is greatly reduced under the condition of considering the false alarm rate and the false alarm rate.
The patent refers to a 4D millimeter wave radar-based automatic parking method and system, which adopts the 4D millimeter wave radar to detect the reference distance between the vehicle tail and each reference object, and does not consider the actual short-range target detection performance and the distance measurement precision of the millimeter wave radar. In the patent 'a vehicle-mounted millimeter wave radar target classifying method, device, system and storage medium', an energy distribution diagram in Doppler frequency-time dimension is used as network input to classify targets, but the Doppler stationary clutter is filtered through vector average cancellation in the signal preprocessing process, and important targets are lost when the radar is stationary or runs at a low speed, particularly in a parking mode, the targets are mostly stationary or low-speed targets relative to the radar. And the patent adopts single frame images for classification, so that the accuracy is low. The paper "human detection based on time-varying signature on range-doppler diagram using deep neural networks" adopts multi-frame images as input, so that the classification accuracy is improved, but the calculated amount is large, the occupied memory is more, and the method is not suitable for vehicle-mounted equipment.
Disclosure of Invention
The invention mainly provides an automatic parking obstacle sensing method based on millimeter wave radar for an automatic parking scene, which meets the requirement of an intelligent driving system for accurately sensing obstacles around a vehicle body in the parking process, thereby realizing efficient and safe automatic parking.
The vehicle millimeter wave radar transmits a linear frequency modulation continuous wave signal:
Wherein the method comprises the steps ofFor fast time, T is the single chirp (which can be understood as pulse) duration (i.e. pulse width),As the carrier frequency of the signal,Is a fast time frequency, andIn correspondence with the fact that,For frequency modulation. rect represents a rectangular function expressed asExp is an exponential function, and the expression isJ represents the imaginary part.
The target reflection echo is as follows:
Wherein the method comprises the steps ofThe distance from the target to the radar array is the speed of light, c. After receiving the back wave, the method carries out the processing of the de-line tone, namely, the method carries out conjugate multiplication with the reference signal, thus obtaining a single-frequency signal withI.e. the frequency of the single frequency signal is proportional to the distance of the target. The echo is processed by a line-removing tone and is subjected to Fourier transformation to obtain a one-dimensional range profile, and each sampling frequency point corresponds to targets with different distances.
The vehicle radar has a short acting distance, and the target echo delay at the maximum acting distance is smaller than the pulse width, so that the sampling starting distance is zero, and the corresponding starting frequency is zero.
The sampling initial frequency is zero, direct current and low-frequency noise are inevitably introduced into the echo, and when the noise power is too high, radar near-distance targets are submerged, so that detection blind areas are generated. The detection blind area changes along with the distance resolution of the radar, and the worse the resolution is, the larger the detection blind area range is. Under the automatic parking scene, the surrounding environment of the vehicle body needs to be accurately perceived, so that collision is avoided.
The invention provides an automatic parking obstacle sensing method based on millimeter wave radar, which aims to solve the problem of radar near detection blind areas and comprises the following specific steps:
Step S1. No target data set is created,
Collecting a plurality of non-target scenes (the non-target scenes comprise a plurality of idle scenes with a static radar, a plurality of idle scenes with a low-speed radar motion and a high-speed radar motion scene, wherein the speed of the low-speed radar motion is lower than 2m/s, the speed of the high-speed radar motion is higher than 10 m/s), acquiring a 2DFFT result (2 DFFT refers to a core algorithm for processing a linear frequency modulation continuous wave signal, and is used for extracting the distance and speed information of a target) of each receiving channel under the condition that the distance of each receiving channel is lower than 2m/s, and taking out the data of a plurality of distance dimension sampling units of each receiving channel (the radar short distance region refers to a region with a distance of 0-5 m, and the radar short distance region can be adjusted according to practical application), recording and taking out the temperature of a radar radio frequency chip while collecting the data, ensuring that a non-target data set covers as many scenes as possible and containing all possible working temperatures (upper limit and lower limit) of the radar radio frequency chip. Finally, the fetched data forms a non-target data sample.
The distance dimension refers to a dimension in a radar wave propagation direction, and is used for measuring a linear distance between a target and a radar, and the distance dimension is a unit of an actual distance represented by each sampling point. The sampling starting distance is the radar array plane, and the space is sampled according to the resolution ratio. The working temperature of the radar radio frequency chip is not the ambient temperature, the radar heats during working, the internal temperature rises, the radar radio frequency chip is affected by the ambient temperature to a certain extent, the radar is mainly dependent on the radar power and the heat dissipation performance, and the radar is designed to consider the ambient temperature, so that the working temperature of the radar radio frequency chip is all the working temperatures considering the extreme environment condition.
Step S2: object data set production,
Various targets are set in a radar near-range region, and the targets are classified into stationary strong targets, stationary weak targets and moving targets according to target characteristics. Wherein the moving object is set as an object whose relative moving speed exceeds the radar by one speed resolution unit. The stationary target is set as the target of which the relative movement speed is not more than one speed resolution unit of the radar. The static targets are divided into strong targets and weak targets, the strong targets are mainly distinguished by scattering intensity of electromagnetic waves, the static strong targets are set to be strong scattering static targets which comprise but are not limited to metals and walls, and the static weak targets are set to be weak scattering static targets which comprise but are not limited to cones and green belts.
Each target is provided with a plurality of distance angles, the 2DFFT results of the distances of all receiving channels of the radar under the radar motion state (the radar motion state is divided into a static state and a low-speed motion state, and the speed of the radar in the low-speed motion state is lower than 2 m/s) of the same target are acquired, the data of a plurality of distance dimension sampling units of the radar near-distance region of each receiving channel are taken out, the temperature of a radar radio frequency chip is recorded and taken out while the data are acquired, so that the taken out data form a target data sample, the set target is labeled, and the label corresponds to the target type.
And step S3, constructing a denoising network, taking the non-target data sample in the step S1 as a training sample, inputting the training sample into the denoising network, training, outputting residual errors, and continuing training iteration until the residual errors are equal to zero, thereby obtaining the denoising network with the residual errors equal to zero.
In order to avoid that direct current and low frequency noise affect radar near-distance target detection, namely direct current and low frequency noise in echo are eliminated, in step S1, data of various non-target scenes (including various idle scenes where the radar is stationary, various idle scenes where the radar moves at a low speed and radar high-speed moving scenes) are collected, because under the idle scenes and the radar high-speed moving scenes (the radar high-speed moving scenes are used for collecting noise samples under high-speed movement so as to perform denoising processing better), targets do not exist in radar near-distance areas necessarily, at this moment, the collected 2DFFT data is direct current and low frequency noise, the noise changes along with temperature, so that the temperature of a radar radio frequency chip is input simultaneously, and when the data are input through training, the network output residual error is close to zero (theoretically, the residual error is equal to zero), which is the purpose and expected result of training, and the denoising network is enabled to memorize the characteristics of the noise.
And S4, taking the target data sample in the step S2 as a training sample, inputting the training sample into the denoising network finally obtained in the step S3, training, outputting residual errors, carrying out target detection on the residual error result, judging whether a target exists or not, obtaining a target detection position and a target class, and continuing training iteration until the target detection position and the target class thereof are in one-to-one correspondence with the target position and the target class thereof set in the step S2, thereby obtaining a target detection network (namely the denoising network after training) and a classification network.
The 2DFFT data collected in the step S2 are direct current and low frequency noise plus target data, the noise data in the data can be removed by the denoising network according to noise characteristics through training, and the target data is reserved, so that a target detection network (namely the denoising network after training) and a classification network are obtained.
In summary, when the input data is an open scene or a high-speed motion scene, the residual error is close to zero and represents no target, when the input data is a target echo, the residual error is the target energy after denoising, the residual error result is subjected to target detection (cfar), distance interpolation is carried out, and a proper antenna array element is selected for angle measurement, so that the target position can be obtained.
In steps S1 and S2, when the data of the plurality of distance dimension sampling units in the radar short-distance region of each receiving channel are taken out, if the data size is smaller, a lightweight network can be selected for training, so that on-chip deployment is facilitated.
Step S5, denoising, detecting and identifying the target real-time data,
When the radar works normally, 2DFFT results of the distances of all receiving channels of the radar under the normal real-time working condition of the radar are collected, and at the moment, the radar has data of a radar near-distance area (the radar near-distance area refers to an area which is 0-5 meters away from the radar) and data of a radar far-distance area (the radar far-distance area refers to an area which is 5 meters away from the radar). The data of the radar remote area can be used for the radar function of normal running of the automobile, and the data of the radar near area can be used for the radar function of parking of the automobile.
And when the radar movement speed is higher than 2m/s, the radar near-range target detection module is not started to detect the near-range target.
When the radar moving speed is lower than 2m/S, a radar short-range target detection module is started to detect short-range targets, the short-range target detection comprises the steps of taking out data of a plurality of distance dimension sampling units of a radar short-range area of each receiving channel, simultaneously recording and taking out real-time temperature of a radar radio frequency chip, forming target real-time data by the taken out data, inputting the target real-time data into a target detection network finally obtained in the step S4, obtaining target data of each channel after denoising the network, summing the channel data to obtain a distance Doppler map (RDMAP), carrying out target detection on the distance Doppler map, carrying out conventional signal processing (namely, carrying out cfar (target detection), speed measurement, angle measurement (doa) and distance interpolation on RDMAP) to obtain target position information, accumulating multi-frame distance Doppler maps (RDMAP), taking out continuous 3-frame distance Doppler maps, inputting the target category into a classification network, and outputting the target category and the latest target information stored at the moment, thereby facilitating system decision.
If no target is detected in the continuous 3-frame range Doppler diagram in the radar near target detection range, indicating that no target exists in the radar observation range during parking, the target parameter information and the range Doppler diagram of the corresponding frame are cleared.
The range-doppler plot is a two-dimensional spectrum plot generated by 2DFFT, reflecting the range-velocity joint distribution of the target.
Compared with the prior art, the invention has the following technical effects:
Aiming at the problem of a millimeter wave radar close-range detection blind area, the invention provides an automatic parking obstacle sensing method based on a millimeter wave radar. The method comprises the steps of acquiring data, constructing a network model, training the model, finally obtaining a denoising network and a classification network, outputting denoised target data according to RDMAP and corresponding working temperatures which are input in real time, calculating the position of a target relative to a radar through cfar, speed measurement, angle measurement and distance interpolation, accumulating through a plurality of frames RDMAP, outputting the target class by the classification network, removing noise energy by the denoising network, improving the target distance precision, detecting the temperature of a radar radio frequency chip in the process of making a data set and processing in real time, taking the temperature as one of network model inputs, improving the robustness of the denoising network at different temperatures, avoiding noise residues caused by the temperature, outputting the target class, position and speed information when an obstacle exists in a radar observation range, facilitating the decision making of a system, and avoiding the system from making a false decision when no obstacle exists.
The invention will be further described with reference to the drawings and examples.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will briefly explain the embodiments or the drawings needed in the prior art, and it is obvious that the drawings in the following description are only some embodiments of the present invention and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow diagram of the method of the present invention;
FIG. 2 is a graph showing DC noise energy of a second distance sampling unit according to a variation of chip temperature in an embodiment of the present invention;
FIG. 3 is a graph showing target energy as a function of chip temperature in an embodiment of the invention;
FIG. 4 is a schematic diagram of a denoising network according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a classification network in an embodiment of the invention;
FIG. 6 is a schematic diagram of the original RDMAP when the second distance sampling unit is not targeted in an embodiment of the present invention;
FIG. 7 is a schematic diagram of denoising RDMAP when the second distance sampling unit is not targeted in an embodiment of the present invention;
FIG. 8 is a schematic diagram of the original RDMAP of the second distance sampling unit with targets in an embodiment of the present invention;
FIG. 9 is a schematic diagram of a denoising RDMAP when there is a target in the second distance sampling unit according to an embodiment of the present invention;
Fig. 10 is a schematic diagram of a result of denoising cfar when the second distance sampling unit has a target in the embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the invention will be readily understood, a more particular description of the invention will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The present invention may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit of the invention, whereby the invention is not limited to the specific embodiments disclosed below.
Aiming at an automatic parking scene, the embodiment provides an automatic parking obstacle sensing method based on millimeter wave radar, which meets the requirement of an intelligent driving system for accurately sensing obstacles around a vehicle body in the parking process, thereby realizing efficient and safe automatic parking.
As shown in fig. 1, the specific steps are as follows:
(1) The creation of the non-target data set,
Collecting 2DFFT results of distances of all receiving channels of the radar under various non-target scenes (the non-target scenes comprise various open scenes with static radar, various low-speed motion of the radar and various high-speed motion scenes of the radar, wherein the low-speed motion speed of the radar is lower than 2m/s, the high-speed motion speed of the radar is higher than 10 m/s), taking out data of a plurality of distance dimension sampling units of radar near-distance areas (radar near-distance areas refer to areas which are 0-5 m away from the radar and can be adjusted according to practical application) of all receiving channels, recording and taking out the temperature of a radar radio frequency chip while collecting the data, and ensuring that a non-target data set covers as many scenes as possible and comprises all working temperatures (upper limit and lower limit) which can appear on the radar radio frequency chip. Finally, the fetched data forms a non-target data sample.
In order to better illustrate the change of the energy of the direct current and low frequency noise along with the temperature change of the radar radio frequency chip, the embodiment takes out the data of the second distance sampling unit (the second distance sampling unit is located in the distance dimension) in the radar near-distance region, and as shown in fig. 2, the schematic diagram of the change of the direct current noise energy of the second distance sampling unit along with the temperature change of the radar radio frequency chip can be obtained, and the direct current noise energy of the second distance sampling unit is reduced along with the increase of the temperature of the radar radio frequency chip.
(2) The object data set is created such that,
Various targets are set in a radar near-range region, and the targets are classified into stationary strong targets, stationary weak targets and moving targets according to target characteristics. Each target is provided with a plurality of distance angles, the 2DFFT results of the distances of all receiving channels of the radar under the radar motion state (the radar motion state is divided into a static state and a low-speed motion state, and the speed of the radar in the low-speed motion state is lower than 2 m/s) of the same target are acquired, the data of a plurality of distance dimension sampling units of the radar near-distance region of each receiving channel are taken out, the temperature of a radar radio frequency chip is recorded and taken out while the data are acquired, so that the taken out data form a target data sample, the set target is labeled, and the label corresponds to the target type.
In order to better illustrate the change of the target energy with the temperature change of the radar rf chip, the embodiment takes out the target energy at 50m, and as shown in fig. 3, which is a schematic diagram of the change of the target energy with the temperature change of the chip, it can be obtained that the target energy at 50m decreases with the increase of the chip temperature of the radar rf chip.
In summary, the noise energy and the target energy are both related to the working temperature of the radar radio frequency chip, so that the corresponding working temperature of the radar radio frequency chip needs to be recorded when data are acquired, and the working temperature of the radar radio frequency chip is used as input at the same time, so that missed detection and false detection caused by temperature change can be avoided.
(3) The network model is constructed and the network model is constructed,
Firstly, constructing a denoising network, taking a non-target data sample as a training sample, inputting the training sample into the denoising network, training, outputting residual errors, and continuing training iteration until the residual errors are equal to zero, thereby obtaining the denoising network with the residual errors equal to zero (namely, a lightweight denoising network).
In this embodiment, the 2DFFT results of the second distance sampling units (the second distance sampling units are located in the distance dimension) of the radar near-distance region of each receiving channel are taken as input, the two-dimensional images are obtained by arranging the doppler and the channel dimension, the U-net is selected as the denoising network, pruning is performed, and finally the lightweight denoising network is obtained, as shown in fig. 4.
And then constructing a target detection network, namely inputting a target data sample as a training sample into a lightweight denoising network, training, outputting residual errors, performing target detection on residual error results, judging whether targets exist or not to obtain target detection positions and target categories, and continuing training iteration until the target detection positions and the target categories thereof are in one-to-one correspondence with the set target positions and the target categories thereof, thereby obtaining the target detection network and the classification network.
After the denoising network training is completed, a target detection network is constructed. The real target echo can be obtained after the denoising network, a plurality of frames are accumulated to serve as the input of the target detection network, the target detection network is trained, the output result is the target category, and the label obtained by classifying according to the characteristics corresponds to the label. In this embodiment, a range-doppler plot (RDMAP) obtained by accumulating 3 frames is taken as an input, a convolutional neural network is selected as a feature extraction network, and a fully connected layer is taken as a classifier, so as to obtain a fully connected neural network (i.e., a classification network), as shown in fig. 5.
(4) The target real-time data is de-noised, detected and identified,
When the radar works normally, 2DFFT results of the distances of all receiving channels of the radar under the normal real-time working condition of the radar are collected, and at the moment, the radar has data of a radar near-distance area (the radar near-distance area refers to an area which is 0-5 meters away from the radar) and data of a radar far-distance area (the radar far-distance area refers to an area which is 5 meters away from the radar). The data of the radar remote area can be used for the radar function of normal running of the automobile, and the data of the radar near area can be used for the radar function of parking of the automobile.
And when the radar movement speed is higher than 2m/s, the radar near-range target detection module is not started to detect the near-range target.
When the radar motion speed is lower than 2m/s, a radar short-range target detection module is started to detect short-range targets, the short-range target detection comprises the steps of taking out data of a plurality of distance dimension sampling units of a radar short-range area of each receiving channel, simultaneously recording and taking out real-time temperature of a radar radio frequency chip, forming target real-time data by the taken out data, inputting the target real-time data into a target detection network, obtaining target data of each channel after denoising the target real-time data, summing the target data of each channel to obtain a distance Doppler map (RDMAP), carrying out target detection on the distance Doppler map, carrying out distance interpolation and speed measurement to obtain current frame target information (namely carrying out conventional signal processing on RDMAP, namely cfar (target detection), speed measurement and angle measurement (doa) and distance interpolation to obtain target position information), accumulating multi-frame distance Doppler maps (RDMAP), and taking out continuous 3 frames of distance Doppler maps, inputting the target real-time data into a classification network to obtain target category, and outputting the target category and the latest target information stored at the moment, so that a system decision is facilitated.
If no target is detected in the continuous 3-frame range Doppler diagram in the radar near target detection range, indicating that no target exists in the radar observation range during parking, the target parameter information and the range Doppler diagram of the corresponding frame are cleared.
In this embodiment, the 2DFFT result of the second distance sampling unit (the second distance sampling unit is in the distance dimension) of the radar near-distance region of each receiving channel is taken as input, and is shown in fig. 6, which is an original RDMAP schematic diagram when the second distance sampling unit is not targeted, is shown in fig. 7, which is a post-denoising RDMAP schematic diagram when the second distance sampling unit is not targeted, is shown in fig. 8, which is an original RDMAP schematic diagram when the second distance sampling unit is targeted, wherein the target is a pedestrian, is shown in fig. 9, which is a post-denoising RDMAP schematic diagram when the second distance sampling unit is targeted, wherein the target is a pedestrian, and is shown in fig. 10, which is a post-denoising cfar schematic diagram when the second distance sampling unit is targeted.
In summary, the automatic parking obstacle sensing method based on the millimeter wave radar provided by the invention adopts the denoising network to remove the energy of the direct current signal, so that the detection performance of a weak target is improved, the ranging precision of a near-range target is improved, meanwhile, the working temperature of a radar radio frequency chip is detected in the process of manufacturing a data set and processing in real time and is used as one of network inputs, the robustness of the denoising network at different temperatures is improved, and noise residues caused by the temperature are avoided. And (5) carrying out target detection, speed measurement, angle measurement and distance interpolation after denoising to obtain target parameter information. According to the method, the multi-frame energy distribution map is obtained through accumulation, and the target classification is completed through a classification network, unlike the general radar target classification, the Doppler frequency-time dimension energy distribution map (namely the range Doppler map) is directly adopted as network input to perform target classification, the input contains more target characteristic information, and the millimeter wave radar near-range target detection performance and the ranging accuracy are remarkably improved. Compared with an ultrasonic radar, the method can output a high-precision distance measurement result and can output the azimuth, the altitude and the speed information of the target. In the invention, only near targets are classified, only partial Doppler results of the distance dimension sampling units are selected as network input, and multi-frame accumulation is adopted, so that the calculated amount and the memory consumption are greatly reduced.
The above description is only of the preferred embodiment of the present invention, and is not intended to limit the present invention in any way. Any person skilled in the art can make many possible variations and modifications to the technical solution of the present invention or modifications to equivalent embodiments using the methods and technical contents disclosed above, without departing from the scope of the technical solution of the present invention. Therefore, all equivalent changes according to the shape, structure and principle of the present invention are covered in the protection scope of the present invention.