Disclosure of Invention
The invention aims to provide a method for predicting traffic jams caused by abnormal driving of a large vehicle, so as to solve the problems that the traditional method for predicting the traffic jams is insufficient in real time, the abnormal driving behaviors of the large vehicle are difficult to judge and the prediction accuracy is insufficient.
The traffic jam prediction method for predicting traffic jam caused by abnormal driving of a large vehicle comprises the following steps:
Step one, acquiring road traffic information in real time by using a camera and radar equipment, wherein the road traffic information comprises picture information and point cloud information;
processing the picture information and the point cloud information to obtain target information of each target, fusing the target information through the characteristic levels of the camera and the radar equipment, tracking and identifying the targets, wherein the target information comprises information of vehicle positions, types, vehicle speeds, course angles and lane levels, and flow and illegal driving information, and the illegal driving comprises illegal lane changing, illegal stopping and reverse driving;
Thirdly, based on the road traffic information in the first step, carrying out front prediction to detect abnormal driving behaviors of the large vehicle, wherein the abnormal driving behaviors of the large vehicle comprise that the whole speed of the expressway is limited due to illegal driving of the large vehicle into the expressway and congestion is caused by anchoring of the large vehicle on a main road;
And step four, when abnormal behaviors of the large-sized vehicle are detected, inputting traffic information characteristics into a forward prediction model subjected to deep learning training for calculation, and obtaining a traffic jam prediction result.
The beneficial effect of this scheme is:
The method has the advantages that the disadvantages of the camera and the radar can be complemented, the accuracy of data acquisition and late self-learning are guaranteed, the two types of data are fused in characteristic level, comprehensive traffic information can be obtained, automatic forward prediction of traffic conditions is carried out, abnormal driving conditions of large vehicles can be found and responded early, effective data support is provided for early prediction of traffic jam conditions, and traffic problems are effectively reduced for early provision after congestion occurs.
Further, in the second step, identifying pixel coordinate information and a target category of a target in the image information through a YOLOv model in computer vision, wherein the target category comprises a non-motor vehicle, a motorcycle, a car, a bus and a truck;
Preprocessing point cloud information, registering, integrating a plurality of position or time point cloud data, extracting features to obtain edge and plane information in the point cloud, dividing and identifying objects, dividing the point cloud into different objects, and classifying target categories.
The method has the advantages that through data processing, useful information can be extracted from the laser radar point cloud, key data support is provided for space perception and environment understanding of roads, and accurate data support is provided for subsequent data fusion.
Further, in the second step, tracking the targets through Deepsort models, and identifying the motion trail of each target;
Mapping the identified track onto a world coordinate system through Homography homography matrix to display the real motion track of the target on the map;
and comparing the positions of the targets of the front frame and the rear frame through Deepsort models to obtain the target information of each target lane level.
The method has the advantages that the real-time tracking and the corresponding information identification of the target can find the driving state of the target in time, and an accurate data basis is provided for the subsequent congestion prediction.
In the third step, when no abnormal driving behavior of the large-sized vehicle is detected, the abnormal traffic information is automatically analyzed by a reverse prediction model, and the abnormal traffic information comprises short-time and large-scale fluctuation of the traffic flow, large-scale reduction of the vehicle speed and large-scale surge of illegal lane change, and traffic jam is predicted.
The method has the advantages that through carrying out additional traffic jam prediction on the condition that the abnormal driving behavior of the large vehicle is not detected, potential traffic problems can still be detected under the condition that the abnormal driving of the large vehicle is not clearly detected, and therefore smoothness and safety of a road are more comprehensively maintained.
In the fourth step, when the abnormal driving behavior of the large vehicle is not detected, the traffic information features are input into a reverse prediction model trained by deep learning for calculation, and a traffic jam prediction result is obtained.
The traffic jam prediction method has the beneficial effects that the model is used for automatically predicting the traffic jam, and the prediction result is more intelligent and reliable.
In the fourth step, the forward prediction model and the reverse prediction model are obtained through training of the LSTM model by utilizing the history data updated in real time, and the traffic jam prediction result comprises time of occurrence of the jam, jam duration and jam grade.
Further, in the fourth step, the LSTM model includes an input layer, an LSTM layer, and an output layer, where the input layer includes an abnormal driving type of the large vehicle, a traffic flow of each lane, and an average vehicle speed of each lane, the LSTM layer is configured to process time series data, capture a dependency relationship and a change trend in time, and the output layer is configured to output a prediction of a congestion condition, including a congestion occurrence time, a congestion duration, and a congestion level.
Further, in the fourth step, the running process of the LSTM model is as follows:
The degree of influence of new information of the current time step on the cell state is controlled through an input gate, and the output of the input gate is expressed as:
;
Wherein, sigma is a Sigmoid activation function,AndFor the weight matrix and bias term of the input gate,In the hidden state of the last time step,Input of the current time step;
determination of the cell State of the previous time step by means of the forgetting doorThe forget gate is expressed as:
;
Wherein, theAndRespectively a weight matrix and a bias term of the forgetting gate;
updating the cell state according to the output of the input gate and the forget gate, and calculating candidate cell states, wherein the candidate cell states are expressed as follows:
;
the cell state is updated with weighting using the outputs of the forget gate and the input gate, expressed as:
;
controlling hidden state of current time step using output gateThe output of the output gate is:
;
Wherein, theAndThe weight matrix and bias term for the output gate.
Detailed Description
Further details are provided below with reference to the specific embodiments.
Example 1
A method of predicting traffic congestion caused by abnormal driving of a large vehicle, as shown in fig. 1, comprising the steps of:
The first step is to collect road traffic information in real time by using a camera and radar equipment, wherein the road traffic information comprises picture information and point cloud information, and the camera and the radar equipment are all commonly used equipment for collecting the existing traffic data, and are not described in detail herein.
And secondly, processing the picture information and the point cloud information to obtain target information of each target. The processing of the image information comprises the steps of identifying pixel coordinate information and target categories of targets in the image information through a YOLOv model in computer vision, wherein the target categories comprise categories of non-motor vehicles, motorcycles, sedans, buses and trucks, and the motorcycles further comprise categories of bicycles.
And then the following formula is adopted for fusion through the feature level fusion of the camera and the radar equipment:
;
Wherein, theIs a feature of the fusion after the fusion,AndFeatures from the camera and the radar respectively,Is a weight parameter, and takes a value between 0 and 1.
And tracking and identifying a target, wherein the target information comprises information of vehicle position, type, vehicle speed, course angle and lane level, and the target information also comprises traffic and illegal driving information, and the illegal driving comprises illegal lane changing, illegal stopping and reverse driving.
The target information is obtained by tracking the targets through Deepsort models and identifying the motion trail of each target, and DeepSort (DEEP LEARNING-based Object Tracking) models are target tracking models based on deep learning and are mainly used for accurately detecting and tracking the targets in video sequences. The DeepSort model can realize efficient and accurate detection and tracking of the target in the video sequence by combining deep learning feature extraction and Kalman filtering with target tracking of the Hungary algorithm. The key ideas of Kalman filtering are data fusion and iterative updating. By combining prior information with actual measurements, the system can better estimate the current state and gradually increase the accuracy of the estimation by iterative updating. The core assumption is that the dynamic system is linear, expressed as:
;
;
The target state estimation is carried out by learning the appearance characteristics of the target through deep learning and combining Kalman filtering and Hungary algorithm, so that the target state estimation method is suitable for a multi-target tracking scene.
The identified trajectories are mapped to a world coordinate system by Homography homography matrix to display the real motion trajectories of the targets on the map. Homography is a 3x3 transformation matrix mapping points on an image to corresponding points on a world coordinate system, and the principle of the transformation matrix can be written as a 3x3 matrix H, which is expressed as:
;
For a corresponding point in the graph, (x 1, y 1) in the graph and (x 2, y 2) in the world coordinate system, the mapping relation between the two points is established through H and expressed as:
;
For all corresponding points, a one-to-one mapping relation from the image pixel coordinates to the world coordinates is established, so that the targets detected by the camera can be displayed on the map.
And directly carrying out depth analysis on each target to obtain information including the position, the type, the speed and the like of the vehicle, and comparing the positions of the targets of the front frame and the rear frame through a Deepsort model to obtain the target information of each target lane level. These data not only help to monitor traffic flow in real time, but also provide comprehensive insight into the behavior of vehicles on roads. Meanwhile, the system also carefully monitors each element in the traffic flow, captures the key information of the lane level, such as traffic, illegal driving and the like, wherein the definition of the illegal driving covers the irregular behaviors of illegal lane changing, illegal stopping, reverse running and the like. The following are the specific information obtained:
traffic flow, i.e., the total number of vehicles passing through per unit time, is a key indicator for measuring traffic smoothness. In order to provide more detailed traffic information, the present invention subdivides the traffic data and analyzes it onto different lanes. By the method, more accurate flow information can be provided, so that the change trend of the intersection can be more comprehensively known.
And the data collection mode of the traffic flow is that lane-level data collection is realized through each camera. Specifically, a stop line is taken as one side in each lane, a rectangular area is divided forwards, a counter is increased by 1 after a target passes through the area, and the number of passes in unit time is taken as the flow of the lane.
The speed of the vehicle, i.e. the speed of travel of the vehicle, is an important parameter for assessing traffic conditions and road section smoothness. And the vehicle speed data collection mode is to track the moving track of the target through Deepsort tracking models and calculate the real-time speed of the target. Potential congestion conditions are identified in advance by monitoring the change of the vehicle speed in real time.
Vehicle category-a more detailed classification is introduced covering five major classes of non-motor vehicles, bicycles, carts, buses and trucks. Through the classification mode with finer granularity, the diversity of traffic participants can be more comprehensively known, the vision is focused on the truck, and potential traffic jam caused by the truck can be perceived in advance by grasping the related information of the truck in real time. This is critical for the prediction of traffic congestion, since trucks typically have a large volume and different driving characteristics, the driving status of which has a significant impact on the road segment traffic capacity.
Illegal driving includes a series of unnormal behaviors to more fully monitor and evaluate the behavior of traffic participants. Specifically, driving violations are divided into three main aspects, lane changes, traffic violations, and reverse traffic. Illegal lane changes encompass the act of making a lane change in an inappropriate location or situation. Including frequent lane changes in congested road segments, intersections, or prohibited lane change areas, resulting in traffic confusion and hazards. Especially, the traffic is greatly influenced by the illegal lane change of the large-sized vehicle. By tracking the target in real time, the target is marked when the target large vehicle is changed from one lane to another lane. A parking violation refers to a violation of parking in a parking-prohibited area, intersection, or other non-parking-permitted location. Such behavior may impede normal traffic flow, cause congestion and increase accident risk. Through real-time tracking, if the position information of the target is found to be unchanged within 3 minutes, the target is marked to be illegal, and meanwhile, the target is recorded, so that data support is provided for subsequent judgment. Reverse travel refers to a vehicle traveling in a direction other than the prescribed direction, which may occur on a single-way road or other limited direction area. The retrograde behavior not only violates the traffic rules, but also may cause confusion at intersections and traffic flow disruption. By tracking the target, the motion trail moves in the opposite direction, and the target is marked as retrograde. The data obtained above can provide data support for subsequent judgment.
The camera is responsible for capturing visual information, while the radar detects the position and motion state of the vehicle through reflection of radio waves. By combining and fusing the information of the two different sensors, the system can more comprehensively and accurately understand the traffic condition, fuse the image and the data identified by the laser radar, enable the target identification and tracking to be more accurate, and capture real-time traffic information by the fused data. The feature level fusion not only improves the real-time performance of traffic information, but also greatly enhances the perception and understanding capability of the system on complex traffic scenes, ensures the accuracy of target capturing, and lays a solid foundation for subsequent further processing.
Thirdly, based on the road traffic information in the first step, carrying out front prediction to detect abnormal driving behaviors of the large vehicle, wherein the abnormal driving behaviors of the large vehicle comprise that the whole speed of the expressway is limited due to illegal driving of the large vehicle into the expressway and congestion is caused by anchoring of the large vehicle on a main road;
when no abnormal driving behavior of the large-scale vehicle is detected, the abnormal traffic information is automatically analyzed by a reverse prediction model, wherein the abnormal traffic information comprises short-time and large-scale fluctuation of the traffic flow, large-scale reduction of the vehicle speed, large-scale increase of illegal lane change and prediction of traffic jam.
And step four, when abnormal behaviors of the large-sized vehicle are detected, inputting traffic information characteristics into a forward prediction model subjected to deep learning training for calculation, and obtaining a traffic jam prediction result. The forward prediction model is trained to receive traffic information characteristics of detected abnormal behaviors of the large vehicle and automatically calculate the probability of traffic congestion by the LSTM model. The model is input into abnormal driving types of large vehicles, traffic flow of each lane and average speed of each lane, and output is prediction of congestion conditions, wherein the prediction comprises congestion occurrence time, congestion duration and congestion level. Congestion levels include light congestion (primary), medium congestion (secondary), and heavy congestion (tertiary). The first-level characteristic describes that the road section has slight traffic jam, the traffic flow is slightly slowed down, but the whole road section can still keep a normal running speed. The first-order possible reasons are that some traffic events, large or short-time vehicle deceleration at intersections and the like can lead to light congestion. The second-level characteristic description is that the traffic flow on the road section is obviously affected, the speed of the vehicle is reduced, the density of the vehicle is increased, and the running speed is obviously reduced. The possible reasons are that traffic flow is blocked due to the reasons of illegal driving of vehicles, accidents, road engineering and the like, and moderate congestion is formed. Three-level characterization is that traffic on a road segment is completely blocked, the running speed of the vehicle is very slow, or even the vehicle is stopped before the traffic is stopped, and traffic jam is formed. The road section cannot normally pass due to serious accidents, road closure, large-scale vehicle inflow and the like, and severe congestion is formed. The three-level congestion classification system can reflect different degrees of traffic conditions more accurately, and is beneficial to traffic management departments to take measures more pertinently so as to relieve congestion and improve road traffic efficiency. By means of real-time monitoring and support of a forward prediction model, the system can quickly and accurately identify congestion at different levels and provide powerful references for traffic management decisions.
When the abnormal driving behavior of the large vehicle is not detected, inputting the traffic information characteristics into a reverse prediction model trained by deep learning for calculation, and obtaining a traffic jam prediction result. By combining forward and reverse prediction, the sensitivity of the system to abnormal traffic behaviors can be improved, more comprehensive information support is provided for traffic management, and potential traffic problems can be effectively predicted and processed.
As shown in fig. 2, the forward prediction model and the reverse prediction model are obtained through training of the LSTM model by using the historical data updated in real time, and the traffic congestion prediction result includes the time of congestion occurrence, the congestion duration and the congestion level. The LSTM model comprises an input layer, an LSTM layer and an output layer, wherein the input layer comprises abnormal driving types of large vehicles, traffic flows of all lanes and average speeds of all lanes, the LSTM layer is used for processing time sequence data and capturing time dependency and change trend, and the output layer is used for outputting prediction of congestion conditions, including congestion occurrence time, congestion duration and congestion level. The LSTM model operates as follows:
The degree of influence of new information of the current time step on the cell state is controlled through an input gate, and the output of the input gate is expressed as:
;
Wherein, sigma is a Sigmoid activation function,AndFor the weight matrix and bias term of the input gate,In the hidden state of the last time step,Input of the current time step;
determination of the cell State of the previous time step by means of the forgetting doorThe forget gate is expressed as:
;
Wherein, theAndRespectively a weight matrix and a bias term of the forgetting gate;
updating the cell state according to the output of the input gate and the forget gate, and calculating candidate cell states, wherein the candidate cell states are expressed as follows:
;
the cell state is updated with weighting using the outputs of the forget gate and the input gate, expressed as:
;
controlling hidden state of current time step using output gateThe output of the output gate is:
;
Wherein, theAndThe weight matrix and bias term for the output gate.
The forward prediction model and the reverse prediction model utilize the historical data updated in real time to carry out deep learning training so as to adapt to the continuously-changing dynamic traffic environment and driving behavior. This includes acquiring up-to-date traffic information and data of abnormal behavior of the large vehicle, etc. By adopting an efficient optimization algorithm, it is ensured that two models can provide highly accurate traffic congestion predictions in various scenarios. The dynamic training mechanism enables forward and reverse prediction models to flexibly adapt to novel driving behaviors and changes of complex traffic scenes. They can capture the evolution of the traffic system in real time, maintaining sensitivity to traffic congestion problems. Through continuous real-time training and optimization, the two models not only provide reliable prediction capability, but also lay a solid foundation for the reliability and stability of the intelligent traffic system. The method has important practical application value for timely responding to different traffic management challenges and ensuring road traffic efficiency.
Compared with the prior art, the scheme of the embodiment specifically extracts information related to target driving in the acquired pictures and the point cloud information, classifies the information, distinguishes large vehicles, directly monitors abnormal driving of the large vehicles, and carries out forward prediction of traffic jam based on abnormal driving behaviors, when abnormal driving is not monitored, monitors abnormal traffic information, carries out reverse prediction of traffic jam, monitors traffic jam caused by different classification, has smaller data processing quantity and shorter processing time, and can rapidly, timely and real-timely monitor the jam condition in a limited time. The abnormal driving running of the large-sized vehicle is directly monitored, the monitoring result is more accurate on the premise of not using a complex model, and the accuracy of traffic jam result prediction is improved.
Example two
In the third step, when the front prediction detects the abnormal driving behavior of the large vehicle, the target class of the large vehicle in the picture information is identified, when the target class is the large vehicle, such as a truck or a bus, the load information of the target is judged according to the target class, the identification of the target class can be performed through the vehicle outline, the load information comprises underloading, full loading, overload and no load, the load information is judged according to the size that the load object outline of the target is positioned on the original vehicle outline, the bus generally has a rule that the running path cannot be overloaded and the common running path cannot violate, for example, the load object outline is positioned in the vehicle outline, the load information is underloading, the load object outline is positioned outside the vehicle outline, the load information is overloaded, the load information is full loading when the load object outline is equal to the vehicle outline, and the load information is no load when the load object outline is not present. The method comprises the steps of adding an abnormal degree of the abnormal driving behavior of the large vehicle according to load information, wherein the abnormal degree of the abnormal driving behavior of the large vehicle is [0% -30%) when the load information is empty, the abnormal degree of the abnormal driving behavior of the large vehicle is [30% -50%) when the load information is not full, the abnormal degree of the abnormal driving behavior of the large vehicle is [50% -70%) when the load information is full, the abnormal degree of the abnormal driving behavior of the large vehicle is [70% -100%) when the load information is full, and specific abnormal degree is determined according to rated load of the large vehicle.
The method comprises the steps of carrying out average value calculation of abnormal degree when abnormal driving behaviors of the large-scale vehicle are detected through multiple predictions on road sections with preset length, setting the preset length according to actual requirements, for example, 1 km, judging that the large-scale vehicle drives abnormally, continuously tracking the running of the large-scale vehicle when the average value is larger than a threshold value, correcting the abnormal driving behaviors of the large-scale vehicle to be non-large-scale vehicle when the average value is smaller than or equal to the threshold value, and setting the preset duration according to the actual judging requirements, for example, ten minutes when traffic jam does not occur in the preset duration.
Because of a certain difference between specific sizes of large vehicles running on a road and different specific loading conditions of the large vehicles with different sizes in actual road running, even if part of the large vehicles have abnormal driving behaviors, congestion is not caused in some cases, and misjudgment of the congestion is caused. Therefore, in the second embodiment, after the abnormal driving behavior of the large vehicle is identified, by judging the load information and judging the abnormal degree of the abnormal driving behavior of the large vehicle, the corresponding correction can be performed under different conditions of the large vehicle of different target classes, and the accuracy of the subsequent congestion judgment can be improved.
The foregoing is merely exemplary of the present application, and specific structures and features that are well known in the art are not described in detail herein. It should be noted that modifications and improvements can be made by those skilled in the art without departing from the structure of the present application, and these should also be considered as the scope of the present application, which does not affect the effect of the implementation of the present application and the utility of the patent. The protection scope of the present application is subject to the content of the claims, and the description of the specific embodiments and the like in the specification can be used for explaining the content of the claims.