Background
In the future, the industrial scale of smart cities will be continuously expanded, and smart transportation and public safety will be vigorous as key fields of smart cities. Some traffic illegal behaviors which are not easy to be supervised, such as frequent lane change behaviors, seriously threaten public safety, and prove that a certain vulnerability exists in the existing traffic supervision system. Traditional "electronic police" camera position is fixed, and quantity is limited, and the coverage is less, and the traffic violation action that the driver evaded the camera often can appear, leads to the traffic violation action frequent in the supervision blind area.
At present, the supervision and the report of the traffic violation behaviors are mostly captured by means of a fixed camera arranged at a crossroad, or a reporter who witnesses the traffic violation behaviors shoots with a mobile phone and uploads the shot to a report system of a traffic management network. Such supervision and reporting are not only inefficient, small in coverage, but also lack sufficient evidence, lacking deterrence to illegal vehicles. In special road sections such as an expressway which lack fixed cameras, traffic illegal behaviors such as no safety belts are tied, too short distance between vehicles and the like occur frequently, huge potential safety hazards are caused, and a more healthy traffic illegal behavior supervision system is urgently needed for supervision.
CN110136447A discloses a method for detecting lane change during driving and identifying illegal lane change, which includes: (1) acquiring a video image shot in a depression manner of a road section through a monitoring camera of the road section to be monitored; (2) marking and storing all lane line information needing to be detected in the road section; (3) and identifying and positioning all vehicles in the real-time video, marking and capturing images of the vehicles judged to have the lane change with the solid lines for evidence collection, and recording the license plate information of the vehicles. The method has obvious defects, still needs a monitoring camera at a fixed position, has larger dead angle and smaller coverage range.
CN111047875A discloses a system and method for rapidly reporting traffic violation behaviors during driving, the system includes the following steps: (1) before driving, the mobile phone is fixed in the vehicle, so that the position in front of the driving can be shot and real-time shooting can be carried out; (2) storing video information of 10 seconds before and after reporting in real time as reporting evidence; (3) when the driver finds the traffic violation in front, the reporting function is started through sliding the screen or voice control, and the system judges whether the traffic violation belongs to the traffic violation or not through video information and uploads the traffic violation to the traffic police reporting platform. However, this system still requires the driver to manually determine whether there is a traffic violation in front, which has a certain hysteresis and a traffic hazard.
Disclosure of Invention
The invention aims to provide a decentralized traffic integrity evaluation system, which realizes the functions of automatically identifying and reporting front and rear vehicle traffic violations in the vehicle driving process and evaluating the traffic integrity of drivers with traffic violations.
The technical solution for realizing the purpose of the invention is as follows: a decentralized traffic integrity evaluation system comprises a vehicle-mounted recording module, a server judgment module, an APP integrity query module and a traffic management network.
The vehicle-mounted recording module comprises: the device comprises a power supply module, a main control module, a video capturing module, a video processing module, a 4G communication module and a GPS positioning module; the power supply module, the video capture module, the video processing module, the 4G communication module and the GPS positioning module are respectively connected with the main control module through circuits; the power supply module is connected with an automobile power supply circuit, and the automobile power supply supplies voltage to the power supply module; the GPS positioning module reads the positioning of the vehicle in real time, and the position of the vehicle is divided into two conditions of an expressway and a common road according to the positioning; the video processing module judges whether the vehicle runs on the highway or not according to the positioning information of the GPS positioning module, then carries out image processing on the videos captured by the video capturing module frame by frame and judges whether traffic illegal behaviors exist in the front vehicle and the rear vehicle or not; the 4G communication module is in communication connection with the server judgment module through a 4G network, and uploads videos with traffic illegal behaviors to the server judgment module.
The server judgment module is in communication connection with a traffic management network, further judges whether traffic illegal behaviors exist in the video by means of a neural network algorithm, and uploads the video with the traffic illegal behaviors and the license plate information of illegal vehicles to the traffic management network; the neural network used by the server judgment module comprises: the system comprises a vehicle identification GoogleNET network, a license plate number identification CNN network, a signal lamp identification Faster RCNN network, a lane line identification SSD network and a safety belt identification DBN network.
The APP integrity query module is in communication connection with the server judgment module and synchronizes the traffic violation behaviors judged by the server judgment module into the APP; the APP integrity query module comprises: the system comprises a registration interface, a login interface, a query interface and an intelligent contract interface; the vehicle owner fills in identity information such as a driving license number, a name, an identification card number, a license plate number and the like through a registration interface to register, and uses the identification card number and a password to log in on the registration interface; the APP integrity query module generates a traffic integrity score for the vehicle owner according to the traffic violation of the vehicle owner, and the vehicle owner can query the traffic violation, the violation video and the traffic integrity score on a query interface after logging in; the APP intelligent contract interface has the following functions: when the car owner applies for renting a shared car, becomes a taxi driver, becomes a designated driver or even inquires personal credit information, a corresponding mechanism has the right to inquire the traffic integrity score of the car owner and make a contract on an intelligent contract interface, and the APP integrity inquiry module assists the mechanism to automatically release the contract with the car owner when the integrity score of the car owner falls to a certain value.
Preferably, the video capturing module comprises: the alarm device comprises a first camera, a second camera, a third camera and an alarm bell; the first camera is arranged on the right side of a left headlamp of the vehicle and faces the front of the vehicle, the second camera is arranged on the left side of a right headlamp of the vehicle and faces the front of the vehicle, the third camera is arranged on the top of the vehicle and faces the back of the vehicle, and the third camera is connected with the alarm bell circuit; the video processing module judges whether the front vehicle has traffic violation behaviors or not according to the image information of the first camera and the second camera, and judges whether the rear vehicle has traffic violation behaviors or not according to the image information acquired by the third camera.
Preferably, the specific steps of the video processing module for processing the video are as follows:
(1) identifying lane lines and judging the false or true of the lane lines
Converting a video into an image frame, cutting the lower half part of the image frame, carrying out graying, noise filtering and edge extraction, selecting a threshold value according to a white lane line and a yellow lane line, respectively carrying out binarization to obtain a black and white color block, and then carrying out Hough transform algorithm and conic curve fitting algorithm to obtain the position and the slope (curvature) of the lane line. And then judging the truth of the lane line by a method of calculating the duty ratio.
(2) Identifying the position of a vehicle and a license plate
Graying, noise filtering and edge extracting are carried out on the image frames, and for the front vehicle, the position of the vehicle is positioned according to the contour characteristics of the vehicle body; for the rear vehicle, the vehicle position is located according to the contour features of the windshield. And then selecting a threshold value according to the blue license plate and the yellow license plate to carry out binarization, eliminating an interference color block according to the similarity of the outline of the license plate and a rectangle, positioning the position of the license plate, and if the license plate is not detected, proving that the vehicle is not provided with the license plate or is covered by the license plate.
(3) Judging whether the front vehicle carries out the illegal lane change of the solid line
Whether the front vehicle changes lanes is judged according to the relative position relation of the front vehicle and the lane line, and whether the line is pressed is judged according to the intersection relation of the contour edge of the front vehicle body and the lane line. And judging whether the vehicle carries out illegal lane change by a solid line according to the virtual and real lane lines.
(4) Judging whether the front vehicle runs the red light
When the GPS positioning module judges that the vehicle runs on a common road, cutting the upper half part of an image frame, namely the area where a traffic signal lamp is located, carrying out graying, noise filtering and edge extraction, respectively carrying out binarization by adopting three threshold values according to the color of the signal lamp, eliminating an interference color block according to the similarity of the contour of the signal lamp and the circle, determining the position and the color of the signal lamp, and judging whether the front vehicle has red light running traffic violation or not according to the relative position change of the front vehicle and the color of the signal lamp; and if the signal lamp is not detected, the judgment of running the red light is not carried out.
(5) Judging whether the rear vehicle is too close to the vehicle distance
When the GPS positioning module judges that the vehicle runs on the expressway, firstly, the outline characteristics of the windshield of the rear vehicle are identified, the distance between the rear vehicle and the vehicle is judged according to the change of the relative position of the rear vehicle and the vehicle, after the distance is continuously smaller than the safe distance for a period of time, the traffic violation behavior that the vehicle on the expressway is too close to the rear vehicle is judged, and the main control module controls the alarm bell on the roof to give a long-whistle alarm.
(6) Judging whether a driver of a rear vehicle and a copilot wear safety belts or not
The phenomenon that a highway driver and a copilot do not wear safety belts is still common, and great potential safety hazards are generated; when the GPS positioning module judges that the vehicle is near a toll station of the expressway (the distance is short at the moment, and the video collected by the roof camera is relatively clear), the video processing module starts to judge whether a driver and a copilot of the rear vehicle wear safety belts or not. Firstly, according to the size and the position of the outline of the windshield of the rear vehicle, a driver picture frame and a copilot picture frame are captured from an image frame, graying, noise filtering and edge extraction are respectively carried out, the position of a human face is judged according to the outline characteristics of the human face, the outline of the shoulders is extracted by taking the position of the human face as a reference, and whether a safety belt is worn or not is judged by comparing the outline lengths of the shoulders on the left side and the right side.
Compared with the prior art, the invention has the beneficial effects that: the video capturing module is arranged on a running vehicle, so that the defect that the traditional electronic police can only be fixed at a certain position is overcome, the supervision blind area is greatly reduced, and a stronger deterrent effect on traffic offenders can be generated; the system automatically carries out video capture and judgment of traffic violation behaviors without the participation of a driver in the reporting process, so that the potential safety hazard of driving is reduced; the system is arranged at a fixed position of the vehicle body, so that the adjustment and the maintenance are convenient; the server judgment module carries out second audit on the reported video through a neural network algorithm, so that the accuracy of reported information is greatly improved, and the traffic violation handling pressure of a traffic management network is reduced; the APP integrity query module can query whether a driver has a traffic violation behavior in real time, query the traffic integrity score of the driver, and increase supervision and management on the driver.
Detailed Description
The present invention is further illustrated by the following examples, which are intended to be purely exemplary and are not intended to limit the scope of the invention, as various equivalent modifications of the invention will occur to those skilled in the art upon reading the present disclosure and fall within the scope of the appended claims.
As shown in fig. 1, the decentralized traffic integrity evaluation system disclosed by the present invention includes a vehicle-mountedrecording module 1, aserver determination module 2, an APP integrity query module 3, and atraffic management network 4. When a private car is bought, registered or maintained, or when public vehicles such as a taxi and a bus work, the traffic police installs the vehicle-mountedrecording module 1 on the car, and the car provided with the module can be supplemented properly by a traffic police department and is maintained and upgraded regularly; theserver judgment module 2 is in communication connection with a database of a traffic management network, further judges whether traffic illegal behaviors exist in videos by means of a neural network algorithm, reduces false reports, uploads the videos with the traffic illegal behaviors and license plate information of illegal vehicles to the traffic management network, and finishes a reporting process; APP integrity inquiry module 3 be connected withserver judgment module 2 communication, in synchronizing the traffic violation action thatserver judgment module 2 judged to APP, the car owner accomplishes APP identity registration through information such as the driving license that provides oneself, license plate number, the module generates traffic integrity score for the car owner according to how much of car owner's traffic violation action, the car owner can inquire traffic violation information and the video of violating the law of oneself at APP.
As shown in fig. 2, the vehicle-mountedrecording module 1 includes: the system comprises apower supply module 11, a main control module 12, avideo capture module 13, avideo processing module 14, a4G communication module 15 and aGPS positioning module 16; thepower supply module 11, thevideo capture module 13, thevideo processing module 14, the4G communication module 15 and theGPS positioning module 16 are respectively connected with the main control module 12 through circuits; thepower module 11 is connected with an automobile power circuit, and the automobile power supplies provide voltage for thepower module 11; theGPS positioning module 16 reads the positioning of the vehicle in real time, and divides the position of the vehicle into two conditions of an expressway and a common road according to the positioning; thevideo processing module 14 judges whether the vehicle is running on the expressway according to the positioning information of theGPS positioning module 16, and then performs image processing on the video captured by the video capturingmodule 13 frame by frame to judge whether the vehicle has traffic violations before and after; the4G communication module 15 is in communication connection with theserver judgment module 2 through a 4G network, and uploads videos with traffic illegal behaviors to theserver judgment module 2.
As shown in fig. 2, thevideo capturing module 13 includes: afirst camera 131, asecond camera 132, athird camera 133 and analarm bell 134; thefirst camera 131 is arranged on the right side of the left headlamp of the vehicle and faces the front of the vehicle, thesecond camera 132 is arranged on the left side of the right headlamp of the vehicle and faces the front of the vehicle, thethird camera 133 is arranged on the roof and faces the back of the vehicle, and thethird camera 133 is in circuit connection with thealarm bell 134; thevideo processing module 14 determines whether the front vehicle has a traffic violation or not according to the image information of thefirst camera 131 and thesecond camera 132, and determines whether the rear vehicle has a traffic violation or not according to the image information acquired by thethird camera 133.
The specific steps of thevideo processing module 14 for processing the video are as follows:
(1) identifying lane lines and judging the false or true of the lane lines
The lane lines are divided into a straight lane line and a curve lane line. The identification of the straight line is realized based on Hough transform. The basic principle is that a straight line in an original image space is converted into a point in a Hough parameter space by utilizing the duality of the point and the line, and the problem of detecting the straight line in the original image is converted into the problem of searching a peak value in the parameter space. Therefore, the video is firstly converted into an image frame, the lower half part (the part containing the lane line) of the image frame is cut, graying, noise filtering and edge extraction are respectively carried out, threshold values are respectively selected according to a white lane line and a yellow lane line for binarization, a plurality of black and white straight line segments are obtained, the intercept and the slope of each straight line segment are obtained through a Hough transform algorithm, and the position of the lane line is screened according to the slope characteristics of the lane line.
The identification of the curve line is realized based on a conic curve fitting algorithm. Because the curve can be seen as an elliptical shape locally, a conic curve can be used for fitting, and the steps are as follows: firstly, converting a video into an image frame, carrying out gray level, image filtering and edge extraction on an effective area respectively, selecting a threshold value according to a white lane line and a yellow lane line to carry out binarization respectively to obtain black and white color blocks containing curve sections, using points of each color block area to solve parameters of a conic curve respectively, then substituting points of other color blocks into an equation of the conic curve, and enabling points with established equations to be regarded as points on a curve line within a certain error, wherein the conic curve with the largest point set corresponds to the position of the curve line.
After the position of the lane line is determined, the false and true of the lane line are judged by a method of calculating the black-white duty ratio of the lane line after binarization, the lane line with the duty ratio close to 1 is a solid line, and the lane line with the duty ratio close to 0.5 is a dotted line.
(2) Identifying the position of a vehicle and a license plate
Graying, noise filtering and edge extracting are carried out on the image frames, and for the front vehicle, the position of the vehicle is positioned according to the contour characteristics of the vehicle body; for the rear vehicle, the vehicle position is located according to the contour features of the windshield. And then selecting a threshold value according to the blue license plate and the yellow license plate to carry out binarization, eliminating an interference color block according to the similarity of the outline of the license plate and a rectangle, positioning the position of the license plate, and if the license plate is not detected, proving that the vehicle is not provided with the license plate or is covered by the license plate.
(3) Judging whether the front vehicle carries out the illegal lane change of the solid line
Whether the front vehicle changes lanes is judged according to the relative position relation of the front vehicle and the lane line, and whether the line is pressed is judged according to the intersection relation of the contour edge of the front vehicle body and the lane line. And judging whether the vehicle carries out illegal lane change by a solid line according to the virtual and real lane lines.
(4) Judging whether the front vehicle runs the red light
When theGPS positioning module 16 judges that the vehicle runs on a common road, the upper half part of the image frame, namely the area where the traffic signal lamp is located, is cut, graying, noise filtering and edge extraction are carried out, binarization is respectively carried out by adopting three threshold values according to the colors (red, yellow and green) of the signal lamp, an interference color block is eliminated according to the similarity of the outline of the signal lamp and the circle, whether the signal lamp exists is judged, and the position and the color of the signal lamp are determined. Judging whether the front vehicle has a red light running traffic violation or not according to the relative position change of the front vehicle and the color of the signal lamp; and if the signal lamp is not detected, the judgment of running the red light is not carried out.
(5) Judging whether the rear vehicle is too close to the vehicle distance
When the motor vehicle runs on the expressway, the motor vehicle should keep a distance of more than 50 meters with the vehicle on the same lane. When theGPS positioning module 16 judges that the vehicle runs on the expressway, firstly, the outline characteristics of the windshield of the rear vehicle are identified, the distance between the rear vehicle and the vehicle is judged according to the change of the relative position of the rear vehicle and the vehicle, after the distance is continuously less than 50 meters and reaches 20 seconds, the traffic violation behavior that the distance between the rear vehicle and the vehicle on the expressway is too short is judged, and the main control module controls the alarm bell on the roof to give a long-whistle alarm.
(6) Judging whether a driver of a rear vehicle and a copilot wear safety belts or not
The phenomenon that a highway driver and a copilot do not wear safety belts is still common, and great potential safety hazards are generated. When theGPS positioning module 16 judges that the vehicle is near a toll station of the expressway (the distance is short at this time, and the video collected by the roof camera is relatively clear), the video processing module starts to judge whether a driver and a copilot of the rear vehicle wear safety belts or not. According to the size and the position of the rear vehicle windshield contour, a driver picture frame and a copilot picture frame are captured from an image frame, graying, noise filtering and edge extraction are respectively carried out, the position of a human face is judged according to the human face contour characteristic, the shoulder contour is extracted by taking the position of the human face as a reference, and whether a safety belt is worn or not is judged by comparing the lengths of the left shoulder contour and the right shoulder contour.
As shown in fig. 3, theserver determining module 2 identifies the traffic violation in the video by using a neural network algorithm, where the neural network used includes: the system comprises a vehicle identification GoogleNET network (21), a license plate number identification CNN network (22), a signal lamp identification RCNN network (23), a lane line identification SSD network (24) and a safety belt identification DBN network (25). The training period of the neural network is longer, so that the module adopts a neural network model which is trained in advance, and online updating is not carried out.
The training of identifying the vehicle GoogleNET network (21) is based on an N-CARS data set, pictures of the front side and the back side of the vehicle in the data set are screened as a training set, the picture of the side surface of the vehicle is removed, a GoogleNET network model in a CNN neural network is selected for training, the angle of a camera is used, the picture of the vehicle at the roadside is collected as a test set, and the network is evaluated.
The training of the license plate number identifying CNN network (22) is based on a CCPD data set, and the CNN neural network is selected because the CNN neural network can integrally process the whole image frame and avoid the segmentation of license plate numbers. The training process cuts the license plate photo of the data set, separates out single numbers and puts the numbers in different categories. The CNN neural network was trained using 34 different classes of data sets (26 capital English letters with "I", "O" and numbers 0-10 removed) and evaluated using the test set
The training of the signal lamp fast RCNN network (23) is based on a data set of 'Traffic Lights Registration (TLR) public keys', and the fast RCNN neural network in the CNN neural network is selected for training in consideration of small targets of the signal lamp in picture frames, so that the neural network has advantages in the aspect of detecting small targets. And collecting pictures of signal lamps in different weathers to form a test set, and evaluating the network.
The training of the identification lane line SSD network (24) is based on a data set consisting of images of lane lines acquired from various environments on the network and the surrounding, the SSD neural network in the CNN neural network is used for identifying a straight lane line and a curved lane line in consideration of the advantage of the processing of linear characteristics, a smaller detection frame is selected as much as possible according to the characteristics of the lane lines, the straight lane line and the curved lane line are identified and marked in blocks, and the straight curve and the virtual reality of the lane lines can be judged according to the slope and the density of the blocks.
The training of the safety belt DBN (digital video network) (25) is recognized based on a data set consisting of pictures collected on the network, and in order to remove invalid areas, picture sets of parts below the necks of a driver and a copilot are selected as training sets and are divided into two types of wearing safety belts and not wearing safety belts; firstly, mapping the feature vectors of a training set picture to different feature spaces by adopting an RBM (radial basis function) network, realizing layer-by-layer learning of features, and reserving feature information of the picture as much as possible; then, the BP network is used as the last layer of the DBN network, the characteristic vector output by the RBM network is received, and the whole DBN network is finely adjusted in a supervision mode; when the network is tested, firstly, haar feature detection algorithm is carried out on a driver picture frame and a copilot picture frame in a picture to detect human face features, the picture frame below the neck is selected as a test object to be input into the network, and the network is evaluated.
The operation process of theserver judgment module 2 is as follows: when theserver judging module 2 receives the illegal video information sent by the vehicle-mountedrecording module 1, the image frames of the videos are respectively imported into each network for recognition, whether traffic illegal behaviors exist is judged, the video information with the traffic illegal behaviors and the recognized license plate information are uploaded to thetraffic management network 4, and the reporting process is completed.
As shown in fig. 4, the APP integrity query module 3 includes aregistration interface 31, alogin interface 32, aquery interface 33, and anintelligent contract interface 34. The vehicle owner fills in identity information such as a driving license number, a name, an identification card number, a license plate number and the like through theregistration interface 31 to register, the vehicle owner logs in through theregistration interface 32 by using the identification card number and a password, after logging in, the vehicle owner can inquire own traffic violation behaviors and related violation video certificates through theinquiry interface 33, and when an error report condition occurs, the vehicle owner can apply complaints through theinquiry interface 33, and a manual background is used for screening and canceling the erroneously judged traffic violation behaviors. The APP integrity query module 3 generates a traffic integrity score according to the traffic violation of the vehicle owner, and can check the traffic integrity score on thequery interface 33. The APPintelligent contract interface 34 has the following functions: when the car owner applies for renting a shared car, becomes a taxi driver, becomes a designated driver or even inquires personal credit information, a corresponding mechanism has the right to inquire the traffic integrity score of the car owner and make a contract on theintelligent contract interface 34, and the APP integrity inquiry module 3 assists the mechanism to automatically release the contract with the car owner when the integrity score of the car owner is reduced to a certain value.
In conclusion, the invention discloses a decentralized traffic integrity evaluation system, which mainly explains the composition and the operation process of the system. Firstly, when a private car is bought, registered or maintained, or when public vehicles such as a taxi and a bus work, the traffic police installs the vehicle-mounted recording module on the car. In the driving process, after the video processing module carries out image processing on the videos captured by the video capturing module frame by frame, whether the front vehicle has traffic illegal behaviors such as solid line illegal lane changing, red light running, no license plate or shielding license plate and the like is judged according to image information acquired by two cameras in the front of the vehicle; according to the image information behind the vehicle collected by the roof camera, whether the rear vehicle has traffic illegal behaviors such as drivers or copilot who do not wear safety belts, too short distance, no license plates or shielding plates and the like is judged, videos containing the illegal behaviors are uploaded to the server judgment module through the 4G communication module, then the server judgment module further judges whether the traffic illegal behaviors exist in the videos by means of the neural network, corresponding vehicle owner information is found according to license plate numbers, dishonest information is displayed to vehicle owners through the APP information query module, meanwhile, the video information with the traffic illegal behaviors and the license plate information are uploaded to a traffic management network, the reporting process is completed, the video information is used as stimulus for monitoring the traffic illegal behaviors, and vehicle owners who successfully complete the reporting can obtain the reporting reward of the traffic management bureau. Compared with the existing system, the system is installed on the traveling vehicles such as private cars, so that the monitoring blind areas are greatly reduced, a stronger deterrent effect on traffic offenders can be generated, meanwhile, the system automatically carries out video capture and judgment on traffic offences, a driver does not need to participate in the reporting process, and the potential safety hazards of traveling vehicles are reduced. The system is installed in the fixed position of the vehicle body, adjustment and maintenance are convenient to conduct, the server judgment module conducts second audit on the reported video through the neural network algorithm, accuracy of reported information is greatly improved, and traffic violation behavior processing pressure of a traffic management network is relieved. The APP integrity query module can query whether a driver has a traffic violation behavior in real time, query the traffic integrity score of the driver, and increase supervision and management on the driver. The invention ensures that the supervision of the traffic illegal action is not executed by specific people (traffic police) or equipment (electronic police), realizes the decentralization of traffic supervision, and increases the scope and the restriction force of traffic law enforcement.