Disclosure of Invention
The invention aims to provide a decentralised traffic integrity evaluation system which realizes the functions of automatically identifying and reporting traffic illegal behaviors of vehicles in front and behind in the running process of the vehicles and evaluating traffic integrity of drivers with traffic illegal behaviors.
The technical solution for realizing the purpose of the invention is as follows: a decentralised traffic integrity evaluation system comprises a vehicle-mounted recording module, a server judging module, an APP integrity inquiring module and a traffic management network.
The vehicle-mounted recording module comprises: the system comprises a power supply module, a main control module, a video capturing module, a video processing module, a 4G communication module and a GPS positioning module; the power module, the video capturing module, the video processing module, the 4G communication module and the GPS positioning module are respectively connected with the main control module through circuits; the power supply module is connected with an automobile power supply circuit, and the automobile power supply provides voltage for the power supply module; the GPS positioning module reads the positioning of the vehicle in real time, and divides the position of the vehicle into two conditions of a highway and a common road according to the positioning; the video processing module judges whether the vehicle runs on the expressway according to the positioning information of the GPS positioning module, then carries out image processing on the video captured by the video capturing module frame by frame, and judges whether traffic illegal behaviors exist on the front and rear vehicles; the 4G communication module is in communication connection with the server judgment module through a 4G network, and uploads the video with traffic illegal behaviors to the server judgment module.
The server judging module is in communication connection with the traffic management network, further judges whether traffic illegal behaviors exist in the video by means of a neural network algorithm, and uploads the video with the traffic illegal behaviors and license plate information of illegal vehicles to the traffic management network; the neural network used by the server judging module comprises: the system comprises a vehicle identification GoogleNET network, a license plate number CNN network, a signal lamp fast RCNN network, a lane line identification SSD network and a safety belt identification DBN network.
The APP integrity query module is in communication connection with the server judgment module, and synchronizes the traffic illegal behaviors judged by the server judgment module into the APP; the APP honest query module comprises: a registration interface, a login interface, a query interface and an intelligent contract interface; the vehicle owner fills in the identity information such as the driving license number, the name, the identity card number, the license plate number and the like through a registration interface to register, and uses the identity card number and the password to register on a registration interface; the APP honest inquiring module generates traffic honest scores for the vehicle owners according to the traffic illegal behaviors of the vehicle owners, and the vehicle owners can inquire the traffic illegal behaviors, the illegal behavior videos and the traffic honest scores of the vehicle owners after logging in the inquiring interface; the APP intelligent contract interface has the following functions: when the owner applies for renting and sharing the automobile, becomes a taxi driver, becomes a pilot driver and even inquires personal credit information, the corresponding mechanism is authorized to inquire the traffic integrity score of the owner and contract the owner on an intelligent contract interface, and the APP integrity inquiry module assists the mechanism to automatically release the contract with the owner when the integrity score of the owner is reduced to a certain value.
Preferably, the video capturing module includes: the first camera, the second camera, the third camera and the alarm bell; the first camera is arranged on the right side of the left headlight of the vehicle and faces the front of the vehicle, the second camera is arranged on the left side of the right headlight of the vehicle and faces the front of the vehicle, the third camera is arranged on the roof and faces the rear of the vehicle, and the third camera is connected with the alarm bell circuit; the video processing module judges whether the front vehicle has traffic illegal behaviors according to the image information of the first camera and the second camera, and judges whether the rear vehicle has traffic illegal behaviors according to the image information acquired by the third camera.
Preferably, the specific steps of the video processing module for processing video are as follows:
(1) Identifying lane lines and judging the deficiency and the excess of the lane lines
Converting the video into an image frame, cutting the lower half part of the image frame, carrying out graying, noise filtering, extracting edges, respectively carrying out binarization according to a white lane line and a yellow lane line selection threshold value to obtain a black and white color block, and then obtaining the position and the slope (curvature) of the lane line through a Hough transform algorithm and a conical curve fitting algorithm. And then judging the deficiency and the excess of the lane line by a duty ratio calculation method.
(2) Identifying the position of a vehicle and a license plate
Graying the image frame, filtering noise, extracting edges, and positioning the vehicle position according to the contour features of the vehicle body for the front vehicle; for a rear vehicle, the vehicle position is located according to the contour features of the windshield. And then binarizing according to a threshold value selected by the blue license plate and the yellow license plate, eliminating an interference color block according to the similarity between the outline of the license plate and the rectangle, positioning the license plate position, and proving that the vehicle has no license plate or is shielded if the license plate is not detected.
(3) Judging whether the front vehicle is subjected to solid line illegal lane change
Judging whether the front vehicle changes lanes or not according to the relative position relation between the front vehicle and the lane lines, and judging whether to press lines or not according to the intersecting relation between the contour edge of the front vehicle body and the lane lines. And judging whether the vehicle performs the solid line illegal lane change according to the deficiency and the excess of the lane line.
(4) Judging whether the front vehicle runs the red light
When the GPS positioning module judges that a vehicle runs on a common highway, the upper half part of an image frame, namely an area where a traffic signal lamp is located, is cut, subjected to grey scale, noise filtering and edge extraction, respectively subjected to binarization according to the color of the signal lamp, eliminating interference color blocks according to the similarity between the outline and the circle of the signal lamp, determining the position and the color of the signal lamp, and judging whether the front vehicle runs red light traffic illegal behaviors according to the relative position change of the front vehicle and the color of the signal lamp; if the signal lamp is not detected, the red light running judgment is not carried out.
(5) Judging whether the distance between the rear vehicles is too close
When the GPS positioning module judges that the vehicle runs on the expressway, firstly, the contour features of the rear windshield are identified, the distance between the rear vehicle and the vehicle is judged according to the change of the relative positions of the rear vehicle and the vehicle, after the distance is continuously smaller than the safety distance for a period of time, the rear vehicle is judged to have traffic illegal behaviors that the distance between the vehicles on the expressway is too close, and the main control module controls the warning bell length and howling warning of the roof.
(6) Judging whether the driver and the copilot of the rear vehicle wear the safety belt
The phenomenon that the expressway driver and the copilot do not wear safety belts is still common, and a large potential safety hazard is generated; when the GPS positioning module judges that the vehicle is near a toll of a highway (the vehicle distance is relatively close, and the video acquired by the roof camera is relatively clear), the video processing module starts to judge whether the rear vehicle driver and the copilot wear the safety belt. Firstly, according to the size and the position of a rear windshield contour, a driver picture frame and a co-driver picture frame are cut in an image frame, gray-scale treatment, noise filtering and edge extraction are respectively carried out, the position of a human face is judged according to the contour characteristics of the human face, the position of the human face is taken as a reference, shoulder contours are extracted, and whether a safety belt is worn or not is judged through comparison of the lengths of the left shoulder contour and the right shoulder contour.
Compared with the prior art, the invention has the beneficial effects that: the video capturing module is arranged on a running vehicle, so that the defect that the traditional electronic police can only be fixed at a certain position is overcome, the supervision blind area is greatly reduced, and a stronger deterrent effect can be generated for traffic offenders; the system automatically performs video capturing and judgment of traffic illegal behaviors, does not need drivers to participate in the reporting process, and reduces the driving safety hidden trouble; the system is arranged at a fixed position of the vehicle body, so that the system is convenient to adjust and maintain; the server judging module carries out second auditing on the reported video through a neural network algorithm, so that the accuracy of reported information is greatly improved, and the traffic illegal action processing pressure of a traffic management network is reduced; the APP integrity query module can query whether the driver has traffic illegal behaviors or not in real time, and query the traffic integrity score of the driver, so that the supervision and management of the driver are increased.
Detailed Description
The present invention is further illustrated below in conjunction with specific embodiments, it being understood that the embodiments are merely illustrative of the present invention and not limiting the scope of the present invention, and that modifications of various equivalent forms of the present invention will fall within the scope of the appended claims to the person skilled in the art after reading the present invention.
As shown in fig. 1, the invention discloses a decentralised traffic integrity evaluation system, which comprises a vehicle-mounted recording module 1, a server judging module 2, an APP integrity query module 3 and a traffic management network 4. When a private car is purchased, hung or maintained, or a taxi works with public vehicles such as buses, the vehicle-mounted recording module 1 is mounted on the car by a traffic police, and the traffic police department can give proper assistance to the car on which the module is mounted and regularly maintain and upgrade the module; the server judging module 2 is in communication connection with a database of the traffic management network, further judges whether traffic illegal behaviors exist in the video by means of a neural network algorithm, reduces false alarm, and uploads the video with the traffic illegal behaviors and license plate information of illegal vehicles to the traffic management network to complete the reporting process; the APP integrity query module 3 is in communication connection with the server judging module 2, the traffic illegal actions judged by the server judging module 2 are synchronized into the APP, an owner of the APP completes the APP identity registration by providing information such as own driving license, license plate number and the like, the module generates traffic integrity scores for the owners according to the traffic illegal actions of the owners, and the owners can query own traffic illegal information and illegal videos at the APP.
As shown in fig. 2, the in-vehicle recording module 1 includes: the system comprises a power supply module 11, a main control module 12, a video capturing module 13, a video processing module 14, a 4G communication module 15 and a GPS positioning module 16; the power module 11, the video capturing module 13, the video processing module 14, the 4G communication module 15 and the GPS positioning module 16 are respectively connected with the main control module 12 in a circuit manner; the power supply module 11 is connected with an automobile power supply circuit, and the automobile power supply supplies voltage to the power supply module 11; the GPS positioning module 16 reads the positioning of the vehicle in real time, and divides the position of the vehicle into two conditions of expressways and common roads according to the positioning; the video processing module 14 judges whether the vehicle runs on the expressway according to the positioning information of the GPS positioning module 16, then carries out image processing on the video captured by the video capturing module 13 frame by frame, and judges whether traffic illegal behaviors exist in the front and rear vehicles; the 4G communication module 15 is in communication connection with the server judgment module 2 through a 4G network, and uploads the video with traffic illegal behaviors to the server judgment module 2.
As shown in fig. 2, the video capturing module 13 includes: the first camera 131, the second camera 132, the third camera 133 and the alarm bell 134; the first camera 131 is arranged on the right side of the left headlight of the vehicle and faces the front of the vehicle, the second camera 132 is arranged on the left side of the right headlight of the vehicle and faces the front of the vehicle, the third camera 133 is arranged on the roof of the vehicle and faces the rear of the vehicle, and the third camera 133 is in circuit connection with the alarm bell 134; the video processing module 14 determines whether the front vehicle has traffic law enforcement according to the image information of the first camera 131 and the second camera 132, and determines whether the rear vehicle has traffic law enforcement according to the image information collected by the third camera 133.
The specific steps of the video processing module 14 for processing video are as follows:
(1) Identifying lane lines and judging the deficiency and the excess of the lane lines
The lane lines are divided into straight lines and curved lines. The identification of the straight line is based on the hough transform. The basic principle is that the point-to-line duality is utilized to convert the straight line of the original image space into a point of the Hough parameter space, and the problem of detecting the straight line in the original image is converted into the problem of searching the peak value in the parameter space. The method comprises the steps of converting video into image frames, cutting the lower half part (part containing lane lines) of the image frames, respectively carrying out graying, noise filtering, edge extraction, respectively carrying out binarization according to a white lane line and a yellow lane line selection threshold value to obtain a plurality of black-white straight line segments, obtaining the intercept and the slope of each straight line segment through a Hough transform algorithm, and screening the position of the lane line according to the slope characteristics of the lane line.
The identification of the curve line is realized based on a conic curve fitting algorithm. Because the curve can be regarded as an elliptical shape locally, a conic curve can be used for fitting, as follows: firstly converting video into image frames, respectively carrying out graying, image filtering and edge extraction on an effective area, respectively carrying out binarization according to a white lane line and a yellow lane line selection threshold value to obtain black-white color blocks containing curve segments, respectively solving parameters of a conic curve by using points of each color block area, substituting points of the remaining color blocks into an equation of the conic curve, and enabling points of which the equation is established within a certain error to be regarded as points on the curved line, wherein the conic curve with the largest point set is the position corresponding to the curved line.
After the positions of the lane lines are determined, the reality of the lane lines is judged by a method of calculating the black-white duty ratio of the lane lines after binarization, the lane lines with duty ratio close to 1 are solid lines, and the lane lines with duty ratio close to 0.5 are broken lines.
(2) Identifying the position of a vehicle and a license plate
Graying the image frame, filtering noise, extracting edges, and positioning the vehicle position according to the contour features of the vehicle body for the front vehicle; for a rear vehicle, the vehicle position is located according to the contour features of the windshield. And then binarizing according to a threshold value selected by the blue license plate and the yellow license plate, eliminating an interference color block according to the similarity between the outline of the license plate and the rectangle, positioning the license plate position, and proving that the vehicle has no license plate or is shielded if the license plate is not detected.
(3) Judging whether the front vehicle is subjected to solid line illegal lane change
Judging whether the front vehicle changes lanes or not according to the relative position relation between the front vehicle and the lane lines, and judging whether to press lines or not according to the intersecting relation between the contour edge of the front vehicle body and the lane lines. And judging whether the vehicle performs the solid line illegal lane change according to the deficiency and the excess of the lane line.
(4) Judging whether the front vehicle runs the red light
When the GPS positioning module 16 determines that the vehicle is traveling on a common road, the upper half of the image frame, that is, the area where the traffic signal is located, is cut, and is subjected to graying, noise filtering, edge extraction, binarization is respectively performed by adopting three thresholds according to the color (red, yellow and green) of the signal, interference color blocks are eliminated according to the similarity between the contour of the signal and the circle, whether the signal is present is determined, and the position and color of the signal are determined. Judging whether the front vehicle runs the traffic law according to the relative position change of the front vehicle and the color of the signal lamp; if the signal lamp is not detected, the red light running judgment is not carried out.
(5) Judging whether the distance between the rear vehicles is too close
The motor vehicle should be kept at a distance of at least 50 meters or more from the same lane vehicle when traveling on the highway. When the GPS positioning module 16 judges that the vehicle runs on the expressway, firstly, the outline features of the rear windshield are identified, the distance between the rear vehicle and the vehicle is judged according to the change of the relative positions of the rear vehicle and the vehicle, after the distance is continuously less than 50 meters and reaches 20 seconds, the traffic illegal action that the vehicle distance on the expressway is too close is judged to occur on the rear vehicle, and the main control module controls the warning bell and the howling warning on the roof.
(6) Judging whether the driver and the copilot of the rear vehicle wear the safety belt
The phenomenon that the expressway driver and the copilot do not wear safety belts is still common, and a large potential safety hazard is generated. When the GPS positioning module 16 determines that the vehicle is near a toll of a highway (where the vehicle distance is relatively short and the video captured by the roof camera is relatively clear), the video processing module begins to determine whether the rear driver and the copilot wear a safety belt. According to the size and the position of the contour of the rear windshield, a driver picture frame and a co-driver picture frame are cut in an image frame, gray-scale treatment, noise filtering and edge extraction are respectively carried out, the position of a human face is judged according to the contour characteristics of the human face, the position of the human face is taken as a reference, shoulder contours are extracted, and whether a safety belt is worn or not is judged through comparison of the lengths of the left shoulder contour and the right shoulder contour.
As shown in fig. 3, the server judging module 2 adopts a neural network algorithm to identify traffic illegal actions in the video, and the used neural network includes: the system comprises a recognition vehicle GoogleNET network (21), a recognition license plate number CNN network (22), a recognition signal lamp Faster RCNN network (23), a recognition lane line SSD network (24) and a recognition safety belt DBN network (25). The training period of the neural network is longer, so that the module adopts a neural network model trained in advance and does not update online.
The training of the recognition vehicle GoogleNET network (21) is based on an N-CARS data set, pictures of the front and the back of the vehicle in the data set are screened as a training set, pictures of the side face of the vehicle are removed, a GoogleNET network model in a CNN neural network is selected for training, the angle of a camera is used, pictures of roadside vehicles are collected as a test set, and the network is evaluated.
The training of the recognition license plate number CNN network (22) is based on the CCPD data set, and the CNN neural network is selected because the CNN neural network can integrally process the whole image frame, and the segmentation of license plate numbers is avoided. The training process firstly cuts license plate photos of the data set, separates out single numbers and places the single numbers in different categories. CNN neural networks were trained using 34 different classes of data sets (26 capital English letters with "I" and "O" removed and numbers 0-10), and evaluated using test sets
The training for identifying the signal lamp Faster RCNN network (23) is based on a data set of 'Traffic Lights Recognition (TLR) public benchmarks', and the training is performed by selecting the Faster RCNN neural network in the CNN neural network in consideration of the fact that the signal lamp has a small target in a picture frame, and the neural network is more advantageous in the aspect of detecting the small target. And collecting pictures of signal lamps in different weather to form a test set, and evaluating the network.
The training of the SSD network (24) for identifying the lane lines is based on a data set formed by pictures of the lane lines collected by various surrounding environments on the network, the SSD neural network in the CNN neural network is considered to be advantageous in processing linear characteristics, the SSD neural network is used for identifying the straight lane lines and the curved lane lines, smaller detection frames are selected as much as possible according to the characteristics of the lane lines, the straight lane lines and the curved lane lines are identified and marked in blocks, and the bending, straight, virtual and actual of the lane lines can be judged according to the slope and the intensity of the blocks.
The training of the identification safety belt DBN network (25) is based on a data set formed by pictures collected on the network, and in order to remove invalid areas, the picture set of the part below the neck of a driver and a copilot is selected as a training set, and the training set is divided into two types of wearing safety belts and not wearing safety belts; firstly, mapping feature vectors of training set pictures to different feature spaces by adopting an RBM network, realizing layer-by-layer learning of features, and reserving feature information of images as much as possible; then, by means of BP network as the last layer of DBN network, the RBM network is received to output characteristic vector, and the whole DBN network is finely tuned in a supervision manner; when the network is tested, firstly, a haar feature detection algorithm is carried out on a driver picture frame and a co-driver picture frame in the picture to detect the face features, and the picture frame of the part below the neck is selected as a test object to be input into the network, and the network is evaluated.
The operation process of the server judging module 2 is as follows: when the server judging module 2 receives the illegal video information sent by the vehicle-mounted recording module 1, the image frames of the video are respectively imported into each network for identification, whether traffic illegal behaviors exist or not is judged, and the video information with the traffic illegal behaviors and the identified license plate information are uploaded to the traffic management network 4 to complete the reporting process.
As shown in fig. 4, the APP integrity query module 3 includes a registration interface 31, a login interface 32, a query interface 33, and an intelligent contract interface 34. The vehicle owner fills in the identity information such as the driving license number, the name, the identity card number, the license plate number and the like through the registration interface 31 to register, the registration interface 32 uses the identity card number and the password to register, the query interface 33 can query own traffic illegal behaviors and related illegal video evidence after the registration, and when the situation of false report occurs, the vehicle owner can conduct complaint on the query interface 33, and the manual background can screen and cancel the traffic illegal behaviors with false judgment. The APP integrity query module 3 generates traffic integrity scores according to traffic violations of vehicle owners, and can be checked at the query interface 33. The APP smart contract interface 34 has the following functions: when the owner applies for renting the shared automobile, becomes a taxi driver, becomes a pilot driver and even inquires personal credit information, the corresponding organization has the right to inquire the traffic integrity score of the owner and contract the intelligent contract interface 34, and the APP integrity inquiry module 3 assists the organization to automatically release the contract with the owner when the integrity score of the owner falls to a certain value.
In summary, the invention discloses a decentralised traffic integrity evaluation system, and mainly describes the composition and the operation process of the system. First, the vehicle-mounted recording module is mounted on a private car by a traffic police when the private car is purchased, registered or maintained, or when a taxi and a public car such as a bus are operated. In the driving process, after the video processing module carries out image processing on the video captured by the video capturing module frame by frame, judging whether traffic illegal behaviors such as solid-line illegal lane changing, red light running, no license plate or license plate exist on a front vehicle or not according to the image information acquired by the two cameras in front of the vehicle; according to the vehicle rear image information collected by the vehicle roof camera, judging whether a driver or a copilot does not wear a safety belt, the vehicle distance is too close, no license plate or license plate exists in the rear vehicle, uploading a video containing the illegal behaviors to a server judging module through a 4G communication module, then further judging whether the traffic illegal behaviors exist in the video by the server judging module through a neural network, finding out corresponding vehicle owner information according to license plate numbers, displaying the illegal information to the vehicle owner at an APP honest inquiring module, uploading the video information and license plate information with the traffic illegal behaviors to a traffic management network, completing the reporting process, and successfully completing the reported vehicle owner to obtain reporting rewards of traffic management authorities as incentives for supervising the traffic illegal behaviors. Compared with the prior system, the system is arranged on a private car and other travelling crane, greatly reduces the supervision blind area, can generate stronger deterrence effect on traffic offenders, and simultaneously automatically performs video capturing and judgment of traffic offender behaviors without the participation of drivers in the reporting process, thereby reducing the travelling safety hidden trouble. The system is arranged at a fixed position of the vehicle body, so that the adjustment and the maintenance are convenient, the server judging module carries out second auditing on the reported video through a neural network algorithm, the accuracy of reported information is greatly improved, and the traffic illegal action processing pressure of the traffic management network is reduced. The APP integrity query module can query whether the driver has traffic illegal behaviors or not in real time, and query the traffic integrity score of the driver, so that the supervision and management of the driver are increased. The invention makes the supervision of traffic illegal actions no longer executed by specific crowd (traffic police) or equipment (electronic police), realizes the decentralization of traffic supervision, and increases the range and restriction force of traffic law enforcement.