CN102685516A - Active safety type assistant driving method based on stereoscopic vision - Google Patents

Abstract

The invention discloses an active safety type assistant driving method based on stereoscopic vision. An active safety type assistant driving system comprehensively utilizes an OME information technology, consists of a stereoscopic vision subsystem, an image immediate processing subsystem and a safety assistant driving subsystem and comprises two sets of high resolution CCD (charge-coupled device) cameras, an ambient light sensor, a two-channel video collecting card, a synchronous controller, a data transmission circuit, a power supply circuit, an image immediate processing algorithms library, a voice reminding module, a screen display module and an active safety type driving control module. According to the active safety type assistant driving method, separation lines and parameters such as relative distance, relative speed, the relative acceleration and the like of dangerous objects such as front vehicles, front bicycles, front pedestrians and the like can be accurately identified in real time in sunny days, cloudy days, at nigh and under the severe weather conditions such as rain with snow, dense fog and the like, so that the system can prompt a driver to adopt countermeasure through voice and can realize automatic deceleration and emergency brake at emergency situation, thereby ensuring safe travel in a whole day.

Description

Stereoscopic vision active safety auxiliary driving method

Technical field

The invention belongs to the automobile assistant driving field, relate to a kind of active safety formula auxiliary driving method based on stereovision technique.This active safety formula DAS (Driver Assistant System) comprehensive utilization ray machine teleinformatic technique; Form by stereoscopic vision subsystem, image fast processing subsystem and safe driver assistance subsystem, comprise two high resolution CCD video cameras, ambient light illumination transducer, binary channels video frequency collection card, isochronous controller, data transmission circuit, power supply circuits, image fast processing algorithms library, voice reminder module, screen display module and active safe driving control module etc.Under light application ratio was fine fully, in the vehicle ' process, the present invention obtained the visible image of target on the travel direction through the binocular solid camera system; Under atrocious weather conditions such as cloudy day, night, sleet, dense fog; Realize obtaining of clear road conditions image through algorithm for image enhancement, and image information and synchronous environment illuminance information are delivered to image fast processing subsystem, then under the support of image fast processing algorithm through the binary channels video frequency collection card; At first to the real-time detection of lines; Integrated use image fast processing algorithm again identifies the parameters such as relative distance, relative velocity and relative acceleration of risk objects such as front vehicles, bicycle, pedestrian real-time and accurately in image, and result images is input on the Vehicular display device; Supply the driver to observe use; The counter-measure that should take with regard to current road surface situation through the voice suggestion driver simultaneously realizes under the emergency situation slowing down automatically and snub, guarantees traffic safety round-the-clockly.

Background technology

Rapid expansion along with high development of social economy and city size; Transport need increases day by day; The high speed development of transportation promoted on the one hand logistics, greatly shortened people travel time, improved operating efficiency, also brought the traffic problems of many sternnesses on the other hand.The continuity that traffic jam, traffic accident take place frequently, the continuous deterioration of traffic environment not only destroys road transport causes traffic congestion, and jeopardizes the country and people's the security of the lives and property.

In a lot of traffic accidents that take place in recent years, the accident that is taken place under night or the cloudy situation is particularly serious.According to the traffic accidents statistics, though night traffic accident number of times only accounts for about 40% of traffic accident sum, fatal accident accounts for 60% of whole accident.Cause the dangerous factor of driving at night to mainly contain two; The one, sense of vision factor, because the light problem, more visual blind areas make driving at night to bring some difficulties to driving; Bad visibility, circle, road are unclear, often make vehicle depart from the proper motion track or run into fortuitous event and take measures too late.This is the main cause that the night traffic accident takes place.The 2nd, physiologic factor, be driver's physiological condition the poorest the time night, easy fatigue, eyesight reduces, and is inaccurate to the judgement of speed and distance, absent minded.Owing to the view that can not see the road both sides, little in addition to driver's excitatory stimulation thing, the main driving fatigue of the easiest product, the driver is careless slightly will to lead to serious traffic accident.Therefore, the frequent problem that takes place of solution night traffic accident is extremely urgent.

Safety is that Hyundai Motor is learned upward most important subject under discussion.In order to keep automobile consumer's safety, let it obtain best guarantee, Safety Design has become a most important ring among the Hyundai Motor design, and the cost that safety is equipped with is also increasingly high among auto-producing proportion.Automotive safety equipment has the branch of active safety and passive security outfit, mainly is that bump when meeting accident is as differentiation.So-called passive safety feature then is to take place in that traffic accident is unexpected, under the vehicle situation out of control; Carry out passive protective effect for passenger, hope to see through fixture, let the indoor occupant of car; Be fixed on safe position; And utilize structural guiding and crumple, and absorb the strength of bump as far as possible, guarantee the indoor occupant's of car safety.Active safety is equipped with and is meant the servicing unit of being done before the generation bump.These devices just can begin start at vehicle when out of control; Get involved the action of driving in every way, hope can utilize machinery and electronic installation, keeps the state of controlling of vehicle; Let the driver can recover control with all strength, avoid traffic accident to take place unexpectedly for vehicle.

For the traffic accident that reduces growing traffic accident and avoid Driver's Factors to bring; Alleviate driver's manipulation strength; Automobile active safety driver assistance Study on Technology receives the common concern of various countries; And dropped into great amount of manpower and material resources and financial resources and carried out the systematic research exploitation, to improve the fail safe of automobile.Automotive safety driver assistance technology is to utilize sensor technology, signal processing technology, mechanics of communication, computer technology etc.; Residing environment of identification vehicle and state; And make according to the information that each transducer obtains and to analyze and judge; Perhaps send and advise and warning message, remind the driver to note evading danger to the driver; Perhaps in case of emergency, help the driver operation vehicle, prevent the generation of accident, make vehicle get into the state of a safety; Perhaps replace driver's operation, realize the automation of vehicle operating.

From the origin cause of formation of road traffic accident, the driver is the principal element that causes traffic accident, and the pedestrian is the colony that is injured main in traffic accident.In Japan, pedestrian's number of casualties accounts for 27% of whole traffic accident injures and deaths sum; In Europe, annual, vehicle causes about 9,000 people death because of crashing with pedestrian or cyclist, and 200,000 people are injured.The road traffic quantity of China, death toll and the ten thousand car death rates all come first of the world; And death toll is higher with the ratio of number of injured people in the traffic accident; China's ten thousand car death rates were 9.2 in 2004, and the ten thousand car death rates of developed country are between 1.2～5.9.Because the pedestrian is the Primary Actor of road traffic; In order to protect pedestrian's safety effectively; Carry out vehicle front pedestrian's detection, in time inform driver's vehicle front pedestrian's existence, and carry out safe early warning; To reducing or avoid vehicle and the pedestrian significant meaning that crashed, and have potential economic worth and application prospects.

This project is exactly under such background; With active safety driver assistance technology is the breach; The vehicle-mounted combination stereoscopic vision of key breakthrough each item key technology; At avenue, highway and standard high-grade highway, under atrocious weather conditions such as fine day, cloudy day, night, sleet, dense fog, also can realize keeping round-the-clock auxiliary security such as safe track, the safe speed of a motor vehicle, safe distance between vehicles to drive function.

Domestic and international present situation

The potential hazard thing of detection this car in the environment that goes; Mainly be the environmental information of the front, rear, left and right of going through the sensor acquisition car, be delivered to central controller, central controller adopts multi-sensor information fusion technology; Whether there is the potential hazard thing in the environmental information around judging; And follow the tracks of the potential hazard thing, detect distance in real time, again according to different weather, come driver assistance person to judge vehicle safety travel from car vehicle, different relative velocity, relative acceleration; This system is also referred to as automotive safety auxiliary system, anti-collision system for automobile, also has the automobile active safety early warning system.Present Research mainly contains:

(1) U.S.

In the seventies in 20th century, the U.S. has been placed on attentiveness on the road, puts into tens million of dollars in the advanced bus or train route system.In recent years, along with establishment, the development and perfect of ITS system, the U.S. has strengthened especially safety guarantee The Application of Technology research of The intelligent vehicles technology.DOT is beginning one 5 yearly plan at the end of last century, drop into 3,500 ten thousand dollars with General Motors Corporation developed a kind of automobile front and back CAS cooperatively.

The IVI plan of the U.S. and automatic anti-collision warning of General Motors Corporation's joint study exploitation one cover and the experimental system that prevents.At present, the existing 50,000 cover collision warning systems that surpass of the U.S. are used for heavy vehicle.The Daimler. Chrysler's research and development automobile anti-collision device mainly is two rangefinders and an image system, can measure safe distance, if find before the car barrier is arranged, computer can cause brake gear automatically.Night vision device has been installed additional in Volvo AB on car, night vision device can show the object beyond the range of headlamp, shows the distance of the headlamp and the preceding barrier of car of car, the prompting brake driver.

(2) Japan

Japan came into effect the Smartway demonstration project in China in 2003, expected 2015 and built up.This provide in the works such as the track keep, the crossroad is crashproof, the pedestrian dodges with the spacing assurance etc.Simultaneously, Japan pays special attention to the recent actual gains that technology is brought, and the safety guarantee technology is progressively added on the automobile, makes automobile progressively intelligent.About 21 century advanced safety vehicle project demand, each automobile production enterprise of Japan is developing and has the good active safety and the advanced safety vehicle of passive safety according to Japanese Ministry of Communications.It is said that the main purpose of developing this car is to avoid the generation of car accident and reduce extent of injury, the major technique measure is a large amount of advanced safety pre-warning systems that use.For example, Mitsubishi is planned to help the driver certifying risk of collision with the passive beam transducer on two scanning laser radars, several video camera and 6 directions.Axela Inc. utilizes scanning laser radar and ultrasonic sensor to detect whether the place ahead has the pedestrian or whether direction has the vehicle of sailing at the oblique angle.Nissan's a kind of novel radameter of also having gone into operation; When the vehicle closing speed of automobile and front was too fast, the generating laser that is installed on car the place ahead and laser sensor and the velocity transducer that links to each other with speed changer can be noted through the prompting of the display unit in driver's cabin driver.Honda company has developed a kind of new computer radar system, and it can sound a warning to the driver before car accident in the twisting with slight brake and safety belt.

(3) Europe

In Europe, countries such as Germany, France, Italy are also carrying out the exploitation of safety guarantee Study on Technology energetically.Perception, the vision that the research of Europe open fund concentrates on driver's monitoring, road environment strengthens, front truck is apart from control and sensor fusion aspect.Europe entrusts fund supporting vertically and Lateral Collision Avoidance research.EU Committee determines a few days ago, since the second half year in 2005, will carry out automobile in European Union member countries anticollision radar is installed, and use all Europe unified special frequency band, and is interference-free to guarantee this short distance radar technically, thereby improves highway traffic safety.

" the specific lane obstructions thing early warning system " of Volkswagen's development can forecast whether the vehicle of reverse driving constitutes danger to oneself overtaking other vehicles.This system is by laser range sensor and the common monitoring vehicle road ahead of image system situation; Measure front part of vehicle to barrier apart from the basis on; Car-mounted computer calculates relatively the travel speed near vehicle, thereby can forecast whether contrary volume of traffic constitutes danger to oneself.Advanced " stopping/walk " system (StopAnd Go) is housed, i.e. the low coverage radar on Audi's laboratory vehicle.The low coverage radar has 8 eyes, and search radius is 30 meters, and the people who drives this automobile will feel more comfortable, safer.

(4) China

Domestic starting late, some R and D have also been done by the automobile vendor of China such as Changchun one vapour, Shanghai Volkswagen, Wuhan Second Automobile Works and some scientific research institutions such as Science Institute of Ministry of Communications, Wuhan automotive research institute, automobile system of Tsing-Hua University, machinery system of Beijing Institute of Technology, man-machine institute of Xi'an Communications University, Jilin University, Chang An University.The industry plan of State Planning Commission, economic and commercial committee and the plan of the Department of Science and Technology all do not relate to content in this respect.But automobile automatic bumper technology has caused the each side attention, and medium are actively introduced external situation, and government official and scientific research personnel actively follow the tracks of the development abroad state.That it is reported and apply for a patent has: the millimeter-wave automotive CAS of Shanghai Inst. of Microsystem and Information Technology, Chinese Academy of Sci; Automobile radar anti-collision and intelligence control system that Zhenjiang, Zhejiang communication Broadcasting Equipment Plant develops voluntarily; The automobile anti-rear end collision early warning system of anti-collision radar system Beijing Kai Ruide image technique company development of Saibo Electronic Co., Ltd., Jiangsu Prov.'s development; The anti-collision prewarning apparatus for vehicle of Kang Hong scientific & technical corporation of University Of Chongqing development; The automobile automatic bumper of the safe automobile automatic bumper Manufacturing Co., Ltd far away in Beijing exploitation; The express highway intelligent type automobile active safety early warning system of speed Science and Technology Ltd. and the joint development of navigational guidance research institute of automation institute of University Of Chongqing is pacified in Chongqing, has developed nearly ten years, has model machine to occur.

Technology trends

Along with the development of optical instrument and embedded computer technology, in the coming years, will shoulder important duty such as assisting driver, " supervision " are driven, monitoring road conditions based on the on-vehicle safety DAS (Driver Assistant System) of machine vision.Based on the on-vehicle safety DAS (Driver Assistant System) of machine vision, because clear concept, advanced technology, expense is lower, and need plurality of advantages such as not change to original vehicle interior structure, has good application prospects.

Famous big companies such as Bosch, Delphi, Hella, Siermens VDO and Valeo are all stepping up the development of this respect.Wherein " compound eye (Multifaceted Eyes) " system of Nissan Motor exploitation adopts 4 cameras that the image that photographs is synthesized; Demonstrate the picture of overlooking from the automobile top then; The driver can clearly observe all situations of motor vehicle environment thus; Almost do not have the dead angle, therefore shut-down operation also becomes easier, safer; Matsushita Electric Industries's industry and Sanyo Electric are being developed at present and are being used for the auxiliary low price vehicle-borne CCD camera system of back visibibility; Honda company has been equipped with LDW lane identification system in their Inspire, Accord, Legend vehicle; The vehicle-mounted monitoring system of Nissan Motor's exploitation comprises the camera and the embedded computer of analyzing vehicle location and speed that are positioned on the rearview mirror, matches as automobile; Germany benz and Bayerische Motorne Werke Aktiengeellschaft have researched and developed the novel intelligent Night View Assist, utilize infrared imagery technique that information such as the pedestrian of 150 meters distant places, animal, roadblock are had a panoramic view, and the driver are carried out voice reminder, so that it is made a response early; Daimler-Kreisler Corp has researched and developed the lane identification system based on video technique.

More than the on-vehicle safety DAS (Driver Assistant System) based on machine vision of numerous producers exploitation; Perhaps to utilizing visual light imaging to carry out the road conditions monitoring under the good situation of daylight; Perhaps utilize infrared technique in automobile night running process, to keep watch on the place ahead target; Therefore all can only under single weather condition, work, make its driver assistance function be restricted, this Project Study and being suitable under the complicated weather road conditions of daytime, night and the sleet condition of developing vehicle, bicycle and pedestrian's the full-automatic fast identification and the round-the-clock stereoscopic vision DAS (Driver Assistant System) of location technology; To remedy above defective; Bring the round-the-clock comprehensive brand-new safe driving of driver and experience, the present invention is intended to promote the autonomous innovation of Chinese automobile electronic technology, finally realizes the great-leap-forward development of Chinese Automobile Industry ' technology.

Summary of the invention

The present invention is a kind of stereoscopic vision active safety auxiliary driving method; Towards all kinds of roads; Like avenue, standard high-grade highway and highway, fine fully in illumination, and all can realize multinomial driver assistance function under the atrocious weather conditions such as cloudy day, night, sleet, dense fog.Native system adopts visible light stereoscopic vision monitoring scheme, and hardware system mainly partly is made up of stereoscopic vision subsystem, image fast processing subsystem and safe driver assistance subsystem etc. that two high resolution CCD video cameras and ambient light illumination transducer are formed jointly.Content of the present invention comprises:

1) stereoscopic vision subsystem

(1) system forms

The stereoscopic vision subsystem comprises two high resolution CCD video cameras, ambient light illumination transducer, binary channels video frequency collection card, isochronous controller, data transmission circuit and power supply circuits composition.(Charge Coupled Device is a kind of semiconductor imaging device CCD) to charge coupled device wherein, has that the noise of reading is low, dynamic range is big, the response sensitivity advantages of higher; Highgrade integration has reduced the complexity of ccd sensor, has advantages such as volume is little, in light weight, low in energy consumption, life-span length, anti-vibration.The working method of ccd video camera is that the image of subject passes through lens focus to the CCD chip; CCD is according to the electric charge of the power accumulation corresponding proportion of light, and the electric charge of each pixel accumulation is under the video time sequence control, and pointwise moves outward; After filtering, processing and amplifying, form vision signal output.Use two high resolution CCD video cameras to build the stereoscopic vision subsystem and both can resolve hardware support is provided, can enlarge field range again, the comprehensive monitoring information of road surface for the three-dimensional spatial information of realizing the place ahead risk object.The ambient light illumination transducer is used for the variation of monitoring of environmental illuminance, judges whether vehicle environment of living in is fine day, cloudy day, night, sleet or dense fog.Isochronous controller is used to control two high resolution CCD video cameras and sees through front windshield, the road conditions image in synchronous acquisition vehicle ' the place ahead, and synchronous acquisition ambient light illumination information.In stereoscopic vision subsystem work process; Two ccd video cameras of isochronous controller control are gathered the road conditions image; The road conditions image passes through data transmission circuit with ambient light illumination information;, be transferred to after treatment on the vehicle-carrying display screen to image fast processing subsystem via the synchronous driving of binary channels video frequency collection card, the driver just can see the picture rich in detail that conforms to surrounding environment and condition of road surface; Power supply circuits are that whole system provides supply of electric power, realize that stereoscopic vision multi-source image information obtains.

(2) system calibrating

Camera parameters is the bridge that connects camera review and three-dimensional scence.Camera calibration is meant through experiment and Calculation Method, confirms the process of each parameter in the video camera imaging model.Measured these parameters, also with regard to given in fixed ccd video camera the transformation relation between object point and picture point.The purpose of camera calibration is exactly to confirm the parameter of video camera; Through these parameter lists the relation of the mutual alignment between each coordinate system in the imaging system is shown; Thereby express the relation between image pixel positions and road scene point and road vehicles, the pedestrian position, the precision of demarcation will directly influence the overall precision of DAS (Driver Assistant System).

The parameter of stereoscopic vision subsystem comprises two types, is respectively inner parameter and external parameter.Wherein inner parameter is meant the parameter that characterizes the distinctive imaging model of stereoscopic vision subsystem, is to be determined by the CCD face battle array of stereoscopic vision subsystem employing and the inherent characteristic of optical system, and is separate between each parameter; And external parameter is meant the parameter that characterizes the position of stereoscopic vision subsystem in world coordinate system, is the transformation relation between stereoscopic vision subsystem and the world coordinate system, for position and the attitude parameter of discerning travel route the place ahead risk object lays the foundation.Said scaling method carries out according to following steps:

A) stereoscopic vision subsystem calibrating template pattern is the chess checkerboard pattern, quadrate, and by the square grid of 17 * 17 of black-and-white two colors alternate composition, each grid is of a size of 30 (mm) * 30 (mm); B) use the stereoscopic vision subsystem to take the image on the calibrating template plane of some width of cloth (>=15) from different perspectives; C) the black and white grid summit in the detected image; D) obtain homography matrix between calibrating template plane and its plane of delineation; E) distortion factor is under 0 the prerequisite making, and utilizes the homography matrix of obtaining to calculate the intrinsic parameter and outer parameter of video camera; F) with e) in inside and outside parameter be initial value, make that the distortion factor initial value is 0, use the Levenberg-Marquardt algorithm and carry out non-linear minimum optimization, thereby obtain one group of camera intrinsic parameter value that precision is higher, calculate each item distortion factor simultaneously; G) utilize the structural parameters of the three-dimensional sub-vision system of left and right cameras calculation of parameter calibrate.

2) image fast processing subsystem

(1) in the daytime with the pattern of working at night

Through the vehicle-periphery light conditions that the analysis environments illuminance sensor obtains, the image fast processing subsystem pattern of can selecting automatically to be operated in the daytime or work at night.Light application ratio is down fine fully, after image fast processing subsystem carries out denoising to video image, directly handles the road conditions image that is provided by the stereoscopic vision subsystem; Under atrocious weather conditions such as cloudy day, night, sleet, dense fog; Many, the poor contrast of road conditions image noise that illumination is not enough, sleet mist particle causes the stereoscopic vision subsystem to obtain; Image fast processing subsystem is carrying out image on the basis of denoising, also will carry out image enhancement processing.

The purpose of figure image intensifying is with original unsharp image clear or emphasical some characteristic of paying close attention in using that becomes, and suppresses the characteristic of non-concern, makes it to improve picture quality, abundant information amount, strengthens image interpretation and recognition effect.The present invention is according to the needs of driver assistance; On purpose stress road surface risk object local characteristics in the image; Original unsharp the place ahead lines profile information, motor vehicle, bicycle and pedestrian's image of going in the image become clear, and stress the enhancing to the moving target characteristic, the difference in the expanded view picture between the different target characteristic; Suppress not have the detailed characteristic that changes (for example region of partial sky, zone, the road surface except that lines); Make it to improve picture quality, abundant information amount, strengthen image interpretation and recognition effect, satisfy the needs of risk object position and posture analysis.

Said image enchancing method is regarded image as a kind of 2D signal, and it is carried out strengthening based on the signal of two-dimensional Fourier transform.Carry out LPF (promptly only letting low frequency signal pass through) method earlier, remove the noise among the figure; Carry out high-pass filtering method again, strengthen high-frequency signals such as edge, make fuzzy picture become clear.Carry out gray level correction, greyscale transformation and histogram modification, enlarge dynamic range of images, expanded contrast, the edge contour of outburst danger target is convenient to target identification.Guarantee that native system is not that very desirable situation can better be got rid of environmental impact under such as the situation of night, cloudy day, sleet, dense fog in natural lighting, gathers and recover road conditions image clearly.

(2) identification of road surface risk object

Vehicle goes on highway, and risk of collision is mainly from the place ahead motor vehicle, bicycle and the pedestrian of going on the highway.Particularly the place ahead speed is lower than the vehicle of this car, and wherein the most dangerous be the vehicle that is on the same track, in addition, bicycle and pedestrian on the same track also constitute a threat to traffic safety.Based on this, the present invention at first identifies lines profile on the road surface, thus this car lines of living in is positioned; Carry out moving object then and cut apart and identification, its target is vehicle, bicycle and the pedestrian who is positioned at this car the place ahead on the same track; Vehicle, bicycle and pedestrian for the place ahead, same track of accurately identifying; Again according to the relative distance, relative velocity and the relative acceleration information that calculate; The driver is carried out voice suggestion, and in case of emergency slow down automatically and snub, avoid collision.

The risk object identification of described road surface at first is to use image processing algorithm to carry out lines identification, detects according to road edge and lines, realizes the detection of road through the identification to lines or edge.Said method is according to the obvious ID of trace route path line on road surface; Dynamically absorb pavement image through video camera; Through Computer Processing identification road markings line; And the distance of longitudinal direction of car axis runout sign and and tag line between angle, said method has faster image processing speed and better control real-time.

Described lines recognition methods is: the base of at first detecting lines at this road surface, the place ahead, track application unit pixel grey scale number of degrees criterion; Delimit a window area according to the lane width on the lines base of finding then; In this zone, detect the edge of lines again with the Robinson edge detection operator, then in inferior window area the detection pulse to left and right edges with definite lines.If only detect the border of lines on one side, then the translation window detects another side again.Can extract following information through the identification track: whether the track of vehicle ', track bearing of trend, vehicle the surveyed area of deviation and definite risk object.

Described risk object recognition methods is: confirm to be the detection range constraint with left and right sides lines after the lines of this car place.With risk objects such as image processing method identification front vehicles, bicycle and pedestrians; Concrete grammar is: at first in the road surface scope of this car place lines; Application unit pixel grey scale number of degrees criterion detects the base of risk object; Delimit a window area according to the width on the risk object base of finding then; In this zone, detect the edge of risk object again with the Robinson edge detection operator, then in inferior window area the detection pulse to left and right edges with definite risk object.If only detect risk object border on one side, then the translation window detects another side again, realizes that the risk object integrity profile detects.

(3) pose of road surface risk object resolves

The pose of described road surface risk object resolves; Be that the risk object utilization Kalman filtering of confirming as front vehicles, bicycle and pedestrian is followed the tracks of; The position of real-time resolving the place ahead risk object (comprising vehicle, bicycle and pedestrian) and attitude change, and according to accurate position of the risk object that calculates and attitude, calculate relative distance, relative velocity and the relative acceleration of itself and this car; Carry out the control of this car speed of a motor vehicle, prevent that collision from taking place.

This car is on the pretreated basis of image with the process nature of relative distance, relative velocity and the relative acceleration measuring and calculating of the place ahead risk object, and through the understanding to two dimensional image, realization is to road environment and go up the cognitive process of Three-dimension Target.This process need is accurately discerned on the basis in the effective detection of described lines and front vehicles, bicycle and pedestrian, uses the binocular stereo vision 3-D information fetching method, obtains front vehicles, bicycle and pedestrian's three-dimensional space position and attitude parameter.

Binocular stereo vision (Binocular Stereo Vision) is a kind of important form of machine vision; It is based on principle of parallax and utilizes imaging device to obtain two width of cloth images of testee from different positions; Through the position deviation between the computed image corresponding points, obtain the method for object dimensional geological information.

Stereoscopic vision subsystem of the present invention by about two ccd video cameras form.As shown in Figure 1, mark the relevant parameter of left and right video camera among the figure respectively withsubscript 1 and r.1 A in the world space (X, Y, Z) the imaging surface Cl of left and right cameras and the picture point on the Cr be respectively al (ul, vl) and ar (ur, vr).These two pictures that picture point is same object-point A in the world space are called " conjugate point ".Known this two conjugation picture points, made they and the photocentre Ol of camera separately and the line of Or respectively, i.e. projection line alOl and arOr, their intersection point be object-point A in the world space (X, Y, Z).The basic image-forming principle of Here it is stereoscopic vision subsystem of the present invention.

In the stereoscopic vision subsystem of described parallel optical axis (as shown in Figure 2); About focal length and other inner parameter of two ccd video cameras all equate; Optical axis is vertical with the imaging plane of video camera; The x axle of two video cameras overlaps, and the y axle is parallel to each other, and therefore left video camera is overlapped with right video camera along its x direction of principal axis translation one segment distance b (being called baseline) back.By spatial point A and about the polar plane confirmed of the photocentre Ol, Or of two video cameras be that the conjugation polar curve is right with intersection pl, the pr of left and right sides imaging plane Cl, Cr respectively, they respectively with the parallel and conllinear of reference axis ul, ur of imaging plane separately.In stereoscopic vision subsystem structure form of the present invention; The geometrical relationship of left and right cameras configuration is the simplest; Polar curve has had good character, at subpoint al on the imaging plane of the left and right sides and the matching relationship between the ar condition very easily is provided for seeking object-point A.

In the vehicle ' process; The stereoscopic vision subsystem is gathered the road conditions image information in real time, calculates through parallax, obtain the anaglyph of full screen after; Adopt the mode of background modeling; Obtain the anaglyph of sport foreground object, expand again with erosion algorithm carry out the image preliminary treatment, obtain the complete foreground moving object disparity map of supply analyzing.Adopt the motion tracking algorithm; Detect distance, the movement locus of risk objects such as road vehicles, bicycle and pedestrian in real time; And with the rule of prior setting (this rule and this car vehicle and weather pattern are relevant; For example this vehicle skidding distance and rainy day cause factors such as braking distance prolongation) compare, if relative distance, relative velocity, relative velocity relation enter in the alarm range of setting, system then carries out voice suggestion to the driver; If relative distance and relative velocity relation enter in the risk range of setting, system then slows down and snub automatically.

3) safe driver assistance subsystem

Safe driver assistance subsystem is connected with automobile bus, comprising: voice reminder module, screen display module, active safe driving control module.The function that safe driver assistance subsystem possesses comprises:

(1) keep safe track: the present invention's detection and Identification in real time lines, have the navigation channel and depart from the audio alert function, can in time remind the driver to correct direction, prevent accidents such as vehicle is turned on one's side, side collision.

(2) keep the safe speed of a motor vehicle to drive; The present invention possesses relative velocity and the relative acceleration that synchronous real-time measurement is driven risk objects such as the place ahead motor vehicle, bicycle and pedestrian, can more effectively realize the intelligent cruise driver assistance of automobile, crashproof driver assistance function.

(3) remain a safe distance behind the car in front: the present invention reaches sub-pixel to the profile accuracy of detection of the place ahead risk object that goes, and the relative distance calculation accuracy of the place ahead risk object that goes is superior to 10cm, and the resolving time of relative distance is superior to 0.1s.Risk object within the safe distance in going is carried out distance prompt, intelligent sound prompting and dangerous situation report to the police, can in time remind the driver to adjust the speed of a motor vehicle, prevent the accident of types such as vehicle knocks into the back, head-on crash.

Description of drawings

Fig. 1 is the basic imaging schematic diagram of binocular tri-dimensional vision system.

Fig. 2 is the binocular stereo vision system imaging principle and the corresponding image points matching relationship figure of parallel optical axis of the present invention.

Fig. 3 is the scheme of installation of the embodiment of the invention.Comprising: (1) right video camera, (2) left video camera, (3) ambient lighting transducer, (4) binary channels video frequency collection card, (5) display screen, (6) system central processor.

Fig. 4 is the method flow diagram of embodiments of the invention.

Embodiment

When the present invention works; Abundant in illumination, weather conditions are (for example fine) under the situation preferably, the stereoscopic camera of forming through two high resolution CCD video cameras demarcating; Obtain the advance visible image of risk object such as motor vehicle, bicycle, pedestrian in the place ahead of automobile; Foundation is demarcated the distance that the camera s internal and external orientation that obtains calculates the place ahead target, and voice suggestion the driver note, the clutch self-actuating brake.Under the sufficient environment of illumination; The stereoscopic vision subsystem directly obtains the visible image of target on the travel direction, under the inadequate situation of illumination (for example night, cloudy day, sleet and foggy weather), and the parameter that provides based on the ambient lighting transducer; The road conditions image that the stereoscopic vision subsystem is obtained passes through image enhancement technique; Under the limited situation of pilot's line of vision, supply driver's observation, guarantee driving safety; Get access to the road conditions image and the ambient lighting parameter is delivered to high speed digital signal processor through the binary channels video frequency collection card together; Under the support of high speed image Processing Algorithm, visible image is carried out data fusion then, and relevant analysis, processing; Through real-time detection to road; Methods such as utilization rim detection, Threshold Segmentation identify the parameters such as position, distance, relative velocity, relative acceleration of risk objects such as lines, front vehicles, bicycle, pedestrian in real time, exactly in image, and point out drivers to take appropriate measures with regard to current road surface situation according to these parameters; And then assurance traffic safety; Simultaneously with the objective contour that identifies, unite the target range that calculates, be input on the Vehicular display device, supply the driver to observe use.

Beneficial effect

The present invention provides automobile active safety driver assistance function under round-the-clock situation, possess following ability:

(1) use stereo visual system, identify the lines information on the vehicle forward direction, the control vehicle keeps safe track under steam;

(2) identify risk object profiles such as lines on the vehicle forward direction, motor vehicle, bicycle, pedestrian, calculate the distance of these potential hazard targets fast, remain a safe distance behind the car in front under steam;

(3) calculate motor vehicle on the vehicle forward direction, bicycle, pedestrian's relative velocity and relative acceleration fast, keep the safe speed of a motor vehicle under steam;

(4) work at night under the pattern,, realize that under the bad situation of illumination auxiliary security drives function through image enhancement technique.

Performance

(1) the present invention can detect lines information in real time, possesses the function that keeps safe track, has the navigation channel and departs from the audio alert function, can in time remind the driver to correct direction, prevents accidents such as vehicle is turned on one's side, side collision.

(2) road conditions IMAQ speed >=15fps of the present invention (frame/second), vehicle relative distance calculation accuracy is superior to 10cm, and the profile accuracy of detection of the place ahead risk object that goes reaches sub-pixel, the relative distance resolving time＜0.1s of the place ahead risk object that goes.

(3) the present invention possesses the function that remains a safe distance behind the car in front: risk object within the safe distance in going is carried out distance prompt, intelligent sound prompting and dangerous situation report to the police; Can in time remind the driver to adjust the speed of a motor vehicle, prevent the accident of types such as vehicle knocks into the back, head-on crash.

Claims (10)

1. active safety formula auxiliary driving method based on stereovision technique; It is characterized in that; Active safety formula DAS (Driver Assistant System) is made up of stereoscopic vision subsystem, image fast processing subsystem and safe driver assistance subsystem, comprises two high resolution CCD video cameras, ambient light illumination transducer, binary channels video frequency collection card, isochronous controller, data transmission circuit, power supply circuits, image fast processing algorithms library, voice reminder module, screen display module and active safe driving control module etc.

2. active safety formula auxiliary driving method according to claim 1; It is characterized in that its stereoscopic vision subsystem comprises two high resolution CCD video cameras, ambient light illumination transducer, binary channels video frequency collection card, isochronous controller, data transmission circuit and power supply circuits composition.

3. stereoscopic vision subsystem according to claim 2; It is characterized in that; The stereoscopic vision subsystem that two high resolution CCD video cameras are built both can resolve for the three-dimensional spatial information of realizing the place ahead risk object hardware support was provided; Can enlarge field range again, the comprehensive monitoring information of road surface.The ambient light illumination transducer is used for the variation of monitoring of environmental illuminance, judges whether vehicle environment of living in is fine day, cloudy day, night, sleet or dense fog.Isochronous controller is used to control two high resolution CCD video cameras and sees through front windshield, the road conditions image in synchronous acquisition vehicle ' the place ahead, and synchronous acquisition ambient light illumination information.In stereoscopic vision subsystem work process; Two ccd video cameras of isochronous controller control are gathered the road conditions image; The road conditions image passes through data transmission circuit with ambient light illumination information;, be transferred to after treatment on the vehicle-carrying display screen to image fast processing subsystem via the synchronous driving of binary channels video frequency collection card, the driver just can see the picture rich in detail that conforms to surrounding environment and condition of road surface; Power supply circuits are that whole system provides supply of electric power, realize that stereoscopic vision multi-source image information obtains.

4. active safety formula auxiliary driving method according to claim 1; It is characterized in that; Its image fast processing subsystem comprises in the daytime and the pattern of working at night, and under the pattern of working at night, needs carry out the figure image intensifying to the road conditions image; Concrete grammar is: carry out low pass filtering method earlier, remove the noise among the figure; Carry out high-pass filtering method again, strengthen high-frequency signals such as edge, make fuzzy picture become clear.Carry out gray level correction, greyscale transformation and histogram modification, enlarge dynamic range of images, expanded contrast, the edge contour of outburst danger target is convenient to target identification.Guarantee that native system is not that very desirable situation can better be got rid of environmental impact under such as the situation of night, cloudy day, sleet, dense fog in natural lighting, gathers and recover road conditions image clearly.

5. active safety formula auxiliary driving method according to claim 1; It is characterized in that; The recognition methods of the road surface risk object of its image fast processing subsystem; At first be to use image processing algorithm to carry out lines identification, detect, realize the detection of road through identification lines or edge according to road edge and lines.Said method is according to the obvious ID of trace route path line on road surface; Dynamically absorb pavement image through video camera; Through Computer Processing identification road markings line; And the distance of longitudinal direction of car axis runout sign and and tag line between angle, said method has faster image processing speed and better control real-time.

6. active safety formula auxiliary driving method according to claim 1; It is characterized in that; The lines detection method of its image fast processing subsystem is: the base of at first detecting lines at this road surface, the place ahead, track application unit pixel grey scale number of degrees criterion; Delimit a window area according to the lane width on the lines base of finding then; In this zone, detect the edge of lines again with the Robinson edge detection operator, then in inferior window area the detection pulse to left and right edges with definite lines.If only detect the border of lines on one side, then the translation window detects another side again.Can extract following information through the identification track: whether the track of vehicle ', track bearing of trend, vehicle the surveyed area of deviation and definite risk object.

7. active safety formula auxiliary driving method according to claim 1 is characterized in that, the risk object recognition methods of its image fast processing subsystem is: after the lines of definite this car place, be the detection range constraint with left and right sides lines.With risk objects such as image processing method identification front vehicles, bicycle and pedestrians; Concrete grammar is: at first in the road surface scope of this car place lines; Application unit pixel grey scale number of degrees criterion detects the base of risk object; Delimit a window area according to the width on the risk object base of finding then; In this zone, detect the edge of risk object again with the Robinson edge detection operator, then in inferior window area the detection pulse to left and right edges with definite risk object.If only detect risk object border on one side, then the translation window detects another side again, realizes that the risk object integrity profile detects.

8. active safety formula auxiliary driving method according to claim 1; It is characterized in that the pose calculation method of the road surface risk object of its image fast processing subsystem is: in the vehicle ' process, the stereoscopic vision subsystem is gathered the road conditions image information in real time; Calculate through parallax; After obtaining the anaglyph of full screen, adopt the mode of background modeling, obtain the anaglyph of sport foreground object; Expand again with erosion algorithm carry out the image preliminary treatment, obtain the complete foreground moving object disparity map of supply analyzing.Adopt the motion tracking algorithm; Detect distance, the movement locus of risk objects such as road vehicles, bicycle and pedestrian in real time; And with the rule of prior setting (this rule and this car vehicle and weather pattern are relevant; For example this vehicle skidding distance and rainy day cause factors such as braking distance prolongation) compare, if relative distance, relative velocity, relative velocity relation enter in the alarm range of setting, system then carries out voice suggestion to the driver; If relative distance and relative velocity relation enter in the risk range of setting, system then slows down and snub automatically.

9. active safety formula auxiliary driving method according to claim 1 is characterized in that, its safe driver assistance subsystem is connected with automobile bus, comprising: voice reminder module, screen display module, active safe driving control module.The function that safe driver assistance subsystem possesses comprises: keep safe track, keep that the safe speed of a motor vehicle is driven, function remains a safe distance behind the car in front.

10. active safety formula auxiliary driving method according to claim 1; It is characterized in that; Active safety formula DAS (Driver Assistant System) can detect lines information in real time, possesses the function that keeps safe track, has the navigation channel and departs from the audio alert function; Can in time remind the driver to correct direction, prevent accidents such as vehicle is turned on one's side, side collision.The road conditions IMAQ speed >=15fps (frame/second) of system, vehicle relative distance calculation accuracy is superior to 10cm, and the profile accuracy of detection of the place ahead risk object that goes reaches sub-pixel, the relative distance resolving time＜0.1s of the place ahead risk object that goes.Can carry out distance prompt, intelligent sound prompting and dangerous situation to risk object within the safe distance in going and report to the police, can in time remind the driver to adjust the speed of a motor vehicle, prevent the accident of types such as vehicle knocks into the back, head-on crash.

Images