Summary of the invention
The full liquid crystal instrument system based on ADAS and AR technology that the purpose of the present invention is to provide a kind of, can be applied to large sizeOn new energy vehicle or vehicle with strong electromagnetic, by the system can in automobile instrument display set vehicle driving peripheryVehicle and road conditions, Real-time Traffic Information, traffic marking instruction, safe distance between vehicles prompt and specific real-time road condition information are reminded in oneAR panoramic table, and provide danger early warning for driver, guidance and auxiliary driver are more acurrate, safely drive vehicle.
To achieve the goals above, the technical solution adopted by the present invention is a kind of full liquid crystal based on ADAS and AR technologyInstrument system, including full liquid crystal instrument, ADAS information acquisition system, Vehicular entertainment navigation system, BCM body control system,ADAS information acquisition system, BCM body control system connect the input terminal of Vehicular entertainment navigation system, vehicle by CAN busThe output end for carrying amusement navigation system connects full liquid crystal instrument by dual-channel LVDS signal wire, and ADAS information acquisition system passes throughCAN bus connects full liquid crystal instrument to realize that synchronizing information, BCM body control system connect full liquid crystal instrument by CAN busTo realize that car status information is shown.
As an improvement of the present invention, the ADAS information acquisition system includes sighting distance image-forming module, the first CAN receiptsDevice is sent out, sighting distance image-forming module includes multiple millimetre-wave radars and an image acquisition device, and all millimetre-wave radars pass through the first CANTransceiver connects Vehicular entertainment navigation system and full liquid crystal instrument, and image acquisition device connects car entertainment by CVBS signal wire and leadsBoat system.
As an improvement of the present invention, the Vehicular entertainment navigation system includes SOC module, amusement MCU, GPS and northStruggle against satellite positioning module, LVDS encoder, video encoder, the second CAN transceiver, the second CAN transceiver connection amusement MCU'sRX and the port TX, amusement MCU connect SOC module by spi bus, and SOC module connects LVDS by FPD-Link transfer busEncoder and video encoder, the port RX and TX of GPS and big-dipper satellite locating module connection SOC module, image acquisition device are logicalCVBS signal wire connection video encoder is crossed, the first CAN transceiver connects the second CAN transceiver by CAN bus.
As an improvement of the present invention, the full liquid crystal instrument includes instrument MCU, third CAN transceiver, LVDS solutionCode device, digital zoom input, liquid crystal display, control button, third CAN transceiver connect the port RX and TX of instrument MCU,Instrument MCU connects digital zoom input by spi bus, and LVDS decoder connects image by FPD-Link transfer busDisplay driver, digital zoom input connect liquid crystal display, and control button connects the port I/O of instrument MCU, the first CANTransceiver connection connects third CAN transceiver by CAN bus, and LVDS encoder connects LVDS solution by dual-channel LVDS signal wireCode device.
As an improvement of the present invention, the BCM body control system includes BCM module, PEPS module, ABS mouldBlock, ECU module, BCM module, PEPS module, ABS module and ECU module pass through CAN bus connect second and the 3rd CAN receiveSend out device.
As an improvement of the present invention, the sighting distance image-forming module includes four millimetre-wave radars, is separately positioned on vehicleAll around, image acquisition device includes four high-definition image cameras to body, is separately positioned on vehicle body all around.
As an improvement of the present invention, the amusement MCU and instrument MCU is all made of RH850 microprocessor and is setMeter, SOC module are designed using J6-DRA725 processor, and digital zoom input is designed using IMX6D industrial control board,First to third CAN transceiver use TJA1042 CAN transceiver.
Above system carries out the method that instrument AR panoramic table is shown, mould is imaged using the sighting distance of ADAS information acquisition systemBlock obtains real time panoramic image and the imaging on vehicle body periphery, is sent in Vehicular entertainment navigation system by entertaining MCU and SOC mouldThe data analysis module of block composition, after this data analysis module receives data, by the image for acquiring and capturing (obstacle in kindObject/object) do preset display label after, the blending algorithm of combined data analysis module carries out continuing real-time tracing, and sameWhen by the displacement information of image material object and vehicle running state information binding analysis, analysis the data obtained is led with GPS and BeidouThe data/BCM data of navigating carry out algorithm fusion, in conjunction with barrier speed in vehicle current driving speed, image, object speed intoRow fusion calculation obtains vehicle and barrier/object relative distance, after all data of fusion calculation gained, by dataIt is sent to full liquid crystal instrument, full liquid crystal instrument fusion real time panoramic image and the fused data of real vehicle carry out AR real time panoramic againImage co-registration, and full liquid crystal instrument again merges real vehicle status information with the AR real time parsing module in instrument MCU again, mostAR image is shown in the display screen of full liquid crystal instrument afterwards, while full liquid crystal instrument provides corresponding danger early warning and prompting.
Compared with the existing technology, full liquid crystal instrument overall system architecture of the invention is ingenious in design, structurally reasonable stabilization, easilyIt is operated in installation, the ADAS information acquisition system in the full liquid crystal instrument system of automobile is to the vehicle week in driving processIt encloses barrier/object related data to be acquired, and data is supplied in Vehicular entertainment navigation system by CAN busFurther fusion calculation processing is carried out with vehicle real time status information detected by BCM body control system, it finally will fusionData generation image after calculation processing is sent to progress AR panoramic table in full liquid crystal instrument and shows, and is in danger in vehicleDanger early warning prompt is generated when warning line, in full liquid crystal instrument and Vehicular entertainment navigation system, jointly to remind driver in timeIt removes a hidden danger and imminent danger, to instruct and assist that driver is more acurrate, safely drives vehicle.
Specific embodiment
In order to deepen the understanding of the present invention and recognize, the invention will be further described below in conjunction with the accompanying drawings and introduces.
As shown in Figure 1, a kind of full liquid crystal instrument system based on ADAS and AR technology provided for the preferred embodiment of the present inventionSystem, including full liquid crystal instrument, ADAS information acquisition system, Vehicular entertainment navigation system, BCM body control system, ADAS informationAcquisition system, BCM body control system connect the input terminal of Vehicular entertainment navigation system by CAN bus, and car entertainment is ledThe output end of boat system connects full liquid crystal instrument by dual-channel LVDS signal wire, and ADAS information acquisition system is connected by CAN busFull liquid crystal instrument is connect to realize that synchronizing information, BCM body control system connect full liquid crystal instrument by CAN bus to realize vehicleStatus information is shown.The full liquid crystal instrument system has AR(Augmented Reality, augmented reality) function, ADASImage acquisition device in information acquisition system to obstacles around the vehicle/object related data in driving process intoRow acquisition, forms 360 ° of stereoscopic image datas, and data are supplied in Vehicular entertainment navigation system and BCM by CAN busThe further fusion calculation processing of the progress of vehicle real time status information detected by body control system, i.e. progress parameter calibration,The processing such as point cloud and elimination redundancy, and the vehicle measured in conjunction with the image data and millimetre-wave radar for synchronizing the same time comeData (including the current location information of vehicle, direction, angle etc.) render the current vehicle with enhancing visual effectLocation information, direction, angle are sent to progress AR panoramic table in full liquid crystal instrument according to vehicle image and show, and at vehicleDanger early warning prompt is generated, jointly when dangerous warning line, in full liquid crystal instrument and Vehicular entertainment navigation system to remind in timeDriver removes a hidden danger and imminent danger.
As shown in Fig. 2, the ADAS information acquisition system includes sighting distance image-forming module, the first CAN transceiver, sighting distance imagingModule includes multiple millimetre-wave radars and an image acquisition device, and all millimetre-wave radars connect vehicle by the first CAN transceiverAmusement navigation system and full liquid crystal instrument are carried, image acquisition device connects Vehicular entertainment navigation system by CVBS signal wire.Specifically, the sighting distance image-forming module includes four millimetre-wave radars, is separately positioned on vehicle body all around, image acquisition device includesFour high-definition image cameras, are separately positioned on vehicle body all around.Vehicle body periphery is obtained in real time by sighting distance image-forming moduleInclude lane road conditions, traffic lane line, front vehicles, road sign, traffic identification marking board, vehicle body periphery pedestrian, vehicle body periphery barrierHinder the panoramic picture and radar image (comprising night vision image) including object etc., and by the first CAN transceiver to the image of acquisitionData are transmitted.
The Vehicular entertainment navigation system includes SOC module, amusement MCU, GPS and big-dipper satellite locating module, LVDS volumeCode device, video encoder, the second CAN transceiver, amusement MCU are mainly responsible for electric power starting and the pass of Vehicular entertainment navigation systemClose, control and switch the starting between CAN chip, LVDS chip and SOC module and closing function, the connection of the second CAN transceiverThe port RX and TX of MCU is entertained, amusement MCU connects SOC module by spi bus, and SOC module transmits total by FPD-LinkLine connects LVDS encoder and video encoder, the port RX and TX of GPS and big-dipper satellite locating module connection SOC module, figureVideo encoder is connected as collector passes through CVBS signal wire, the first CAN transceiver passes through CAN bus and connects the 2nd CAN transmitting-receivingDevice, by the second CAN transceiver receive from the first CAN transceiver transmission come image data, and be transmitted to amusement MCU progressSOC module is sent into after pretreatment, while SOC module receives the virtual navigation that GPS and big-dipper satellite locating module transmitMap roadmap data, by video encoder treated vehicle body periphery live video data and by BCM body control systemFour received partial datas are carried out Data Fusion by the real-time vehicle status information data transmitted, SOC moduleVirtual navigation map route guiding model is generated afterwards to be added on true road, to realize the function of ADAS, and is passed throughLVDS signal data is formed after LVDS coder processes to be transmitted in full liquid crystal instrument by LVDS signal wire.
The full liquid crystal instrument includes instrument MCU, third CAN transceiver, LVDS decoder, digital zoom input, liquidCrystal display screen, control button, third CAN transceiver connect the port RX and TX of instrument MCU, and instrument MCU is connected by spi busDigital zoom input, LVDS decoder connect digital zoom input, image display driving by FPD-Link transfer busDevice connects liquid crystal display, and control button connects the port I/O of instrument MCU, and the connection of the first CAN transceiver is connected by CAN busThird CAN transceiver is connect, LVDS encoder connects LVDS decoder by dual-channel LVDS signal wire.LVDS decoder receives numberIt is sent to after in digital zoom input and display screen progress AR panoramic table is on the one hand driven to show, be on the other hand sent into instrumentData clock synchronization process is carried out in MCU, while instrument MCU also receives the relevant information from control button input, and will letterBreath is sent into display screen by digital zoom input after being handled and is shown.
The BCM body control system includes BCM(car body controller) it module, PEPS(keyless entry and is opened without keyDynamic system) module, ABS(anti-lock braking system) module, ECU(electronic control unit/car running computer) module, BCM module,PEPS module, ABS module and ECU module pass through CAN bus and connect second and third CAN transceiver.BCM body control systemBy all status informations of vehicle (including speed state, car light state, engine condition, vehicle door status, tire pressure state, skylightState, boot state etc.) it is sent in Vehicular entertainment navigation system and full liquid crystal instrument by CAN bus and is handled and shownShow.
The amusement MCU and instrument MCU is all made of RH850 microprocessor and is designed, and RH850 microprocessor uses double-coreRH850G3 processor, CPU frequency maximum 120MHz support memory protection unit, support double-precision floating point unit (FPU), branchIt holds memory protection unit (MPU), supports internal peripherals protection (IPG), support processor elements protection (PEG), haveThe advantages that low-power consumption, high throughput and a variety of peripheral functionalities.SOC module is designed using J6-DRA725 processor, J6-DRA725 Infotainment application processor can meet the dense process demand of present information car for amusement experience, can be extended to upwardsDRA74x device, it is upper all with corresponding software frame and SDK in the application such as Infotainment, instrument, ADAS, support 1920*720 3D QNX pattern number instrument and the full-scale high definition Android information entertainment of 1920*1080, pass through operating system layerThe isolation realization in face is not interfere with each other.It can be used as 3D by DSP with superelevation integrated level and look around hardware accelerator, it can be in not shadowRing Infotainment, instrument show and interactive application under the premise of, realize and additional look around display.It can also be achieved AR technology, pass throughAcquisition, identification to camera to outdoor scene, and driving auxiliary information is added to outdoor scene using upper.Also it is integrated with ISP, Ke YijieAbout fall traditional ISP chip, reduces whole system BOM.Digital zoom input is designed using IMX6D industrial control board, first toThird CAN transceiver uses TJA1042 CAN transceiver, which is specially designed for the high-speed CAN application of automobile industry,The function of sending and receiving differential signal, low electromagnetic (EME) can be provided for (in microcontroller) CAN protocol controllerWith high electromagnetic interference tolerance (EMI), there is extremely low electric current standby mode, and there is host and bus arousal function.
As shown in figure 3, above system carries out the method that instrument AR panoramic table is shown, ADAS information acquisition system is utilizedSighting distance image-forming module obtains real time panoramic image and the millimetre-wave radar imaging on vehicle body periphery, is sent to Vehicular entertainment navigation systemIn by amusement MCU and the data analysis module that forms of SOC module, after data analysis module receives data, acquisition and will captureTo image in kind (barrier/object) do preset display label after, the blending algorithm progress of combined data analysis moduleContinue real-time tracing, while by the displacement information of image material object and vehicle running state information binding analysis, to analysis institute's totalAlgorithm fusion is carried out according to GPS and Beidou navigation data/BCM data, in conjunction with barrier speed in vehicle current driving speed, imageDegree, object speed carry out fusion calculation and obtain vehicle and barrier/object relative distance, by institute obtained by fusion calculationThere are data, and sends the data to full liquid crystal instrument, full liquid crystal instrument fusion real time panoramic image and the fused data of real vehicleAR real time panoramic image co-registration is carried out again, and full liquid crystal instrument is real with the AR in instrument MCU again by real vehicle status information againWhen parsing module merge, finally show AR image in the display screen of full liquid crystal instrument, while liquid crystal instrument provides accordingly entirelyDanger early warning and prompting.
This system is will to acquire real time road information by the sighting distance image-forming module of ADAS information acquisition system, and by the realityScape image is integrated in traditional map navigation, carries out fusion calculation by reuse algorithm and real-time on-site traffic information, is generated emptyQuasi- navigation map route map guides model to be added on true road, more acurrate with guiding driver, safer, more intuitiveGround drives vehicle, and when vehicle is in dangerous warning line, reminds driver to remove a hidden danger and imminent danger in time.
This system is by the deviation system (LDW) in ADAS, automatic emergency brake system (AEB), front truck collision reportAlert system (FCW), adaptive cruise function (ACC), traffic jam auxiliary (TJA) and the full frame of Vehicular entertainment navigation system are ledFunction of navigating carries out aggregation of data calculation process by LVDS communication and CAN communication, to provide for vehicle stable intuitive, clear completeThe AR of face and safety is shown.Wherein, deviation system is the Road that real-time vehicle running is acquired by sighting distance image-forming moduleTraffic lane line image on diameter carries out image procossing by SOC module and virtual navigation map road graticule compares, and working as vehicleWhen head direction or vehicle body live conductor etc. are deviateed, or when having the tendency that deviation, is issued by full liquid crystal instrument and combine AR and ADAS skillThe image of art is reminded, and shows automotive run-off-road image, is issued early warning when can be potentially encountered danger, on full liquid crystal instrument and is mentionedShow signal, while can also issue corresponding early warning sound in Vehicular entertainment navigation system.
Automatic emergency brake system is the traffic behavior letter that driving front vehicles or pedestrian are acquired by sighting distance image-forming moduleBreath, when be in an emergency or calculated by SOC module data fusion will have emergency occurrence tendency when, full liquid crystalInstrument is sent out an emergency signal by rapid fusion calculations incorporated SOC module and gives emergency braking controller, and emergency brake of vehicle is carried out,Image can be shown on full liquid crystal instrument simultaneously, and is combined outdoor scene traffic behavior to issue highest safety measure signal and given BCM vehicle body controlSystem processed, BCM body control system receive and after confirmation signals, issue emergency brake command, and hydraulic driving system carries out urgentBraking.When receiving the highest safety signal of SOC module sending when full liquid crystal instrument, in conjunction with the letter live on the spot of Image AcquisitionBreath carries out the risk object for showing there may be or having generated after data fusion calculating.
Front truck collision warning system is to acquire real vehicle front vehicles and pedestrian and road conditions by sighting distance image-forming module, urgentThe data such as combined high precision navigation information and real vehicle travel speed carry out fusion calculation when turning, front vehicles or row occur working asThe safe distance of people is unsatisfactory for requirement and when there is collision potential range, and anti-collision warning is issued in full liquid crystal instrument, and combines real(such as travel speed, GPS and Beidou position longitude and latitude, road-scene, all around other vehicles or pedestrians to vehicle real-time status dataTravel speed and distance etc.) carry out showing full scale vehicle collision early warning image by full liquid crystal instrument after fusion calculation processing, and issueEarly warning sound.
Adaptive cruise function is to acquire vehicle traveling lane state traffic information, front truck road by sighting distance image-forming moduleWith auxiliary road driving states information, the real vehicle real time running speed provided in conjunction with BCM body control system carries out fusion calculation processingReal-time display real vehicle is and real at a distance from target vehicle (all vehicles specifically immediately ahead of real vehicle) on full liquid crystal instrument afterwardsWhen adjust running speed, with the safe distance for keeping running speed and between target vehicle.Specific speed adjusts logic1) full liquid crystal instrument can also provide in real time in real-time display real vehicle running speed and while with the range image of target vehicleDanger early warning image, while prompt tone is issued by Vehicular entertainment navigation system and carries out early warning, driver is receiving full liquid crystal instrumentPrompt image and Vehicular entertainment navigation system early warning sound when, it is artificial to carry out speed adjustment；2) working as SOC module early warningMay dangerous driving trend or when being in dangerous driving state, generate corresponding deceleration instruction and give BCM Body ControlSystem is to carry out forced deceleration.
Traffic jam auxiliary is the GPS provided by Vehicular entertainment navigation system and Beidou navigation data in full liquid crystal instrumentMiddle offer city peak period road conditions, emergency road dredge information (such as road ahead has congestion, and road ahead, which has, promptly repairs the roads, orHave road emergency episode processing road closure period etc.) and city hollow road scene information etc. carry out early warning, to assist drivingPerson plans optimal traffic route in advance.
Automobile AR instrument system proposed by the invention passes through the AR skill in full liquid crystal instrument and Vehicular entertainment navigation systemArt and Embedded A/D AS interconnection technique pass through cleverly algorithm fusion, and by fused AR panoramic picture in full liquid crystal instrumentIt shows, provides more intuitive, clear, accurate and safe lane situation, traffic marking instruction, safe distance between vehicles to driverIt prompts and specifically and accurately real-time road condition information is reminded, it is time saving and laborsaving, to instruct and assist the more accurate safety of driverGround drives vehicle, and when vehicle is in dangerous warning line, reminds driver to remove a hidden danger and imminent danger in time,To avoid serious dangerous driving.
The technical means disclosed in the embodiments of the present invention is not limited only to technological means disclosed in above embodiment, further includesTechnical solution consisting of any combination of the above technical features.It should be pointed out that for those skilled in the artFor, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also considered asProtection scope of the present invention.