A kind of automated driving system and method based on monocular cam and raspberry pieTechnical field
The invention belongs to the technical fields of image procossing, and in particular to it is a kind of run in raspberry pie based on monocular imageThe automated driving system and method for the depth convolutional neural networks of head.
Background technique
Currently, automatic Pilot technology is fast-developing, it is typically based on more mesh cameras or laser radar, and needRun on the server of height configuration, still, the computing capability that traditional raspberry pie provides is smaller, can not provide tradition nobody driveComputing capability needed for sailing technology, in hardware situation not abundant, i.e., in the case where only providing a monocular cam, byMore detailed traffic information can not be provided in model car traveling in monocular cam, therefore, it is desirable to realize automatic Pilot justIncreasingly complex neural network is needed to handle the image of monocular cam shooting, and still, raspberry pie just as stated earlier providesComputing capability be limited, therefore, more complicated neural network can not be run in raspberry pie, currently without suitable mindThe balance that the two may be implemented through network model is generally imaged using binocular camera or three mesh in existing network modelHead is assisted using modes such as other ancillary equipments such as radar with simple multilayer convolutional neural networks, and Lai Shixian is in raspberry pie meterAutomatic Pilot in calculation ability, but hardware cost is increased in this way, so in the hardware foundation of monocular cam and raspberry pieOn cannot achieve automatic Pilot at this stage.
Therefore, it is badly in need of a kind of scheme that automatic Pilot can be realized under simple environment.
Summary of the invention
In view of the deficiencies of the prior art, the present invention provides a kind of automatic Pilot system based on monocular cam and raspberry pieSystem, the automatic Pilot method based on monocular cam and raspberry pie that the present invention also provides a kind of；
Term is explained:
OpenCV is the cross-platform computer vision library based on BSD license (open source) distribution, may operate inIn Linux, Windows, Android and Mac OS operating system.Its lightweight and efficiently -- by a series of C functions and a small amount ofC++ class is constituted, while providing the interface of the language such as Python, Ruby, MATLAB, realizes image procossing and computer visionMany general-purpose algorithms of aspect.
The technical solution of the present invention is as follows:
A kind of automated driving system based on monocular cam and raspberry pie, including sequentially connected data collection module,Data pre-processing unit, depth convolutional neural networks, control unit；
For obtaining model car, (model car is to be provided with monocular cam and raspberry pie to the data collection module4WD model of mind vehicle) operation when traffic information, traffic information refers to road conditions picture, and the traffic information that will acquire is sent toData pre-processing unit；The data pre-processing unit for being pre-processed to the road conditions picture received, refer to successively intoRow gray processing, noise reduction, binaryzation, character cutting and normalized；The depth convolutional neural networks are in training for instructingThe data set for practicing pretreated multiple road conditions pictures composition, obtains mature depth convolutional neural networks, runs in model carIn the process, multiple above-mentioned pretreated road conditions pictures are inputted, the control information of model car, the control packet of model car are obtainedInclude steering direction (turn left or turn right), steering angle (left-hand rotation or how many degree of turning right), throttle size；Described control unit is by modelThe control information of vehicle passes to model car, completes model car automatic Pilot.
Preferred according to the present invention, the data collection module refers to the 4WD intelligence for being provided with monocular cam and raspberry pieEnergy model car, monocular cam obtain road conditions picture for shooting.
Preferred according to the present invention, the traffic information of acquisition is transmitted to data by opencv with the format of byte stream and locates in advanceManage unit.
Preferred according to the present invention, raspberry pie model car provides the model car control interface of hardware aspect, passes throughPython writes interface, and the control information of model car is passed to the hardware interface of model car, completes the control information of model carTransmitting.
A kind of automatic Pilot method based on monocular cam and raspberry pie, for realizing model car on model lane fromIt is dynamic to drive, it comprises the following steps that
(1) data set is collected；Data set includes a large amount of road conditions pictures；
(2) data set pre-processes；
(3) using step (2) pretreated data set training depth convolutional neural networks, maturation is obtained by trainingDepth convolutional neural networks；The road conditions picture of mature depth convolutional neural networks input shooting, the control letter of output model vehicleBreath, the control information of model car include that steering direction (turn left or turn right), steering angle (turn left or how many degree of turning right), throttle are bigIt is small；
(4) the mature depth convolutional neural networks obtained using step (3) training, keep model car enterprising in model laneRow automatic Pilot.
Preferred according to the present invention, the step (1) collects data set, refers to: being provided with monocular cam and raspberry pie4WD model of mind vehicle captured in real-time road conditions, obtain a large amount of road conditions pictures.
It is preferred according to the present invention, the step (2), data prediction, comprising:
A, when obtaining road conditions picture, the speed of model car, throttle size, steering direction, steering angle at this time are recorded in real timeIt spends (angular dimension that model car deviates direction of advance at this time), the title of every road conditions picture of mark, storage position, throttle are bigSmall, speed, steering angle and steering direction are recorded as one, and all records are placed in the same .csv file；
B, gray processing, noise reduction, binaryzation, character cutting and normalizing are successively carried out to the road conditions picture after processing of step AChange processing.
Preferred according to the present invention, depth convolutional neural networks structure is as follows: successively including that a tomographic image normalizes layer, fourThe convolutional layer and four layers of full articulamentum of the convolutional layer of layer 5x5, two layers 3x3；
Image normalization
Convolution:5x5,filter:12,strides:2x2,activation:ELU
Convolution:5x5,filter:24,strides:2x2,activation:ELU
Convolution:5x5,filter:36,strides:2x2,activation:ELU
Convolution:5x5,filter:48,strides:2x2,activation:ELU
Convolution:3x3,filter:64,strides:1x1,activation:ELU
Convolution:3x3,filter:64,strides:1x1,activation:ELU
Drop out(0.5)
Fully connected:neurons:100,activation:ELU
Fully connected:neurons:50,activation:ELU
Fully connected:neurons:10,activation:ELU
Fully connected:neurons:1(output)
The depth convolutional neural networks are convolutional neural networks end to end, and entire depth convolutional neural networks are as blackBox, after inputting road conditions picture, firstly, carrying out image by first layer image normalization layer (normalization layers of Image)Pretreatment, then, after four layers of 5x5 convolutional layer and two layers of 3x3 convolutional layer, picture becomes high-dimensional feature figure, passes through 6The convolution of layer completes the feature extraction to road conditions picture, i.e. local feature ratio in extraction image, such as extracts lane lineBending direction and degree export last deflection ratio finally, by four layers of full articulamentum, deflection ratio is prediction mouldThe type vehicle angle next to be deviateed is turned left divided by 90 degree of obtained values, value positive number representative, and negative representative is turned right, greatlyDivided by 90 degree, full articulamentum is equivalent to the feature for extracting convolution and carries out always the small angular dimension for deviateing current driving direction that representsKnot, and predict next steering angle.
It is preferred according to the present invention, the step (4), the mature depth convolutional Neural obtained using step (3) trainingNetwork makes model car carry out automatic Pilot on model lane, refers to:
C, the road conditions picture in model car operational process by monocular cam shooting is passed to mature depth convolutional Neural netNetwork；
D, mature depth convolutional neural networks complete deduction process, export deflection ratio, deflection ratio is prediction modelThe vehicle angle next to be deviateed is turned left divided by 90 degree of obtained values, value positive number representative, and negative representative is turned right, sizeThe angular dimension for deviateing current driving direction is represented divided by 90 degree；
E, after obtaining deflection ratio, throttle size is obtained by calculating；Model car will be by braking and unclamping oil when excessively curvedDoor reduces speed, adjusts throttle after then adjusting the angle again by crossing, therefore steering angle and throttle are closely bound up, when turningWhen very big to angle throttle must unclamp reduce speed come through bend, vice versa.It so can after predicting deviation angleThrottle size (size is percentages), calculation formula is obtained by calculation are as follows: a=1.0-t^2- (v1 ÷ v2) ^2, aRefer to throttle size, t refers to deflection ratio, and v1 refers to speed, and v2 refers to limitation speed, permits when model car present speed v1 is greater thanWhen maximum speed (speed be artificial setting) perhaps, limitation speed v2 be that (speed is manually sets for the minimum speed that allowsSet), otherwise, limitation speed v2 is the maximum speed allowed；
F, the throttle size of acquisition and deflection ratio are passed into raspberry pie model car, so the steering of Controlling model vehicle andAcceleration-deceleration by obtained throttle size and is biased to angle, is connect by the hardware that python code is transmitted to raspberry pie model carMouthful, after which obtains throttle size and be biased to angle, speed and the direction of model car are adjusted, implementation model vehicle is driven automaticallyIt sails.
The invention has the benefit that
It is expensive that the present invention has abandoned complicated deep layer network model and radar etc. compared with current traditional automatic Pilot algorithmHardware realizes the automatic Pilot under lesser computing capability and poor hardware condition, allows to lesser costAnd cost uses end-to-end nerve by simple visual identity that is, in the case where monocular cam and raspberry pie is used onlyNetwork implementations is unmanned.
Detailed description of the invention
Fig. 1 is the structural block diagram of the automated driving system based on monocular cam and raspberry pie；
Fig. 2 is that the present invention is based on the flow diagrams of monocular cam and the automatic Pilot method of raspberry pie；
Specific embodiment
The present invention is further qualified with embodiment with reference to the accompanying drawings of the specification, but not limited to this.
Embodiment 1
A kind of automated driving system based on monocular cam and raspberry pie, as shown in Figure 1, including sequentially connected dataCollector unit, data pre-processing unit, depth convolutional neural networks, control unit；
For obtaining model car, (model car is the 4WD intelligence for being provided with monocular cam and raspberry pie to data collection moduleCan model car) operation when traffic information, it is pre- that the traffic information that traffic information refers to road conditions picture, and will acquire is sent to dataProcessing unit；Data pre-processing unit refers to for pre-processing to the road conditions picture received and successively carries out gray processing, dropIt makes an uproar, binaryzation, character cutting and normalized；Depth convolutional neural networks in training for train it is pretreated multipleThe data set of road conditions picture composition, obtains mature depth convolutional neural networks, in model car operational process, inputs above-mentioned pre-Treated multiple road conditions pictures, obtain the control information of model car, the control information of model car include steering direction (turn left orTurn right), steering angle (turn left or turn right how many degree), throttle size；The control information of model car is passed to mould by control unitType vehicle completes model car automatic Pilot.
Data collection module refers to that the 4WD model of mind vehicle for being provided with monocular cam and raspberry pie, monocular cam are usedRoad conditions picture is obtained in shooting.
The traffic information of acquisition is transmitted to data pre-processing unit by opencv with the format of byte stream.
Raspberry pie model car provides the model car control interface of hardware aspect, writes interface by python, by modelThe control information of vehicle passes to the hardware interface of model car, completes the transmitting of the control information of model car.
Depth convolutional neural networks structure is as follows: successively including tomographic image normalization layer, the convolutional layer of four layers of 5x5, twoThe convolutional layer and four layers of full articulamentum of layer 3x3；
Image normalization
Convolution:5x5,filter:12,strides:2x2,activation:ELU
Convolution:5x5,filter:24,strides:2x2,activation:ELU
Convolution:5x5,filter:36,strides:2x2,activation:ELU
Convolution:5x5,filter:48,strides:2x2,activation:ELU
Convolution:3x3,filter:64,strides:1x1,activation:ELU
Convolution:3x3,filter:64,strides:1x1,activation:ELU
Drop out(0.5)
Fully connected:neurons:100,activation:ELU
Fully connected:neurons:50,activation:ELU
Fully connected:neurons:10,activation:ELU
Fully connected:neurons:1(output)
The depth convolutional neural networks are convolutional neural networks end to end, and entire depth convolutional neural networks are as blackBox has taken into account the computing capability of raspberry pie and the accuracy of model；After inputting road conditions picture, firstly, returning by the first tomographic imageOne changes layer (Image
Normalization layers) image preprocessing is carried out, then, by four layers of 5x5 convolutional layer and 3x3 volumes of two layersAfter lamination, picture becomes high-dimensional feature figure, by 6 layers of convolution, completes the feature extraction to road conditions picture, that is, extracts figureLocal feature ratio as in, such as the bending direction and degree of lane line are extracted, finally, by four layers of full articulamentum, it is defeatedLast deflection ratio out, deflection ratio are the prediction model vehicle angles next to be deviateed divided by 90 degree of obtained values, shouldThe representative of value positive number is turned left, and negative representative is turned right, and size represents the angular dimension for deviateing current driving direction divided by 90 degree, entirelyArticulamentum, which is equivalent to, summarizes the feature that convolution is extracted, and predicts next steering angle.
Embodiment 2
A kind of automatic Pilot method based on monocular cam and raspberry pie, as shown in Fig. 2, for realizing model car in mouldAutomatic Pilot on type lane, comprises the following steps that
(1) data set is collected；Data set includes about 100,000 or so road conditions pictures；
(2) data set pre-processes；
(3) using step (2) pretreated data set training depth convolutional neural networks, maturation is obtained by trainingDepth convolutional neural networks；The road conditions picture of mature depth convolutional neural networks input shooting, the control letter of output model vehicleBreath, the control information of model car include that steering direction (turn left or turn right), steering angle (turn left or how many degree of turning right), throttle are bigIt is small；
(4) the mature depth convolutional neural networks obtained using step (3) training, keep model car enterprising in model laneRow automatic Pilot.
Step (1) collects data set, and refer to: the 4WD model of mind vehicle for being provided with monocular cam and raspberry pie is clapped in real timeRoad conditions are taken the photograph, a large amount of road conditions pictures are obtained.
Step (2), data prediction, comprising:
A, when obtaining road conditions picture, the speed of model car, throttle size, steering direction, steering angle at this time are recorded in real timeIt spends (angular dimension that model car deviates direction of advance at this time), the title of every road conditions picture of mark, storage position, throttle are bigSmall, speed, steering angle and steering direction are recorded as one, and all records are placed in the same .csv file；
B, gray processing, noise reduction, binaryzation, character cutting and normalizing are successively carried out to the road conditions picture after processing of step AChange processing.
Step (4), the mature depth convolutional neural networks obtained using step (3) training, makes model car in model carAutomatic Pilot is carried out on road, is referred to:
C, the road conditions picture in model car operational process by monocular cam shooting is passed to mature depth convolutional Neural netNetwork；
D, mature depth convolutional neural networks complete deduction process, export deflection ratio, deflection ratio is prediction modelThe vehicle angle next to be deviateed is turned left divided by 90 degree of obtained values, value positive number representative, and negative representative is turned right, sizeThe angular dimension for deviateing current driving direction is represented divided by 90 degree；
E, after obtaining deflection ratio, throttle size is obtained by calculating；Model car will be by braking and unclamping oil when excessively curvedDoor reduces speed, adjusts throttle after then adjusting the angle again by crossing, therefore steering angle and throttle are closely bound up, when turningWhen very big to angle throttle must unclamp reduce speed come through bend, vice versa.It so can after predicting deviation angleThrottle size (size is percentages), calculation formula is obtained by calculation are as follows: a=1.0-t^2- (v1 ÷ v2) ^2, aRefer to throttle size, t refers to deflection ratio, and v1 refers to speed, and v2 refers to limitation speed, permits when model car present speed v1 is greater thanWhen maximum speed (speed be artificial setting) perhaps, limitation speed v2 be that (speed is manually sets for the minimum speed that allowsSet), otherwise, limitation speed v2 is the maximum speed allowed；
F, the throttle size of acquisition and deflection ratio are passed into raspberry pie model car, so the steering of Controlling model vehicle andAcceleration-deceleration by obtained throttle size and is biased to angle, is connect by the hardware that python code is transmitted to raspberry pie model carMouthful, after which obtains throttle size and be biased to angle, speed and the direction of model car are adjusted, implementation model vehicle is driven automaticallyIt sails.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the artFor member, without departing from the principle of the present invention, it can also make several improvements and retouch, these improvements and modifications are also answeredIt is considered as protection scope of the present invention.
Main target of the present invention is to complete automatic Pilot in the raspberry pie of monocular cam and lower computing capability, thereforeThis experiment handles the traffic information of monocular cam shooting using simple end-to-end multilayer convolutional neural networks, thusTo the steering angle information of model car, the throttle size predicted value and speed of model car are obtained by relevant calculation on this basisModel car control information is finally transmitted to model car control module, the acceleration-deceleration of Controlling model vehicle and turning by predicted value.