Summary of the invention
It is an object of the invention to provide the method and device of a kind of car plate detection, utilization first corrects and identifies afterwards, reduces angleDegree tilts the impact on recognizer, increases the discrimination of Car license recognition；Make full use of car plate spatial positional information and characterKind of information, strengthens the algorithm recall rate to car plate, it is to avoid flase drop.
For solving above-mentioned technical problem, the present invention provides a kind of method that car plate detects, including:
Obtain the image information that image capture device gathers；
Utilize character machining device that described image information is detected, obtain the position, region of each character in described image informationConfidence breath and the character kind of information of correspondence；
Zone position information according to each described character and the character kind of information of correspondence and car plate rule are to described imageCar plate in information is fitted obtaining matching car plate, and described matching car plate carries out License Plate obtains License Plate knotReally；
According to described License Plate result, described car plate is carried out angle correct；
Described car plate after correction is carried out character recognition, obtains described license board information.
Wherein, regular to described according to the zone position information of each described character and the character kind of information of correspondence and car plateCar plate in image information is fitted obtaining matching car plate, including:
Zone position information according to each described character and the character kind of information of correspondence, determine chinese character and correspondenceThe zone position information of described chinese character；
Whether alphabetic character is there is in the range of the zone position information preset distance of chinese character described in judging distance；
If existing, then according to the zone position information of described chinese character and the zone position information of the alphabetic character of correspondenceDetermine predetermined car plate direction；
Described predetermined car plate direction determines the zone position of the alphabetic character of remaining five alphabetic character and correspondenceBreath, obtains meeting the matching car plate of seven characters of car plate rule.
Wherein, described matching car plate is carried out License Plate and obtains License Plate result, including:
According to the zone position information of each character in described matching car plate, ask for the boundary rectangle of described matching car plate；
Described boundary rectangle is enlarged according to predetermined ratio, obtains License Plate result.
Wherein, according to described License Plate result, described car plate is carried out angle correct, including:
Determine the central point of the zone position information of each character in described matching car plate, fit to according to each described central pointFirst straight line, and the level inclination of car plate is determined according to described first straight line and horizontal angle；
Calculate the vertical dip angle of each character in described matching car plate, and will be according to the calculated meansigma methods of each vertical dip angleVertical dip angle as car plate；
According to described level inclination and described vertical dip angle, utilize affine transformation that described car plate is carried out angle correct.
Wherein, the training method of described character machining device, including:
Obtain the character data of various different angles as nominal data；
Obtain multiple character image datas of various different visual angles as training sample；
According to described nominal data, utilize CNN convolutional neural networks model that described training sample is trained, obtain wordSymbol detector.
The present invention also provides for the device of a kind of car plate detection, including:
Acquisition module, for obtaining the image information that image capture device gathers；
Character machining module, is used for utilizing character machining device to detect described image information, obtains described image letterThe zone position information of each character and the character kind of information of correspondence in breath；
Car plate matching locating module, for the zone position information according to each described character and the character kind of information of correspondenceWith car plate rule, the car plate in described image information is fitted obtaining matching car plate, and described matching car plate is carried out car plateLocation obtains License Plate result；
VLP correction module, for carrying out angle correct according to described License Plate result to described car plate；
Recognition of License Plate Characters module, for the described car plate after correction is carried out character recognition, obtains described license board information.
Wherein, described car plate matching locating module includes car plate fitting unit, and wherein, described car plate fitting unit includes:
Chinese character subelement, for the zone position information according to each described character and the character kind of information of correspondence,Determine the zone position information of the described chinese character of chinese character and correspondence；
First alphabetic character judgment sub-unit, for the zone position information preset distance of chinese character described in judging distanceIn the range of whether there is alphabetic character；
Car plate direction determines subelement, if for existing, then according to zone position information and the correspondence of described chinese characterThe zone position information of alphabetic character determine predetermined car plate direction；
Car plate matching subelement, for determining remaining five alphabetic characters and word of correspondence on described predetermined car plate directionThe zone position information of alphabetic character, obtains meeting the matching car plate of seven characters of car plate rule.
Wherein, described car plate matching locating module includes License Plate unit, and wherein, described License Plate unit includes:
Boundary rectangle obtains subelement, for according to the zone position information of each character in described matching car plate, asks for instituteState the boundary rectangle of matching car plate；
License Plate subelement, for being enlarged according to predetermined ratio by described boundary rectangle, obtains License Plate knotReally.
Wherein, described VLP correction module, including:
Level inclination unit, for determining the central point of the zone position information of each character in described matching car plate, according toEach described central point fits to the first straight line, and determines that the level of car plate is inclined according to described first straight line and horizontal angleAngle；
Vertical dip angle unit, for calculating the vertical dip angle of each character in described matching car plate, and vertically will incline according to eachThe calculated meansigma methods in angle is as the vertical dip angle of car plate；
VLP correction unit, for according to described level inclination and described vertical dip angle, utilizes affine transformation to described carBoard carries out angle correct.
Wherein, also include: training module, for described character machining device is trained；Specifically include:
Nominal data unit, for obtaining the character data of various different angles as nominal data；
Training sample unit, for obtaining multiple character image datas of various different visual angles as training sample；
Training unit, for according to described nominal data, utilizes CNN convolutional neural networks model to enter described training sampleRow training, obtains character machining device.
The method of car plate provided by the present invention detection, including: obtain the image information that image capture device gathers；UtilizeImage information is detected by character machining device, obtains the zone position information of each character in image information and the character kind of correspondenceCategory information；Zone position information according to each character and the character kind of information of correspondence and car plate rule are to the car in image informationBoard is fitted obtaining matching car plate, and matching car plate is carried out License Plate obtains License Plate result；According to License PlateResult carries out angle correct to car plate；Car plate after correction is carried out character recognition, obtains license board information；
The method uses first to correct and identifies afterwards, reduces angle and tilts the impact on recognizer, increases Car license recognitionDiscrimination；And take full advantage of car plate spatial positional information and character kind of information carries out car plate matching and License Plate, increaseThe strong algorithms recall rate to car plate, it is to avoid flase drop；Adapt to the detection of the vehicle license plate to multiple visual angle；Solve to examine on roadWhen surveying license board information, because the restriction of camera position, there is the biggest angle of inclination in the car plate of collection, causes detection vehicle carThe problem that board information is failed；The invention also discloses the device of car plate detection, there is the effect above, do not repeat them here.
Detailed description of the invention
The core of the present invention is to provide the method and device of a kind of car plate detection, and utilization first corrects and identifies afterwards, reduces angleDegree tilts the impact on recognizer, increases the discrimination of Car license recognition；Make full use of car plate spatial positional information and characterKind of information, strengthens the algorithm recall rate to car plate, it is to avoid flase drop.
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearer, below in conjunction with the embodiment of the present inventionIn accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment isThe a part of embodiment of the present invention rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the artThe every other embodiment obtained under not making creative work premise, broadly falls into the scope of protection of the invention.
Refer to the flow chart of the method for the car plate detection that Fig. 1, Fig. 1 are provided by the embodiment of the present invention；The method is permissibleIncluding:
The image information that S100, acquisition image capture device gather；
Wherein, the method for this car plate detection may be used for the process of the vehicle image captured, it is also possible to for regarding of gatheringFrequently the process of two field picture；The most here image information can be video image information can also be picture image information.HereinThe concrete form of image information and image capture device is not defined.
S110, utilize character machining device that described image information is detected, obtain each character in described image informationThe character kind of information of zone position information and correspondence；
Wherein, this step, primarily to detect the character in image information, obtains each in image informationThe zone position information of character and the character kind of information of correspondence；The such as characters on license plate of China can be divided into three kinds, Chinese CharacterSymbol, alphabetic character and numerical character.All characters in image can be detected during whole image information detects,Detection can obtain the zone position information of character and the kind of information of character.
Herein for improving further to various characters especially recall rate to wide-angle character, it is possible to use each classCharacter machining device is trained by the training sample data of the magnanimity of oblique viewing angle.But the not tool to training character machining deviceBody Model is defined；Preferably, the training method of character machining device, including:
Obtain the character data of various different angles as nominal data；
Obtain multiple character image datas of various different visual angles as training sample；
According to described nominal data, utilize CNN convolutional neural networks model that described training sample is trained, obtain wordSymbol detector.
Wherein, character machining device learns CNN convolutional neural networks model construction according to the degree of depth, can utilize each during trainingPlant the mass data of visual angle kinds of characters as training sample.Because each class oblique viewing angle has the number of training of magnanimityAccording to, the nominal data of utilization has the character data of various different angles, so detection model i.e. tilts word to wide-angle characterSymbol has good recall rate.
When utilizing character machining device that image information is detected, if image failing find character information, then under carrying outOne frame analysis, until spreading out of number and the correspondence position information of character after finding character information.
S120, according to the zone position information of each described character and the character kind of information of correspondence and car plate rule to describedCar plate in image information is fitted obtaining matching car plate, and described matching car plate is carried out License Plate obtains License PlateResult；
Wherein, here according to the zone position information of each character and the character kind of information of correspondence and car plate rule to imageWhen car plate in information is fitted and positions, to be fitted with specific reference to the car plate rule that every country is concrete；HereThe character quantity of car plate of rule various countries to be comprised, character kind and the queueing discipline of character.According to such arrangement ruleThen carry out the character location information obtained and kind of information selecting until being determined for compliance with a string character letter of all car plates ruleA string character information that breath will determine is as the license board information of matching, it is possible to utilize the position of a string character information determinedInformation carries out License Plate.
Concrete process may is that
From the zone position information of each described character and the character kind of information of correspondence, first is determined according to car plate ruleSpcial character；
The second spcial character is determined in the preset range of the first spcial character described in distance determined by；
In the preset range of the front character once determined, character late is determined successively, until obtaining according to car plate ruleMeet the matching car plate of car plate rule；
Or owing to characters on license plate carries out horizontally arranged according to a direction, accordingly it is also possible to special according to firstCharacter and the second spcial character determine predetermined car plate direction, carry out the position kind of other characters in the range of the direction determinesDetermine, thus obtain matching car plate.
Such as comprise 7 characters at China Coach car plate, and first character is the Chinese character representing province, second characterFor alphabetic character, remaining five is alphabetic character or numerical character；It is preferred, therefore, that according to the position, region of each described characterCar plate in described image information is fitted obtaining matching car by confidence breath and the character kind of information of correspondence and car plate ruleBoard, including:
Zone position information according to each described character and the character kind of information of correspondence, determine chinese character and correspondenceThe zone position information of described chinese character；
Whether alphabetic character is there is in the range of the zone position information preset distance of chinese character described in judging distance；
If existing, then according to the zone position information of described chinese character and the zone position information of the alphabetic character of correspondenceDetermine predetermined car plate direction；
Described predetermined car plate direction determines the zone position of the alphabetic character of remaining five alphabetic character and correspondenceBreath, obtains meeting the matching car plate of seven characters of car plate rule.
Wherein, the occurrence of preset distance according to distance between characters on license plate in practical situation and can incline variousIn the case of Xie, the actual value of this distance is determined.Here on predetermined car plate direction, determine remaining five alphabetic characters and correspondenceThe zone position information of alphabetic character time can be on the line direction from chinese character to alphabetic character.
Wherein, when chinese character more than one, can proceed as described above successively, it is also possible to carry out above-mentioned mistake side by sideJourney.
Detailed process may is that
So when car plate matching, the first step is location chinese character, searches out Chinese Character in the most all charactersSymbol, if can not find, directly carrying out next frame analysis, if finding chinese character, at search word alphabetic character about, determiningMay determine that the general orientation of car plate after the two character, look for remaining five characters in corresponding orientation.To eachChinese character performs above operation, repeats this process n times, completes all car plate matchings.So complete seven characters on license plateMatching and the location of each character.
I.e. from the beginning of a Chinese character position, finding a neighbouring English alphabet about.According to Chinese character and letterSeat relatively speculates the position of character late, and searches in character machining result.Successively according to it has been determined that character pre-Survey the position of character late.If 5 letter or numbers can be found continuously, the most successfully simulate a car plate.
Next need that described matching car plate is carried out License Plate and obtain License Plate result, every according to matching car plateThe zone position information of individual character may determine that the zone position information of the position shared by this car plate, in order to ensure that car plate position is believedCease completely can also carry out a certain degree of expansion as License Plate result using the zone position information of this shared position.Preferably, described matching car plate is carried out License Plate and obtains License Plate result, including:
According to the zone position information of each character in described matching car plate, ask for the boundary rectangle of described matching car plate；
Described boundary rectangle is enlarged according to predetermined ratio, obtains License Plate result.
Wherein, after obtaining meeting the character that car plate is regular, ask for according to the zone position information of character each in matching car plateBoundary rectangle also extends in proportion to surrounding, obtains License Plate result.
Here predetermined ratio is actually needed according to user and is determined, and does not the most limit concrete proportional numerical valueFixed.
S130, according to described License Plate result, described car plate is carried out angle correct；
Wherein, the matching car plate obtained through said process is the car plate with certain inclination angle, in order to improve dump carThe recall rate of board and detection effect, to carry out angle correct here, form the matching car plate of a level matching car plate.I.e. rootCar plate and horizontal line and/or the inclination angle of vertical curve is may determine that, then according to these inclination angles to car plate angle according to License Plate resultDegree is corrected.Preferably, according to described License Plate result, described car plate is carried out angle correct, including:
Determine the central point of the zone position information of each character in described matching car plate, fit to according to each described central pointFirst straight line, and the level inclination of car plate is determined according to described first straight line and horizontal angle；
Calculate the vertical dip angle of each character in described matching car plate, and will be according to the calculated meansigma methods of each vertical dip angleVertical dip angle as car plate；
According to described level inclination and described vertical dip angle, utilize affine transformation that described car plate is carried out angle correct.
Wherein, the level inclination of car plate can simulate straight line by the central point of characters on license plate and obtain, car plateVertical dip angle by calculate single characters on license plate vertical dip angle, 7 angles can be obtained, then these angles are averagingValue can be obtained by the vertical dip angle of whole car plate, as shown in Figure 2；Affine change can be utilized according to level angle and vertical angleChange and car plate is corrected.
S140, to correction after described car plate carry out character recognition, obtain described license board information.
Wherein, characters on license plate can be identified by character recognition here according to OCR model, obtains license plate recognition resultI.e. license board information.
Turn next to Fig. 3, it is provided that a kind of method that concrete oblique viewing angle Car license recognition judges；First have to character is supervisedSurvey device to initialize, after having initialized, the frame of video obtained is carried out character machining and classification, obtain according to monitoring resultChinese character number and positional information, obtain each chinese character of N；Now cross and do not get chinese character next frame to be carried outThe process of video frame images；Obtain a chinese character position, to search word alphabetic character around this chinese character, by this Chinese CharacterSymbol and alphabetic character determine car plate direction in space, search for remaining five car plate according to car plate spatial information on the basis of result aboveCharacter, if not finding remaining five character, does not fits to car plate, and utilize second repeat to n-th chinese character position onProcess of stating is until to remaining five characters on license plate；If finding remaining five characters on license plate, then navigate in license plate area and car plateEach character zone, and according to the license plate sloped angle of character zone position matching, carry out affine transformation, and to the image after conversionCarry out Recognition of License Plate Characters, export recognition result, complete the detection of oblique viewing angle car plate.
Based on technique scheme, the method for the car plate detection that the embodiment of the present invention provides, after the method uses first correctionIdentify, reduce angle and tilt the impact on recognizer, increase the discrimination of Car license recognition；And it is empty to take full advantage of car plateBetween positional information and character kind of information carry out car plate matching and License Plate, strengthen the algorithm recall rate to car plate, it is to avoidFlase drop；Adapt to the detection of the vehicle license plate to multiple visual angle；When solving to detect license board information on road, because camera positionRestriction, there is the biggest angle of inclination in the car plate of collection, causes the problem that detection vehicle license plate information is failed.
There is the biggest angle of inclination for car plate, cause the problem identifying that license board information is failed, (such as road boundary detection car plateTime) the method has the advantage that the first employing video analysis mode realizes high inclination-angle Car license recognition, has saved hardware costs,Reduce labor cost.Second employing is first corrected and is identified afterwards, reduces angle and tilts the impact on recognizer, increases carThe discrimination of board identification.3rd takes full advantage of car plate spatial positional information and character kind of information, strengthens algorithm to car plateRecall rate, it is to avoid flase drop.4th character machining utilizes the degree of depth to learn CNN convolutional neural networks model, utilizes in the training stageKinds of characters data under various visual angles, increase the algorithm recall rate to big tilted character.
The method embodiments providing car plate detection, utilization first corrects and identifies afterwards, reduces angle and tilts knowledgeThe impact of other algorithm, increases the discrimination of Car license recognition.
The device of the car plate detection provided the embodiment of the present invention below is introduced, the dress of car plate described below detectionPutting the method detected with above-described car plate can be mutually to should refer to.
Refer to the structured flowchart of the device of the car plate detection that Fig. 4, Fig. 4 are provided by the embodiment of the present invention.This device canTo include:
Acquisition module 100, for obtaining the image information that image capture device gathers；
Character machining module 200, is used for utilizing character machining device to detect described image information, obtains described imageThe zone position information of each character and the character kind of information of correspondence in information；
Car plate matching locating module 300, for the zone position information according to each described character and the character kind of correspondenceInformation and car plate rule are fitted obtaining matching car plate to the car plate in described image information, and carry out described matching car plateLicense Plate obtains License Plate result；
VLP correction module 400, for carrying out angle correct according to described License Plate result to described car plate；
Recognition of License Plate Characters module 500, for the described car plate after correction is carried out character recognition, obtains described car plate letterBreath.
Optionally, described car plate matching locating module 300 includes car plate fitting unit, wherein, described car plate fitting unitIncluding:
Chinese character subelement, for the zone position information according to each described character and the character kind of information of correspondence,Determine the zone position information of the described chinese character of chinese character and correspondence；
First alphabetic character judgment sub-unit, for the zone position information preset distance of chinese character described in judging distanceIn the range of whether there is alphabetic character；
Car plate direction determines subelement, if for existing, then according to zone position information and the correspondence of described chinese characterThe zone position information of alphabetic character determine predetermined car plate direction；
Car plate matching subelement, for determining remaining five alphabetic characters and word of correspondence on described predetermined car plate directionThe zone position information of alphabetic character, obtains meeting the matching car plate of seven characters of car plate rule.
Optionally, described car plate matching locating module 300 includes License Plate unit, wherein, described License Plate unitIncluding:
Boundary rectangle obtains subelement, for according to the zone position information of each character in described matching car plate, asks for instituteState the boundary rectangle of matching car plate；
License Plate subelement, for being enlarged according to predetermined ratio by described boundary rectangle, obtains License Plate knotReally.
Optionally, described VLP correction module 400, including:
Level inclination unit, for determining the central point of the zone position information of each character in described matching car plate, according toEach described central point fits to the first straight line, and determines that the level of car plate is inclined according to described first straight line and horizontal angleAngle；
Vertical dip angle unit, for calculating the vertical dip angle of each character in described matching car plate, and vertically will incline according to eachThe calculated meansigma methods in angle is as the vertical dip angle of car plate；
VLP correction unit, for according to described level inclination and described vertical dip angle, utilizes affine transformation to described carBoard carries out angle correct.
Based on any of the above-described embodiment, this device also includes: training module, for instructing described character machining devicePractice；Specifically include:
Nominal data unit, for obtaining the character data of various different angles as nominal data；
Training sample unit, for obtaining multiple character image datas of various different visual angles as training sample；
Training unit, for according to described nominal data, utilizes CNN convolutional neural networks model to enter described training sampleRow training, obtains character machining device.
Wherein, based on each embodiment above-mentioned, this device can be integrated in intellectual analysis one ball machine, or embeds intelligenceIn energy analytical equipment, embedded intelligence analytical equipment can be connected formation car plate detecting system by switch with common ball machine.
In description, each embodiment uses the mode gone forward one by one to describe, and what each embodiment stressed is real with otherExecuting the difference of example, between each embodiment, identical similar portion sees mutually.For device disclosed in embodimentSpeech, owing to it corresponds to the method disclosed in Example, so describe is fairly simple, relevant part sees method part explanation?.
Professional further appreciates that, in conjunction with the unit of each example that the embodiments described herein describesAnd algorithm steps, it is possible to electronic hardware, computer software or the two be implemented in combination in, in order to clearly demonstrate hardware andThe interchangeability of software, the most generally describes composition and the step of each example according to function.TheseFunction performs with hardware or software mode actually, depends on application-specific and the design constraint of technical scheme.SpecialtyTechnical staff specifically should can be used for using different methods to realize described function to each, but this realization should notThink beyond the scope of this invention.
The method described in conjunction with the embodiments described herein or the step of algorithm can direct hardware, processor be heldThe software module of row, or the combination of the two implements.Software module can be placed in random access memory (RAM), internal memory, read-only depositReservoir (ROM), electrically programmable ROM, electrically erasable ROM, depositor, hard disk, moveable magnetic disc, CD-ROM or technologyIn any other form of storage medium well known in field.
The method and device detected car plate provided by the present invention above is described in detail.Tool used hereinPrinciple and the embodiment of the present invention are set forth by body example, and the explanation of above example is only intended to help and understands thisBright method and core concept thereof.It should be pointed out that, for those skilled in the art, without departing from the present inventionOn the premise of principle, it is also possible to the present invention is carried out some improvement and modification, these improve and modification also falls into right of the present inventionIn the protection domain required.