Summary of the invention
Propose the present invention to solve the above-mentioned problems.According to an aspect of the present invention, a kind of recognition of face side is providedMethod, the face identification method include：Image queue is formed for same object to be identified acquisition facial image；Determine the teamWhether face images are qualified facial image in column；Top-quality face figure is selected in qualified facial imagePicture；And recognition of face is carried out to the top-quality facial image.
In one embodiment of the invention, whether face images are qualified face in the determination queueThe step of image includes：For the acquired facial image of each frame in the queue, determine that at least one the following isNo satisfaction identification requires：The 3 d pose of face in the acquired facial image；The acquired facial imageFog-level；The occlusion state of face in the acquired facial image；The brightness of the acquired facial image.
In one embodiment of the invention, whether full to the 3 d pose, the fog-level, the occlusion stateThe determination that foot identification requires is carried out based on depth convolutional network.
In one embodiment of the invention, determine the face in the acquired facial image 3 d pose whetherMeet identification to require to include：Determine the angle per the one-dimensional positive face of deviation of the face in three dimensions；And it is if describedIt is not more than predetermined threshold per the one-dimensional angle for deviateing positive face, it is determined that the 3 d pose of the face meets identification and requires, conversely,Identification is then unsatisfactory for require.
In one embodiment of the invention, determine whether the fog-level of the acquired facial image meets identificationIt requires to include：Motion blur and Gaussian Blur based on the acquired facial image determine the acquired facial imageFog-level；And if the fog-level of the acquired facial image is not more than predetermined threshold, it is determined that the peopleThe fog-level of face image meets identification and requires, and requires conversely, being then unsatisfactory for identification.
In one embodiment of the invention, determine the face in the acquired facial image occlusion state whetherMeet identification to require to include：Determine whether the key position of the face is blocked；And if the face key positionIt is not blocked, it is determined that the occlusion state of the face in the facial image meets identification and requires, and wants conversely, being then unsatisfactory for identificationIt asks.
In one embodiment of the invention, whether what identification required is met to the brightness of the acquired facial imageDetermination is carried out based on grey level histogram.
In one embodiment of the invention, the face identification method further includes：It lives to the object to be identifiedPhysical examination is surveyed.
In one embodiment of the invention, the In vivo detection is to be based on executing instruction movement to the object to be identifiedFacial image carry out, including judge whether instruction movement qualified and divides using preparatory trained skin elasticityClass device judge the object to be identified execute instruction movement before and after skin area image whether be in skin of living body at leastOne of.
In one embodiment of the invention, the In vivo detection is based on structured light patterns in the object to be identifiedWhat the sub-surface scattering degree in face carried out.
In one embodiment of the invention, the face identification method further includes：Before acquiring facial image orIt is whether suitable that current illumination condition is detected during acquiring facial image, if illumination condition is improper, starts benefitElectro-optical device carries out light filling.
In one embodiment of the invention, the face identification method further includes：Before acquiring facial image first reallyDetermine whether object to be identified enters shooting area.
In one embodiment of the invention, the face identification method further includes：Scanning people is shown to object to be identifiedThe animation of face is acquiring and is identifying facial image with prompt.
In one embodiment of the invention, the face identification method further includes：When failing to acquire in the given timeWhen to qualified facial image or to facial image recognition failures, prompt the object to be identified adjustment posture to resurvey peopleFace image.
In one embodiment of the invention, described to select top-quality facial image packet in qualified facial imageIt includes：Quality score is carried out to the facial image of the qualification, selects the highest facial image of score as top-quality faceImage；Wherein, carrying out quality score to the facial image of the qualification includes：The face figure is determined based on depth convolutional networkThe fog-level of picture, the mass fraction are defined as 1 and subtract fog-level.
According to a further aspect of the invention, a kind of face identification device is provided, the face identification device includes：Image is adoptedCollect module, for forming image queue for same object to be identified acquisition facial image, determines all faces in the queueWhether image is qualified facial image, and selects top-quality facial image to send identification in qualified facial imageModule；And the identification module, for carrying out recognition of face to the top-quality facial image.
In one embodiment of the invention, described image acquisition module is further used for：For every in the queueThe acquired facial image of one frame, determines whether at least one the following meets identification and require：The acquired face figureThe 3 d pose of face as in；The fog-level of the acquired facial image；In the acquired facial imageThe occlusion state of face；The brightness of the acquired facial image.
In one embodiment of the invention, described image acquisition module is to the 3 d pose, the fog-level, instituteStating occlusion state whether to meet the determination that identification requires is carried out based on depth convolutional network.
In one embodiment of the invention, described image acquisition module determines the people in the acquired facial imageWhether the 3 d pose of face, which meets identification, requires to include：Determine the angle per the one-dimensional positive face of deviation of the face in three dimensionsDegree；And if described be not more than predetermined threshold per the one-dimensional angle for deviateing positive face, it is determined that the 3 d pose of the face is fullFoot identification requires, and requires conversely, being then unsatisfactory for identification.
In one embodiment of the invention, described image acquisition module determines the fuzzy of the acquired facial imageWhether degree, which meets identification, requires to include：Described in motion blur and Gaussian Blur based on the acquired facial image determineThe fog-level of acquired facial image；And if the fog-level of the acquired facial image is not more than predetermined thresholdValue, it is determined that the fog-level of the facial image meets identification and requires, and requires conversely, being then unsatisfactory for identification.
In one embodiment of the invention, described image acquisition module determines the people in the acquired facial imageWhether the occlusion state of face, which meets identification, requires to include：Determine whether the key position of the face is blocked；And if instituteThe key position for stating face is not blocked, it is determined that the occlusion state of the face in the facial image meets identification and requires, insteadIt, then be unsatisfactory for identification and require.
In one embodiment of the invention, described image acquisition module is to the brightness of the acquired facial imageThe no determination for meeting identification requirement is carried out based on grey level histogram.
In one embodiment of the invention, the face identification device further includes：In vivo detection module, for describedObject to be identified carries out In vivo detection.
In one embodiment of the invention, the In vivo detection module is based on executing instruction the object to be identified dynamicThe facial image of work carries out In vivo detection, and the In vivo detection includes judging whether the instruction movement is qualified and uses pre-First trained skin elasticity classifier judges that skin area image of the object to be identified before and after executing instruction movement isNo is at least one of skin of living body.
In one embodiment of the invention, the In vivo detection module is based on structured light patterns in the object to be identifiedFace in sub-surface scattering degree carry out In vivo detection.
In one embodiment of the invention, the face identification device further includes：Illumination detection module, for describedWhether current illumination condition is detected before image capture module acquisition facial image or during acquiring facial imageProperly, if illumination condition is improper, start light compensating apparatus and carry out light filling.
In one embodiment of the invention, the face identification device further includes：Apart from detection module, for describedFirst determine whether object to be identified enters shooting area before image capture module acquisition facial image.
In one embodiment of the invention, the face identification device further includes：Display module is used for to be identified rightAs the animation of display scanning face, facial image is being acquired and identified with prompt.
In one embodiment of the invention, the display module is also used to：When failing to collect conjunction in the given timeThe facial image of lattice or when to facial image recognition failures, prompts the object to be identified adjustment posture to resurvey face figurePicture.
In one embodiment of the invention, described image acquisition module is further used for：To the face figure of the qualificationAs carrying out quality score, select the highest facial image of score as top-quality facial image；Wherein, to the qualificationFacial image carries out quality score：The fog-level of the facial image, the quality are determined based on depth convolutional networkScore definition subtracts fog-level for 1.
According to a further aspect of the present invention, a kind of face identification system is provided, the face identification system includes that image passesSensor, storage device and processor, described image sensor are stored on the storage device for acquiring facial image by instituteThe computer program of processor operation is stated, the computer program executes described in any of the above-described when being run by the processorFace identification method.
Another aspect according to the present invention provides a kind of computer-readable medium, stores on the computer-readable mediumThere is computer program, the computer program executes face identification method described in any of the above embodiments at runtime.
Face identification method, device, system and computer-readable medium according to an embodiment of the present invention are in recognition of faceWithout all handling acquired each frame facial image, but only to top-quality face in qualified facial imageImage is handled, and not only can guarantee the accuracy of face recognition result, while also greatly reducing calculation amount, is savedComputing resource, improves recognition efficiency.
Specific embodiment
In order to enable the object, technical solutions and advantages of the present invention become apparent, root is described in detail below with reference to accompanying drawingsAccording to example embodiments of the present invention.Obviously, described embodiment is only a part of the embodiments of the present invention, rather than this hairBright whole embodiments, it should be appreciated that the present invention is not limited by example embodiment described herein.Based on described in the present inventionThe embodiment of the present invention, those skilled in the art's obtained all other embodiment in the case where not making the creative laborIt should all fall under the scope of the present invention.
Firstly, describing the face identification method for realizing the embodiment of the present invention, device, system and calculating referring to Fig.1The exemplary electronic device 100 of machine readable medium.
As shown in Figure 1, electronic equipment 100 include one or more processors 102, it is one or more storage device 104, defeatedEnter device 106, output device 108 and imaging sensor 110, these components pass through bus system 112 and/or other formsThe interconnection of bindiny mechanism's (not shown).It should be noted that the component and structure of electronic equipment 100 shown in FIG. 1 are only exemplary, andUnrestricted, as needed, the electronic equipment also can have other assemblies and structure.
The processor 102 can be central processing unit (CPU) or have data-handling capacity and/or instruction executionThe processing unit of the other forms of ability, and the other components that can control in the electronic equipment 100 are desired to executeFunction.
The storage device 104 may include one or more computer program products, and the computer program product canTo include various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.It is described easyThe property lost memory for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non-Volatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..In the computer readable storage mediumOn can store one or more computer program instructions, processor 102 can run described program instruction, to realize hereafter instituteThe client functionality (realized by processor) in the embodiment of the present invention stated and/or other desired functions.In the meterCan also store various application programs and various data in calculation machine readable storage medium storing program for executing, for example, the application program use and/orThe various data etc. generated.
The input unit 106 can be the device that user is used to input instruction, and may include keyboard, mouse, wheatOne or more of gram wind and touch screen etc..
The output device 108 can export various information (such as image or sound) to external (such as user), andIt may include one or more of display, loudspeaker etc..
Described image sensor 110 can be shot facial image (such as photo, video etc.), and by captured imageIt is stored in the storage device 104 for the use of other components.Image collecting device 110 can be camera.It should be appreciated thatImage collecting device 110 is only example, and electronic equipment 100 can not include image collecting device 110.In this case, may be usedTo utilize other image acquisition device facial images, and the facial image of acquisition is sent to electronic equipment 100.
Illustratively, for realizing face identification method according to an embodiment of the present invention, device, system and computer-readableThe exemplary electronic device of medium may be implemented as smart phone, tablet computer etc..
In the following, face identification method 200 according to an embodiment of the present invention will be described with reference to Fig. 2.
In step S210, image queue is formed for same object to be identified acquisition facial image.
In step S220, determine whether face images are qualified facial image in the queue.
In one embodiment, figure can be formed for same object to be identified acquisition facial image by image collecting deviceJudge whether the face images in image queue are qualified facial image as queue, and by image collecting device.GenerallyGround, image collecting device may be for an object acquisition multiple images or video flowing to be identified, in the embodiment of the present inventionIn, image collecting device, which may not need, to be completely transferred to identification module for its acquired image and handles, and can only byQualified facial image is transmitted to identification module and is handled, and can not only reduce the data volume of transmission in this way, moreover it is possible to reduce and knowThe data volume of other places reason further, since transmission is qualified facial image, therefore can also guarantee the accurate of identifying processingProperty.In other embodiments, determine whether acquired facial image is that qualified this work of facial image can not also be byImage collecting device is completed, and is completed by other modules or device.
In one embodiment, qualified facial image can be understood as the facial image for meeting recognition of face requirement.Such asThe acquired facial image of fruit meets recognition of face requirement, it is determined that acquired facial image is qualified facial image；InsteadIt, if acquired facial image is unsatisfactory for recognition of face requirement, it is determined that acquired facial image is underproof peopleFace image.According to the different application scene of recognition of face, recognition of face requires to be slightly different.Generally, recognition of face is wantedAsking may include the basic demand for enabling to face normally to be identified.
Illustratively, determine that the step of whether acquired facial image is qualified facial image can include determining that downWhether column items, which meet identification, requires：The 3 d pose of face in acquired facial image；Acquired facial imageFog-level (i.e. fuzziness)；The occlusion state of face in acquired facial image；The brightness of acquired facial image.In one embodiment, these are all satisfied identification and require, and just determine that acquired facial image is qualified facial image.?In other embodiments, at least partly satisfaction in these, which identifies, to be required, then acquired facial image can be determined for qualificationFacial image.It describes to determine whether facial image closes in face identification method according to an embodiment of the present invention below with reference to Fig. 3The schematic flow chart of the method 300 of lattice.
As shown in figure 3, determining whether the brightness of acquired facial image meets identification and require in step S310.IfThe brightness of acquired facial image meets identification and requires, then continues to step S320；, whereas if acquired faceThe brightness of image is unsatisfactory for identification and requires, then skips to step S360.
In one embodiment, it can be based on to whether the brightness of acquired facial image meets the determination that identification requiresGrey level histogram carries out.It in one example, can be to the face in facial image in face entirety, eye part, right eye portionDivide and mouth respectively extracts grey level histogram feature, obtains four histograms, calculate aforementioned four histogram and its 30% and 70%The brightness of quantile differs greatly if there is two or more numerical value and normal illumination face corresponding data, is then judged asThe brightness of facial image is unsatisfactory for identification and requires, and is otherwise judged as that satisfaction identification requires.It in other examples, can also be by appointingWhat his suitable mode requires to determine whether the brightness of acquired facial image meets identification.
In step S320, determine whether the fog-level of acquired facial image meets identification and require.If acquiredFacial image fog-level meet identification require, then continue to step S330；, whereas if acquired face figureThe fog-level of picture is unsatisfactory for identification and requires, then skips to step S360.
It in one embodiment, can be with to whether the fog-level of acquired facial image meets determination that identification requiresIt is carried out based on depth convolutional network.In one example, determine whether the fog-level of acquired facial image meets identificationIt is required that may include：Motion blur and Gaussian Blur based on acquired facial image determine the mould of acquired facial imagePaste degree；If the fog-level of acquired facial image is not more than predetermined threshold, it is determined that the fog-level of facial imageMeet identification to require, be required conversely, being then unsatisfactory for identification.It can be implemented based on the good depth convolutional network model of off-line trainingThe process.Wherein, the setting of the predetermined threshold can be based on specific application scenarios.It in other examples, can also be by appointingWhat his suitable mode requires to determine whether the fog-level of acquired facial image meets identification.
In step S330, determine whether the occlusion state of the face in acquired facial image meets identification and require.Such asThe occlusion state of face in the acquired facial image of fruit meets identification and requires, then continues to step S340；Conversely, such asThe occlusion state of face in the acquired facial image of fruit is unsatisfactory for identification and requires, then skips to step S360.
In one embodiment, whether what identification required is met to the occlusion state of the face in acquired facial imageDetermination can be carried out based on depth convolutional network.In one example, blocking for the face in acquired facial image is determinedWhether state meets identification requirement：Determine whether the key position of face is blocked；If the key position of faceIt is not blocked, it is determined that the occlusion state of the face in facial image meets identification and requires, and requires conversely, being then unsatisfactory for identification.Wherein, the key position of face may include at least one of organs such as eyes, mouth.For example, in one example, it can be rightThe eyes and mouth of face carry out shadowing.Using the good depth convolutional network model of off-line training, according to the face figure of inputWhether picture, output three key positions of left-eye/right-eye/mouth are blocked.If any one position is blocked, facial imageIdentification is unsatisfactory for require.In other examples, acquired face figure can also be determined by any other suitable modeWhether the occlusion state of the face as in, which meets identification, requires.
In step S340, determine whether the 3 d pose of the face in acquired facial image meets identification and require.Such asThe 3 d pose of face in the acquired facial image of fruit meets identification and requires, then continues to step S350；Conversely, such asThe 3 d pose of face in the acquired facial image of fruit is unsatisfactory for identification and requires, then skips to step S360.
In one embodiment, whether what identification required is met to the 3 d pose of the face in acquired facial imageDetermination can be carried out based on depth convolutional network.In one example, the three-dimensional of the face in acquired facial image is determinedWhether posture meets identification requirement：Determine the angle per the one-dimensional positive face of deviation of face in three dimensions；IfIt is not more than predetermined threshold per the one-dimensional angle for deviateing positive face, it is determined that the 3 d pose of face meets identification and requires, conversely, then notMeet identification to require.It can implement the process based on the good depth convolutional network model of off-line training.If side face to the left and rightAngle be greater than or equal to predetermined threshold (such as 30 degree), or bow and face upward brilliance degree and be greater than or equal to predetermined threshold (such as 30Degree), it is determined that the 3 d pose of the face in acquired facial image is unsatisfactory for identification and requires.In other examples, may be usedDetermine whether the 3 d pose of the face in acquired facial image meets identification in a manner of suitable by any otherIt is required that.
In step S350, determine that acquired facial image is qualified facial image.
In step S360, determine that acquired facial image is underproof facial image.
It describes above exemplarily and determines whether facial image closes in face identification method according to an embodiment of the present inventionThe schematic flow of the method for lattice.Although being walked it is worth noting that, being described as in the process includes step S310 to S360The sequence of rapid S310 to S340 is merely exemplary and not restrictive, and step S310 to step S340 can be suitable in no particular orderSequence.In actual application, different priority or weight can also be arranged to the judgement of step S310 to step S340,It in addition it is also possible to need not include the whole of these steps, or also may include that other additional steps are further to carry outJudgement, to meet the needs of practical application, the invention is not limited in this regard.
The step of continuing to describe face identification method 200 according to an embodiment of the present invention referring back to Fig. 2 below.
In step S230, top-quality facial image is selected in qualified facial image.
In one embodiment, when for an object to be identified there are when the facial image of multiframe qualification, can be therefromTop-quality facial image is selected to carry out recognition of face for sending identification module to.It in one example, can be to thisA little facial images carry out quality score, and (such as mass fraction can subtract the value that fuzziness obtains for 1, and wherein fuzziness is 0 to 1Between numerical value), it is highest for being handled in subsequent identification step then therefrom to select score, after being further reducedThe calculation amount of continuous processing.Wherein, fuzziness can be used housebroken neural net regression and obtain, for example, one image of inputTo neural network, the fuzzy score of this image is exported as fuzziness.Further, if 3 d pose, coverage extent, lightIf line judgement all qualifications, mass fraction is 1-fuzziness, and otherwise mass fraction is 0-fuzziness, if obtaining one negative point,It is just directly filtered, quality score is carried out to all images in image queue in this way and sorts below, then selectTop-quality facial image.
In other examples, it can also be selected from qualified facial image using other suitable modes top-qualityFacial image.
In one embodiment, step S230 can be implemented by image collecting device, can also be by other modules or dressIt sets to implement.
In step S240, recognition of face is carried out to the top-quality facial image.
In an embodiment of the present invention, recognition of face described in step S240 can be the method for known recognition of face.It will be appreciated, however, that the present invention is not limited by the method for recognition of face, the method for either existing recognition of face or futureThe method of the recognition of face of exploitation can be applied in face identification method 200 according to an embodiment of the present invention, and also answerIncluding within the scope of the present invention.
Based on above description, face identification method according to an embodiment of the present invention is in recognition of face without to acquiredEach frame facial image all handled, but only to top-quality facial image in qualified facial image atReason, not only can guarantee the accuracy of face recognition result, while also greatly reducing calculation amount, save calculating moneySource improves recognition efficiency.
Illustratively, face identification method according to an embodiment of the present invention can be in setting with memory and processorIt is realized in standby, device or system.
In addition, face identification method according to an embodiment of the present invention can also be deployed in server end (or cloud).SubstitutionGround, face identification method according to an embodiment of the present invention can also be deployed in server end (or cloud) and personal terminal with being distributedPlace.
In a further embodiment, face identification method according to an embodiment of the present invention can also include：To be identifiedObject carries out In vivo detection.Wherein the step of In vivo detection can the recognition of face the step of before implement, can also be in faceImplement after the step of identification.Preferably, the recognition of face the step of before implement In vivo detection, to improve the property of recognition of faceEnergy and efficiency.
In a further embodiment, face identification method according to an embodiment of the present invention can also include：In acquisition peopleIt is whether suitable that current illumination condition is detected before face image or during acquiring facial image, if illumination condition is notProperly, then start light compensating apparatus and carry out light filling.
In a further embodiment, face identification method according to an embodiment of the present invention can also include：In acquisition peopleFirst determine whether object to be identified enters shooting area before face image.
In a further embodiment, face identification method according to an embodiment of the present invention can also include：To be identifiedThe animation of object display scanning face, is acquiring and is identifying facial image with prompt.
In a further embodiment, face identification method according to an embodiment of the present invention can also include：When predeterminedFail to collect qualified facial image in time or when to facial image recognition failures, prompts the object to be identified adjustment appearanceState is to resurvey facial image.
These further embodiments can be individually or real together with above-mentioned face identification method 200 in combination with each otherIt applies, it is real according to the present invention to be advanced optimized in terms of improving recognition of face performance, saving computing resource, raisingThe face identification method for applying example is described in detail it below with reference to Fig. 4.
Fig. 4 shows the schematic flow chart of face identification method 400 according to another embodiment of the present invention.As shown in figure 4,Face identification method 400 may include steps of：
In step S410, determine whether object to be identified enters shooting area.If it is determined that object to be identified enters shootingRegion then advances to step S420, conversely, then continuing implementation steps S410 itself.Step S410 can be used for determine toIdentification object restarts the image collecting device for acquiring facial image when entering shooting area, can save power consumption in this way.In one embodiment, which can be realized by the way of infrared induction.
In step S420, whether suitable current illumination condition is detected.If current illumination condition is suitable, advance toStep S440, conversely, then advancing to step S430.Wherein, suitable illumination condition can be understood as such illumination condition,The brightness of facial image collected meets identification and requires under the illumination condition.Whether detect illumination condition properly can be based on pre-Threshold value is determined to judge.Step S420 can provide preferable basic condition for acquisition, the recognition of face of subsequent facial image,So that subsequent processing is highly efficient.In one embodiment, this can be implemented using sensor or other suitable modesStep.
In step S430, current illumination condition is adjusted, suitable degree is adjusted to, then proceeds byStep S440.For example, can for example start lighting device if detecting that current illumination is darker in step S420 and be mendedLight.If detecting that current illumination is excessively bright, can take appropriate measures implementation adjustment.
In step S440, the facial image of object to be identified is acquired, and In vivo detection is carried out to object to be identified.If reallyDetermining object to be identified is living body, then step S450 is continued to, conversely, then skipping to step S490.Object to be identified is carried outIn vivo detection can effectively guard against the attack of the various ways such as photo, video, 3D faceform or mask.Specifically, rightThe In vivo detection that object to be identified carries out can be carried out using following manner.
In one embodiment, photo or video can be only shot, in addition vacation face in cloud judges, for some pairs of safetyIt is required that weak scene.Illustratively, the photo or video taken for object to be identified can be uploaded to cloud serviceDevice detects the face in photo or video by cloud server and judges when detecting face the authenticity of face.For example, may include having trained true face classifier and false face classifier in cloud server.It is described in detail below by example logicalIt crosses shooting photo or video carries out the embodiment of In vivo detection, in order to understand.
In one example, it can indicate that object to be identified reads aloud passage, by acquiring facial image, judge its lipDynamic whether move with the lip of corresponding text matches, if matching, In vivo detection success.
In one example, can indicate object to be identified make required movement (required movement be, for example, finger pressingIt gulps down gas in two cheek skins or mouth to heave two cheeks).In an exemplary example, when object to be identified has done oneOr when multiple instructions movement, acquire its facial image, judge whether its actions taken qualified, if so, In vivo detection success,Conversely, In vivo detection fails.In another exemplary example, when object to be identified has done one or more instruction movementsWhen, the skin area image before capturing object to be identified movement in image respectively and after movement, and by skin area imageIt is transferred to skin elasticity classifier, which is the disaggregated model succeeded in school in advance.For example, if it is workBody skin, then model output is 1, and otherwise output is 0.In this embodiment it is possible to based on referring to object to be identified in executionShow that the comparison of the skin area image of movement front and back carries out In vivo detection.
Illustratively, the study of skin elasticity classifier can carry out offline.A kind of possible embodiment is to search in advanceCollection living body true man do the before and after frames image of compulsory exercise, while collecting using photo, video playback, scraps of paper mask and 3D modelEtc. the attack image for doing compulsory exercise.The former as positive sample, the latter as negative sample, then use deep learning, support toThe statistical learning methods such as amount machine train skin elasticity classifier.
It illustratively, can be based on Face datection and face key point location algorithm come real to the capture of skin area imageIt is existing, such as a large amount of facial images are collected in advance, the canthus of face is manually marked out in every image, the corners of the mouth, the wing of nose, cheekbone is mostHigh point, a series of key points such as outer profile point use machine learning algorithm (such as deep learning, or returning based on local featureReduction method) and using the aforementioned image marked as input training Face datection, face key point location model.It will be collectedAfter the facial image of movement front and back inputs trained Face datection, face key point location model, will output face location andHuman face region is cut into a series of triangular plate members according to key point position coordinates, will be located at chin, cheekbone by key point position coordinatesThe triangular plate member image block in the regions such as bone, two cheeks is as face skin area.
In another embodiment, living body acquisition device, such as binocular camera can be done using special hardware, for oneThe higher scene of a little safety requirements.In this embodiment it is possible to the judgement based on the sub-surface scattering degree to face to be identifiedCarry out In vivo detection.Due to the sub-surface scattering degree of 3D mask etc. and true man's face it is different (when sub-surface scatters stronger, imageGradient is smaller, so that diffusion is smaller), for example, the sub-surface scattering degree of the mask of the materials such as general paper or plastics is remoteIt is weaker than face, and the sub-surface of the mask of the materials such as general silica gel scatters degree much stronger than face, therefore by diffusionJudgement can effectively defend mask attacker.It therefore, in embodiments of the present invention, can be by binocular camera and structure light knotIt closes, has the 3D face of structured light patterns by binocular camera acquisition, then according to structured light patterns in 3D face sub-surfaceScattering degree carries out living body judgement.
The In vivo detection that may include in face identification method according to an embodiment of the present invention is described above exemplarilySpecific example.The embodiment of face identification method 400 according to an embodiment of the present invention is continued to describe now referring back to Fig. 4.
In step S450, determine whether acquired facial image is qualified facial image, if it is, continuing onTo step S460, if it is not, then skipping to step S490.Wherein, the recognition of face that step S450 can be described with aforementioned combination Fig. 2The step S220 of method 200 is similar, for sake of simplicity, details are not described herein again.
In step S460, recognition of face is carried out to qualified facial image.In one embodiment, before this step,The step S230 that can also implement the face identification method 200 of aforementioned combination Fig. 2 description, i.e., select from qualified facial imageTop-quality facial image is can be further reduced calculation amount in this way, improve for carrying out recognition of face in step S460Recognition accuracy.
In step S470, it is determined whether there are matched face recognition results.If it is, continuing to stepS480, whereas if matched face recognition result is not present, i.e. recognition of face fails, then skips to step S490.
In step S480, face recognition result is exported.
In step S490, prompt object to be identified adjustment posture to resurvey facial image.If acquired faceImage is unqualified or face recognition result fails, it may be possible to, can be not since the posture of object to be identified needs to adjustQualified facial image can be collected or when to facial image recognition failures, prompt object to be identified adjustment posture to resurveyFacial image.
In addition, completing acquisition satisfaction identification requirement from starting to collect after object to be identified comes into coverageFacial image process, and be transmitted to identification module and identified, probably need regular hour (such as 1-2 seconds), can be withIt shows the process of human face scanning in the form of animation on display terminal within this time, is prompting object to be identified systemAcquisition and identification facial image, if failing to collect the image or knowledge for meeting that identification requires after the predetermined time (such as 2 seconds)Do not fail, then object to be identified adjustment posture is prompted to resurvey facial image.
The schematic flow of face identification method according to another embodiment of the present invention is described above exemplarily.It is based onAbove description, face identification method according to another embodiment of the present invention can not only save computing resource, moreover it is possible to improve peopleFace recognition performance simultaneously improves user experience.
The face identification device of another aspect of the present invention offer is described below with reference to Fig. 5.Fig. 5 shows real according to the present inventionApply the schematic block diagram of the face identification device 500 of example.
As shown in figure 5, face identification device 500 according to an embodiment of the present invention includes image capture module 510 and identificationModule 520.The modules can execute each step/function of the face identification method above in conjunction with Fig. 2 description respectively.Only the major function of each module of face identification device 500 is described below, and omits the details having been described aboveContent.
Image capture module 510 is used to form image queue for same object to be identified acquisition facial image, determines instituteIt states whether face images in queue are qualified facial image, and selects top-quality people in qualified facial imageFace image sends identification module 520 to.Identification module 520 is used to carry out recognition of face to the top-quality facial image.Image capture module 510 and identification module 520 can the operation storage dresses of processor 102 in electronic equipment as shown in Figure 1The program instruction that stores in 104 is set to realize.
In one embodiment, image capture module 510 may be implemented in image collecting device.Based on this, image is adoptedAcquisition means, which may not need, to be completely transferred to identification module 520 for its acquired image and handles, and can only will be qualifiedTop-quality facial image is transmitted to identification module 520 and is handled in facial image, can not only reduce transmission in this wayData volume, moreover it is possible to the data volume of identifying processing is reduced, further, since that transmission is top-quality people in qualified facial imageFace image, therefore can also guarantee the accuracy of identifying processing.
In one embodiment, qualified facial image can be understood as the facial image for meeting recognition of face requirement.Such asThe acquired facial image of fruit meets recognition of face requirement, it is determined that acquired facial image is qualified facial image；InsteadIt, if acquired facial image is unsatisfactory for recognition of face requirement, it is determined that acquired facial image is underproof peopleFace image.According to the different application scene of recognition of face, recognition of face requires to be slightly different.Generally, recognition of face is wantedAsking may include the basic demand for enabling to face normally to be identified.
Illustratively, image capture module 510 determine acquired facial image whether be qualified facial image stepSuddenly it can include determining that whether the following meets identification and require：The 3 d pose of face in acquired facial image；ThroughThe fog-level of the facial image of acquisition；The occlusion state of face in acquired facial image；Acquired facial imageBrightness.In one embodiment, these are all satisfied identification and require, and just determine that acquired facial image is qualified faceImage.In other embodiments, at least partly satisfaction in these, which identifies, requires, then can determine acquired facial imageFor qualified facial image.It can understand in conjunction with Fig. 3 and determine facial image in face identification method according to an embodiment of the present inventionWhether He Ge method, for sake of simplicity, details are not described herein again.
In one embodiment, face identification device 500 can also include In vivo detection module (not shown in FIG. 5),In vivo detection module is used to carry out In vivo detection to object to be identified.In one example, In vivo detection module can based on pairThe facial image that the object to be identified executes instruction movement carries out In vivo detection, and the In vivo detection includes judging the instructionMovement it is whether qualified and using preparatory trained skin elasticity classifier judge the object to be identified execute instruction it is dynamicWhether the skin area image for making front and back is at least one of skin of living body.In another example, In vivo detection module canTo carry out In vivo detection based on sub-surface scattering degree of the structured light patterns in the face of the object to be identified.It can combineDescription understands the work of the In vivo detection module in face identification device according to an embodiment of the present invention as described in step S440 in Fig. 4Body detection process, for sake of simplicity, details are not described herein again.In vivo detection module may be implemented in image collecting device, can also be withRealize other than image collecting device other modules or device in.
In one embodiment, face identification device 500 can also include illumination detection module (not shown in FIG. 5),Illumination detection module is used to examine before image capture module 510 acquires facial image or during acquiring facial imageIt whether suitable surveys current illumination condition, if illumination condition is improper, starts light compensating apparatus and carry out light filling.It can be in conjunction with figureDescription understands the behaviour of the illumination detection module in face identification device according to an embodiment of the present invention as described in step S420 in 4Make, for sake of simplicity, details are not described herein again.Illumination detection module may be implemented in image collecting device, also may be implemented schemingAs other than acquisition device other modules or device in.
In one embodiment, face identification device 500 can also include apart from detection module (not shown in FIG. 5),It is used to before image capture module 510 acquires facial image first determine whether object to be identified enters shooting apart from detection moduleRegion.Can in conjunction in Fig. 4 as described in step S410 in description understanding face identification device according to an embodiment of the present invention away fromOperation from detection module, for sake of simplicity, details are not described herein again.It may be implemented in image collecting device apart from detection module,Also may be implemented other than image collecting device other modules or device in.
In one embodiment, face identification device 500 can also include display module (not shown in FIG. 5), displayModule is used to show the animation of scanning face to object to be identified, is acquiring and identifying facial image with prompt.In a realityIt applies in example, when failing to collect qualified facial image in the given time or to facial image recognition failures, the displayModule is also used to prompt the object to be identified adjustment posture to resurvey facial image.It can combine in Fig. 4 about stepThe description of S490 understands the operation of the display module in face identification device according to an embodiment of the present invention, for sake of simplicity, hereinIt repeats no more.Display module may be implemented on the display terminal of recognition of face front end or realize in other suitable modules or dressIn setting.
The structure composition of face identification device according to an embodiment of the present invention is described above exemplarily, for sake of simplicity,Only the major function of each module is described, and omits the detail content having been noted above previously in conjunction with Fig. 2 to Fig. 4.
Based on above description, face identification device according to an embodiment of the present invention is in recognition of face without to acquiredEach frame facial image all handled, but only to top-quality facial image in qualified facial image atReason, not only can guarantee the accuracy of face recognition result, while also greatly reducing calculation amount, save calculating moneySource improves recognition efficiency.In addition, face identification device according to an embodiment of the present invention can also improve recognition of face performance, andImprove user experience.
Fig. 6 shows the schematic block diagram of face identification system 600 according to an embodiment of the present invention.Face identification system600 include imaging sensor 610, storage device 620 and processor 630.
Wherein, imaging sensor 610 is used to acquire the facial image of object to be identified.The storage of storage device 620 is for realThe program code of corresponding steps in existing face identification method according to an embodiment of the present invention.Processor 630 is for running storageThe program code stored in device 620 to execute the corresponding steps of face identification method according to an embodiment of the present invention, and is usedCorresponding module in realization face identification device according to an embodiment of the present invention.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedFollowing steps：Determining the face images in the image queue for the facial image formation of same object to be identified acquisition isThe no facial image for qualification；Top-quality facial image is selected in qualified facial image；And most to the qualityGood facial image carries out recognition of face.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedThe determination queue in face images the step of whether being qualified facial image include：For in the queueThe acquired facial image of each frame, determine whether at least one the following meets identification and require：The acquired peopleThe 3 d pose of face in face image；The fog-level of the acquired facial image；The acquired facial imageIn face occlusion state；The brightness of the acquired facial image.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedTo the 3 d pose, the fog-level, the occlusion state whether meet identification require determination be based on depth roll upProduct network carries out.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedThe determination acquired facial image in the 3 d pose of face whether meet identification and require to include：Determine the faceThe angle per the one-dimensional positive face of deviation in three dimensions；And make a reservation for if the angle per the one-dimensional positive face of deviation is not more thanThreshold value, it is determined that the 3 d pose of the face meets identification and requires, and requires conversely, being then unsatisfactory for identification.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedThe fog-level of the determination acquired facial image whether meet identification and require to include：Based on the acquired faceThe motion blur and Gaussian Blur of image determine the fog-level of the acquired facial image；And it is if described acquiredThe fog-level of facial image be not more than predetermined threshold, it is determined that the fog-level of the facial image meets identification and requires,It is required conversely, being then unsatisfactory for identification.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedThe determination acquired facial image in the occlusion state of face whether meet identification and require to include：Determine the faceKey position whether be blocked；And if the key position of the face is not blocked, it is determined that in the facial imageFace occlusion state meet identification require, conversely, be then unsatisfactory for identification require.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedTo the brightness of the acquired facial image whether meet identification require determination be to be carried out based on grey level histogram.
In one embodiment, hold face identification system 600 when said program code is run by processor 630Row following steps：In vivo detection is carried out to object to be identified.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedThe In vivo detection be carried out based on the facial image for executing instruction movement to the object to be identified, including judgement described inWhether instruction movement is qualified and judges that the object to be identified refers in execution using preparatory trained skin elasticity classifierWhether the skin area image for showing movement front and back is at least one of skin of living body.
In one embodiment, when said program code is run by processor 630 face identification system 600 is executedThe In vivo detection be based on structured light patterns in the face of the object to be identified sub-surface scattering degree carry out.
In one embodiment, hold face identification system 600 when said program code is run by processor 630Row following steps：Whether current illumination condition is detected before acquiring facial image or during acquiring facial imageProperly, if illumination condition is improper, start light compensating apparatus and carry out light filling.
In one embodiment, hold face identification system 600 when said program code is run by processor 630Row following steps：First determine whether object to be identified enters shooting area before acquiring facial image.
In one embodiment, hold face identification system 600 when said program code is run by processor 630Row following steps：The animation of scanning face is shown to object to be identified, is acquiring and identifying facial image with prompt.
In one embodiment, hold face identification system 600 when said program code is run by processor 630Row following steps：When failing to collect qualified facial image in the given time or to facial image recognition failures, promptThe object to be identified adjustment posture is to resurvey facial image.
In addition, according to embodiments of the present invention, additionally providing a kind of storage medium, storing program on said storageInstruction, when described program instruction is run by computer or processor for executing the face identification method of the embodiment of the present inventionCorresponding steps, and for realizing the corresponding module in face identification device according to an embodiment of the present invention.The storage mediumIt such as may include the storage card of smart phone, the storage unit of tablet computer, the hard disk of personal computer, read-only memory(ROM), Erasable Programmable Read Only Memory EPROM (EPROM), portable compact disc read-only memory (CD-ROM), USB storage,Or any combination of above-mentioned storage medium.The computer readable storage medium can be one or more computer-readable depositAny combination of storage media.
In one embodiment, the computer program instructions may be implemented real according to the present invention when being run by computerEach functional module of the face identification device of example is applied, and/or recognition of face according to an embodiment of the present invention can be executedMethod.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorIt manages device and executes following steps：Determine the owner in the image queue for the facial image formation of same object to be identified acquisitionWhether face image is qualified facial image；Top-quality facial image is selected in qualified facial image；And to instituteIt states top-quality facial image and carries out recognition of face.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorManage that the step of whether face images are qualified facial image in the determination queue that device executes includes：For instituteThe acquired facial image of each frame in queue is stated, determines whether at least one the following meets identification and require：The warpThe 3 d pose of face in the facial image of acquisition；The fog-level of the acquired facial image；It is described acquiredThe occlusion state of face in facial image；The brightness of the acquired facial image.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorWhat reason device executed is base to whether the 3 d pose, the fog-level, the occlusion state meet the determination that identification requiresIt is carried out in depth convolutional network.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorWhether the 3 d pose for the face in the determination acquired facial image that reason device executes, which meets identification, requires to include：It determinesThe angle per the one-dimensional positive face of deviation of the face in three dimensions；And if it is described per the one-dimensional angle for deviateing positive face notGreater than predetermined threshold, it is determined that the 3 d pose of the face meets identification and requires, and requires conversely, being then unsatisfactory for identification.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorWhether the fog-level for the determination acquired facial image that reason device executes, which meets identification, requires to include：Based on described through adoptingThe motion blur and Gaussian Blur of the facial image of collection determine the fog-level of the acquired facial image；And if instituteThe fog-level of acquired facial image is stated no more than predetermined threshold, it is determined that the fog-level of the facial image, which meets, to be knownIt does not require, is required conversely, being then unsatisfactory for identification.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorWhether the occlusion state for the face in the determination acquired facial image that reason device executes, which meets identification, requires to include：It determinesWhether the key position of the face is blocked；And if the key position of the face is not blocked, it is determined that the peopleThe occlusion state of face in face image meets identification and requires, and requires conversely, being then unsatisfactory for identification.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorWhat reason device executed is based on grey level histogram to whether the brightness of the acquired facial image meets the determination that identification requiresIt carries out.
In one embodiment, the computer program instructions also make when being run by computer or processor computer orProcessor executes following steps：In vivo detection is carried out to object to be identified.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorReason device execute the In vivo detection be carried out based on the facial image for executing instruction movement to the object to be identified, includingJudge whether the instruction movement is qualified and judges the object to be identified using preparatory trained skin elasticity classifierWhether the skin area image before and after executing instruction movement is at least one of skin of living body.
In one embodiment, the computer program instructions make computer or place when being run by computer or processorThe In vivo detection that reason device executes is the sub-surface scattering journey based on structured light patterns in the face of the object to be identifiedWhat degree carried out.
In one embodiment, the computer program instructions also make when being run by computer or processor computer orProcessor executes following steps：Current illumination is detected before acquiring facial image or during acquiring facial imageWhether condition is suitable, if illumination condition is improper, starts light compensating apparatus and carries out light filling.
In one embodiment, the computer program instructions also make when being run by computer or processor computer orProcessor executes following steps：First determine whether object to be identified enters shooting area before acquiring facial image.
In one embodiment, the computer program instructions also make when being run by computer or processor computer orProcessor executes following steps：The animation of scanning face is shown to object to be identified, is acquiring and identifying face figure with promptPicture.
In one embodiment, the computer program instructions also make when being run by computer or processor computer orProcessor executes following steps：When failing to collect qualified facial image in the given time or to facial image recognition failuresWhen, prompt the object to be identified adjustment posture to resurvey facial image.
Each module in face identification device according to an embodiment of the present invention can pass through people according to an embodiment of the present inventionThe processor computer program instructions that store in memory of operation of the electronic equipment of face identification realize, or can be in rootThe computer instruction stored in computer readable storage medium according to the computer program product of the embodiment of the present invention is by computerIt is realized when operation.
Face identification method, device, system and computer-readable medium according to an embodiment of the present invention are in recognition of faceShi Wuxu handles acquired each frame facial image, but only to top-quality people in qualified facial imageFace image is handled, and not only can guarantee the accuracy of face recognition result, while also greatly reducing calculation amount, sectionAbout computing resource, improves recognition efficiency.In addition, face identification method according to an embodiment of the present invention, device, system andComputer-readable medium can also improve recognition of face performance, and improve user experience.
Although describing example embodiment by reference to attached drawing here, it should be understood that above example embodiment are only exemplary, and be not intended to limit the scope of the invention to this.Those of ordinary skill in the art can carry out various changes whereinAnd modification, it is made without departing from the scope of the present invention and spiritual.All such changes and modifications are intended to be included in appended claimsWithin required the scope of the present invention.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosureMember and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actuallyIt is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technicianEach specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceedThe scope of the present invention.
In several embodiments provided herein, it should be understood that disclosed device and method can pass through itIts mode is realized.For example, apparatus embodiments described above are merely indicative, for example, the division of the unit, onlyOnly a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can be tiedAnother equipment is closed or is desirably integrated into, or some features can be ignored or not executed.
In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the inventionExample can be practiced without these specific details.In some instances, well known method, structure is not been shown in detailAnd technology, so as not to obscure the understanding of this specification.
Similarly, it should be understood that in order to simplify the present invention and help to understand one or more of the various inventive aspects,To in the description of exemplary embodiment of the present invention, each feature of the invention be grouped together into sometimes single embodiment, figure,Or in descriptions thereof.However, the method for the invention should not be construed to reflect following intention：It is i.e. claimedThe present invention claims features more more than feature expressly recited in each claim.More precisely, such as corresponding powerAs sharp claim reflects, inventive point is that the spy of all features less than some disclosed single embodiment can be usedSign is to solve corresponding technical problem.Therefore, it then follows thus claims of specific embodiment are expressly incorporated in this specificEmbodiment, wherein each, the claims themselves are regarded as separate embodiments of the invention.
It will be understood to those skilled in the art that any combination pair can be used other than mutually exclusive between featureAll features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed any methodOr all process or units of equipment are combined.Unless expressly stated otherwise, this specification (is wanted including adjoint rightAsk, make a summary and attached drawing) disclosed in each feature can be replaced with an alternative feature that provides the same, equivalent, or similar purpose.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodimentsIn included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the inventionWithin the scope of and form different embodiments.For example, in detail in the claims, embodiment claimed it is one of anyCan in any combination mode come using.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processorsSoftware module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practiceMicroprocessor or digital signal processor (DSP) realize some moulds in article analytical equipment according to an embodiment of the present inventionThe some or all functions of block.The present invention is also implemented as a part or complete for executing method as described hereinThe program of device (for example, computer program and computer program product) in portion.It is such to realize that program of the invention can storeOn a computer-readable medium, it or may be in the form of one or more signals.Such signal can be from internetDownloading obtains on website, is perhaps provided on the carrier signal or is provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and abilityField technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims,Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of notElement or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple suchElement.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer realIt is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branchTo embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fameClaim.
The above description is merely a specific embodiment or to the explanation of specific embodiment, protection of the inventionRange is not limited thereto, and anyone skilled in the art in the technical scope disclosed by the present invention, can be easilyExpect change or replacement, should be covered by the protection scope of the present invention.Protection scope of the present invention should be with claimSubject to protection scope.