Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, some embodiments of the present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The first embodiment of the present application relates to a face registration method.
The embodiment can be applied to a face recognition device, taking an intelligent door lock as an example, when a new user (namely, a new user is registered), the intelligent door lock needs to enter new user information, specifically, face data of the new user is collected, processed and stored in a user database, so that when the new user needs to open or close the intelligent door lock, the user stands in front of the intelligent door lock, and the face information is collected and recognized by a camera device of the intelligent door lock.
The specific flow of the face registration method in this embodiment is shown in fig. 1.
Step 101, collecting a face picture.
Specifically, in the embodiment, a picture of a face is acquired through a camera device carried by the intelligent door lock, in one example, the camera device starts shooting, after a picture is acquired, 2D face detection is firstly performed to judge whether the face exists, if the face exists, subsequent steps are performed, if the face does not exist, shooting errors may occur, and if an incomplete face is shot or an insufficiently clear face is shot, the picture is returned to be acquired again.
In one example, a deep learning method can be used for 2D face detection, the face features required to be included in a complete 2D face are obtained through the deep learning method, the face features obtained through the deep learning method are used for comparison detection during detection, if the collected picture conforms to the face features, the collected picture is considered to have a face, otherwise, the collected picture is considered to have no face. The face features may be whether there are eyes, whether the number of eyes is two, whether there are noses, whether the number of noses is one, the contour shape may be a possible shape of the eyes, a possible shape of the noses, and the relative positions of the eyes and the noses may be included, which is not listed here.
102, detecting whether the face pictures acquired for multiple times are from the same person; if the face pictures acquired for multiple times are determined to be from the same person, executing step 103; if not, the step 101 is executed again.
Taking the collected face picture as a floodlight image as an example, the step of detecting whether the face pictures collected for multiple times are from the same person may be as shown in fig. 2, and specifically the following steps are performed:
step 201, feature data of the face pictures acquired for many times are obtained.
Step 202, comparing the characteristic data obtained from each face picture.
And step 203, determining whether the face pictures acquired for multiple times are from the same person or not according to the comparison result.
Specifically, the steps can determine whether the multiple face pictures are from the same person by comparing feature data of the faces in the floodlight image, wherein the feature data comprises an aspect ratio of eyes, a distance between two eyes, a curve length and a radian of eyebrows, an aspect ratio of mouths, a radian of chin and the like, and features of eyes, eyebrows, mouths and the like in the feature data can be identified from the floodlight image and feature data values are obtained through measurement. Since the contour sizes of the same person generally remain substantially unchanged, it is determined whether the same person is accurately feasible according to the comparison result of the feature data.
It should be noted that, in the registration process of a new user, the face may be collected many times, and when the face pictures are collected for the third time, the fourth time, and so on, the face pictures may be compared with the stored face pictures, so as to ensure that the face data obtained in one registration are all from the same person as much as possible.
Taking 3D face recognition as an example, fusing a plurality of face pictures to form a 3D face, so when the face picture is collected for the first time, the face picture can be directly stored, when the face picture is collected for the second time, the face picture needs to be compared with the stored face picture to determine whether two face pictures are the same person, when the two face pictures are determined to be the same person, the subsequent steps are continued, and if the two face pictures are not the same person, invalid data can be considered to be collected, so that the face pictures are discarded.
In one example, the embodiment specifically detects whether the face pictures acquired multiple times are from the same person as follows: and detecting whether the face pictures acquired twice in the two adjacent times are from the same person. In practical application, whether the adjacent or more than two consecutive collected face pictures come from the same person or not can be detected, whether the non-adjacent two collected face pictures come from the same person or not can also be detected, and various detection rules can be set according to actual needs and are not listed one by one.
Step 103, registration data is obtained.
Specifically, the face images acquired for multiple times are fused, wherein the fusion comprises the steps of overlaying, combining and removing the face feature data to form a fused face model, and the face data in the registration data is obtained according to the fused data. In addition, the registration data may include information such as an account name and a password, which is not described herein again.
In one example, whether the acquisition times reach or not can be judged after fusion, and a certain amount of acquisition times can be preset in the registration process of the 3D face so as to ensure that enough data volume is acquired. And if the acquisition number is judged to be not equal to the preset value after one-time fusion, returning to the step 101 to continue acquisition until the acquisition times are reached, and obtaining the data after the last fusion, namely the face data in the registered data.
The above steps 102 to 103 are to obtain the registration data according to the face pictures acquired many times if it is determined that the face pictures acquired many times are from the same person.
It should be noted that, in this embodiment, steps 101 to 103 are a flow of one-time acquisition in the registration process, and multiple acquisitions may be performed in the actual registration process, so as to obtain multiple face pictures. Wherein, a threshold value of the collection times can be set, and when the collection times reach the threshold value, the whole registration process is ended.
The structure and the operation principle of the above steps 101 to 103 may be as shown in fig. 3, where a person 4 sends acquisition information to a controller (or a processor) 2 through a human-computer interaction device 3 (for example, a touch screen), where the controller may be an AP (application processor for short), the controller 2 sends an acquisition instruction to the camera module 1, the camera module 1 receives the instruction, projects structured light to the face of the person 4, and after being reflected, the camera module 1 acquires a picture and sends the picture to the controller 2 for processing, and the controller 2 is specifically configured to implement functions of face detection, recognition, 3D reconstruction, data fusion, and the like.
Compared with the prior art, the method and the device have the advantages that whether the front image and the rear image are detected by the same person or not is added in the registration process, and the face data are fused after the face data are determined, so that the face data in the obtained registration data come from the same person, the person-changing registration in the registration process can be avoided, the problem that different persons can be verified and passed in the identification process is avoided, and the potential safety hazard in the identification process is greatly reduced. The feature data are obtained according to the floodlight image of the face, and then the feature data are compared, so that whether the face pictures acquired for many times are from the same person or not is determined according to the comparison result, and the method is clear and has good realizability.
In this embodiment, a face registration method is applied to a face recognition device as an example, in another example, the face registration method in this embodiment may also be applied to a server, and the specific process may include: the server collects face pictures through the face recognition device, obtains a single collected picture, then carries out face recognition, judges whether the pictures containing the faces collected for many times in one registration are the same person, carries out data fusion on a plurality of face pictures judged as the same person, and finally obtains registration data.
The second embodiment of the present application relates to a face registration method. This embodiment is substantially the same as the first embodiment, and mainly differs therefrom in that: when the detection is carried out on the same person or not in the first embodiment, the collected floodlight image is used for detection, but in the embodiment, the floodlight image and the structured light image are collected, the detection is carried out according to the combination of the floodlight image and the structured light image, another detection method is provided, 3D data can be obtained, and the accuracy of the detection result is further improved.
The face registration method in this embodiment is shown in fig. 1, and specifically includes the following steps:
step 101, collecting a face picture.
Specifically, in the step, besides the floodlight image corresponding to the face, the structured light image can be collected. The structured light can be projected to the face through a projector arranged in the camera shooting equipment, and then collected through a camera in the camera shooting equipment to obtain a corresponding structured light image. Where the set of projected rays of known spatial direction is called structured light, such as speckle.
102, detecting whether the face pictures acquired for multiple times are from the same person; if the face pictures acquired for multiple times are determined to be from the same person, executing step 103; if not, the step 101 is executed again.
Specifically, the detection process shown in fig. 2 can be adopted in the step of detection, and specifically, the following steps are adopted:
step 201, feature data of the face pictures acquired for many times are obtained.
Specifically, the feature data extraction for the floodlight image of the face in this step is similar to that in the first embodiment, and is not described herein again.
When the feature data of the structured light image of the human face is extracted in the step, the structured light image is subjected to 3D reconstruction, and the feature data is extracted from the reconstructed image. Specifically, the data form of the reconstruction map obtained by 3D reconstruction may include: the depth map or three-dimensional point cloud, in one example, may be a combination of both. And then, calculating the feature data of the reconstructed image to obtain the feature data of the human face.
Continuing, the process of 3D reconstruction of the structured light image may be embodied as follows: and calculating the three-dimensional coordinates of the object corresponding to the structured light image according to the parameters of the image pickup equipment, wherein the parameters of the image pickup equipment comprise: internal parameters (such as camera focal length, principal point position, etc.) and external parameters (rotational and translational relationships between the camera and the projector). More specifically, the system prestores a prestore image (which can be a speckle pattern) of the camera equipment, matches the acquired image with the prestore image to obtain parallax, and calculates the three-dimensional coordinates of the face according to the parallax, the internal reference and the external reference. And then, extracting the feature data of the human face according to the calculated three-dimensional coordinates.
Continuing to explain, the feature data obtained in this step may include both the feature data extracted from the flood image and the feature data extracted from the point cloud or the depth map obtained from the structured light image.
Step 202, comparing the characteristic data obtained from each face picture.
Specifically, during comparison, the feature data acquired later and the feature data acquired first can be compared, wherein the face picture acquired first can be selected according to the acquisition time, for example, the face picture with the closest acquisition time is selected for comparison.
And step 203, determining whether the face pictures acquired for multiple times are from the same person or not according to the comparison result.
Specifically, the comparison result may be the similarity of the feature data, a similarity threshold is preset, and if the similarity in the comparison result is higher than the threshold, it is determined that each face image is from the same person.
In one example, the comparison may be performed once, and whether the comparison result is the same person may be determined according to the comparison result of this time, and if the comparison result meets the preset condition, the comparison result is determined to be the same person, and if the comparison result does not meet the preset condition, the comparison result is determined to be not the same person. In another example, the comparison may be performed multiple times, and whether the comparison result is the same person or not may be determined according to the comparison results of multiple times.
In short, the above steps 201 to 203 specifically detect whether the plurality of face pictures collected are from the same person according to the template generated by each structured light image. And continuing to execute subsequent steps after determining the same person. And 103, fusing the data of the face pictures to obtain registration data.
Specifically, before the face image data is fused, the feature data can be stored as templates, and then, during the fusion, the templates can be fused. Specifically, a plurality of face pictures correspond to different angles of a face, and during fusion, all templates are deeply fused according to the change of the face angles, and the obtained data after the last fusion is the face data in the registered data. The data volume in the template data is simplified, so that the data volume in the processing process is reduced.
It is thus clear that not only through floodlight image, still obtain characteristic data jointly through the structured light image in this embodiment, compare characteristic data again and confirm whether each face picture is same person, contained three-dimensional information in the structured light image, contained two-dimensional information in the floodlight image, so both combine the back, and the information is more abundant, just also makes the testing result more accurate, and the credibility is higher.
In one example, the feature data may be obtained only from the structured light image, and will not be described herein.
The third embodiment of the present application relates to a face registration method. The embodiment is further improved on the basis of the first embodiment, and the main improvement is as follows: the embodiment detects whether a plurality of face pictures come from the same person, and also detects whether the detected face pictures are real persons at each time, so that the condition that data acquisition is carried out by adopting a portrait model and the like to influence the safety of a subsequent identification process is avoided.
A flowchart of the face registration method in this embodiment is shown in fig. 4, and specifically follows:
step 401, collecting a face picture.
In particular, the present embodiment may include capturing a flood image.
Step 402, detecting whether the face in the face picture is a living body; if yes, go on to step 403; if not, the process returns to step 401.
Specifically, the living body of the human face can be called a real human face, in the prior art, the registration by using the portrait model is feasible, so that the stored data is derived from the photographed portrait model, the real data of the registered user is not stored, the stored registered data does not have credibility, and in the later identification stage, anyone can identify the human face by only holding the portrait model. In order to improve the reliability of the registration data and increase the reliability of the recognition result, the present embodiment increases the detection of whether the face is a living body, and avoids the registration stage using a portrait model.
In one example, this step processes a flood image of the face using spectral analysis or local binarization; and determining whether the human face is a living body according to the processing result. Because the human faces are different in color reflection rate or absorption rate, images acquired under the irradiation of the same light source present different details, and the details can be analyzed by adopting spectral analysis or LBP (Local Binary Pattern) characteristics or a combination of the two, wherein the spectral analysis (such as Discrete Fourier Transform, DFT for short) and LBP (Local Binary Pattern) can be combined with deep learning and other modes to carry out living body detection on the images so as to distinguish authenticity.
Taking fig. 5 to 8 as examples, fig. 5 and 6 are respectively a face picture obtained by collecting a real person and a corresponding frequency spectrum graph after DFT processing, and fig. 7 and 8 are respectively a face picture obtained by collecting a fake person and a corresponding frequency spectrum graph after DFT processing. It can be seen from the above figures that the face picture obtained according to the real person has obvious gray level gradual change and fuzzy color block boundary after processing, and the face picture obtained according to the fake person has obvious color block boundary after being processed by DFT, so that the acquired face picture can be distinguished from the real person or the fake person by the spectral graph obtained by DFT processing.
In another example, the face picture can be subjected to LBP to distinguish authenticity. Fig. 9 is a schematic diagram of the principle of LBP conversion performed on a picture, and fig. 10 and 11 are a face picture obtained by acquiring a real person and a corresponding map after LBP conversion, respectively, so as to determine a gray level feature according to the LBP map, and it can be seen that, after LBP processing is performed on the face picture, gray level gradient has a certain feature, and the feature can be used to distinguish whether the acquired face picture comes from a real person or a fake person.
Steps 403 to 404 are similar to steps 102 to 103 in the first embodiment, and are not described again here.
In the present embodiment, the live body detection is performed by using a floodlight image, and in an example, the live body detection may also be performed by using a structured light image, which specifically includes: 3D reconstruction is carried out on the structured light image to obtain a reconstruction map; and detecting whether the face in the face picture is a living body according to the reconstructed picture. More specifically, when the structured light image is captured by the imaging device, whether the captured face is a real person face or a photograph can be determined from the 3D image generated by the conversion, and since the photograph is a two-dimensional object, if the photograph is used as the captured object, a 3D image with a normal stereoscopic effect cannot be obtained, and thus, in one example, whether the captured object is a real person or a photograph can be determined from the 3D image generated by the conversion. Of course, it is also possible to replace the structured light image with a TOF image. In one example, multiple 2D images (infrared flood images or RGB images) may be collected before collecting the structured light image for live body detection, and whether the multiple 2D images are from the same person is determined, so as to prevent that a photo or a video of a certain first is used to obtain a 2D face detection pass before registration, and then the photo or the video is quickly changed to a certain second to obtain a live body detection pass, so as to forge the certain first to complete registration. Certainly, the structured light image may be obtained, or a 2D image is additionally taken after the three-dimensional reconstruction is completed through the live body detection step to determine whether the two images are the same person, that is, in the present case, the face image taken for the first time in the registration process is used as a reference to verify whether the same person is present in the registration process, so as to prevent the occurrence of successful registration for replacement in the registration process.
In addition, in practical applications, combinations of the above-mentioned methods for detecting living bodies can be utilized, and are not described herein again.
Therefore, the living body detection of the face picture is added in the embodiment, the use of models, photos and other non-living bodies in the registration process is avoided, the safety of face recognition is further ensured, meanwhile, the embodiment provides various living body detection methods, so that different detection methods can be selected according to needs in practical application, and the method is very flexible.
The fourth embodiment of the present application relates to a face registration method. The embodiment is further improved on the basis of the first embodiment, and the main improvement is as follows: in the embodiment, a step of judging the integrity of the fused data is added, so that the acquisition times are reduced as much as possible while the acquisition of the complete data is ensured.
A flowchart of the face registration method in the present embodiment is shown in fig. 12, and specifically includes the following steps:
steps 1201 to 1202 in this embodiment are similar to steps 101 to 102 in the first embodiment, and are not described again.
And 1203, fusing the face picture data.
Specifically, the face pictures from the same person are determined to be subjected to depth fusion.
Step 1204, judge whether the data after fusing include the intact human face characteristic; if yes, go to step 1205; if not, the process returns to the step 1201.
Specifically, in the integrity judgment, the integrity judgment can be determined according to whether the features of the face are complete, such as whether the contour is complete, whether the whole face is covered, and the like. If the face is not complete enough, the registration is considered to be failed, and the process can return to the step 1201 to instruct the camera device to continue to acquire the face picture until the face data is complete.
In one example, the integrity judgment may utilize a preset set of feature points, and then determine whether the data of the set of feature points exists in the fused data, and if each feature point has data, the fused data may be considered to be complete. Wherein, the feature points may include: the left eye canthus, the right eye canthus, etc.
In an example, incomplete face data after fusion may also be caused by an apparatus error, so to avoid the apparatus entering a dead loop, it may be set to end the flow of the face registration method in this embodiment after attempting to reacquire a certain number of times, and at the same time, an error prompt is given accordingly.
In an example, even if the number of acquisition times is small, complete data may still be acquired, and if it is determined that the face data is complete, even if the number of acquisition times does not reach the preset number of acquisition times, the acquisition may not be continued, and accordingly the acquisition is ended, so as to simplify the face registration process in the present embodiment. In another example, it may also be determined whether the number of times of acquisition reaches a preset number of times, and if so, the integrity determination is performed, and if not, the process returns to continue acquiring the picture, or the registration process is ended.
Step 1205, registration data is obtained.
Specifically, after the complete face features are determined in step 1204, it is described that complete face data is obtained, and then registration data can be obtained in combination with other registration information.
Therefore, according to the judgment of the feature integrity, the acquisition times can be reduced while accurate and complete data can be acquired, and the method is more suitable for practical application scenarios.
It should be further noted that the technical solutions in the second to fourth embodiments may be combined and used as needed, and an example of the combined use is described below, and a flowchart of the face registration method is shown in fig. 13. In this example, the step of collecting the face picture includes collecting a flood image and a structured light image of the face, performing 3D face reconstruction on the structured light image, determining whether a template exists, if not, performing living body detection, detecting whether the face in the collected face picture is a real person, if the living body detection passes, storing feature data of the currently collected face picture as the template, where the series of steps describes a processing process of collecting the face of the real person at the first time, and then continuing to describe other situations that may occur. And if the existing template does not exist, directly returning to reacquire when the in-vivo detection fails.
In another case, if an existing template exists, it indicates that a face picture is not acquired for the first time, then the face picture acquired for the current time is processed to acquire feature data, the acquired feature data is compared with the feature data in the existing template to determine whether the face picture is the same as the face picture acquired before, then if the comparison is determined to be passed, living body detection is continued, if the living body detection is passed, template fusion is performed, and the series of steps describe a general processing process of acquiring the face picture for the second time and after in one registration. And then, continuously judging whether the number of the templates is full, if so, judging whether a complete face is obtained, if so, determining that the registration is successful, and if not, determining that the registration is failed. On the other hand, if the number of the templates is not full, the face picture acquired at the current time is stored as the template, and then the face picture is returned to be continuously acquired. After comparing the feature data, if the comparison fails, determining whether the total acquisition times is less than a threshold (which may be a preset threshold), if the total acquisition times is greater than or equal to the threshold, determining that a certain number of times has been acquired, and without continuing to try, ending the registration process in this embodiment, and if the total acquisition times is less than the threshold, determining that the try can be continued, and returning to reacquire the face picture.
The flow described above is the total flow of one registration process.
A fifth embodiment of the present application relates to a face registration apparatus.
Fig. 14 shows a schematic diagram of an apparatus in this embodiment, which specifically includes:
and the acquisition module is used for acquiring the face picture.
And the detection module is used for detecting whether the face pictures acquired for multiple times are from the same person.
And the fusion module is used for acquiring registration data according to the face pictures acquired for many times when the face pictures acquired for many times are determined to be from the same person.
It should be understood that this embodiment is an example of the apparatus corresponding to the first embodiment, and may be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the first embodiment.
It should be noted that each module referred to in this embodiment is a logical module, and in practical applications, one logical unit may be one physical unit, may be a part of one physical unit, and may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, elements that are not so closely related to solving the technical problems proposed by the present invention are not introduced in the present embodiment, but this does not indicate that other elements are not present in the present embodiment.
A sixth embodiment of the present invention relates to a server, as shown in fig. 15, including:
at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the face registration methods according to the first to fourth embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
A seventh embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method according to the above embodiments may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps in the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the present application, and that various changes in form and details may be made therein without departing from the spirit and scope of the present application in practice.