WO2023019927A1 - Facial recognition method and apparatus, storage medium, and electronic device - Google Patents

Abstract

The present application discloses a facial recognition method and apparatus, a storage medium, and an electronic device. The method comprises: collecting a facial image containing a face, and extracting a face feature corresponding to the facial image; searching, in a plurality of pre-stored face feature groups, for a target face feature that matches with the face feature; when the target face feature is found, acquiring the number of acquisition instances of the facial image; and if the number of acquisition instances is greater than or equal to a preset frequency threshold, then determining an additionally recorded face feature on the basis of the face feature, and inputting the additionally recorded face feature into a face feature group corresponding to the target face feature. According to the present embodiments, successfully recognized face features of a user that can only be successfully recognized by means of multiple instances of comparison are used as additionally recorded face features for inputting into a face feature group, so that face features in an external environment that require multiple instances of comparison are stored in the face feature group, in this way, the recognition passing rate of the user in the external environment may be increased, thereby reducing the impact of the external environment on facial recognition.

Description

Translated fromChinese

一种人脸识别方法、装置、存储介质及电子设备A face recognition method, device, storage medium and electronic equipment

本申请要求于2021年8月20日提交中国专利局，申请号为202110962703.2，发明名称为“一种人脸识别方法、装置、存储介质及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202110962703.2 and the title of the invention "a face recognition method, device, storage medium and electronic equipment" submitted to the China Patent Office on August 20, 2021, and its entire content Incorporated in this application by reference.

技术领域technical field

本申请涉及人脸识别技术领域，特别涉及一种人脸识别方法、装置、存储介质及电子设备。The present application relates to the technical field of face recognition, and in particular to a face recognition method, device, storage medium and electronic equipment.

背景技术Background technique

人脸识别(Facial Recognition)技术被广泛应用于各个领域，例如，安防系统，门禁系统，支付系统，自主服务系统等。人脸识别技术是通过视频采集设备采集包含人脸的面部图像，并提取面部图像携带的人脸特征(例如，五官位置、脸型等)，然后再与人脸特征数据库进行比对以识别用户身份。然而，由于人脸识别技术容易受外部环境(例如，光照、姿态以及遮挡等)影响，从而人脸识别技术的实际应用过程中容易出现因外部环境改变而导致的识别失败的问题。Face recognition (Facial Recognition) technology is widely used in various fields, such as security systems, access control systems, payment systems, autonomous service systems, etc. Face recognition technology is to collect facial images containing human faces through video acquisition equipment, and extract facial features carried by facial images (such as facial features, face shape, etc.), and then compare them with facial feature databases to identify users. . However, since the face recognition technology is easily affected by the external environment (eg, illumination, posture, and occlusion, etc.), the problem of recognition failure caused by changes in the external environment is prone to occur during the actual application of the face recognition technology.

发明内容Contents of the invention

本申请要解决的技术问题在于，针对现有技术的不足，提供一种人脸识别方法、装置、存储介质及电子设备。The technical problem to be solved in this application is to provide a face recognition method, device, storage medium and electronic equipment for the deficiencies of the prior art.

为了解决上述技术问题，本申请实施例第一方面提供了一种人脸识别方法，所述方法包括：In order to solve the above technical problems, the first aspect of the embodiment of the present application provides a face recognition method, the method comprising:

获取包含人脸的面部图像，并提取所述面部图像对应的人脸特征；Obtaining a facial image containing a human face, and extracting facial features corresponding to the facial image;

基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征；其中，若干所述人脸特征组中的每组人脸特征组均 包括注册人脸特征；或者包括所述注册人脸特征和补录人脸特征，所述注册人脸特征与所述补录人脸特征的录入时间不同；Based on preset matching conditions, search for target facial features that match the facial features in several pre-stored facial feature groups; wherein, each group of facial feature groups in the several facial feature groups includes the registered person Facial features; or include the registered facial features and supplementary recording facial features, and the registration time of the registered facial features is different from that of the supplementary recording facial features;

当查找到目标人脸特征时，获取所述面部图像的采集次数；When the target face feature is found, the number of acquisitions of the facial image is obtained;

若所述采集次数大于或者等于预设次数阈值，则将所述人脸特征作为补录人脸特征,并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。If the number of acquisitions is greater than or equal to the preset number of times threshold, the facial feature is used as a supplementary facial feature, and the supplementary facial feature is entered into the facial feature group corresponding to the target facial feature .

所述人脸识别方法，其中，所述预设匹配条件包括相似度大于预设相似度阈值和/或差异度小于预设差异度阈值。In the face recognition method, the preset matching condition includes that the similarity is greater than a preset similarity threshold and/or the difference is smaller than a preset difference threshold.

所述人脸识别方法，其中，所述补录人脸特征包括困难人脸特征和/或融合人脸特征，其中，所述困难人脸特征为提取到的人脸特征；所述融合人脸特征为基于所述注册人脸特征以及所述困难人脸特征融合得到的。The face recognition method, wherein, the added facial features include difficult facial features and/or fusion facial features, wherein the difficult facial features are extracted facial features; the fusion facial features The features are obtained based on the fusion of the registered facial features and the difficult facial features.

所述人脸识别方法，其中，所述融合人脸特征为所述注册人脸特征和所述困难人脸特征加权得到，其中，所述注册人脸特征的加权系数与所述困难人脸特征的加权系数的和为1。The face recognition method, wherein, the fused face feature is obtained by weighting the registered face feature and the difficult face feature, wherein the weighting coefficient of the registered face feature and the difficult face feature The sum of the weighting coefficients is 1.

所述人脸识别方法，其中，所述补录人脸特征包括困难人脸特征和融合人脸特征；所述若所述采集次数大于或者等于预设次数阈值，则基于所述人脸特征确定所述人脸特征对应的补录人脸特征，并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内具体包括：The face recognition method, wherein, the additionally recorded face features include difficult face features and fusion face features; if the number of acquisitions is greater than or equal to a preset number of times threshold, then determine based on the face features The supplementary recording of human facial features corresponding to the described human facial features, and entering the supplementary recording of human facial features into the corresponding human facial feature group of the target human facial features specifically includes:

若所述采集次数大于或者等于预设次数阈值，将所述人脸特征作为困难人脸特征，并基于所述困难人脸特征和所述目标人脸特征所对应的人脸特征组内的注册人脸特征确定融合人脸特征；If the number of acquisitions is greater than or equal to the preset number of times threshold, the face feature is regarded as a difficult face feature, and based on the registration in the face feature group corresponding to the difficult face feature and the target face feature Facial features determine the fusion of facial features;

将所述困难人脸特征和所述融合人脸特征作为补录人脸特征，并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。The difficult facial features and the fused facial features are used as supplementary facial features, and the supplementary facial features are entered into the facial feature group corresponding to the target facial features.

所述人脸识别方法，其中，所述将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内具体包括：Said face recognition method, wherein said adding said additionally recorded face feature into the face feature group corresponding to said target face feature specifically includes:

检测所述目标人脸特征对应的人脸特征组中是否包含补录人脸特征；Detecting whether the face feature group corresponding to the target face feature contains supplementary face features;

当未包含补录人脸特征时，将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内；When not including supplementary recording of human face features, said supplementary recording of human face features is entered in the facial feature group corresponding to the target facial features;

当包含补录人脸特征时，采用所述补录人脸特征替换所述目标人脸特征对应的人脸特征组内的补录人脸特征。When the supplementary facial feature is included, the supplementary facial feature in the facial feature group corresponding to the target facial feature is replaced by the supplementary facial feature.

所述人脸识别方法，其中，所述方法还包括：The face recognition method, wherein the method also includes:

当未查找到目标人脸特征时，记录所述面部图像的采集次数，并重新执行所述采集包含人脸的面部图像的步骤。When the target face feature is not found, record the number of times the facial image is collected, and re-execute the step of collecting a facial image containing a human face.

本申请实施例第二方面提供了一种人脸识别装置，所述的人脸识别装置包括：The second aspect of the embodiment of the present application provides a face recognition device, and the face recognition device includes:

提取单元，用于获取包含人脸的面部图像，并提取所述面部图像对应的人脸特征；An extraction unit, configured to obtain a facial image containing a human face, and extract facial features corresponding to the facial image;

查找单元，用于基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征；其中，若干所述人脸特征组中的每组人脸特征组均包括注册人脸特征；或者包括所述注册人脸特征和补录人脸特征，所述注册人脸特征与所述补录人脸特征的录入时间不同；A search unit, configured to search for target facial features that match the facial features in several pre-stored facial feature groups based on preset matching conditions; wherein, each set of facial features in the several facial feature groups The groups all include registered facial features; or include the registered facial features and supplementary facial features, and the registration time of the registered facial features is different from that of the supplementary facial features;

获取单元，用于当查找到目标人脸特征时，获取所述面部图像的采集次数；An acquisition unit, configured to acquire the number of acquisitions of the facial image when the target face feature is found;

录入单元，用于当所述采集次数大于或者等于预设次数阈值时，将所述人脸特征作为补录人脸特征,并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。The entry unit is configured to use the face feature as a re-recorded face feature when the number of acquisitions is greater than or equal to a preset number of times threshold, and record the re-recorded face feature into the target face feature corresponding In the face feature group.

本申请实施例第三方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如上任一所述的人脸识别方法中的步骤。The third aspect of the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to Realize the steps in the face recognition method as described above.

本申请实施例第四方面提供了一种电子设备，其包括：采集装置、存储器及分别与所述采集装置、所述存储器连接的处理器；其中：The fourth aspect of the embodiment of the present application provides an electronic device, which includes: a collection device, a memory, and a processor respectively connected to the collection device and the memory; wherein:

所述采集装置，用于采集包含人脸的面部图像；The collection device is used to collect facial images including human faces;

存储器，用于存储所述面部图像及可被所述处理器执行的计算机可读程序；a memory for storing the facial image and a computer-readable program executable by the processor;

所述处理器，用于执行所述计算机可读程序时实现如上任一所述的人脸识别方法中的步骤。The processor is configured to implement the steps in any one of the face recognition methods described above when executing the computer-readable program.

有益效果：与现有技术相比，本申请提供了一种人脸识别方法、装置、存储介质及电子设备，所述方法包括采集包含人脸的面部图像并提取面部图像对应的人脸特征；在预存的若干人脸特征组中查找与人脸特征相匹配的目标人脸特征；当查找到目标人脸特征时获取面部图像的采集次数；若采集次数大于或等于预设次数阈值，则基于人脸特征确定补录人脸特征，并将补录人脸特征录入目标人脸特征对应的人脸特征组内。本实施例通过将需要通过多次对比才能识别成功的用户的识别成功的人脸特征作为补录人脸特征录入人脸特征组内，使得人脸特征组内存储有需要多次比对的外部环境下的人脸特征，这样可以增加用户在该外部环境下的识别通过率，从而可以减少外部环境对人脸识别的影响。Beneficial effects: Compared with the prior art, the present application provides a face recognition method, device, storage medium and electronic equipment, the method includes collecting a facial image containing a human face and extracting facial features corresponding to the facial image; Search for target face features that match the face features in several pre-stored face feature groups; when the target face features are found, obtain the number of acquisitions of facial images; if the number of acquisitions is greater than or equal to the preset number of times threshold, then based on The face feature determines the added face feature, and enters the added face feature into the face feature group corresponding to the target face feature. In this embodiment, the successfully recognized face features of users who need to be compared multiple times to be successfully identified are entered into the face feature group as supplementary face features, so that the external facial features that require multiple comparisons are stored in the face feature group. The face features in the environment can increase the recognition pass rate of the user in the external environment, thereby reducing the impact of the external environment on face recognition.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员而言，在不符创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings under the premise of not conforming to creative work.

图1为本申请提供的人脸识别方法的流程图。FIG. 1 is a flow chart of the face recognition method provided by the present application.

图2为本申请提供的人脸识别方法中向目标人脸特征组中录入补录人脸特征和融合人脸特征的一个例子的流程示意图。FIG. 2 is a schematic flowchart of an example of inputting supplementary facial features and fusing facial features into the target facial feature group in the face recognition method provided by the present application.

图3为本申请提供的人脸识别装置的结构原理图。FIG. 3 is a structural schematic diagram of a face recognition device provided by the present application.

图4为本申请提供的电子设备的结构原理图。FIG. 4 is a schematic diagram of the structure of the electronic device provided by the present application.

具体实施方式Detailed ways

本申请提供一种人脸识别方法、装置、存储介质及电子设备，为使本申请的目的、技术方案及效果更加清楚、明确，以下参照附图并举实施例对本申请进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。This application provides a face recognition method, device, storage medium, and electronic equipment. In order to make the purpose, technical solution, and effect of this application clearer and clearer, the following describes this application in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式 “一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。Those skilled in the art will understand that the singular forms "a", "an", "said" and "the" used herein may also include plural forms unless otherwise stated. It should be further understood that the word "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Additionally, "connected" or "coupled" as used herein may include wireless connection or wireless coupling. The expression "and/or" used herein includes all or any elements and all combinations of one or more associated listed items.

本技术领域技术人员可以理解，除非另外定义，这里使用的所有术语(包括技术术语和科学术语)，具有与本申请所属领域中的普通技术人员的一般理解相同的意义。还应该理解的是，诸如通用字典中定义的那些术语，应该被理解为具有与现有技术的上下文中的意义一致的意义，并且除非像这里一样被特定定义，否则不会用理想化或过于正式的含义来解释。Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have meanings consistent with their meaning in the context of the prior art, and unless specifically defined as herein, are not intended to be idealized or overly Formal meaning to explain.

发明人经过研究发现，人脸识别(Facial Recognition)技术被广泛应用于各个领域，例如，安防系统，门禁系统，支付系统，自主服务系统等。人脸识别技术是通过视频采集设备采集包含人脸的面部图像，并提取面部图像携带的人脸特征(例如，五官位置、脸型等)，然后再与人脸特征数据库进行比对以识别用户身份。然而，由于人脸识别技术容易受外部环境(例如，光照、姿态以及遮挡等)影响，从而人脸识别技术的实际应用过程中容易出现因外部环境改变而导致的识别失败的问题。例如，人脸特征数据库中的人脸特征在室内环境下录入的，用户的面部图像在室外环境下采集的，由于室内环境中的光照与室外环境中的光照不同，从而在将基于该面部图像获取到人脸特征与人脸特征数据库进行比对时容易出现识别失败的问题。The inventor found through research that face recognition (Facial Recognition) technology is widely used in various fields, such as security systems, access control systems, payment systems, and autonomous service systems. Face recognition technology is to collect facial images containing human faces through video acquisition equipment, and extract facial features carried by facial images (such as facial features, face shape, etc.), and then compare them with facial feature databases to identify users. . However, since the face recognition technology is easily affected by the external environment (eg, illumination, posture, and occlusion, etc.), the problem of recognition failure caused by changes in the external environment is prone to occur during the actual application of the face recognition technology. For example, the face features in the face feature database are entered in an indoor environment, and the user's facial image is collected in an outdoor environment. Since the illumination in the indoor environment is different from that in the outdoor environment, the facial image will be When the facial features are obtained and compared with the facial feature database, the problem of recognition failure is prone to occur.

为了解决上述问题，在本申请实施例中，采集包含人脸的面部图像并提取面部图像对应的人脸特征；在预存的若干人脸特征组中查找与人脸特征相匹配的目标人脸特征；当查找到目标人脸特征时，判定人脸识别成功并获取面部图像的采集次数；若采集次数大于或等于预设次数阈值，则基 于人脸特征确定补录人脸特征，并将补录人脸特征录入目标人脸特征对应的人脸特征组内。本实施例通过将需要通过多次对比才能识别成功的用户的识别成功的人脸特征作为补录人脸特征录入人脸特征组内，使得人脸特征组内存储有需要多次比对的外部环境下的人脸特征，这样可以增加用户在该外部环境下的识别通过率，从而可以减少外部环境对人脸识别的影响。In order to solve the above problems, in the embodiment of the present application, the facial image containing the human face is collected and the facial features corresponding to the facial image are extracted; the target facial features matching the facial features are searched in several pre-stored facial feature groups ; When the target face feature is found, it is determined that the face recognition is successful and the number of acquisitions of the facial image is obtained; The face feature is entered into the face feature group corresponding to the target face feature. In this embodiment, the successfully recognized face features of users who need to be compared multiple times to be successfully identified are entered into the face feature group as supplementary face features, so that the external facial features that require multiple comparisons are stored in the face feature group. The face features in the environment can increase the recognition pass rate of the user in the external environment, thereby reducing the impact of the external environment on face recognition.

下面结合附图，通过对实施例的描述，对申请内容作进一步说明。The content of the application will be further explained by describing the embodiments below in conjunction with the accompanying drawings.

本实施例提供了一种人脸识别方法，本实施例提供的人脸识别方法可以装配有人脸识别功能的电子设备上，电子设备配置有图像采集装置以便通过图像采集装置来采集包含人脸的面部图像。其中，所述电子设备可以包括但不限于诸如具有图像采集装置(例如，前置摄像头和/或后置摄像头)的移动电话，膝上形计算机或平板计算机，以及智能门锁等。This embodiment provides a face recognition method. The face recognition method provided by this embodiment can be equipped with an electronic device with a face recognition function. The electronic device is equipped with an image acquisition device so that the image acquisition device can be used to collect images containing human faces. facial image. Wherein, the electronic device may include, but not limited to, a mobile phone with an image acquisition device (for example, a front camera and/or a rear camera), a laptop computer or a tablet computer, and a smart door lock.

如图1所述，本实施例提供的人脸识别方法具体可以包括：As shown in Figure 1, the face recognition method provided in this embodiment may specifically include:

S10、获取包含人脸的面部图像，并提取所述面部图像对应的人脸特征。S10. Acquire a facial image including a human face, and extract facial features corresponding to the facial image.

具体地，面部图像为携带有人脸的面部器官的图像，也就是说，面部图像携带有用户的眼睛、鼻子、嘴部以及耳朵等面部器官。其中，所述面部图像可以通过运行有本实施例提供的人脸识别方法的电子设备采集得到的，也可以是通过外部设备采集并发送给运行有本实施例提供的人脸识别方法的电子设备的，还可以是通过云端或者后台服务器获取得到的。在一个实现方式中，所述面板图像通过运行有本实施例提供的人脸识别方法的电子设备采集得到的，其中，该电子设备连接有图像采集设备(例如，摄像头等)，该图像采集设备可以装置于电子设备上，也可以是通过有线或者无线于该电子设备相连接等。Specifically, the facial image is an image carrying facial organs of a human face, that is, the facial image carries facial organs such as the user's eyes, nose, mouth, and ears. Wherein, the facial image can be collected by an electronic device running the face recognition method provided in this embodiment, or can be collected by an external device and sent to the electronic device running the face recognition method provided in this embodiment It can also be obtained through the cloud or a background server. In one implementation, the panel image is acquired by an electronic device running the face recognition method provided in this embodiment, wherein the electronic device is connected to an image acquisition device (for example, a camera, etc.), and the image acquisition device It may be installed on an electronic device, or may be connected to the electronic device by wire or wirelessly.

在本实施例的一个实现方式中，人脸特征可以包括人脸关键部位的特征点，以及特征点之间的相对位置以及相对距离等，例如，人脸特征包括人脸上眼睛、鼻子、嘴以及下巴等关键部位的特征点，以及各特征点之间的相对位置和相对距离；或者是，人脸特征可以包括人脸的68点人脸特征点的位置信息等。其中，人脸特征可以采用传统人脸识别算法提取到的，例如，基于图像分割的人脸特征提取算法等，或者是，人脸特征可以通过 基于深度学习的神经网络模型获取提到的，例如，经过训练的卷积神经网络、循环神经网络模型以及双向循环神经网络模型等。In an implementation of this embodiment, the facial features may include feature points of key parts of the human face, as well as the relative positions and relative distances between the feature points. For example, the facial features include eyes, nose, mouth, etc. And the feature points of key parts such as the chin, and the relative positions and relative distances between the feature points; or, the face features can include the position information of 68 face feature points of the face. Among them, the face features can be extracted by using traditional face recognition algorithms, for example, face feature extraction algorithms based on image segmentation, etc., or, face features can be obtained through deep learning-based neural network models, such as , trained convolutional neural network, recurrent neural network model, and bidirectional recurrent neural network model.

S20、基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征。S20. Based on preset matching conditions, search for a target face feature matching the face feature in several pre-stored face feature groups.

具体地，所述预设匹配条件为预先设置的，用于确定人脸特征对应的目标人脸特征的依据，其中，预设匹配条件包括相似度大于预设相似度阈值和/或差异度小于预设差异度阈值。所述差异度用于反映人脸特征与目标人脸特征之间的差异程度，所述相似度用于反映人脸特征与目标人脸特征之间的相似程度，其中，所述相似度均可以为通过计算人脸特征中的特征点与目标人脸特征中的特征点的欧式距离得到，或者是，通过统计人脸特征中的特征点与目标人脸特征中的特征点的匹配数量得到的，或者是通过经过训练的神经网络模型确定得到等。所述差异度可以通过计算人脸特征中的特征点与目标人脸特征中的特征点的位置差得到，或者是，通过统计人脸特征中的特征点与目标人脸特征中的特征点的不匹配数量得到的，或者是，通过经过训练的神经网络模型确定得到等。Specifically, the preset matching condition is preset and is used to determine the basis of the target facial feature corresponding to the facial feature, wherein the preset matching condition includes that the similarity is greater than the preset similarity threshold and/or the difference is less than Preset difference threshold. The degree of difference is used to reflect the degree of difference between the facial features and the target facial features, and the similarity is used to reflect the degree of similarity between the facial features and the target facial features, wherein the similarity can be It is obtained by calculating the Euclidean distance between the feature points in the face feature and the feature points in the target face feature, or by counting the number of matching points between the feature points in the face feature and the feature points in the target face feature , or determined by the trained neural network model, etc. The degree of difference can be obtained by calculating the position difference between the feature points in the facial features and the feature points in the target facial features, or by counting the difference between the feature points in the facial features and the feature points in the target facial features The number of mismatches is obtained, or it is determined by the trained neural network model, etc.

所述若干人脸特征组中的每组人脸特征组均包括注册人脸特征，或者包括注册人脸特征和补录人脸特征，其中，若干人脸特征组中的各人脸特征组各自包括的注册人脸特征互不相同，各人脸特征组中包括的注册人脸特征和补录人脸特征为同一用户不同录入时间录入的人脸特征。可以理解的是，若干人脸特征组中的每组人脸特征组对应一个人脸，各人脸特征组各自对应的人脸互不相同，并且当人脸特征组包括注册人脸特征和补录人脸特征时，注册人脸特征和补录人脸特征均为该人脸特征组所属人脸的人脸特征，并且注册人脸特征的录入时间与补录人脸特征的录入时间不相同。Each group of facial feature groups in the several facial feature groups includes registered facial features, or includes registered facial features and supplementary recorded facial features, wherein each facial feature group in several facial feature groups The included registered facial features are different from each other, and the registered facial features and re-recorded facial features included in each facial feature group are facial features entered by the same user at different recording times. It can be understood that each group of human face feature groups in several human face feature groups corresponds to a human face, and the faces corresponding to each human face feature group are different from each other, and when the human face feature group includes registered human face features and supplementary When recording facial features, both the registered facial features and supplementary recorded facial features are the facial features of the face to which the facial feature group belongs, and the input time of registered facial features is different from that of supplementary recorded facial features .

举例说明：假设若干人脸特征组包括人脸特征组A和人脸特征组B，人脸特征组A包括注册人脸特征，人脸特征组B包括注册人脸特征和补录人脸特征，那么人脸特征组A对应的人脸a与人脸特征组B对应人脸b不相同。人脸特征组B中的注册人脸特征和补录人脸特征均为人脸b的人脸特征，并且注册人脸特征和补录人脸特征的录入时间不同。For example: Assuming that several face feature groups include face feature group A and face feature group B, face feature group A includes registered face features, face feature group B includes registered face features and supplementary face features, Then the face a corresponding to the face feature group A is different from the face b corresponding to the face feature group B. The registered facial features and supplementary facial features in facial feature group B are both the facial features of face b, and the registration time of the registered facial features and supplementary facial features are different.

在本实施例的一个实现方式中，注册人脸特征的录入时间早于补录人脸特征的录入时间，例如，注册人脸特征的录入时间为用户第一次注册时录入的人脸特征，补录人脸特征为在基于注册人脸特征对应人脸进行人脸验证的过程中录入的人脸特征。此外，注册人脸特征的录入场景可以与补录人脸特征的录入场景不同，例如，注册人脸特征的录入场景为室内场景，补录人脸特征的录入场景为室外场景，或者是，注册人脸特征的录入场景为亮光场景，补录人脸特征的录入场景为暗光场景等。这样人脸特征组中携带有不同录入场景下的人脸特征，从而可以减少录入场景中的外部因素对人脸识别的影响，进而可以提高识别成功率。In an implementation of this embodiment, the entry time of the registered face features is earlier than the entry time of the additionally recorded face features, for example, the entry time of the registered face features is the face features entered by the user when he first registered, The supplementary facial features are the facial features entered during the process of face verification based on the faces corresponding to the registered facial features. In addition, the entry scene of registered facial features can be different from the entry scene of supplementary recording of human features. The recording scene of facial features is a bright scene, and the recording scene of supplementary recording of facial features is a dark scene, etc. In this way, the face feature group carries the face features in different recording scenarios, thereby reducing the impact of external factors in the recording scene on face recognition, thereby improving the recognition success rate.

在本实施例的一个实现方式中，所述补录人脸特征可以为困难人脸特征和/或融合人脸特征，其中，所述困难人脸特征为根据人脸识别成功时采集到的面部图像提取到的人脸特征；所述融合人脸特征为基于所述注册人脸特征以及所述困难人脸特征融合得到的。可以理解的是，所述补录人脸特征可以仅包括通过对采集到的面部图像识别得到的困难人脸特征，也可以仅包括是基于注册人脸特征以及困难人脸特征融合得到的融合人脸特征，还可以同时包括困难人脸特征和融合人脸特征。在一个典型实施例中，所述补录人脸特征困难人脸特征和融合人脸特征。In an implementation of this embodiment, the supplementary facial features may be difficult facial features and/or fused facial features, wherein the difficult facial features are based on facial features collected when face recognition is successful. The facial features extracted from the image; the fused facial features are obtained based on the fusion of the registered facial features and the difficult facial features. It can be understood that the supplementary facial features may only include difficult facial features obtained through recognition of the collected facial images, or may only include fused facial features based on the fusion of registered facial features and difficult facial features. Face features can also include difficult face features and fusion face features. In a typical embodiment, the supplementary recording of difficult facial features and fusion of facial features.

在本实施例的一个实现方式中，所述融合人脸特征可以注册人脸特征和困难人脸特征加权得到。其中，所述加权过程具体可以为在获取到困难人脸特征后将注册人脸特征和困难人脸特征做加权融合求得两个特征的特征中心，并将求得的特征中心作为融合人脸特征录入人脸特征组中，这样可以提高人脸识别的可识别范围，如角度，光照等，比仅存储注册人脸特征的人脸特征组的识别效果好。在一个典型实施例中，在将注册人脸特征和困难人脸特征做加权融合时，注册人脸特征的加权系数与困难人脸特征的加权系数的和为1。相应的，融合人脸特征的计算公式可以为：In an implementation of this embodiment, the fused facial features can be obtained by weighting registered facial features and difficult facial features. Wherein, the weighting process may specifically be to perform weighted fusion of registered face features and difficult face features to obtain the feature center of the two features after obtaining the difficult face feature, and use the obtained feature center as the fusion face The features are entered into the face feature group, which can improve the recognizable range of face recognition, such as angle, illumination, etc., which is better than the recognition effect of the face feature group that only stores registered face features. In a typical embodiment, when performing weighted fusion of the registered facial features and the difficult facial features, the sum of the weighting coefficients of the registered facial features and the weighting coefficients of the difficult facial features is 1. Correspondingly, the calculation formula for fusing facial features can be:

embd_mixed＝α·embd_reg+(1-α)·embd_recembd_mixed=α·embd_reg+(1-α)·embd_rec

其中，embd_mixed表示融合人脸特征，α表示注册人脸特征的加权系数，其取值范围为0-1，1-α表示补录人脸特征的加权系数，embd_reg表 示注册人脸特征，embd_rec表示补录人脸特征。在一典型实现方式中，α＝0.5。Among them, embd_mixed indicates the fusion of facial features, α indicates the weighting coefficient of registered facial features, and its value range is 0-1, 1-α indicates the weighting coefficient of re-recorded facial features, embd_reg indicates registered facial features, and embd_rec indicates Supplementary facial features. In a typical implementation, α=0.5.

在本实施例的一个实现方式中，由于若干人脸特征组中可以存在包括注册人脸特征以及补录人脸特征的人脸特征组，补录人脸特征可以包括困难人脸特征和/或融合人脸特征，从而基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征时，对于每组人脸特征组，将人脸特征与该人脸特征组中的所有人脸特征进行比对，其中，所有人脸特征可以包括注册人脸特征，或者包括注册人脸特征和补录人脸特征；当人脸特征组中至少存在一个候选人脸特征，候选人脸特征与所述人脸特征满足预设匹配条件时，将该候选人脸特征作为与所述人脸特征相匹配的目标人脸特，则表示人脸识别成功。可以理解的是，若人脸特征组中包含注册人脸特征、困难人脸特征以及融合人脸特征时，在进行人脸识别时，可以同时将人脸特征与人脸特征组中的该三个人脸特征进行比较，如果有人脸特征组中的任意一个人脸特征与识别到的人脸特征满足预设匹配条件，则认为人脸识别成功。In an implementation of this embodiment, since there may be facial feature groups including registered facial features and supplementary facial features in several facial feature groups, supplementary facial features may include difficult facial features and/or Fusion of facial features, so that when searching for target facial features matching the facial features in several pre-stored facial feature groups based on preset matching conditions, for each group of facial feature groups, combining the facial features with the All facial features in the facial feature group are compared, wherein, all facial features can include registered facial features, or include registered facial features and supplementary facial features; when there is at least one candidate in the facial feature group Face feature, when the candidate face feature and the face feature meet the preset matching condition, the candidate face feature is used as the target face feature matched with the face feature, which means the face recognition is successful. It can be understood that if the face feature group includes registered face features, difficult face features and fusion face features, when performing face recognition, the face features and the three face features in the face feature group can be combined simultaneously. If any face feature in the face feature group meets the preset matching condition with the recognized face feature, the face recognition is considered successful.

例如，若干人脸特征组中包括人脸特征组A和人脸特征组B，人脸特征组A包括注册人脸特征a，人脸特征组B包括注册人脸特征b，困难人脸特征c以及融合人脸特征d，那么基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征时，可以分别将所述人脸特征分别与注册人脸特征a、注册人脸特征b，困难人脸特征c以及融合人脸特征d进行比较，如果所述人脸特征与注册人脸特征a满足预设匹配条件，则注册人脸特征a为与所述人脸特征相匹配的目标人脸特征，并判定人脸识别成功；如果所述人脸特征与注册人脸特征b和/或困难人脸特征c和/或融合人脸特征d满足预设匹配条件，则注册人脸特征b和/或困难人脸特征c和/或融合人脸特征d为与所述人脸特征相匹配的目标人脸特征，并判定人脸识别成功。For example, several face feature groups include face feature group A and face feature group B, face feature group A includes registered face feature a, face feature group B includes registered face feature b, difficult face feature c and fusion face feature d, then when searching for target face features matching the face features in several pre-stored face feature groups based on preset matching conditions, the face features can be respectively matched with the registered person Face feature a, registered face feature b, difficult face feature c and fusion face feature d are compared, if the face feature and registered face feature a meet the preset matching conditions, then the registered face feature a is the same as The target face feature that described face feature matches, and judges that face recognition is successful; If the matching condition is set, the face feature b and/or the difficult face feature c and/or the fusion face feature d are registered as the target face feature matching the face feature, and the face recognition is determined to be successful.

S30、当查找到目标人脸特征组时，获取采集包含人脸的面部图像的采集次数。S30. When the target face feature group is found, acquire the number of acquisitions of facial images including human faces.

具体地，采集次数为采集到所述面部图像的次数，可以理解的是，采集次数等于该人脸进行人脸识别时人脸识别装置采集到的包含该人脸的面部图像的图像数量。所述采集次数用于反映用户在该识别场景中执行人脸识别的难易程度，其中，采集次数越多，说明该识别场景中执行人脸识别的难度越大，反之，采集次数越少，说明该识别场景中执行人脸识别的难度越小。由此，可以基于所述采集次数来确定是需要为该识别场景补录人脸特征，以增大该识别场景下人脸识别的识别通过率。此外，所述采集次数可以是在对进行人脸识别的过程中记录，也可以是基对人脸识别的过程中存储的所述面部图像的图像数量确定等。Specifically, the number of collection times is the number of times the facial images are collected, and it can be understood that the number of collection times is equal to the number of facial images including the face collected by the face recognition device when the face is recognized. The number of times of collection is used to reflect the degree of difficulty for the user to perform face recognition in the recognition scene. The more times of collection, the greater the difficulty of performing face recognition in the recognition scene. Conversely, the fewer the number of times of collection, It shows that the difficulty of performing face recognition in this recognition scene is smaller. Therefore, based on the number of acquisitions, it can be determined whether it is necessary to additionally record face features for the recognition scene, so as to increase the recognition pass rate of face recognition in the recognition scene. In addition, the number of acquisitions may be recorded during the face recognition process, or may be determined based on the number of images of the facial images stored during the face recognition process.

在本实施例的一个实现方式中，在基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征时，还可以存在未查找到目标人脸特征的情况，在未查找到目标人脸特征时，可能因面部图像的采集场景与目标人脸特征对应的人脸特征组中的各人脸特征的采集场景不同，而导致识别失败，也可能是因是人脸特征数据集中未存储该用户对应的目标人脸特征组。基于此，所述方法还包括：当未查找到目标人脸特征时，记录所述面部图像的采集次数，并重新执行所述采集包含人脸的面部图像的步骤，以便于在查找到目标人脸特征时，可以获取到面部图像的采集次数。In an implementation of this embodiment, when searching for a target face feature that matches the face feature in several pre-stored face feature groups based on preset matching conditions, there may also be cases where no target face is found. In the case of features, when the target face feature is not found, the acquisition scene of the face image may be different from that of each face feature in the face feature group corresponding to the target face feature, resulting in recognition failure, or The reason is that the target face feature group corresponding to the user is not stored in the face feature dataset. Based on this, the method also includes: when the target face feature is not found, record the number of times the facial image is collected, and re-execute the step of collecting a facial image containing a human face, so that when the target person is found When facial features are used, the number of acquisitions of facial images can be obtained.

S40、若所述采集次数大于或者等于预设次数阈值，则将所述人脸特征作为补录人脸特征,并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。S40. If the number of acquisitions is greater than or equal to the preset number of times threshold, then use the facial feature as a supplementary facial feature, and record the supplementary facial feature into the facial feature corresponding to the target facial feature s.

具体地，当采集次数大于或者等于预设次数阈值时，说明该人脸识别场景为人脸识别的困难场景，此时可以基于当前成功识别时根据采集到的面部图像所提取的人脸特征确定补录人脸特征，并将补录人脸特征作为该困难场景的补入人脸特征，这样在该困难场景下，可以通过补录的补录人脸特征来提高该困难场景的识别通过率，从而可以提高人脸识别的识别通过率。此外，本实施例是在查找到所述人脸特征对应的目标人脸特征时，才会为确定补录人脸特征，这样可以保证该面部图像为注册用户的面部图 像，进而可以人脸特征组中的各人脸特征均为该注册用户的人脸特征，避免将错误的人脸特征录入到该注册用户的人脸特征组内。此外，当采集次数小于预设次数阈值时，说明该人脸识别场景不为人脸识别的困难场景，此时无需为该人脸识别场景录入特征，从而可以保持人脸特征组不变并进行下一帧识别。Specifically, when the number of collection times is greater than or equal to the preset number of times threshold, it indicates that the face recognition scene is a difficult scene for face recognition. Record the face features, and use the re-recorded face features as the supplementary face features of the difficult scene, so that in this difficult scene, the recognition pass rate of the difficult scene can be improved by supplementing the re-recorded face features, Thereby, the recognition pass rate of face recognition can be improved. In addition, in this embodiment, when the target face feature corresponding to the face feature is found, the face feature will be added for determination, so as to ensure that the face image is the face image of the registered user, and then the face feature Each face feature in the group is the face feature of the registered user, so as to avoid entering wrong face features into the face feature group of the registered user. In addition, when the number of acquisitions is less than the preset threshold, it means that the face recognition scene is not a difficult scene for face recognition. At this time, there is no need to enter features for the face recognition scene, so that the face feature group can be kept unchanged and the next step can be performed. A frame recognition.

在一个实施例中，由于补录人脸特征可以是通过面部图像提取的困难人脸特征，和/或基于困难人脸特征与注册人脸特征的融合人脸特征。从而，在将补录人脸特征录入人脸特征组后人脸特征组可能包括以下情况，一、仅包括注册人脸特征、困难人脸特征；二、仅包括注册人脸特征、融合人脸特征、三同时包括人脸特征、困难人脸特征、融合人脸特征。In one embodiment, the supplementary facial features may be difficult facial features extracted from facial images, and/or fusion facial features based on difficult facial features and registered facial features. Thereby, the face feature group may include the following situations after the supplementary face features are entered into the face feature group, one, only include registered face features, difficult face features; two, only include registered face features, fusion face features Features, three include face features, difficult face features, and fusion face features.

在一个实施例中，所述补录人脸特征包括困难人脸特征和融合人脸特征；所述若所述采集次数大于或者等于预设次数阈值，则基于所述人脸特征确定所述人脸特征对应的补录人脸特征，并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内具体包括：In one embodiment, the re-recorded facial features include difficult facial features and fusion facial features; if the number of acquisitions is greater than or equal to a preset number of thresholds, then determine the human face based on the facial features Supplementary recording of human face features corresponding to facial features, and entry of said supplementary recording of human facial features into the corresponding human face feature group of said target human facial features specifically includes:

将所述人脸特征作为困难人脸特征，并基于所述困难人脸特征和所述目标人脸特征所对应的人脸特征组内的注册人脸特征确定融合人脸特征；Using the facial features as difficult facial features, and determining fusion facial features based on the registered facial features in the facial feature group corresponding to the difficult facial features and the target facial features;

将所述困难人脸特征和所述融合人脸特征作为补录人脸特征，并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。The difficult facial features and the fused facial features are used as supplementary facial features, and the supplementary facial features are entered into the facial feature group corresponding to the target facial features.

具体地，当所述采集次数大于或者预设次数阈值时，说明所述人脸特征为困难场景下采集的人脸特征，从而可以将该人脸特征作为困难人脸特征，然后在基于困难人脸特征以及目标人脸特征所对应的人脸特征组内的注册人脸特征确定融合人脸特征，以得到补录人脸特征。其中，特征融合的方式以及过程可以采用上述的特征融合的方式以及过程，这里就不再赘述Specifically, when the number of times of collection is greater than or the preset number of times threshold, it indicates that the face feature is a face feature collected in a difficult scene, so that the face feature can be used as a difficult face feature, and then based on the difficult person The facial features and the registered facial features in the facial feature group corresponding to the target facial features determine and fuse the facial features to obtain supplementary facial features. Among them, the method and process of feature fusion can adopt the above-mentioned method and process of feature fusion, which will not be repeated here.

在一个实施例中，所述将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内具体包括：In one embodiment, the entering the supplementary facial feature into the facial feature group corresponding to the target facial feature specifically includes:

检测所述目标人脸特征对应的人脸特征组中是否包含补录人脸特征；Detecting whether the face feature group corresponding to the target face feature contains supplementary face features;

当未包含补录人脸特征时，将所述补录人脸特征录入所述目标人脸特 征对应的人脸特征组内；When not including supplementary record human face feature, described supplementary record human face feature is entered in the human face feature group corresponding to described target human face feature;

当包含补录人脸特征时，采用所述补录人脸特征替换所述目标人脸特征对应的人脸特征组内的补录人脸特征。When the supplementary facial feature is included, the supplementary facial feature in the facial feature group corresponding to the target facial feature is replaced by the supplementary facial feature.

具体地，所述目标人脸特征对应的人脸特征组内可以仅包括注册人脸特征，也可以包括注册人脸特征和补录人脸特征，从而在目标人脸特征对应的人脸特征组中录入补录人脸特征时，可以检测目标人脸特征对应的人脸特征组中是否存在补录人脸特征，如果未存在补录人脸特征则可以直接将补录人脸特征录入目标人脸特征对应的人脸特征组，如果存在补录人脸特征则可以将已存储的补录人脸特征删除，并将基于人脸特征确定的补录人类特征录入目标人脸特征对应的人脸特征组。Specifically, the face feature group corresponding to the target face feature may only include registered face features, or may include registered face features and supplementary face features, so that the face feature group corresponding to the target face feature When entering supplementary facial features, it can detect whether there are supplementary facial features in the face feature group corresponding to the target facial feature. If there is no supplementary facial feature, you can directly enter the supplementary facial features into the target The face feature group corresponding to the face feature. If there is a supplementary facial feature, the stored supplementary facial feature can be deleted, and the supplementary human feature determined based on the facial feature can be entered into the face corresponding to the target facial feature. feature group.

例如，在查找到目标人脸特征且采集次数大于或者等于预设次数阈值时确定得到补录人脸特征a，若目标人脸特征对应的人脸特征组ID1中存放注册人脸特征(即原始注册特征)，注册人脸特征为室内场景录入的人脸特征，可以直接将补录人脸特征(即该困难场景对应的困难人脸特征和/或融合人脸特征)录入人脸特征组ID1内；若目标人脸特征对应的人脸特征组ID1中存放注册人脸特征(即原始注册特征)和补录人脸特征b，那么将补录人脸特征b从人脸特征组ID1中删除，并将补录人脸特征a存储于人脸特征组ID1，以更新人脸特征组ID1中的补录人脸特征，以使得人脸特征组ID1中的补录人脸特征更好的满足当前人脸识别设备的使用场景。For example, when the target face feature is found and the number of acquisitions is greater than or equal to the preset times threshold, it is determined to obtain the added face feature a, if the face feature group ID1 corresponding to the target face feature stores the registered face feature (i.e. Registration feature), the registered face feature is the face feature entered in the indoor scene, and the supplementary face feature (that is, the difficult face feature corresponding to the difficult scene and/or the fusion face feature) can be directly entered into the face feature group ID1 If the face feature group ID1 corresponding to the target face feature stores the registered face feature (that is, the original registration feature) and the added face feature b, then the added face feature b will be deleted from the face feature group ID1 , and store the re-recorded face feature a in the face feature group ID1, to update the re-recorded face feature in the face feature group ID1, so that the re-recorded face feature in the face feature group ID1 better satisfies Current usage scenarios of face recognition devices.

在一个实施例中，为了确保补录人脸特征的有效性，在对人脸识别成功时采集到的包含人脸的面部图像进行特征提取得到困难人脸特征，并将困难人脸特征与注册人脸特征进行融合得到融合特征后，还需要将融合人脸特征与注册人脸特征进行相似度比较，只有当融合人脸特征与注册人脸特征的相似度大于或等于预设相似度阈值时，前述的困难人脸特征和/或融合人脸特征才会作为有效特征补录进对应的目标人脸的人脸特征组内，或者，若该目标人脸的人脸特征组内已存有注册人脸特征、困难人脸特征和融合人脸特征，则对其中的困难人脸特征和融合人脸特征进行更新，注册人脸特征始终保持不变，否则，即使人脸识别成功，但对该人脸识别成功 时采集的面部图像进行提取的困难人脸特征并据其得到的融合人脸特征也不会被被采用，即不会作为有效的补录人脸特征。In one embodiment, in order to ensure the validity of supplementary facial features, feature extraction is performed on the facial images containing human faces collected when the face recognition is successful to obtain difficult facial features, and the difficult facial features are combined with the registration After the facial features are fused to obtain the fused features, it is necessary to compare the similarity between the fused facial features and the registered facial features. Only when the similarity between the fused facial features and the registered facial features is greater than or equal to the preset similarity threshold , the aforementioned difficult face features and/or fused face features will be added as effective features into the face feature group of the corresponding target face, or, if there is already a face feature group in the target face Register face features, difficult face features and fusion face features, then update the difficult face features and fusion face features, and the registered face features will remain unchanged. Otherwise, even if the face recognition is successful, but the The difficult facial features extracted from the facial images collected when the face recognition is successful and the fused facial features obtained therefrom will not be adopted, that is, they will not be used as effective supplementary facial features.

举例说明：如图2所示，假设用户ID为1的用户在注册时配置有人脸特征组ID1，人脸特征组ID1中设置有三个人脸特征存储位，第一预设匹配条件为相似度大于预设第一相似度阈值；当第一次注册时，将识别到的人脸特征作为注册人脸特征embd_reg录入人脸特征组ID1，当后续人脸识别过程中，当采集用户1的面部图像的采集次数大于预设次数阈值(例如，20次等)后识别成功时，将识别成功的人脸特征作为困难人脸特征，并将注册人脸特征和识别成功的人脸特征进行融合以得到融合人脸特征embd_mixed，然后检测融合人脸特征与注册人脸特征embd_reg相似度是否大于预设相似度阈值，若大于预设相似度阈值，则将困难人脸特征embd_difficult和融合人脸特征embd_mixed作为补录人脸特征录入人脸特征组ID1，以使得人脸特征组ID1中存储有注册人脸特征、困难人脸特征以及融合人脸特征，若小于获取等于预设相似度阈值，则丢弃困难人脸特征和融合人脸特征，即保持人脸特征组ID1不变。For example: as shown in Figure 2, assume that the user whose user ID is 1 is configured with a face feature group ID1 when registering, and there are three face feature storage bits set in the face feature group ID1, and the first preset matching condition is that the similarity is greater than Preset the first similarity threshold; when registering for the first time, enter the recognized face feature as the registered face feature embd_reg into the face feature group ID1. During the subsequent face recognition process, when the facial image of user 1 is collected When the number of acquisitions is greater than the preset number of thresholds (for example, 20 times, etc.), when the recognition is successful, the successfully recognized face features will be regarded as difficult face features, and the registered face features will be fused with the successfully recognized face features to obtain Fuse the face feature embd_mixed, and then detect whether the similarity between the fused face feature and the registered face feature embd_reg is greater than the preset similarity threshold, and if it is greater than the preset similarity threshold, use the difficult face feature embd_difficult and the fused face feature embd_mixed as Supplementary facial features are entered into the facial feature group ID1, so that the registered facial features, difficult facial features and fusion facial features are stored in the facial feature group ID1. Face features and fused face features, that is, keep the face feature group ID1 unchanged.

综上所述，本实施例提供了一种人脸识别方法，所述方法包括采集包含人脸的面部图像并提取面部图像对应的人脸特征；在预存的若干人脸特征组中查找与人脸特征相匹配的目标人脸特征；当查找到目标人脸特征时获取面部图像的采集次数；若采集次数大于或等于预设次数阈值，则基于人脸特征确定补录人脸特征，并将补录人脸特征录入目标人脸特征对应的人脸特征组内。本实施例通过将需要通过多次对比才能识别成功的用户的识别成功的人脸特征作为补录人脸特征录入人脸特征组内，使得人脸特征组内存储有需要多次比对的外部环境下的人脸特征，这样可以增加用户在该外部环境下的识别通过率，从而可以减少外部环境对人脸识别的影响。In summary, the present embodiment provides a face recognition method, the method includes collecting a facial image containing a human face and extracting facial features corresponding to the facial image; The target face feature that matches the face feature; when the target face feature is found, the number of acquisitions of the facial image is obtained; if the number of acquisitions is greater than or equal to the preset number of times threshold, the supplementary facial feature is determined based on the face feature, and the Supplementary facial features are entered into the facial feature group corresponding to the target facial features. In this embodiment, the successfully recognized face features of users who need to be compared multiple times to be successfully identified are entered into the face feature group as supplementary face features, so that the external facial features that require multiple comparisons are stored in the face feature group. The face features in the environment can increase the recognition pass rate of the user in the external environment, thereby reducing the impact of the external environment on face recognition.

基于上述人脸识别方法，本实施例提供了一种人脸识别装置，如图3所示，所述的人脸识别装置包括：Based on the above face recognition method, this embodiment provides a face recognition device, as shown in Figure 3, the face recognition device includes:

提取单元100，用于获取包含人脸的面部图像，并提取所述面部图像对应的人脸特征；Anextraction unit 100, configured to acquire a facial image containing a human face, and extract facial features corresponding to the facial image;

查找单元200，用于基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征，其中，若干所述人脸特征组中的每组人脸特征组均包括注册人脸特征，或者包括所述注册人脸特征和补录人脸特征，所述注册人脸特征与所述补录人脸特征的录入时间不同；Thesearch unit 200 is configured to search for target facial features matching the facial features in several pre-stored facial feature groups based on preset matching conditions, wherein each group of human faces in the several facial feature groups The feature groups all include registered facial features, or include the registered facial features and supplementary facial features, and the registration time of the registered facial features is different from that of the supplementary facial features;

获取单元300，用于当查找到目标人脸特征时，获取所述面部图像的采集次数；Anacquisition unit 300, configured to acquire the number of acquisitions of the facial image when the target face feature is found;

录入单元400，用于当所述采集次数大于或者等于预设次数阈值时，Theentry unit 400 is configured to, when the number of collection times is greater than or equal to a preset number of times threshold,

将所述人脸特征作为补录人脸特征,并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。The facial feature is used as a supplementary facial feature, and the supplementary facial feature is entered into the facial feature group corresponding to the target facial feature.

基于上述人脸识别方法，本实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如上述实施例所述的人脸识别方法中的步骤。Based on the face recognition method described above, this embodiment provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be processed by one or more processors Execute to realize the steps in the face recognition method described in the above-mentioned embodiments.

基于上述人脸识别方法，本申请还提供了一种电子设备，如图4所示，其包括采集装置20、存储器21及分别与所述采集装置20、所述存储器21连接的处理器22。其中，采集装置20，用于采集包含人脸的面部图像；存储器21，用于存储采集装置20采集到的面部图像及可被处理器22执行的计算机可读程序；处理器22，可以调用存储器21中的逻辑指令以执行上述实施例中的方法。在各种实施方案中，电子设备可包括台式计算机、膝上型计算机、嵌入式设备、移动电话、平板电脑、个人数字助理等。Based on the above-mentioned face recognition method, the present application also provides an electronic device, as shown in FIG. 4 , which includes acollection device 20, a memory 21, and a processor 22 connected to thecollection device 20 and the memory 21, respectively. Wherein,acquisition device 20 is used for collecting the facial image that comprises human face; Memory 21 is used for storing the facial image thatacquisition device 20 collects and the computer-readable program that can be executed by processor 22; Processor 22 can call memory 21 to execute the method in the above-mentioned embodiment. In various embodiments, electronic devices may include desktop computers, laptop computers, embedded devices, mobile phones, tablet computers, personal digital assistants, and the like.

可以理解的是，采集装置20包括相机系统，该相机系统包括彩色相机、红外相机、深度相机中的至少一种，以用于采集包含用户人脸的彩色图像、红外图像、深度图像中的至少一种。其中，深度相机可以为基于双目测距原理的深度相机、基于结构光原理的深度相机或基于TOF原理的深度相机。It can be understood that thecollection device 20 includes a camera system, and the camera system includes at least one of a color camera, an infrared camera, and a depth camera, for collecting at least one of a color image, an infrared image, and a depth image containing a user's face. A sort of. Wherein, the depth camera may be a depth camera based on the principle of binocular ranging, a depth camera based on the principle of structured light, or a depth camera based on the principle of TOF.

此外，上述的存储器21中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。In addition, the logic instructions in the above-mentioned memory 21 may be implemented in the form of software function units and when sold or used as an independent product, they may be stored in a computer-readable storage medium.

存储器21作为一种计算机可读存储介质，可设置为存储软件程序、计 算机可执行程序，如本公开实施例中的方法对应的程序指令或模块。处理器22通过运行存储在存储器21中的软件程序、指令或模块，从而执行功能应用以及数据处理，即实现上述实施例中的方法。The memory 21, as a computer-readable storage medium, can be configured to store software programs and computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 22 runs software programs, instructions or modules stored in the memory 21 to execute functional applications and data processing, that is, to implement the methods in the above-mentioned embodiments.

存储器21可包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据电子设备的使用所创建的数据等。此外，存储器21可以包括高速随机存取存储器，还可以包括非易失性存储器。例如，U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等多种可以存储程序代码的介质，也可以是暂态存储介质。The memory 21 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the electronic device, and the like. In addition, the memory 21 may include a high-speed random access memory, and may also include a non-volatile memory. For example, various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., can also be temporary state storage medium.

在一个实施例中，处理器22可包括一个或多个处理单元，处理器22可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器22中。在一些实施例中，处理器22和存储器21可以在同一芯片上实现，在一些实施例中，它们也可以在独立的芯片上分别实现。In one embodiment, the processor 22 may include one or more processing units, and the processor 22 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc. The modem processor primarily handles wireless communications. It can be understood that the modem processor may not be integrated into the processor 22 . In some embodiments, the processor 22 and the memory 21 can be implemented on the same chip, and in some embodiments, they can also be implemented on independent chips.

在一个实施例中，处理器22也可以是通用处理器，例如中央处理器(CPU)、数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件，可以实现或者执行本申请实施例中公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本申请实施例所公开的人脸识别方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。In one embodiment, the processor 22 may also be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic Devices and discrete hardware components can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the face recognition method disclosed in the embodiments of the present application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

此外，上述存储介质以及移动终端中的多条指令处理器加载并执行的具体过程在上述方法中已经详细说明，在这里就不再一一陈述。In addition, the specific process of loading and executing multiple instruction processors in the above storage medium and the mobile terminal has been described in detail in the above method, and will not be described one by one here.

最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不 使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims (10)

Translated fromChinese

一种人脸识别方法，其特征在于，包括：A face recognition method, characterized in that, comprising:获取包含人脸的面部图像，并提取所述面部图像对应的人脸特征；Obtaining a facial image containing a human face, and extracting facial features corresponding to the facial image;基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征；其中，若干所述人脸特征组中的每组人脸特征组均包括注册人脸特征；或者包括所述注册人脸特征和补录人脸特征，所述注册人脸特征与所述补录人脸特征的录入时间不同；Based on preset matching conditions, search for target facial features that match the facial features in several pre-stored facial feature groups; wherein, each group of facial feature groups in the several facial feature groups includes the registered person Facial features; or include the registered facial features and supplementary recording facial features, and the registration time of the registered facial features is different from that of the supplementary recording facial features;当查找到目标人脸特征时，获取所述面部图像的采集次数；When the target face feature is found, the number of acquisitions of the facial image is obtained;若所述采集次数大于或者等于预设次数阈值，则将所述人脸特征作为补录人脸特征,并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。If the number of acquisitions is greater than or equal to the preset number of times threshold, the facial feature is used as a supplementary facial feature, and the supplementary facial feature is entered into the facial feature group corresponding to the target facial feature .根据权利要求1所述人脸识别方法，其特征在于，所述预设匹配条件包括相似度大于预设相似度阈值和/或差异度小于预设差异度阈值。The face recognition method according to claim 1, wherein the preset matching condition includes that the similarity is greater than a preset similarity threshold and/or the difference is smaller than a preset difference threshold.根据权利要求1所述人脸识别方法，其特征在于，所述补录人脸特征包括困难人脸特征和/或融合人脸特征，其中，所述困难人脸特征为提取到的人脸特征；所述融合人脸特征为基于所述注册人脸特征以及所述困难人脸特征融合得到的。The face recognition method according to claim 1, wherein the added facial features include difficult facial features and/or fusion facial features, wherein the difficult facial features are extracted facial features ; The fused facial features are obtained based on the fusion of the registered facial features and the difficult facial features.根据权利要求3所述的人脸识别方法，其特征在于，所述融合人脸特征为所述注册人脸特征和所述困难人脸特征加权得到，其中，所述注册人脸特征的加权系数与所述困难人脸特征的加权系数的和为1。The face recognition method according to claim 3, wherein the fusion face feature is obtained by weighting the registered face feature and the difficult face feature, wherein the weighting coefficient of the registered face feature The sum of the weighting coefficients and the difficult face features is 1.根据权利要求3所述人脸识别方法，其特征在于，所述补录人脸特征包括困难人脸特征和融合人脸特征；所述若所述采集次数大于或者等于预设次数阈值，则基于所述人脸特征确定所述人脸特征对应的补录人脸特征，并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内具体包括：According to the described face recognition method of claim 3, it is characterized in that, the re-recorded face features include difficult face features and fusion face features; if the number of acquisitions is greater than or equal to a preset number of thresholds, then based on The facial features determine the supplementary facial features corresponding to the facial features, and entering the supplementary facial features into the facial feature group corresponding to the target facial features specifically includes:若所述采集次数大于或者等于预设次数阈值，将所述人脸特征作为困难人脸特征，并基于所述困难人脸特征和所述目标人脸特征所对应的人脸 特征组内的注册人脸特征确定融合人脸特征；If the number of acquisitions is greater than or equal to the preset number of times threshold, the face feature is regarded as a difficult face feature, and based on the registration in the face feature group corresponding to the difficult face feature and the target face feature Facial features determine the fusion of facial features;将所述困难人脸特征和所述融合人脸特征作为补录人脸特征，并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。The difficult facial features and the fused facial features are used as supplementary facial features, and the supplementary facial features are entered into the facial feature group corresponding to the target facial features.根据权利要求1或5所述人脸识别方法，其特征在于，所述将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内具体包括：According to the described face recognition method of claim 1 or 5, it is characterized in that, the described supplementary recording of human face features into the face feature group corresponding to the target face feature specifically includes:检测所述目标人脸特征对应的人脸特征组中是否包含补录人脸特征；Detecting whether the face feature group corresponding to the target face feature contains supplementary face features;当未包含补录人脸特征时，将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内；When not including supplementary recording of human face features, said supplementary recording of human face features is entered in the facial feature group corresponding to the target facial features;当包含补录人脸特征时，采用所述补录人脸特征替换所述目标人脸特征对应的人脸特征组内的补录人脸特征。When the supplementary facial feature is included, the supplementary facial feature in the facial feature group corresponding to the target facial feature is replaced by the supplementary facial feature.根据权利要求1所述人脸识别方法，其特征在于，所述方法还包括：The face recognition method according to claim 1, wherein the method further comprises:当未查找到目标人脸特征时，记录所述面部图像的采集次数，并重新执行所述采集包含人脸的面部图像的步骤。When the target face feature is not found, record the number of times the facial image is collected, and re-execute the step of collecting a facial image containing a human face.一种人脸识别装置，其特征在于，包括：A face recognition device, characterized in that it comprises:提取单元，用于获取包含人脸的面部图像，并提取所述面部图像对应的人脸特征；An extraction unit, configured to obtain a facial image containing a human face, and extract facial features corresponding to the facial image;查找单元，用于基于预设匹配条件在预存的若干人脸特征组中查找与所述人脸特征相匹配的目标人脸特征；其中，若干所述人脸特征组中的每组人脸特征组均包括注册人脸特征；或者包括所述注册人脸特征和补录人脸特征，所述注册人脸特征与所述补录人脸特征的录入时间不同；A search unit, configured to search for target facial features that match the facial features in several pre-stored facial feature groups based on preset matching conditions; wherein, each set of facial features in the several facial feature groups The groups all include registered facial features; or include the registered facial features and supplementary facial features, and the registration time of the registered facial features is different from that of the supplementary facial features;获取单元，用于当查找到目标人脸特征时，获取所述面部图像的采集次数；An acquisition unit, configured to acquire the number of acquisitions of the facial image when the target face feature is found;录入单元，用于当所述采集次数大于或者等于预设次数阈值时，将所述人脸特征作为补录人脸特征,并将所述补录人脸特征录入所述目标人脸特征对应的人脸特征组内。The entry unit is configured to use the face feature as a re-recorded face feature when the number of acquisitions is greater than or equal to a preset number of times threshold, and record the re-recorded face feature into the target face feature corresponding In the face feature group.一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1-7任意一项所述的人脸识别方法中的步骤。A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors, so as to realize the requirements of claim 1 Steps in any one of the face recognition methods described in -7.一种电子设备，其特征在于，包括：采集装置、存储器及分别与所述采集装置、所述存储器连接的处理器；其中：An electronic device, characterized by comprising: a collection device, a memory, and a processor connected to the collection device and the memory respectively; wherein:所述采集装置，用于采集包含人脸的面部图像；The collection device is used to collect facial images including human faces;存储器，用于存储所述面部图像及可被所述处理器执行的计算机可读程序；a memory for storing the facial image and a computer-readable program executable by the processor;所述处理器，用于控制所述采集装置进行图像采集，并用于执行所述计算机可读程序时实现如权利要求1-7任意一项所述的人脸识别方法中的步骤。The processor is configured to control the acquisition device to acquire images, and to implement the steps in the face recognition method according to any one of claims 1-7 when executing the computer-readable program.

Images