技术领域technical field
本发明实施例涉及图像处理技术领域，尤其涉及一种人脸图像处理方法及装置、计算机可读存储介质。Embodiments of the present invention relate to the technical field of image processing, and in particular, to a method and device for processing a face image, and a computer-readable storage medium.
背景技术Background technique
人脸识别是基于人的脸部特征信息进行身份识别的一种生物识别技术。随着物联网终端的通讯能力、计算能力的提高，人脸识别的应用需求不再仅仅满足于处理单张图片。在某些应用场景下，需要对视频流进行人脸识别，得到视频流中的人脸图像。Face recognition is a kind of biometric identification technology based on human facial feature information. With the improvement of the communication and computing capabilities of IoT terminals, the application requirements of face recognition are no longer limited to processing a single image. In some application scenarios, it is necessary to perform face recognition on the video stream to obtain the face image in the video stream.
目前的对视频流进行人脸识别的方案是将视频视为具有时序的图片，然后针对每张图片进行人脸识别，再将所有得到的人脸与人脸库进行比对，最后将识别结果汇总。The current solution for face recognition on video streams is to regard the video as a picture with time series, and then perform face recognition for each picture, and then compare all the obtained faces with the face database, and finally put the recognition results. summary.
然而，对视频流中的每张图片分别进行人脸识别，这种图像处理方法进行人脸图像识别的效率较低。However, face recognition is performed on each picture in the video stream separately, and this image processing method is less efficient for face image recognition.
发明内容SUMMARY OF THE INVENTION
本发明实施例解决的技术问题是人脸图像识别效率较低。The technical problem solved by the embodiments of the present invention is that the face image recognition efficiency is low.
为解决上述技术问题，本发明实施例提供一种人脸图像处理方法，包括：获取待处理的图像，所述待处理的图像中包含人脸图像；对所述人脸图像进行人脸向量化，得到所述人脸图像对应的人脸特征向量；基于所述人脸图像的质量分，并结合所述人脸图像对应的人脸特征向量，对所述人脸图像进行聚类；确定所述人脸图像的类别及各类别对应的类中心点；对各类别的类中心点对应的人脸图像进行人脸识别。In order to solve the above technical problem, an embodiment of the present invention provides a method for processing a face image, including: acquiring an image to be processed, where the image to be processed includes a face image; performing face vectorization on the face image , obtain the face feature vector corresponding to the face image; based on the quality score of the face image, and in combination with the face feature vector corresponding to the face image, perform clustering on the face image; determine the Describe the category of the face image and the class center point corresponding to each category; perform face recognition on the face image corresponding to the class center point of each category.
可选的，所述对所述人脸图像进行人脸向量化，得到所述人脸图像对应的人脸特征向量，包括：对所述人脸图像进行图像标准化，得到标准化人脸图像；采用人脸向量化算法对所述标准化人脸图像进行人脸向量化处理，获取预设维度的第一人脸特征向量作为所述人脸图像对应的人脸特征向量。Optionally, performing face vectorization on the face image to obtain a face feature vector corresponding to the face image includes: performing image standardization on the face image to obtain a standardized face image; using The face vectorization algorithm performs face vectorization processing on the standardized face image, and obtains a first face feature vector of a preset dimension as the face feature vector corresponding to the face image.
可选的，所述对所述人脸图像进行图像标准化，得到标准化人脸图像之后，还包括：将所述标准化人脸图像进行镜像，得到镜像后的人脸图像；采用人脸向量化算法对所述镜像后的人脸图像进行人脸向量化处理，获取预设维度的第二人脸特征向量；计算所述第一人脸特征向量与所述第二人脸特征向量的均值，将所求的均值作为所述人脸图像对应的人脸特征向量。Optionally, after performing image standardization on the face image to obtain a standardized face image, the method further includes: mirroring the standardized face image to obtain a mirrored face image; using a face vectorization algorithm Perform face vectorization processing on the mirrored face image to obtain a second face feature vector of a preset dimension; calculate the mean value of the first face feature vector and the second face feature vector, and set the The obtained mean value is used as the face feature vector corresponding to the face image.
可选的，采用以下任一种算法对所述人脸图像进行聚类：采用非极大值抑制算法对所述人脸图像进行聚类；采用社区发现算法对所述人脸图像进行聚类。可选的，所述确定各类别对应的类中心点，包括：当采用非极大值抑制算法对所述人脸图像进行聚类时，将各类别中质量分最高的人脸图像作为对应类别的类中心点；当采用社区发现算法对所述人脸图像进行聚类时，将各类别中边最多的人脸图像作为对应类别的类中心点。Optionally, use any one of the following algorithms to cluster the face images: use a non-maximum suppression algorithm to cluster the face images; use a community discovery algorithm to cluster the face images . Optionally, the determining the class center point corresponding to each category includes: when using the non-maximum value suppression algorithm to cluster the face images, taking the face image with the highest quality score in each category as the corresponding category. The class center point of ; when using the community discovery algorithm to cluster the face images, the face image with the most edges in each category is used as the class center point of the corresponding category.
可选的，所述采用非极大值抑制算法对所述人脸图像进行聚类，包括：将所述人脸图像按照质量分倒序排列，记为人脸质量分列队；根据各人脸图像的人脸特征向量，计算每个人脸图像与其他人脸图像的相似度，记为相似度矩阵；在第i次迭代时，将质量分最高的人脸图像作为类别Ci的类中心点；根据所述相似度矩阵中各人脸图像之间的相似度，将与类别Ci的类中心点的相似度阈值达到预设阈值的人脸图像归入类别Ci；将所述类别Ci中的人脸图像从人脸质量分列队Ti删除，得到更新后的人脸质量分列队T(i+1)；将类别Ci中的人脸图像与类别Ci类中心点的相似度从所述相似度矩阵Mi中删除，得到更新后的相似度矩阵M(i+1)；当更新后的人脸质量分列队T(i+1)不为空时，继续从所述更新后的质量分列队T(i+1)中选取质量分最高的人脸图像作为类别C(i+1)的类中心点。Optionally, the clustering of the face images by using a non-maximum value suppression algorithm includes: arranging the face images in reverse order of quality points, and denoting it as a face quality queue; Face feature vector, calculate the similarity between each face image and other face images, and record it as a similarity matrix; in the ith iteration, the face image with the highest quality score is used as the class center point of the category Ci; The similarity between the face images in the similarity matrix, the face images whose similarity threshold with the class center point of the category Ci reaches the preset threshold value are classified into the category Ci; the face images in the category Ci are classified into the category Ci; Delete from the face quality queue Ti to obtain the updated face quality queue T(i+1); the similarity between the face image in the category Ci and the center point of the category Ci is removed from the similarity matrix Mi Delete to obtain the updated similarity matrix M(i+1); when the updated face quality queue T(i+1) is not empty, continue from the updated quality queue T(i+1) 1), select the face image with the highest quality score as the class center point of class C(i+1).
可选的，采用以下任一种算法，计算每个人脸图像与其他人脸图像的相似度：余弦距离算法、欧式距离算法。Optionally, use any one of the following algorithms to calculate the similarity between each face image and other face images: cosine distance algorithm, Euclidean distance algorithm.
可选的，所述采用社区发现算法对所述人脸图像进行聚类，包括：将所有人脸图像划分为不同的类别，形成类别的集合SET(C)0；计算各类别中的所有的人脸图像的人脸相似度并与预设第二阈值进行比较，高于第二阈值权重为1，低于预设第二阈值权重为0，得到边的集合SET(E)0；第i次迭代时，对应的类别的集合SET(C)i，边的集合SET(E)i，计算当前聚类状态下的聚类稳定性Qi；将相邻的两个类别合并至同一个类别中，得到一个新的类别的集合SET(C)(i+1)；计算合并之后的各类别内的边的权重之后以及类别间的边的权重之和，得到边的集合SET(E)(i+1)；计算合并后的聚类稳定性Q(i+1)；当合并后的聚类稳定性Q(i+1)大于合并前的聚类稳定性Qi时，则接受合并，将错误划分状态组成的集合置空，并进入第i+1次迭代；当合并后的聚类稳定性Q(i+1)小于或等于合并前的聚类稳定性Qi时，则不接受合并，将当前划分加入错误划分状态组成的集合中；判断是否完成所有合并可能性，当完成所有的合并可能性，则将第i次迭代时对应的聚类作为最优聚类结果；当没有完成所有的合并可能性，则继续进行合并尝试。Optionally, the clustering of the face images by using a community discovery algorithm includes: dividing all face images into different categories to form a set of categories SET(C)0; The face similarity of the face image is compared with the preset second threshold, the weight is 1 above the second threshold, and the weight is 0 below the preset second threshold, and the set of edges SET(E)0 is obtained; In the second iteration, the corresponding set of categories SET(C)i, the set of edges SET(E)i, calculate the clustering stability Qi under the current clustering state; merge the two adjacent categories into the same category , get a new set of categories SET(C)(i+1); calculate the sum of the weights of the edges within each category after the merger and the weights of the edges between categories, and get the set of edges SET(E)(i +1); Calculate the cluster stability Q(i+1) after merging; when the cluster stability Q(i+1) after merging is greater than the cluster stability Qi before merging, the merging is accepted, and an error The set composed of the divided states is left empty and enters the i+1th iteration; when the cluster stability Q(i+1) after merging is less than or equal to the cluster stability Qi before merging, the merging is not accepted, and the The current division is added to the set composed of the wrong division state; it is judged whether all the merging possibilities are completed, and when all the merging possibilities are completed, the cluster corresponding to the i-th iteration is taken as the optimal clustering result; when all the merging possibilities are not completed Merge possibility, proceed with the merge attempt.
本发明实施例还提供一种人脸图像处理装置，包括：获取单元，适于获取待处理的图像，所述待处理的图像中包含人脸图像；人脸向量化单元，适于对所述人脸图像进行人脸向量化，得到所述人脸图像对应的人脸特征向量；聚类单元，适于基于所述人脸图像的质量分，并结合所述人脸图像对应的人脸特征向量，对所述人脸图像进行聚类；确定单元，适于确定所述人脸图像的类别及各类别对应的类中心点；人脸识别单元，适于对各类别的类中心点对应的人脸图像进行人脸识别。An embodiment of the present invention further provides a human face image processing device, including: an acquisition unit, adapted to acquire an image to be processed, the to-be-processed image includes a human face image; a face vectorization unit, adapted to acquire the image to be processed The face image is vectorized to obtain the face feature vector corresponding to the face image; the clustering unit is adapted to be based on the quality score of the face image and combined with the face feature corresponding to the face image. a vector, for clustering the face images; a determination unit, suitable for determining the categories of the face images and the class center points corresponding to each category; a face recognition unit, suitable for each category face image for face recognition.
可选的，所述人脸向量化单元，适于对所述人脸图像进行图像标准化，得到标准化人脸图像；采用人脸向量化算法对所述标准化人脸图像进行人脸向量化处理，获取预设维度的第一人脸特征向量作为所述人脸图像对应的人脸特征向量。Optionally, the face vectorization unit is adapted to perform image standardization on the face image to obtain a standardized face image; use a face vectorization algorithm to perform face vectorization processing on the standardized face image, A first face feature vector of a preset dimension is obtained as a face feature vector corresponding to the face image.
可选的，所述人脸向量化单元，还适于将所述标准化人脸图像进行镜像，得到镜像后的人脸图像；采用人脸向量化算法对所述镜像后的人脸图像进行人脸向量化处理，获取预设维度的第二人脸特征向量；计算所述第一人脸特征向量与所述第二人脸特征向量的均值，将所求的均值作为所述人脸图像对应的人脸特征向量。Optionally, the face vectorization unit is further adapted to mirror the standardized face image to obtain a mirrored face image; use a face vectorization algorithm to perform a human face image on the mirrored face image. face vectorization processing, obtaining a second face feature vector of preset dimensions; calculating the mean value of the first face feature vector and the second face feature vector, and using the obtained mean value as the corresponding face image face feature vector.
可选的，所述聚类单元，适于采用如下任一种算法对所述人脸图像进行聚类：非极大值抑制算法、社区发现算法。Optionally, the clustering unit is adapted to use any one of the following algorithms to cluster the face images: a non-maximum value suppression algorithm and a community discovery algorithm.
可选的，所述确定单元，适于当采用非极大值抑制算法对所述人脸图像进行聚类时，将各类别中质量分最高的人脸图像作为对应类别的类中心点；当采用社区发现算法对所述人脸图像进行聚类时，将各类别中边最多的人脸图像作为对应类别的类中心点。Optionally, the determining unit is adapted to use the face image with the highest quality score in each category as the class center point of the corresponding category when the non-maximum value suppression algorithm is used to cluster the face images; When using the community discovery algorithm to cluster the face images, the face image with the most edges in each category is used as the class center point of the corresponding category.
可选的，所述聚类单元，适于采用非极大值抑制算法对所述人脸图像进行聚类时，将所述人脸图像按照质量分倒序排列，记为人脸质量分列队；根据各人脸图像的人脸特征向量，计算每个人脸图像与其他人脸图像的相似度，记为相似度矩阵；在第i次迭代时，将质量分最高的人脸图像作为类别Ci的类中心点；根据所述相似度矩阵中各人脸图像之间的相似度，将与类别Ci的类中心点的相似度阈值达到预设阈值的人脸图像归入类别Ci；将所述类别Ci中的人脸图像从人脸质量分列队Ti删除，得到更新后的人脸质量分列队T(i+1)；将类别Ci中的人脸图像与类别Ci类中心点的相似度从所述相似度矩阵Mi中删除，得到更新后的相似度矩阵M(i+1)；当更新后的人脸质量分列队T(i+1)不为空时，继续从所述更新后的质量分列队T(i+1)中选取质量分最高的人脸图像作为类别C(i+1)的类中心点。Optionally, the clustering unit is adapted to use a non-maximum suppression algorithm to cluster the face images, arrange the face images in reverse order of quality points, and record them as face quality queues; The face feature vector of each face image, calculate the similarity between each face image and other face images, and record it as the similarity matrix; in the ith iteration, the face image with the highest quality score is used as the class of the category Ci. Center point; according to the similarity between each face image in the similarity matrix, the face image whose similarity threshold with the class center point of category Ci reaches a preset threshold is classified into category Ci; the category Ci is classified into category Ci; The face images in are deleted from the face quality queue Ti, and the updated face quality queue T(i+1) is obtained; Delete from the similarity matrix Mi to obtain the updated similarity matrix M(i+1); when the updated face quality queue T(i+1) is not empty, continue from the updated quality score. The face image with the highest quality score in the queue T(i+1) is selected as the class center point of the class C(i+1).
可选的，所述聚类单元，适于采用以下任一种算法，计算每个人脸图像与其他人脸图像的相似度：余弦距离算法、欧式距离算法。Optionally, the clustering unit is adapted to use any one of the following algorithms to calculate the similarity between each face image and other face images: cosine distance algorithm, Euclidean distance algorithm.
可选的，所述聚类单元，适于采用社区发现算法对所述人脸图像进行聚类时，将所有人脸图像划分为不同的类别，形成类别的集合SET(C)0；计算各类别中的所有的人脸图像的人脸相似度并与预设第二阈值进行比较，高于第二阈值权重为1，低于预设第二阈值权重为0，得到边的集合SET(E)0；第i次迭代时，对应的类别的集合SET(C)i，边的集合SET(E)i，计算当前聚类状态下的聚类稳定性Qi；将相邻的两个类别合并至同一个类别中，得到一个新的类别的集合SET(C)(i+1)；计算合并之后的各类别内的边的权重之后以及类别间的边的权重之和，得到边的集合SET(E)(i+1)；计算合并后的聚类稳定性Q(i+1)；当合并后的聚类稳定性Q(i+1)大于合并前的聚类稳定性Qi时，则接受合并，将错误划分状态组成的集合置空，并进入第i+1次迭代；当合并后的聚类稳定性Q(i+1)小于或等于合并前的聚类稳定性Qi时，则不接受合并，将当前划分加入错误划分状态组成的集合中；判断是否完成所有合并可能性，当完成所有的合并可能性，则将第i次迭代时对应的聚类作为最优聚类结果；当没有完成所有的合并可能性，则继续进行合并尝试。Optionally, the clustering unit is adapted to use a community discovery algorithm to cluster the face images, divide all face images into different categories, and form a set of categories SET(C)0; The face similarity of all face images in the category is compared with the preset second threshold, the weight is 1 above the second threshold, and the weight is 0 below the preset second threshold, and the set of edges SET(E )0; at the ith iteration, the set of corresponding categories SET(C)i, the set of edges SET(E)i, calculate the clustering stability Qi under the current clustering state; merge the two adjacent categories In the same category, a new category set SET(C)(i+1) is obtained; after calculating the weights of the edges within each category after merging and the sum of the weights of the edges between categories, the set of edges SET is obtained (E)(i+1); Calculate the cluster stability Q(i+1) after merging; when the cluster stability Q(i+1) after merging is greater than the cluster stability Qi before merging, then Accept the merging, empty the set of incorrectly divided states, and enter the i+1th iteration; when the cluster stability Q(i+1) after merging is less than or equal to the clustering stability Qi before merging, then Do not accept the merge, add the current division to the set composed of the wrong division state; judge whether all the merge possibilities are completed, when all the merge possibilities are completed, the cluster corresponding to the i-th iteration will be used as the optimal clustering result; When all merging possibilities have not been completed, merging attempts are continued.
本发明实施例还提供一种人脸图像处理装置，包括存储器和处理器，所述存储器上存储有可在所述处理器上运行的计算机指令，所述处理器运行所述计算机指令时执行上述任一种人脸图像处理方法的步骤。An embodiment of the present invention further provides a face image processing apparatus, including a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and the processor executes the above-mentioned computer instructions when the processor runs the computer instructions. Steps of any face image processing method.
本发明实施例还提供一种计算机可读存储介质，计算机可读存储介质为非易失性存储介质或非瞬态存储介质，其上存储有计算机指令，所述计算机指令运行时执行上述任一种人脸图像处理方法的步骤。An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium is a non-volatile storage medium or a non-transitory storage medium, and stores computer instructions thereon, and the computer instructions execute any of the above when running. The steps of a face image processing method.
与现有技术相比，本发明实施例的技术方案具有以下有益效果：Compared with the prior art, the technical solutions of the embodiments of the present invention have the following beneficial effects:
对待处理图像中的人脸图像进行人脸特征向量化，根据人脸图像对应的质量分及对应的人脸特征向量，对人脸图像进行聚类，得到各人脸图像所属的类别以及各类别的类中心点，在进行人脸识别时，对各类别中的类中心点对应的人脸图像进行识别，从而可以减少在人脸识别过程中需要处理的人脸图像的数量，从而可以提高人脸图像识别效率。Face feature vectorization is performed on the face images in the images to be processed, and the face images are clustered according to the corresponding quality scores of the face images and the corresponding face feature vectors to obtain the category to which each face image belongs and each category. When performing face recognition, the face images corresponding to the class center points in each category can be recognized, thereby reducing the number of face images that need to be processed in the face recognition process, thereby improving human performance. face image recognition efficiency.
进一步，在对人脸进行人脸向量化时，对人脸图像进行标准化处理，从而可以提高人脸检测的容错性，以及提高后续图像处理过程中的图像处理精度及效率。Further, when the face is vectorized, the face image is standardized, so that the fault tolerance of face detection can be improved, and the image processing accuracy and efficiency in the subsequent image processing process can be improved.
进一步，将得到的标准化的人脸图像进行镜像，得到镜像后的人脸图像，并将标准化人脸图像对应的人脸特征向量与镜像后的人脸图像对应的人脸特征向量求均值，作为人脸图像对应的人脸特征向量，可以提高所得到的人脸特征向量的精度。Further, the obtained standardized face image is mirrored to obtain a mirrored face image, and the average value of the face feature vector corresponding to the standardized face image and the face feature vector corresponding to the mirrored face image is taken as The face feature vector corresponding to the face image can improve the accuracy of the obtained face feature vector.
附图说明Description of drawings
图1是本发明实施例中的一种人脸图像处理方法的流程图；1 is a flowchart of a method for processing a human face image in an embodiment of the present invention;
图2是本发明实施例中的一种采用非极大值抑制算法对人脸图像聚类的流程图；Fig. 2 is a kind of flow chart of adopting non-maximum suppression algorithm to cluster face images in an embodiment of the present invention;
图3是本发明实施例中的一种采用社区发现算法对人脸图像进行聚类的流程图；3 is a flow chart of clustering face images using a community discovery algorithm according to an embodiment of the present invention;
图4是本发明实施例中的一种人脸图像处理装置的结构示意图。FIG. 4 is a schematic structural diagram of a face image processing apparatus in an embodiment of the present invention.
具体实施方式Detailed ways
如上所述，目前的对视频流进行人脸识别的方案是将视频视为具有时序的图片，然后针对每张图片进行人脸识别，再将所有得到的人脸与人脸库进行比对，最后将识别结果汇总。然而，对视频流中的每张图片分别进行人脸识别，这种图像处理方法进行人脸图像识别的效率较低。As mentioned above, the current solution for performing face recognition on video streams is to regard the video as a picture with time series, and then perform face recognition for each picture, and then compare all the obtained faces with the face database, Finally, the identification results will be summarized. However, face recognition is performed on each picture in the video stream separately, and this image processing method is less efficient for face image recognition.
本发明实施例中，对待处理图像中的人脸图像进行人脸特征向量化，根据人脸图像对应的质量分及对应的人脸特征向量，对人脸图像进行聚类，得到各人脸图像所属的类别以及各类别的类中心点，在进行人脸识别时，对各类别中的类中心点对应的人脸图像进行识别，从而可以减少在人脸识别过程中需要处理的人脸图像的数量，从而可以提高人脸图像识别效率。In the embodiment of the present invention, the face images in the images to be processed are subjected to face feature vectorization, and the face images are clustered according to the corresponding quality scores of the face images and the corresponding face feature vectors to obtain each face image. The category to which it belongs and the class center points of each category, when performing face recognition, the face images corresponding to the class center points in each category are recognized, so as to reduce the amount of face images that need to be processed in the face recognition process. , so that the efficiency of face image recognition can be improved.
为使本发明实施例的上述目的、特征和有益效果能够更为明显易懂，下面结合附图对本发明的具体实施例做详细的说明。In order to make the above objects, features and beneficial effects of the embodiments of the present invention more clearly understood, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
参照图1，给出了本发明实施例中一种人脸图像处理方法的流程图，所述人脸图像处理方法可以包括以下步骤。Referring to FIG. 1, a flowchart of a method for processing a face image in an embodiment of the present invention is given, and the method for processing a face image may include the following steps.
步骤11，获取待处理图像，所述待处理图像中包含人脸图像。Step 11: Acquire a to-be-processed image, where the to-be-processed image includes a face image.
在具体实施中，可以从视频流中获取待处理图像。在本发明实施例中，可以按照预设的图像抽取规则，从所述视频流中抽取相应帧的图像，并将从抽取到的图像中获取包括人脸图像的图像作为待处理图像。In a specific implementation, the image to be processed may be obtained from the video stream. In this embodiment of the present invention, images of corresponding frames may be extracted from the video stream according to preset image extraction rules, and an image including a face image may be obtained from the extracted images as an image to be processed.
步骤12，将所述人脸图像进行人脸向量化，得到所述人脸图像对应的人脸特征向量。Step 12: Perform face vectorization on the face image to obtain a face feature vector corresponding to the face image.
在具体实施中，为了便于后续进行人脸识别，可以对所述待处理图像中的人脸图像进行图像标准化，得到标准化的人脸图像。In a specific implementation, in order to facilitate subsequent face recognition, image standardization may be performed on the face image in the to-be-processed image to obtain a standardized face image.
在具体实施中，待处理图像中可以包括一个人脸，也可以包括多个人脸。当待处理图像中包括多个人脸时，分别对多个人脸区域进行截取，相应地，得到多个人脸图像。In a specific implementation, the image to be processed may include one human face or multiple human faces. When the image to be processed includes multiple faces, the multiple face regions are intercepted respectively, and correspondingly, multiple face images are obtained.
在本发明实施例中，对所述人脸图像进行标准化可以包括如下步骤：对人脸图像进行尺寸归一化，得到预设尺寸的人脸图像；对得到的预设尺寸的人脸图像中的人脸方向进行旋转等调整操作，使得人脸中的关键点如眼睛、鼻尖以及嘴角等处于预设位置。通过对得到的预设尺寸的人脸图像进行调整操作，可以使得得到的预设尺寸的人脸图像中的人脸处于非倾斜状态。通过对待处理图像中的人脸图像进行标准化处理，可以提高人脸检测的容错性，以及提高后续图像处理过程中的图像处理精度及效率。In the embodiment of the present invention, standardizing the face image may include the following steps: normalizing the size of the face image to obtain a face image of a preset size; Rotate and adjust the orientation of the face, so that the key points in the face, such as the eyes, nose tip, and mouth corners, are in preset positions. By performing an adjustment operation on the obtained face image of preset size, the face in the obtained face image of preset size can be in a non-slanted state. By standardizing the face image in the image to be processed, the fault tolerance of face detection can be improved, and the image processing accuracy and efficiency in the subsequent image processing process can be improved.
在具体实施中，在得到标准化人脸图像后，可以采用人脸向量化算法对所述标准化人脸图像进行人脸向量化处理，获取预设维度的第一人脸特征向量作为所述标准化人脸图像对应的人脸特征向量。一张人脸图像可以看做一个3维的像素矩阵，通过人脸向量化处理之后，可以得到对应的1维向量，1维向量的长度可以根据实际需求进行设定。例如，采用全连接层较小的轻量级的人脸向量化算法对所述标准化的人脸图像进行人脸特征向量化，将3维人脸图像转换成对应的1维向量，并采用512维向量作为所述人脸图像对应的人脸特征向量。In a specific implementation, after a standardized face image is obtained, a face vectorization algorithm may be used to perform face vectorization processing on the standardized face image, and a first face feature vector of a preset dimension is obtained as the standardized face The face feature vector corresponding to the face image. A face image can be regarded as a 3-dimensional pixel matrix. After face vectorization processing, the corresponding 1-dimensional vector can be obtained, and the length of the 1-dimensional vector can be set according to actual needs. For example, using a lightweight face vectorization algorithm with a small fully connected layer to perform face feature vectorization on the standardized face image, convert the 3-dimensional face image into a corresponding 1-dimensional vector, and use 512 The dimension vector is used as the face feature vector corresponding to the face image.
为进一步提高所得到的人脸特征向量的精度，在本发明另一实施例中，得到标准化人脸图像之后，对标准化人脸进行镜像，得到镜像后的人脸图像。采用人脸向量化算法对所述镜像后的人脸图像进行人脸向量化处理，获取预设维度的第二人脸特征向量，计算所述第一人脸特征向量与所述第二人脸特征向量的均值，将所计算的均值作为所述人脸图像对应的人脸特征向量。In order to further improve the accuracy of the obtained face feature vector, in another embodiment of the present invention, after the standardized face image is obtained, the standardized face is mirrored to obtain a mirrored face image. Perform face vectorization processing on the mirrored face image by using a face vectorization algorithm, obtain a second face feature vector of preset dimensions, and calculate the first face feature vector and the second face feature vector. The mean value of the feature vectors, and the calculated mean value is used as the face feature vector corresponding to the face image.
步骤13，基于所述人脸图像的质量分，并结合所述人脸图像对应的人脸特征向量，对所述人脸图像进行聚类。Step 13: Clustering the face images based on the quality scores of the face images and in combination with the face feature vectors corresponding to the face images.
在步骤11中获取待处理图像时，可以对图像进行人脸检测来确定是否为人脸图像。在进行人脸检测时，可以获取人脸区域的坐标位置以及质量分。When the to-be-processed image is acquired in step 11, face detection may be performed on the image to determine whether it is a face image. When performing face detection, the coordinate position and quality score of the face area can be obtained.
在具体实施中，所述人脸图像的质量分是假设某一区域包含人脸的置信度，置信度越高越接受这个区域包含人脸的假设，可以用于描述某一区域包括人脸的可能性。质量分与图像的清晰度、图像中人脸的位置、人脸的姿态(正面人脸、侧面人脸等)等因素相关，人脸图像的清晰度越高、人脸姿态越接近正面人脸所对应的质量分越高。In a specific implementation, the quality score of the face image is the confidence level of assuming that a certain area contains a human face. The higher the confidence level, the more acceptable the assumption that this area contains a human face. It can be used to describe the fact that a certain area includes a human face possibility. The quality score is related to the clarity of the image, the position of the face in the image, the pose of the face (frontal face, side face, etc.) The corresponding quality score is higher.
在具体实施中，可以采用多种方式对人脸图像进行聚类。例如，采用非极大值抑制算法对所述人脸图像进行聚类。又如，采用社区发现(Fast Unfolding)算法对所述人脸图像进行聚类。In a specific implementation, the face images can be clustered in various ways. For example, a non-maximum suppression algorithm is used to cluster the face images. For another example, the face images are clustered by using a community discovery (Fast Unfolding) algorithm.
参照图2，给出了本发明实施例中一种采用非极大值抑制算法对人脸图像聚类的流程图，可以包括以下步骤。Referring to FIG. 2 , a flowchart of clustering face images using a non-maximum value suppression algorithm in an embodiment of the present invention is given, which may include the following steps.
步骤21，将所述人脸图像按照质量分倒序排列，记为人脸质量分列队。Step 21: Arrange the face images in reverse order according to the quality points, and record it as a queue of face quality points.
在具体实施中，可以将所述人脸图像按照质量分从高到低倒序排列，得到人脸质量分列队T0。可以理解的是，也可以将所述人脸图像按照质量分从低到高正序排列，可以根据实际应用需要进行排序即可。In a specific implementation, the face images may be arranged in reverse order according to the quality score from high to low to obtain a face quality queue T0. It can be understood that, the face images can also be arranged in positive order from low to high quality score, and can be sorted according to actual application requirements.
例如，按照标准化的人脸图像的质量分从高到低倒序排列。又如，按照镜像后的人脸图像的质量分从高到低倒序排列。For example, according to the quality score of normalized face images, they are arranged in reverse order from high to low. For another example, the quality points of the mirrored face images are arranged in reverse order from high to low.
步骤22，根据各人脸图像的人脸特征向量，计算每个人脸图像与其他人脸图像的相似度，记为相似度矩阵。Step 22: Calculate the similarity between each face image and other face images according to the face feature vector of each face image, and record it as a similarity matrix.
在具体实施中，可以根据人脸图像的人脸特征向量，依次计算每个人脸图像分别与人脸质量分列队T0中除自身之外的各人脸图像的相似度，得到所有人脸图像两两之间的相似度，得到相似度矩阵M0。In a specific implementation, the similarity between each face image and each face image except itself in the face quality queue T0 can be calculated in turn according to the face feature vector of the face image, and two face images of all face images can be obtained. The similarity between the two is obtained to obtain the similarity matrix M0.
在本发明一实施例中，采用余弦距离算法计算两个人脸图像之间的相似度。在本发明另一实施例中，采用欧式距离算法计算两个人脸图像之间的相似度。可以理解的是，在实际应用中，还可以采用其他算法计算两个人脸图像之间的相似度。In an embodiment of the present invention, the cosine distance algorithm is used to calculate the similarity between two face images. In another embodiment of the present invention, the Euclidean distance algorithm is used to calculate the similarity between two face images. It can be understood that, in practical applications, other algorithms can also be used to calculate the similarity between two face images.
当采用余弦距离算法计算人脸图像的相似度时，可以将计算得到的相似度记录在二维矩阵M0中，矩阵中(a，b)位置记录人脸图像a与人脸图像b的余弦距离，也即人脸图像a和人脸图像b的相似度。When the cosine distance algorithm is used to calculate the similarity of face images, the calculated similarity can be recorded in a two-dimensional matrix M0, and the cosine distance between face image a and face image b is recorded at the position (a, b) in the matrix , that is, the similarity between face image a and face image b.
在具体实施中，步骤21与步骤22之间并没有必须的逻辑上的先后顺序。在本发明实施例中，可以先执行步骤21，然后执行步骤22；也可以先执行步骤22，然后执行步骤21；还可以同时执行步骤21及步骤22。In a specific implementation, there is no necessary logical sequence between step 21 and step 22 . In this embodiment of the present invention, step 21 may be performed first, and then step 22 may be performed; or step 22 may be performed first, and then step 21 may be performed; or step 21 and step 22 may be performed simultaneously.
步骤23，在第i次迭代时，将质量分最高的人脸图像作为类别Ci的类中心点。Step 23, in the ith iteration, the face image with the highest quality score is used as the class center point of the class Ci.
在具体实施中，当第i次迭代时，对应的质量分列队为Ti，对应的相似度矩阵为Mi。将质量分列队为Ti中质量分最高的人脸图像作为类别Ci的类中心点。In a specific implementation, at the ith iteration, the corresponding quality queue is Ti, and the corresponding similarity matrix is Mi. Queue the face image with the highest quality score in Ti as the class center point of the class Ci.
步骤24，生成相似度矩阵Mi。Step 24, generating a similarity matrix Mi.
步骤25，根据所述相似度矩阵中各人脸图像之间的相似度，将与类别Ci的类中心点的相似度阈值达到预设阈值的人脸图像归入类别Ci。Step 25, according to the similarity between the face images in the similarity matrix, classify the face images whose similarity threshold with the class center point of the category Ci reaches a preset threshold value into the category Ci.
根据相似度矩阵Mi中各人脸图像的相似度，确认与类别Ci的类中心点对应的人脸图像的相似度高于预设阈值的人脸图像，并将相似度阈值高于预设阈值的人脸图像归入类别Ci。According to the similarity of each face image in the similarity matrix Mi, confirm that the similarity of the face image corresponding to the class center point of the category Ci is higher than the preset threshold, and set the similarity threshold higher than the preset threshold. The face images of are classified into category Ci.
步骤26，将所述类别Ci中的人脸图像从人脸质量分列队Ti删除，得到更新后的人脸质量分列队T(i+1)。Step 26: Delete the face images in the category Ci from the face quality queue Ti to obtain an updated face quality queue T(i+1).
步骤27，将类别Ci中的人脸图像与类别Ci的类中心点的相似度从所述相似度矩阵Mi中删除。Step 27: Delete the similarity between the face image in the category Ci and the class center point of the category Ci from the similarity matrix Mi.
将类别Ci中的人脸图像与类别Ci的类中心点的相似度从所述相似度矩阵Mi中删除后，可以得到更新后的相似度矩阵M(i+1)。After the similarity between the face image in the category Ci and the class center point of the category Ci is deleted from the similarity matrix Mi, an updated similarity matrix M(i+1) can be obtained.
在具体实施中，步骤26与步骤27之间并没有必须的逻辑上的先后顺序。在本发明实施例中，可以先执行步骤26，然后执行步骤27；也可以先执行步骤27，然后执行步骤26；还可以同时执行步骤26及步骤27。In a specific implementation, there is no necessary logical sequence between step 26 and step 27 . In this embodiment of the present invention, step 26 may be performed first, and then step 27 may be performed; or step 27 may be performed first, and then step 26 may be performed; or step 26 and step 27 may be performed simultaneously.
步骤28，判断更新后的人脸质量分列队T(i+1)是否为空。Step 28, judging whether the updated face quality queue T(i+1) is empty.
当判断结果为否时，即更新后的人脸质量分列队T(i+1)不为空时，此后继续分别执行步骤23及步骤24，进入第i+1次循环。在第i+1次迭代中，继续从所述更新后的质量分列队T(i+1)中选取质量分最高的人脸图像作为类别C(i+1)的类中心点，根据相似度矩阵M(i+1)中的相似度，并将人脸质量分列队T(i+1)中对应的人脸图像进行聚类。When the judgment result is no, that is, when the updated face quality queue T(i+1) is not empty, then continue to execute step 23 and step 24 respectively, and enter the i+1th cycle. In the i+1th iteration, continue to select the face image with the highest quality score from the updated quality queue T(i+1) as the class center point of the class C(i+1), according to the similarity The similarity in the matrix M(i+1), and the corresponding face images in the face quality queue T(i+1) are clustered.
当判断结果为是时，即更新后的人脸质量分列队T(i+1)为空时，则执行步骤14，确定所述人脸图像的类别，并将各类别中质量分最高的人脸图像作为类中心点。When the judgment result is yes, that is, when the updated face quality queue T(i+1) is empty, step 14 is executed to determine the category of the face image, and assign the person with the highest quality score in each category. The face image serves as the class center point.
参照图3，给出了本发明实施例中一种采用社区发现算法对人脸图像进行聚类的流程图，可以包括以下步骤。Referring to FIG. 3 , a flowchart of clustering face images using a community discovery algorithm in an embodiment of the present invention is given, which may include the following steps.
将所有的人脸看成点，所有相似度高于第二阈值的人脸形成边，构造图。利用稳定性衡量聚类的结果，进行迭代，直至划分状态的稳定性无法提升为止。采用社区发现算法对人脸图像进行聚类的流程具体如下：All faces are regarded as points, and all faces whose similarity is higher than the second threshold form edges to construct a graph. Use stability to measure the results of clustering, and iterate until the stability of the partitioned state cannot be improved. The process of clustering face images using the community discovery algorithm is as follows:
步骤301，将所有人脸图像划分为不同的类别，形成类别的集合SET(C)0。Step 301: Divide all face images into different categories to form a set of categories SET(C)0.
在具体实施中，在首次划分时，可以将每张人脸图像作为一个类别。例如，有10张人脸图像，每张人脸图像分别对应一个类别，共10中类别。In a specific implementation, during the first division, each face image can be regarded as a category. For example, there are 10 face images, each face image corresponds to a category, a total of 10 categories.
初始化时，错误的划分状态组成的集合为空At initialization, the set of incorrect partition states is empty
步骤302，计算两人脸图像的相似度，并与预设第二阈值进行比较，高于第二阈值的权重为1，低于预设第二阈值的权重为0，得到边的集合SET(E)0。Step 302, calculate the similarity of the two face images, and compare with the preset second threshold, the weight higher than the second threshold is 1, and the weight lower than the preset second threshold is 0, and the set of edges SET ( E)0.
在具体实施中，当两人脸图像的相似度高于第二阈值时，权重标记为1，两人脸图像之间具有连线，形成边。当两人脸图像的相似度低于第二阈值时，权重标记为0，两人脸图像之间没有边。In a specific implementation, when the similarity of the two-face images is higher than the second threshold, the weight is marked as 1, and there is a connection line between the two-face images to form an edge. When the similarity of the two-face images is lower than the second threshold, the weight is marked as 0, and there is no edge between the two-face images.
步骤303，在第i次迭代时，计算当前聚类状态下的聚类稳定性Qi。Step 303, in the ith iteration, calculate the clustering stability Qi in the current clustering state.
在具体实施中，在第i次迭代时，对应的类别的集合SET(C)i，边的集合SET(E)i。基于当前的聚类状态SET(C)i，边的集合SET(E)i，可以采用公式(1)计算聚类稳定度Qi。In a specific implementation, in the ith iteration, the corresponding category set SET(C)i, and the edge set SET(E)i. Based on the current clustering state SET(C)i and the set of edges SET(E)i, the clustering stability Qi can be calculated by formula (1).
其中，表示顶点与边构成的图网络中所有的权重；Ai,j表示顶点i与及顶点j之间的权重；表示与顶点i连接的边的权重；ci表示顶点被分配到的类别；δ(ci,cj)用于判断顶点i与顶点j是否被划分在同一个类别，若是则返回1，若否则返回0。in, Represents all the weights in the graph network composed of vertices and edges; Ai,j represents the weight between vertex i and vertex j; Represents the weight of the edge connected to vertex i; ci represents the category to which the vertex is assigned; δ(ci , c j) is used to judge whether vertex i and vertex j are classified in the same category, if so, return 1, if Otherwise return 0.
步骤304，将相邻的两个类别合并至同一个类别中，得到一个新的类别的集合SET(C)(i+1)。Step 304: Combine two adjacent categories into the same category to obtain a new category set SET(C)(i+1).
在具体实施中，合并得到的类别的集合SET(C)(i+1)不在SET(S)i中。In a specific implementation, the set of merged categories SET(C)(i+1) is not in SET(S)i.
步骤305，计算合并之后的各类别内的边的权重之和以及类别间的边的权重之和，得到边的集合SET(E)(i+1)。Step 305: Calculate the sum of the weights of the edges within each category and the sum of the weights of the edges between the categories after merging, to obtain a set of edges SET(E)(i+1).
基于新的聚类得到的类别的集合SET(C)(i+1)，合并各边权重，计算合并之后的各类别内的边的权重之和以及类别间的边的权重之和。类内的所有边权重之和为该类自连接权重，两类间的所有边的权重之后，为两类别连接的权重。Based on the set of classes SET(C)(i+1) obtained by the new clustering, the weights of each edge are merged, and the sum of the weights of the edges within each class and the sum of the weights of the edges between classes are calculated. The sum of all edge weights within a class is the self-connection weight of this class, and after the weights of all edges between two classes, is the weight of the connection between the two classes.
步骤307，计算合并后的聚类稳定性Q(i+1)。Step 307: Calculate the combined cluster stability Q(i+1).
基于合并后的SET(C)(i+1)，以及SET(E)(i+1)，计算合并后的聚类稳定性Q(i+1)，具体可以参考Qi的计算方法及过程。Based on the merged SET(C)(i+1) and SET(E)(i+1), the merged cluster stability Q(i+1) is calculated. For details, please refer to the calculation method and process of Qi.
步骤307，判断Q(i+1)-Qi＞0是否成立。Step 307, determine whether Q(i+1)-Qi>0 holds.
当判断结果为是时，则执行步骤308；当判断结果为否时，则执行步骤310。When the judgment result is yes, step 308 is executed; when the judgment result is no, step 310 is executed.
步骤308，接受合并，将错误划分状态组成的集合置空。Step 308 , accept the merging, and empty the set composed of the wrongly divided states.
步骤308之后，执行步骤309。After step 308, step 309 is executed.
步骤309，SET(C)(i+1)&SET(E)(i+1)。Step 309, SET(C)(i+1)&SET(E)(i+1).
进入第i+1次迭代，此后继续执行步骤303。Enter the i+1 th iteration, and then continue to perform step 303 .
步骤310，不接受合并，将当前划分加入错误划分状态组成的集合中。In step 310, the merge is not accepted, and the current division is added to the set composed of the erroneous division states.
步骤311，判断SET(S)i的长度lenC是否等于Step 311, determine whether the length lenC of SET(S)i is equal to
当判断结果为是时，也即已经尝试所有的类别合并可能性，当前的划分为稳定最好的划分，也即为最优聚类，执行步骤14。When the judgment result is yes, that is, all the category merging possibilities have been tried, and the current division is the stable and best division, that is, the optimal clustering, and step 14 is executed.
当判断结果为否时，执行步骤312。When the judgment result is no, step 312 is executed.
步骤312，SET(C)i&SET(E)i。Step 312, SET(C)i & SET(E)i.
停留于第i次迭代，并继续执行步骤303，继续尝试其他可能性的类别合并。Stay at the ith iteration, and continue to perform step 303, and continue to try other possible category merging.
步骤14，确定所述人脸图像的类别及各类别对应的类中心点。Step 14: Determine the category of the face image and the category center point corresponding to each category.
在具体实施中，当采用社区发现算法对所述人脸图像进行聚类时，将各类别中边最多的人脸图像作为对应类别的类中心点。当采用非极大值抑制算法对所述人脸图像进行聚类时，将各类别中质量分最高的人脸图像作为对应类别的类中心点。In a specific implementation, when a community discovery algorithm is used to cluster the face images, the face image with the most edges in each category is used as the class center point of the corresponding category. When the non-maximum value suppression algorithm is used to cluster the face images, the face image with the highest quality score in each category is used as the class center point of the corresponding category.
步骤15，对各类别的类中心点对应的人脸图像进行人脸识别。Step 15: Perform face recognition on the face images corresponding to the class center points of each category.
在具体实施中，可以对各类别的类中心点对应的人脸图像进行人脸识别，而对于与类中心点属于同一类别的其他图像不做人脸识别，采用对人脸图像聚类，达到对人脸图像去重的目的，可以有效的减小需要进行人脸识别的人脸图像的数量，从而提高人脸图像识别效率，尤其是针对视频流图像进行人脸识别的效率。In the specific implementation, face recognition can be performed on the face images corresponding to the class center points of each category, while other images belonging to the same category as the class center points are not subjected to face recognition. The purpose of de-duplication of face images can effectively reduce the number of face images that need to be recognized, thereby improving the efficiency of face image recognition, especially the efficiency of face recognition for video stream images.
由上述方案可知，对待处理图像中的人脸图像进行人脸特征向量化，根据人脸图像对应的质量分及对应的人脸特征向量，对人脸图像进行聚类，得到各人脸图像所属的类别以及各类别的类中心点，在进行人脸识别时，对各类别中的类中心点对应的人脸图像进行识别，采用这种方式可以实现对图像的去重，从而可以减少在人脸识别过程中需要处理的人脸图像的数量，从而可以提高人脸图像识别效率。It can be seen from the above scheme that the face images in the images to be processed are subjected to face feature vectorization, and the face images are clustered according to the corresponding quality scores of the face images and the corresponding face feature vectors to obtain the belonging When performing face recognition, the face images corresponding to the class center points in each category are identified. In this way, the image can be de-duplicated, thereby reducing the number of people in the The number of face images that need to be processed in the face recognition process, so that the efficiency of face image recognition can be improved.
为了便于本领域技术人员更好的理解和实现本发明，本发明实施例还提供一种人脸图像处理装置。In order to facilitate those skilled in the art to better understand and implement the present invention, an embodiment of the present invention further provides a face image processing apparatus.
参照图4，给出了本发明实施例中一种人脸图像处理装置的结构示意图。所述人脸图像处理装置40可以包括：获取单元41、人脸向量化单元42、聚类单元43、确定单元44及人脸识别单元45，其中：Referring to FIG. 4, a schematic structural diagram of a face image processing apparatus in an embodiment of the present invention is given. The face image processing apparatus 40 may include: an acquisition unit 41, a face vectorization unit 42, a clustering unit 43, a determination unit 44 and a face recognition unit 45, wherein:
获取单元41，适于获取待处理的图像，所述待处理的图像中包含人脸图像；an acquisition unit 41, adapted to acquire an image to be processed, the image to be processed includes a face image;
人脸向量化单元42，适于对所述人脸图像进行人脸向量化，得到所述人脸图像对应的人脸特征向量；A face vectorization unit 42, adapted to perform face vectorization on the face image to obtain a face feature vector corresponding to the face image;
聚类单元43，适于基于所述人脸图像的质量分，并结合所述人脸图像对应的人脸特征向量，对所述人脸图像进行聚类；The clustering unit 43 is adapted to perform clustering on the face images based on the quality scores of the face images and in combination with the face feature vectors corresponding to the face images;
确定单元44，适于确定所述人脸图像的类别及各类别对应的类中心点；A determination unit 44, adapted to determine the category of the face image and the class center point corresponding to each category;
人脸识别单元45，适于对各类别的类中心点对应的人脸图像进行人脸识别。The face recognition unit 45 is adapted to perform face recognition on the face images corresponding to the center points of each category.
在具体实施中，所述人脸向量化单元42，适于对所述人脸图像进行图像标准化，得到标准化人脸图像；采用人脸向量化算法对所述标准化人脸图像进行人脸向量化处理，获取预设维度的第一人脸特征向量作为所述人脸图像对应的人脸特征向量。In a specific implementation, the face vectorization unit 42 is adapted to perform image standardization on the face image to obtain a standardized face image; use a face vectorization algorithm to perform face vectorization on the standardized face image process, and obtain a first face feature vector of a preset dimension as a face feature vector corresponding to the face image.
在具体实施中，所述人脸向量化单元42，还适于将所述标准化人脸图像进行镜像，得到镜像后的人脸图像；采用人脸向量化算法对所述镜像后的人脸图像进行人脸向量化处理，获取预设维度的第二人脸特征向量；计算所述第一人脸特征向量与所述第二人脸特征向量的均值，将所求的均值作为所述人脸图像对应的人脸特征向量。In a specific implementation, the face vectorization unit 42 is further adapted to mirror the standardized face image to obtain a mirrored face image; a face vectorization algorithm is used to perform a mirror image of the mirrored face image. Perform face vectorization processing to obtain a second face feature vector of preset dimensions; calculate the mean value of the first face feature vector and the second face feature vector, and use the sought mean value as the face The face feature vector corresponding to the image.
在具体实施中，所述聚类单元43，适于采用如下任一种算法对所述人脸图像进行聚类：非极大值抑制算法、社区发现算法。In a specific implementation, the clustering unit 43 is adapted to use any one of the following algorithms to cluster the face images: a non-maximum value suppression algorithm and a community discovery algorithm.
在具体实施中，所述确定单元44，适于当采用非极大值抑制算法对所述人脸图像进行聚类时，将各类别中质量分最高的人脸图像作为对应类别的类中心点；当采用社区发现算法对所述人脸图像进行聚类时，将各类别中边最多的人脸图像作为对应类别的类中心点。In a specific implementation, the determining unit 44 is adapted to use the face image with the highest quality score in each category as the class center point of the corresponding category when the non-maximum value suppression algorithm is used to cluster the face images ; When using the community discovery algorithm to cluster the face images, the face images with the most edges in each category are used as the class center point of the corresponding category.
在具体实施中，所述聚类单元43，适于采用非极大值抑制算法对所述人脸图像进行聚类时，将所述人脸图像按照质量分倒序排列，记为人脸质量分列队；根据各人脸图像的人脸特征向量，计算每个人脸图像与其他人脸图像的相似度，记为相似度矩阵；在第i次迭代时，将质量分最高的人脸图像作为类别Ci的类中心点；根据所述相似度矩阵中各人脸图像之间的相似度，将与类别Ci的类中心点的相似度阈值达到预设阈值的人脸图像归入类别Ci；将所述类别Ci中的人脸图像从人脸质量分列队Ti删除，得到更新后的人脸质量分列队T(i+1)；将类别Ci中的人脸图像与类别Ci类中心点的相似度从所述相似度矩阵Mi中删除，得到更新后的相似度矩阵M(i+1)；当更新后的人脸质量分列队T(i+1)不为空时，继续从所述更新后的质量分列队T(i+1)中选取质量分最高的人脸图像作为类别C(i+1)的类中心点。In a specific implementation, the clustering unit 43 is adapted to use the non-maximum suppression algorithm to cluster the face images, and arrange the face images in reverse order of quality points, and record them as the face quality queue. ; Calculate the similarity between each face image and other face images according to the face feature vector of each face image, and record it as a similarity matrix; in the i-th iteration, take the face image with the highest quality score as the category Ci The class center point; according to the similarity between each face image in the similarity matrix, the face image whose similarity threshold with the class center point of category Ci reaches the preset threshold is classified into category Ci; The face images in the category Ci are deleted from the face quality queue Ti, and the updated face quality queue T(i+1) is obtained; the similarity between the face images in the category Ci and the center point of the category Ci is changed from Delete from the similarity matrix Mi to obtain the updated similarity matrix M(i+1); when the updated face quality queue T(i+1) is not empty, continue from the updated In the quality queue T(i+1), the face image with the highest quality score is selected as the class center point of the class C(i+1).
在具体实施中，所述聚类单元43，适于采用以下任一种算法，计算每个人脸图像与其他人脸图像的相似度：余弦距离算法、欧式距离算法。In a specific implementation, the clustering unit 43 is adapted to use any one of the following algorithms to calculate the similarity between each face image and other face images: cosine distance algorithm, Euclidean distance algorithm.
在具体实施中，所述聚类单元43，适于采用社区发现算法对所述人脸图像进行聚类时，将所有人脸图像划分为不同的类别，形成类别的集合SET(C)0；计算各类别中的所有的人脸图像的人脸相似度并与预设第二阈值进行比较，高于第二阈值权重为1，低于预设第二阈值权重为0，得到边的集合SET(E)0；第i次迭代时，对应的类别的集合SET(C)i，边的集合SET(E)i，计算当前聚类状态下的聚类稳定性Qi；将相邻的两个类别合并至同一个类别中，得到一个新的类别的集合SET(C)(i+1)；计算合并之后的各类别内的边的权重之后以及类别间的边的权重之和，得到边的集合SET(E)(i+1)；计算合并后的聚类稳定性Q(i+1)；当合并后的聚类稳定性Q(i+1)大于合并前的聚类稳定性Qi时，则接受合并，将错误划分状态组成的集合置空，并进入第i+1次迭代；当合并后的聚类稳定性Q(i+1)小于或等于合并前的聚类稳定性Qi时，则不接受合并，将当前划分加入错误划分状态组成的集合中；判断是否完成所有合并可能性，当完成所有的合并可能性，则将第i次迭代时对应的聚类作为最优聚类结果；当没有完成所有的合并可能性，则继续进行合并尝试。In a specific implementation, the clustering unit 43 is adapted to divide all the face images into different categories when the community discovery algorithm is used to cluster the face images to form a set of categories SET(C)0; Calculate the face similarity of all face images in each category and compare it with a preset second threshold, if the weight is higher than the second threshold, the weight is 1, and if the weight is lower than the preset second threshold, the weight is 0, and the set of edges SET is obtained. (E) 0; at the ith iteration, the set of corresponding categories SET(C)i, the set of edges SET(E)i, calculate the clustering stability Qi in the current clustering state; The categories are merged into the same category, and a new set of categories SET(C)(i+1) is obtained; after calculating the weights of the edges within each category after the merger and the sum of the weights of the edges between categories, the Set SET(E)(i+1); calculate the combined cluster stability Q(i+1); when the combined cluster stability Q(i+1) is greater than the pre-merged cluster stability Qi , then the merge is accepted, the set of incorrectly divided states is empty, and the i+1th iteration is entered; when the cluster stability Q(i+1) after the merge is less than or equal to the cluster stability Qi before the merge , the merge is not accepted, and the current division is added to the set composed of the wrong division state; it is judged whether all the merge possibilities are completed, and when all the merge possibilities are completed, the corresponding cluster at the i-th iteration is regarded as the optimal cluster. As a result; when all merge possibilities have not been completed, the merge attempt is continued.
在具体实施中，所述人脸图像处理装置40的工作原理及流程可以参考本发明上述实施例中提供的任一种人脸图像处理方法中的描述，此处不做赘述。In specific implementation, for the working principle and flow of the face image processing apparatus 40, reference may be made to the description in any one of the face image processing methods provided in the above embodiments of the present invention, which will not be repeated here.
本发明实施例还提供一种人脸图像处理装置，包括存储器和处理器，所述存储器上存储有可在所述处理器上运行的计算机指令，所述处理器运行所述计算机指令时执行本发明实施例中提供的上述任一种人脸图像处理方法的步骤。An embodiment of the present invention further provides a face image processing apparatus, including a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and the processor executes this computer instruction when the processor runs the computer instructions. Steps of any of the above-mentioned methods for processing face images provided in the embodiments of the present invention.
本发明实施例还提供一种计算机可读存储介质，计算机可读存储介质为非易失性存储介质或非瞬态存储介质，其上存储有计算机指令，所述计算机指令运行时执行本发明实施例中提供的上述任一种人脸图像处理方法的步骤。An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium is a non-volatile storage medium or a non-transitory storage medium, on which computer instructions are stored, and the computer instructions execute the implementation of the present invention when running. The steps of any one of the above-mentioned face image processing methods provided in the example.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于任一计算机可读存储介质中，存储介质可以包括：ROM、RAM、磁盘或光盘等。Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in any computer-readable storage medium, and the storage medium can include : ROM, RAM, disk or CD, etc.
虽然本发明披露如上，但本发明并非限定于此。任何本领域技术人员，在不脱离本发明的精神和范围内，均可作各种更动与修改，因此本发明的保护范围应当以权利要求所限定的范围为准。Although the present invention is disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be based on the scope defined by the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910196223.2ACN110069989B (en) | 2019-03-15 | 2019-03-15 | Face image processing method and device, and computer-readable storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910196223.2ACN110069989B (en) | 2019-03-15 | 2019-03-15 | Face image processing method and device, and computer-readable storage medium |
| Publication Number | Publication Date |
|---|---|
| CN110069989Atrue CN110069989A (en) | 2019-07-30 |
| CN110069989B CN110069989B (en) | 2021-07-30 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910196223.2AActiveCN110069989B (en) | 2019-03-15 | 2019-03-15 | Face image processing method and device, and computer-readable storage medium |
| Country | Link |
|---|---|
| CN (1) | CN110069989B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443297A (en)* | 2019-07-30 | 2019-11-12 | 浙江大华技术股份有限公司 | Clustering method, device and the computer storage medium of image |
| CN111128369A (en)* | 2019-11-18 | 2020-05-08 | 创新工场(北京)企业管理股份有限公司 | Method and device for evaluating Parkinson's disease condition of patient |
| CN111694979A (en)* | 2020-06-11 | 2020-09-22 | 重庆中科云从科技有限公司 | Archive management method, system, equipment and medium based on image |
| CN111738120A (en)* | 2020-06-12 | 2020-10-02 | 北京奇艺世纪科技有限公司 | Person identification method, person identification device, electronic equipment and storage medium |
| CN112528809A (en)* | 2020-12-04 | 2021-03-19 | 东方网力科技股份有限公司 | Method, device and equipment for identifying suspect and storage medium |
| CN113920353A (en)* | 2021-11-04 | 2022-01-11 | 厦门市美亚柏科信息股份有限公司 | Unsupervised face image secondary clustering method, unsupervised face image secondary clustering device and unsupervised face image secondary clustering medium |
| CN114255501A (en)* | 2021-12-22 | 2022-03-29 | 以萨技术股份有限公司 | Community monitoring method and system based on face clustering |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110123071A1 (en)* | 2005-09-28 | 2011-05-26 | Facedouble, Inc. | Method And System For Attaching A Metatag To A Digital Image |
| CN102930553A (en)* | 2011-08-10 | 2013-02-13 | 中国移动通信集团上海有限公司 | Method and device for identifying objectionable video content |
| WO2013063736A1 (en)* | 2011-10-31 | 2013-05-10 | Hewlett-Packard Development Company, L.P. | Temporal face sequences |
| CN104036261A (en)* | 2014-06-30 | 2014-09-10 | 北京奇虎科技有限公司 | Face recognition method and system |
| CN104966304A (en)* | 2015-06-08 | 2015-10-07 | 深圳市赛为智能股份有限公司 | Kalman filtering and nonparametric background model-based multi-target detection tracking method |
| CN105404863A (en)* | 2015-11-13 | 2016-03-16 | 小米科技有限责任公司 | Figure feature recognition method and system |
| CN105512685A (en)* | 2015-12-10 | 2016-04-20 | 小米科技有限责任公司 | Object identification method and apparatus |
| CN105868309A (en)* | 2016-03-24 | 2016-08-17 | 广东微模式软件股份有限公司 | A fast image search and self-printing method based on face image clustering and recognition technology |
| CN106503633A (en)* | 2016-10-10 | 2017-03-15 | 上海电机学院 | The method for building up in face characteristic storehouse in a kind of video image |
| CN106570178A (en)* | 2016-11-10 | 2017-04-19 | 重庆邮电大学 | High-dimension text data characteristic selection method based on graph clustering |
| CN107239736A (en)* | 2017-04-28 | 2017-10-10 | 北京智慧眼科技股份有限公司 | Method for detecting human face and detection means based on multitask concatenated convolutional neutral net |
| CN108108658A (en)* | 2016-11-24 | 2018-06-01 | Bioid股份公司 | Biological feather recognition method |
| CN108875522A (en)* | 2017-12-21 | 2018-11-23 | 北京旷视科技有限公司 | Face cluster methods, devices and systems and storage medium |
| CN109063580A (en)* | 2018-07-09 | 2018-12-21 | 北京达佳互联信息技术有限公司 | Face identification method, device, electronic equipment and storage medium |
| CN109388727A (en)* | 2018-09-12 | 2019-02-26 | 中国人民解放军国防科技大学 | A Fast Retrieval Method for BGP Faces Based on Clustering |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110123071A1 (en)* | 2005-09-28 | 2011-05-26 | Facedouble, Inc. | Method And System For Attaching A Metatag To A Digital Image |
| CN102930553A (en)* | 2011-08-10 | 2013-02-13 | 中国移动通信集团上海有限公司 | Method and device for identifying objectionable video content |
| WO2013063736A1 (en)* | 2011-10-31 | 2013-05-10 | Hewlett-Packard Development Company, L.P. | Temporal face sequences |
| CN104036261A (en)* | 2014-06-30 | 2014-09-10 | 北京奇虎科技有限公司 | Face recognition method and system |
| CN104966304A (en)* | 2015-06-08 | 2015-10-07 | 深圳市赛为智能股份有限公司 | Kalman filtering and nonparametric background model-based multi-target detection tracking method |
| CN105404863A (en)* | 2015-11-13 | 2016-03-16 | 小米科技有限责任公司 | Figure feature recognition method and system |
| CN105512685A (en)* | 2015-12-10 | 2016-04-20 | 小米科技有限责任公司 | Object identification method and apparatus |
| CN105868309A (en)* | 2016-03-24 | 2016-08-17 | 广东微模式软件股份有限公司 | A fast image search and self-printing method based on face image clustering and recognition technology |
| CN106503633A (en)* | 2016-10-10 | 2017-03-15 | 上海电机学院 | The method for building up in face characteristic storehouse in a kind of video image |
| CN106570178A (en)* | 2016-11-10 | 2017-04-19 | 重庆邮电大学 | High-dimension text data characteristic selection method based on graph clustering |
| CN108108658A (en)* | 2016-11-24 | 2018-06-01 | Bioid股份公司 | Biological feather recognition method |
| CN107239736A (en)* | 2017-04-28 | 2017-10-10 | 北京智慧眼科技股份有限公司 | Method for detecting human face and detection means based on multitask concatenated convolutional neutral net |
| CN108875522A (en)* | 2017-12-21 | 2018-11-23 | 北京旷视科技有限公司 | Face cluster methods, devices and systems and storage medium |
| CN109063580A (en)* | 2018-07-09 | 2018-12-21 | 北京达佳互联信息技术有限公司 | Face identification method, device, electronic equipment and storage medium |
| CN109388727A (en)* | 2018-09-12 | 2019-02-26 | 中国人民解放军国防科技大学 | A Fast Retrieval Method for BGP Faces Based on Clustering |
| Title |
|---|
| P.ITHAYA RANI等: "Ranking, clustering and fusing the normalized LBP temporal facial features for face recognition in video sequences", 《MULTIMEDIA TOOLS AND APPLICATIONS》* |
| YUJIE ZHONG 等: "GhostVLAD for Set-Based Face Recognition", 《ARXIV》* |
| 周勇: "《复杂多源数据的知识获取与知识发现》", 31 October 2015, 徐州：中国矿业大学出版社* |
| 陈晋音等: "聚类中心自动确定的谱聚类算法研究", 《小型微型计算机系统》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443297A (en)* | 2019-07-30 | 2019-11-12 | 浙江大华技术股份有限公司 | Clustering method, device and the computer storage medium of image |
| CN110443297B (en)* | 2019-07-30 | 2022-06-07 | 浙江大华技术股份有限公司 | Image clustering method and device and computer storage medium |
| CN111128369A (en)* | 2019-11-18 | 2020-05-08 | 创新工场(北京)企业管理股份有限公司 | Method and device for evaluating Parkinson's disease condition of patient |
| CN111694979A (en)* | 2020-06-11 | 2020-09-22 | 重庆中科云从科技有限公司 | Archive management method, system, equipment and medium based on image |
| CN111738120A (en)* | 2020-06-12 | 2020-10-02 | 北京奇艺世纪科技有限公司 | Person identification method, person identification device, electronic equipment and storage medium |
| CN111738120B (en)* | 2020-06-12 | 2023-12-05 | 北京奇艺世纪科技有限公司 | Character recognition method, character recognition device, electronic equipment and storage medium |
| CN112528809A (en)* | 2020-12-04 | 2021-03-19 | 东方网力科技股份有限公司 | Method, device and equipment for identifying suspect and storage medium |
| CN113920353A (en)* | 2021-11-04 | 2022-01-11 | 厦门市美亚柏科信息股份有限公司 | Unsupervised face image secondary clustering method, unsupervised face image secondary clustering device and unsupervised face image secondary clustering medium |
| CN113920353B (en)* | 2021-11-04 | 2022-07-29 | 厦门市美亚柏科信息股份有限公司 | Unsupervised face image secondary clustering method, unsupervised face image secondary clustering device and unsupervised face image secondary clustering medium |
| CN114255501A (en)* | 2021-12-22 | 2022-03-29 | 以萨技术股份有限公司 | Community monitoring method and system based on face clustering |
| Publication number | Publication date |
|---|---|
| CN110069989B (en) | 2021-07-30 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110069989B (en) | Face image processing method and device, and computer-readable storage medium | |
| CN110263774B (en) | A face detection method | |
| US11380119B2 (en) | Pose-aligned networks for deep attribute modeling | |
| CN110580445B (en) | Face key point detection method based on GIoU and weighted NMS improvement | |
| CN110363047A (en) | Method, device, electronic device and storage medium for face recognition | |
| WO2019232866A1 (en) | Human eye model training method, human eye recognition method, apparatus, device and medium | |
| CN105550657B (en) | Improvement SIFT face feature extraction method based on key point | |
| CN109840565A (en) | A kind of blink detection method based on eye contour feature point aspect ratio | |
| WO2019232862A1 (en) | Mouth model training method and apparatus, mouth recognition method and apparatus, device, and medium | |
| WO2017088432A1 (en) | Image recognition method and device | |
| WO2015149534A1 (en) | Gabor binary pattern-based face recognition method and device | |
| CN106407958B (en) | Face feature detection method based on double-layer cascade | |
| WO2022257456A1 (en) | Hair information recognition method, apparatus and device, and storage medium | |
| CN110188630A (en) | A face recognition method and camera | |
| US20240127631A1 (en) | Liveness detection method and apparatus, and computer device | |
| CN110569724A (en) | A Face Alignment Method Based on Residual Hourglass Network | |
| CN108596079A (en) | Gesture identification method, device and electronic equipment | |
| CN115205943A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| WO2022062403A9 (en) | Expression recognition model training method and apparatus, terminal device and storage medium | |
| CN113191195A (en) | Face detection method and system based on deep learning | |
| CN110321935B (en) | Method and device for determining business event relationship, and computer-readable storage medium | |
| CN109902720B (en) | Image Classification and Recognition Method Based on Subspace Decomposition for Depth Feature Estimation | |
| JP6430102B2 (en) | Person attribute estimation device, person attribute estimation method and program | |
| CN114511893A (en) | Convolutional neural network training method, face recognition method and device | |
| CN118537900A (en) | Face recognition method and device, electronic equipment and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |