技术领域technical field
本发明涉及一种人脸验证方法，具体涉及一种跨年龄的人脸验证方法，尤其是基于特征学习的跨年龄人脸验证方法。The invention relates to a face verification method, in particular to a cross-age face verification method, in particular to a cross-age face verification method based on feature learning.
背景技术Background technique
人脸，作为识别一个人最显著的区域，被广泛地应用于各种场合的身份识别。一般来说，人脸的识别方法包括四个步骤：人脸图像采集及检测、人脸图像预处理、人脸图像特征提取、人脸匹配与验证。通常使用一些人工设定的特征描述子，例如LBP、SIFT和Gabor等，来表示人脸数据，利用余弦距离来度量一对图像的相似度，从而实现判断验证。Face, as the most prominent area to identify a person, is widely used in various occasions for identification. Generally speaking, the face recognition method includes four steps: face image acquisition and detection, face image preprocessing, face image feature extraction, face matching and verification. Usually, some artificially set feature descriptors, such as LBP, SIFT, and Gabor, are used to represent face data, and the cosine distance is used to measure the similarity of a pair of images to achieve judgment verification.
但是随着年龄的增长，人的脸部会不可避免地产生变化。在一些场合，只是一个人不同年龄段的照片，例如只有十几年前的照片，需要将备选人员的头像与已有的线索进行比对验证，以达到目的，这就要求进行跨年龄人脸验证。所谓跨年龄人脸验证，就是给定一些不同年龄段的人脸图像，判定这些人脸图像是否属于同一个人。如果人脸验证方法能够应对人脸随着年龄的增长而产生的变化，在档案管理系统、安全验证系统、公安系统的罪犯身份识别、银行和海关的监控等领域，将具有广阔的应用前景。But as we age, our face inevitably changes. In some occasions, it is just a photo of a person of different ages, for example, a photo of only a dozen years ago. It is necessary to compare and verify the avatar of the candidate with the existing clues to achieve the purpose, which requires a cross-age person Face verification. The so-called cross-age face verification is to determine whether these face images belong to the same person given some face images of different age groups. If the face verification method can cope with the changes of the face with age, it will have broad application prospects in the fields of file management systems, security verification systems, criminal identification in public security systems, and monitoring of banks and customs.
为了实现跨年龄验证，大多数传统的方法是对年龄进行建模，通过设计人脸成长模型来进行跨年龄的人脸验证。然而，这类方法往往需要依赖先验，比如说个体的实际年龄，而并不是所有数据集都能够提供年龄信息。In order to achieve cross-age verification, most of the traditional methods are to model age, and carry out cross-age face verification by designing a face growth model. However, such methods often need to rely on priors, such as the actual age of the individual, and not all data sets can provide age information.
深度学习方法模拟人脑的层次处理结构，以简洁的表达方式刻画数据丰富的内在信息，它是一种高度非线性的模型，具有超强的数据拟合能力和学习能力，表达能力更强，更能刻画数据丰富的内在信息。深度网络可以无监督地从数据中学习到特征，这种方式学习到的特征也符合人类感知世界的机理，而且通过深度学习方法学习到的特征往往具有一定的语义特征。中国发明专利申请CN104573679A公开了一种监控场景下基于深度学习的人脸识别系统，包括视频采集单元、人脸检测单元、匹配显示单元、存储单元，其中，检测单元中设置有人脸差别模块，人脸差别模块利用深度学习模块建立的神经网络模型进行人脸判别，该深度学习模块为5层神经网络层。该方法试图利用深度学习方法实现人脸的快速识别，但是采用5层神经网络层难以达到预期目标，因此，该公开文件中并未能提供识别准确率的评估数据。The deep learning method simulates the hierarchical processing structure of the human brain, and describes the rich internal information of the data in a concise way. It is a highly nonlinear model with super data fitting ability and learning ability, and stronger expression ability. It can better describe the intrinsic information rich in data. Deep networks can learn features from data unsupervised. The features learned in this way also conform to the mechanism of human perception of the world, and the features learned through deep learning methods often have certain semantic features. Chinese invention patent application CN104573679A discloses a face recognition system based on deep learning in a monitoring scene, including a video acquisition unit, a face detection unit, a matching display unit, and a storage unit. The face difference module uses the neural network model established by the deep learning module for face discrimination, and the deep learning module is a 5-layer neural network layer. This method attempts to use deep learning methods to achieve rapid face recognition, but it is difficult to achieve the desired goal by using a 5-layer neural network layer. Therefore, the public document does not provide evaluation data for recognition accuracy.
就人脸验证中最关键的步骤特征提取而言，目前主要存在两个问题：As far as feature extraction is the most critical step in face verification, there are currently two main problems:
1、人脸图像的单调性。目前已知的大量人脸数据集中，人脸图像往往是比较单调的，而且目前大多数方法都是在单尺度上做的，这样提取的特征往往不够丰富，不足以表征人脸。1. Monotonicity of face images. In the large number of face datasets currently known, the face images are often relatively monotonous, and most current methods are done on a single scale, so the extracted features are often not rich enough to represent the face.
2、另一个值得关注的问题就是特征的获取。传统的人脸验证采用的都是手工设计的特征，这种特征针对性比较高，但是一般都是低层特征，往往不包含语义信息，而且泛化能力不强。随着大数据时代的到来，数据量也越来越大，如何自动地获取特征成为一个值得研究的课题。2. Another issue worthy of attention is the acquisition of features. Traditional face verification uses hand-designed features, which are highly targeted, but are generally low-level features, often do not contain semantic information, and the generalization ability is not strong. With the advent of the era of big data, the amount of data is also increasing, how to automatically obtain features has become a topic worth studying.
发明内容Contents of the invention
本发明的发明目的是提供一种基于特征学习的跨年龄人脸验证方法，以提高跨年龄人脸验证的准确率。The purpose of the present invention is to provide a feature learning-based cross-age face verification method to improve the accuracy of cross-age face verification.
为达到上述发明目的，本发明采用的技术方案是：一种基于特征学习的跨年龄人脸验证方法，包括如下步骤：In order to achieve the above-mentioned purpose of the invention, the technical solution adopted in the present invention is: a method for face verification across ages based on feature learning, comprising the following steps:
(1)获取待对比的两幅人脸图像；(1) Obtain two face images to be compared;
(2)利用人脸特征点定位的方法对两幅人脸图像进行对齐操作；(2) Utilize the method of facial feature point location to carry out alignment operation to two pieces of human face images;
(3)分别对每幅图像进行特征提取，所述特征提取为：(3) Carry out feature extraction to each image respectively, described feature extraction is:
①通过深度卷积神经网络自动提取高层语义特征；① Automatically extract high-level semantic features through deep convolutional neural networks;
②计算图像的LBP直方图特征；② Calculate the LBP histogram feature of the image;
③将①和②中获得的特征进行融合，得到图像的特征，表达为特征向量；这里采用的融合方法是将两部分的特征向量连接成一个特征向量。③Fuse the features obtained in ① and ② to obtain the features of the image and express them as feature vectors; the fusion method used here is to connect the two parts of the feature vectors into one feature vector.
(4)采用余弦相似度方法计算步骤(3)获得的两幅图像的特征向量之间的距离，据此判断两幅图像是否来自同一人。(4) Calculate the distance between the feature vectors of the two images obtained in step (3) by using the cosine similarity method, and judge whether the two images are from the same person accordingly.
优选的技术方案，步骤(2)中，采用Flandmark方法进行对齐操作。In the preferred technical solution, in step (2), the Flandmark method is used to perform the alignment operation.
上述技术方案中，所述深度卷积神经网络由下列各层依次构成：输入层、第一卷积层、第一最大池化层、第二卷积层、第二最大池化层、第三卷积层、第三最大池化层、第四卷积层、全连接层、输出层。In the above technical solution, the deep convolutional neural network is sequentially composed of the following layers: an input layer, a first convolutional layer, a first maximum pooling layer, a second convolutional layer, a second maximum pooling layer, a third Convolutional layer, third maximum pooling layer, fourth convolutional layer, fully connected layer, output layer.
其中，所述第一卷积层设有20个卷积核大小为4×4波器，第一最大池化层，池化的步长为2，第二卷积层设有40个卷积核大小为3×3的滤波器，第二最大池化层，池化步长为2，第三卷积层设有60个卷积核大小为4×4的滤波器，第三最大池化层，池化步长为2，第四卷积层设有60个卷积核大小为3×3的滤波器。Wherein, the first convolutional layer is provided with 20 convolution kernels with a size of 4×4 filters, the first maximum pooling layer has a pooling step size of 2, and the second convolutional layer is provided with 40 convolutional layers. A filter with a kernel size of 3×3, a second maximum pooling layer with a pooling step size of 2, a third convolutional layer with 60 filters with a convolution kernel size of 4×4, and a third maximum pooling Layer, the pooling step size is 2, and the fourth convolutional layer is equipped with 60 filters with a convolution kernel size of 3×3.
上述技术方案中，所述输出层中设置有一个K类的softmax分类器，K为要分类的数目。In the above technical solution, a K-class softmax classifier is set in the output layer, and K is the number to be classified.
LBP直方图特征的计算方法是，给定一幅图像I，图像I的金字塔表示为： ，其中是高斯核，k是金字塔的层数；G表示得到的图像金字塔，(x,y)表示像素的位置。The calculation method of the LBP histogram feature is, given an image I, the pyramid of the image I is expressed as: ,in Is the Gaussian kernel, k is the number of layers of the pyramid; G represents the obtained image pyramid, and (x, y) represents the position of the pixel.
对于金字塔的每一层分成8×8的块，每个块统计LBP直方图，将每个块的LBP直方图连接成一个向量，图像的LBP金字塔表示为：For each layer of the pyramid, it is divided into 8×8 blocks, and the LBP histogram of each block is counted, and the LBP histogram of each block is connected into a vector, and the LBP pyramid of the image is expressed as:
，其中，L（I）是对图像进行映射的一个表述。 , where L(I) is a representation for mapping images.
余弦相似度计算方法为，The calculation method of cosine similarity is:
图像i的特征向量为，The feature vector of image i is ,
图像j的特征向量为，The feature vector of image j is ,
式中，n为特征的个数，为图像i的第n个特征，为图像j的第n个特征；In the formula, n is the number of features, is the nth feature of image i, is the nth feature of image j;
图像i和图像j的特征向量之间的余弦相似度计算公式为：The cosine similarity calculation formula between the feature vectors of image i and image j is:
，k为特征向量中特征的序号。 , k is the serial number of the feature in the feature vector.
由于上述技术方案运用，本发明与现有技术相比具有下列优点：Due to the use of the above-mentioned technical solutions, the present invention has the following advantages compared with the prior art:
1、本发明采用九层深度卷积神经网络来自动地获取人脸图像的高层语义特征，它的权值共享降低了网络模型的复杂度，这也是首次将深度网络应用到跨年龄人脸验证。1. The present invention uses a nine-layer deep convolutional neural network to automatically obtain high-level semantic features of face images, and its weight sharing reduces the complexity of the network model. This is the first time that a deep network is applied to cross-age face verification .
2、本发明创造性地将手工设计的LBP直方图特征与深度网络自主学习的特征进行融合，实现高层语义特征与低层特征的互补，实验结果表明其具有更好的准确率。2. The present invention creatively fuses the manually designed LBP histogram features with the features learned by the deep network autonomously to realize the complementarity of high-level semantic features and low-level features. Experimental results show that it has better accuracy.
附图说明Description of drawings
图1是本发明实施例中方法的框架构图；Fig. 1 is a framework diagram of the method in the embodiment of the present invention;
图2是实施例中使用的深度卷积神经网络架构图；Fig. 2 is a deep convolutional neural network architecture diagram used in an embodiment;
图3是实施例中使用的CACD数据集示例图；Fig. 3 is the example diagram of the CACD dataset used in the embodiment;
图4是实施例中结果的ROC曲线图。Fig. 4 is a ROC curve graph of the results in Examples.
具体实施方式Detailed ways
下面结合附图及实施例对本发明作进一步描述：The present invention will be further described below in conjunction with accompanying drawing and embodiment:
实施例一：Embodiment one:
一种基于特征学习的跨年龄人脸验证方法，参见图1所示，包括如下步骤：(1)获取待对比的两幅人脸图像；A cross-age face verification method based on feature learning, as shown in Figure 1, comprises the following steps: (1) obtaining two face images to be compared;
(2)利用人脸特征点定位的方法对两幅人脸图像进行对齐操作；(2) Utilize the method of facial feature point location to carry out alignment operation to two pieces of human face images;
在不受约束的环境条件下，人脸图像不可避免地会受到面部表情、光照或者遮挡的影响，当两张图像的人脸部分没有充分对齐的时候，这种影响会被放大。Under unconstrained environmental conditions, face images are inevitably affected by facial expressions, lighting, or occlusions, and this effect is amplified when the face parts of two images are not sufficiently aligned.
人脸特征点定位的目的是在人脸检测的基础上，进一步确定脸部特征点（眼睛、眉毛、鼻子、嘴巴、脸部外轮廓）的位置。定位算法的基本思路是：人脸的纹理特征和各个特征点之间的位置约束结合。早期的面部特征点定位主要集中在几个关键点的定位上，比如说眼睛和嘴巴的中心。后来，研究者们发现加入更多的特征点约束能够有效地提高准确率，增强稳定性。在本实施例中，采用Flandmark方法来进行人脸对齐。它是一个检测面部关键点的开源代码库，而且可以实现实时的对齐操作。The purpose of facial feature point positioning is to further determine the position of facial feature points (eyes, eyebrows, nose, mouth, and facial contour) on the basis of face detection. The basic idea of the positioning algorithm is: the combination of the texture features of the face and the position constraints between each feature point. Early facial feature point localization mainly focused on the localization of several key points, such as the center of eyes and mouth. Later, researchers found that adding more feature point constraints can effectively improve accuracy and enhance stability. In this embodiment, the Flandmark method is used for face alignment. It is an open source code library for detecting facial key points, and can achieve real-time alignment operation.
(3)分别对每幅图像进行特征提取，所述特征提取为：(3) Carry out feature extraction to each image respectively, described feature extraction is:
①通过深度卷积神经网络（DCNN）自动提取高层语义特征；① Automatically extract high-level semantic features through deep convolutional neural network (DCNN);
本实施例中，利用卷积操作和池化操作用来层次地提取图像中的视觉特征，从局部低层特征到全局高层特征。其中包含4个卷积层，每个卷积层后面接一个最大池化层。在每一个卷积层，层内神经元的权值是共享的，而层间权值不存在共享。最后是两个全连接层，倒数第二层的输出即为得到的粗糙的特征，最后一层是输出层，输出的是每幅人脸图像对应的最大概率的人脸图像对应的ID号，也就是每一幅图像对应的类别。In this embodiment, convolution operations and pooling operations are used to extract visual features in an image hierarchically, from local low-level features to global high-level features. It contains 4 convolutional layers, each followed by a maximum pooling layer. In each convolutional layer, the weights of neurons in the layer are shared, but there is no sharing of weights between layers. Finally, there are two fully connected layers. The output of the penultimate layer is the rough feature obtained. The last layer is the output layer, which outputs the ID number corresponding to the face image with the highest probability corresponding to each face image. That is, the category corresponding to each image.
具体地，参见附图2所示，DCNN的整体结构中，一个大小为55×47的经过对齐处理的人脸图像输入到卷积神经网络的第一个卷积层，第一层有20个卷积核大小为4×4的滤波器。得到了20张特征图，然后这些特征图输入到一个最大池化层，池化步长为2。然后池化层的输出作为下一个卷积层的输入，这个卷积层有40个卷积核为3×3的滤波器。这三层主要的工作时提取图像的低层特征比如说简单的边缘特征。在图像局部有小的变动的时候，最大池化使得我们卷积层得到的结果更加鲁棒。而之前提到的2D人脸对齐，使得整个网络对于人脸一些细微的调整有更好的适应性。我们的第三个卷积层设有60个卷积核大小为4×4的滤波器，然后进行第三次最大池化，池化步长为2，最后的第四层卷积层设有60个卷积核大小为3×3的滤波器。经过DCNN的逐层提取，得到了一系列的人脸特征。将特征提取的过程定义为f=Conv（x,q）,其中Conv(.)表示卷积神经网络的特征提取函数，x表示输入的图像, f代表得到的特征向量，q表示的是DCNN中需要学习的参数。Specifically, as shown in Figure 2, in the overall structure of DCNN, an aligned face image with a size of 55×47 is input to the first convolutional layer of the convolutional neural network, and the first layer has 20 A filter with a kernel size of 4×4. 20 feature maps are obtained, and then these feature maps are input into a max pooling layer with a pooling step size of 2. The output of the pooling layer is then used as the input to the next convolutional layer, which has 40 filters with a convolution kernel of 3×3. The main work of these three layers is to extract the low-level features of the image, such as simple edge features. When there are small local changes in the image, the maximum pooling makes the results obtained by our convolutional layer more robust. The 2D face alignment mentioned earlier makes the whole network more adaptable to some minor adjustments of the face. Our third convolutional layer has 60 filters with a convolution kernel size of 4×4, and then performs the third maximum pooling with a pooling step of 2, and the final fourth convolutional layer has 60 filters with a kernel size of 3×3. After layer-by-layer extraction of DCNN, a series of face features are obtained. The process of feature extraction is defined as f=Conv(x,q), where Conv(.) represents the feature extraction function of the convolutional neural network, x represents the input image, f represents the obtained feature vector, and q represents the DCNN parameters to be learned.
最后一个全连接层的输出将被送到一个K类的softmax分类器中，K表示的要分类的数目，这个分类器产生一个类标签的分布。给定一个输入，假设其中第k个输出为，然后softmax分类器的激活函数即为：The output of the last fully connected layer will be fed into a softmax classifier of K classes, where K represents the number of classes to be classified, and this classifier produces a distribution of class labels. Given an input, suppose the kth output is , and then the activation function of the softmax classifier is:
训练的目的是最大化每幅人脸图像被分到正确的face_id的概率，这个问题就可以转化为最小化每个训练样本的交叉熵损失函数。如果说k是一个训练样本正确的标签值，那么这个损失函数可以定义为：. 可以采用随机梯度下降的方法来对这个损失函数进行最小化计算，梯度下降采用的是最经典的BP算法。在一个卷积层，上一层的特征maps被一个可学习的卷积核进行卷积，然后通过一个激活函数，就可以得到输出特征map，每一个输出map可能是多个输入map的值。The purpose of training is to maximize the probability that each face image is assigned to the correct face_id, and this problem can be transformed into minimizing the cross-entropy loss function of each training sample. If k is the correct label value of a training sample, then this loss function can be defined as: . The stochastic gradient descent method can be used to minimize the loss function. The gradient descent adopts the most classic BP algorithm. In a convolutional layer, the feature maps of the previous layer are convoluted by a learnable convolution kernel, and then through an activation function, the output feature map can be obtained. Each output map may be the value of multiple input maps.
②计算图像的LBP直方图特征；② Calculate the LBP histogram feature of the image;
LBP直方图特征的计算方法是，给定一幅图像I，图像I的金字塔表示为：，其中是高斯核，s是金字塔的层数；The calculation method of the LBP histogram feature is, given an image I, the pyramid of the image I is expressed as: ,in Is the Gaussian kernel, s is the number of layers of the pyramid;
对于金字塔的每一层分成8×8的块，每个块统计LBP直方图，将每个块的LBP直方图连接成一个向量，图像的LBP金字塔表示为：For each layer of the pyramid, it is divided into 8×8 blocks, and the LBP histogram of each block is counted, and the LBP histogram of each block is connected into a vector, and the LBP pyramid of the image is expressed as:
③将①和②中获得的特征进行融合，得到图像的特征，表达为特征向量；③Fuse the features obtained in ① and ② to obtain the features of the image and express them as feature vectors;
(4)采用余弦相似度方法计算步骤(3)获得的两幅图像的特征向量之间的距离，据此判断两幅图像是否来自同一人。(4) Calculate the distance between the feature vectors of the two images obtained in step (3) by using the cosine similarity method, and judge whether the two images are from the same person accordingly.
余弦相似度计算方法为，The calculation method of cosine similarity is:
图像i的特征向量为，The feature vector of image i is ,
图像j的特征向量为，The feature vector of image j is ,
式中，n为特征的个数，为图像i的第n个特征，为图像j的第n个特征；In the formula, n is the number of features, is the nth feature of image i, is the nth feature of image j;
图像i和图像j的特征向量之间的余弦相似度计算公式为：The cosine similarity calculation formula between the feature vectors of image i and image j is:
，k为特征向量中特征的序号。 , k is the serial number of the feature in the feature vector.
通过在给定标签的数据集上进行训练，对分类器的参数进行微调，能够使得本实施例的分类器性能更好。By performing training on a data set with a given label and fine-tuning the parameters of the classifier, the performance of the classifier in this embodiment can be improved.
以下进一步通过具体的比对来说明本发明的效果。The effects of the present invention will be further illustrated through specific comparisons below.
实验在CACD(Cross-Age Celebrity Dataset)年龄数据集以及LFW数据集上进行。CACD年龄数据集包含超过160000张来自2000个不同的人的人脸图像，每个人有多张不同年龄阶段的图像，年龄跨度为16到62岁。由于在LFW数据集中，大多数人都只有一张图片，因此用这个数据集来进行训练是不太可行的，因此，采用了CelebFaces+这个数据集进行模型的训练，这个数据集包含来自10177个不同的人的202599张图片，而LFW和CelebFaces数据集中的人基本相同，因此可以在这个上面训练好模型然后在LFW数据集上进行测试。实验硬件环境：Window 7，Core i7处理器，主频为3.4G，内存为8G。代码运行环境是：Matlab 2013a。The experiment is carried out on the CACD (Cross-Age Celebrity Dataset) age dataset and the LFW dataset. The CACD age dataset contains more than 160,000 face images from 2,000 different people, and each person has multiple images of different ages, ranging from 16 to 62 years old. Since most people have only one picture in the LFW data set, it is not feasible to use this data set for training. Therefore, the CelebFaces+ data set is used for model training. This data set contains 10177 different There are 202,599 pictures of people, and the people in the LFW and CelebFaces datasets are basically the same, so the model can be trained on this and then tested on the LFW dataset. Experimental hardware environment: Window 7, Core i7 processor, main frequency is 3.4G, memory is 8G. The code running environment is: Matlab 2013a.
在实验中，用True Positive Rate(TPR)-False Positive Rate(TPR)来衡量实验的效果。TPR和FPR定义如下：In the experiment, True Positive Rate (TPR)-False Positive Rate (TPR) is used to measure the effect of the experiment. TPR and FPR are defined as follows:
1, 在CACD数据集上的实验1, Experiments on the CACD dataset
数据集示例参见附图3所示。An example of the data set is shown in Figure 3.
对于不同的相似性度量方法，结果如表1所示。For different similarity measurement methods, the results are shown in Table 1.
表1 不同相似性度量方法的比较Table 1 Comparison of different similarity measurement methods
对本发明的算法用ROC曲线进行了评估，并与其他方法进行对比。进行对比实验的方法有：梯度方向金字塔（GOP）、范数、Bayesian+PointFive Face(PFF)。在本实验中，测试集包含了2000对正样本和2000对负样本。实验结果的ROC曲线如图4所示。The algorithm of the present invention is evaluated with ROC curve and compared with other methods. The methods for comparative experiments are: Gradient Oriented Pyramid (GOP), Norm, Bayesian+PointFive Face (PFF). In this experiment, the test set contains 2000 pairs of positive samples and 2000 pairs of negative samples. The ROC curve of the experimental results is shown in Figure 4.
从图4可以看出，采用本发明的方法（DCNN+LBPH）相较于其他方法而言，结果上有一定程度的提高。It can be seen from Figure 4 that compared with other methods, the method of the present invention (DCNN+LBPH) has improved the results to a certain extent.
在设置好的CACD数据集子集上，也对验证的准确率进行了评估，与其它方法的比较如表2所示。On the set subset of the CACD dataset, the accuracy of the verification is also evaluated, and the comparison with other methods is shown in Table 2.
表2：Table 2:
2, 在LFW数据集上的实验2, Experiments on the LFW dataset
LFW数据集是一个人脸识别领域的标准数据集。在这个庞大的数据集中，加入一些限制，比如年龄、强光照或者表情的变化之后，任何超越当前算法的小小的进步都是很不容易的。The LFW dataset is a standard dataset in the field of face recognition. In this huge data set, after adding some restrictions, such as age, strong lighting or expression changes, any small progress beyond the current algorithm is not easy.
跑完训练集后，本实施例在LFW view2这个子集中进行验证的工作，这个子集已经列出了所有的同一人脸图像对以及非同一人脸图像对。采取跟CACD数据集上的实验同样的方法，第一步是获取一个准确的人脸表达，然后计算它们的相似度来验证是否属于同一个个体。本发明的方法引入了深度学习框架，与其他算法相比，实验结果如表3所示。可见，本发明的方法在准确率上是优于之前其他方法的。After running the training set, this embodiment performs verification work in the subset of LFW view2, which has listed all the same face image pairs and non-identical face image pairs. Taking the same method as the experiments on the CACD dataset, the first step is to obtain an accurate face expression, and then calculate their similarity to verify whether they belong to the same individual. The method of the present invention introduces a deep learning framework. Compared with other algorithms, the experimental results are shown in Table 3. It can be seen that the method of the present invention is superior to other previous methods in terms of accuracy.
表3table 3
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510270145.8ACN104866829B (en) | 2015-05-25 | 2015-05-25 | A cross-age face verification method based on feature learning |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510270145.8ACN104866829B (en) | 2015-05-25 | 2015-05-25 | A cross-age face verification method based on feature learning |
| Publication Number | Publication Date |
|---|---|
| CN104866829Atrue CN104866829A (en) | 2015-08-26 |
| CN104866829B CN104866829B (en) | 2019-02-19 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510270145.8AActiveCN104866829B (en) | 2015-05-25 | 2015-05-25 | A cross-age face verification method based on feature learning |
| Country | Link |
|---|---|
| CN (1) | CN104866829B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105550658A (en)* | 2015-12-24 | 2016-05-04 | 蔡叶荷 | Face comparison method based on high-dimensional LBP (Local Binary Patterns) and convolutional neural network feature fusion |
| CN105844227A (en)* | 2016-03-21 | 2016-08-10 | 湖南君士德赛科技发展有限公司 | Driver identity authentication method for school bus safety |
| CN105956572A (en)* | 2016-05-15 | 2016-09-21 | 北京工业大学 | In vivo face detection method based on convolutional neural network |
| CN106022287A (en)* | 2016-05-27 | 2016-10-12 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Over-age face verification method based on deep learning and dictionary representation |
| CN106022380A (en)* | 2016-05-25 | 2016-10-12 | 中国科学院自动化研究所 | Individual identity identification method based on deep learning |
| CN106096535A (en)* | 2016-06-07 | 2016-11-09 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | A kind of face verification method based on bilinearity associating CNN |
| CN106203533A (en)* | 2016-07-26 | 2016-12-07 | 厦门大学 | The degree of depth based on combined training study face verification method |
| CN106227718A (en)* | 2016-07-18 | 2016-12-14 | 中国民航大学 | Land based on CNN sky call semantic consistency method of calibration |
| CN106326868A (en)* | 2016-08-26 | 2017-01-11 | 江苏华通晟云科技有限公司 | Face identification method based on cosine similarity measure learning |
| CN106485235A (en)* | 2016-10-24 | 2017-03-08 | 厦门美图之家科技有限公司 | A kind of convolutional neural networks generation method, age recognition methods and relevant apparatus |
| CN106557743A (en)* | 2016-10-26 | 2017-04-05 | 桂林电子科技大学 | A kind of face characteristic extraction system and method based on FECNN |
| CN106599926A (en)* | 2016-12-20 | 2017-04-26 | 上海寒武纪信息科技有限公司 | Expression picture pushing method and system |
| CN106650573A (en)* | 2016-09-13 | 2017-05-10 | 华南理工大学 | Cross-age face verification method and system |
| CN106650650A (en)* | 2016-12-14 | 2017-05-10 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Cross-age face recognition method |
| CN106709448A (en)* | 2016-12-21 | 2017-05-24 | 华南理工大学 | Face convolutional neural network-based traditional Chinese medicine constitution recognition method |
| CN106980834A (en)* | 2017-03-24 | 2017-07-25 | 中山大学 | A kind of automatic classification method of facial image |
| CN107053168A (en)* | 2016-12-09 | 2017-08-18 | 南京理工大学 | A kind of target identification method and hot line robot based on deep learning network |
| CN107341463A (en)* | 2017-06-28 | 2017-11-10 | 北京飞搜科技有限公司 | A kind of face characteristic recognition methods of combination image quality analysis and metric learning |
| CN107480575A (en)* | 2016-06-07 | 2017-12-15 | 深圳市商汤科技有限公司 | The training method of model, across age face identification method and corresponding device |
| CN107507286A (en)* | 2017-08-02 | 2017-12-22 | 五邑大学 | A kind of bi-mode biology feature based on face and handwritten signature is registered system |
| CN107590449A (en)* | 2017-08-31 | 2018-01-16 | 电子科技大学 | A kind of gesture detecting method based on weighted feature spectrum fusion |
| CN107590478A (en)* | 2017-09-26 | 2018-01-16 | 四川长虹电器股份有限公司 | A kind of age estimation method based on deep learning |
| CN107622225A (en)* | 2017-07-27 | 2018-01-23 | 成都信息工程大学 | Face Recognition Method Based on Independent Component Analysis Network |
| CN107818314A (en)* | 2017-11-22 | 2018-03-20 | 北京达佳互联信息技术有限公司 | Face image processing method, device and server |
| CN107871099A (en)* | 2016-09-23 | 2018-04-03 | 北京眼神科技有限公司 | Face detection method and apparatus |
| CN108229269A (en)* | 2016-12-31 | 2018-06-29 | 深圳市商汤科技有限公司 | Method for detecting human face, device and electronic equipment |
| CN108604383A (en)* | 2015-12-04 | 2018-09-28 | 奇跃公司 | Reposition system and method |
| CN108960412A (en)* | 2018-06-29 | 2018-12-07 | 北京京东尚科信息技术有限公司 | Image-recognizing method, device and computer readable storage medium |
| CN109255289A (en)* | 2018-07-27 | 2019-01-22 | 电子科技大学 | A kind of across aging face identification method generating model based on unified formula |
| CN109344774A (en)* | 2018-10-08 | 2019-02-15 | 国网经济技术研究院有限公司 | Thermal power station target identification method in remote sensing image |
| CN109766998A (en)* | 2019-01-14 | 2019-05-17 | 温岭市非普电气有限公司 | A Feature Pyramid Network Structure Based on MobileNet Acceleration |
| CN109815864A (en)* | 2019-01-11 | 2019-05-28 | 浙江工业大学 | A transfer learning-based age recognition method for face images |
| CN109886121A (en)* | 2019-01-23 | 2019-06-14 | 浙江大学 | An Occlusion Robust Face Keypoint Localization Method |
| CN110322509A (en)* | 2019-06-26 | 2019-10-11 | 重庆邮电大学 | Object localization method, system and computer equipment based on level Class Activation figure |
| WO2019223254A1 (en)* | 2018-05-21 | 2019-11-28 | 北京亮亮视野科技有限公司 | Construction method for multi-scale lightweight face detection model and face detection method based on model |
| CN111710424A (en)* | 2020-06-19 | 2020-09-25 | 浙江新芮信息科技有限公司 | Catering personnel health monitoring method, device and computer-readable storage medium |
| CN111881721A (en)* | 2020-06-10 | 2020-11-03 | 广东芯盾微电子科技有限公司 | Cross-age face recognition method, system, device and storage medium |
| CN112287965A (en)* | 2020-09-21 | 2021-01-29 | 卓尔智联(武汉)研究院有限公司 | Image quality detection model training method and device and computer equipment |
| CN112766157A (en)* | 2021-01-20 | 2021-05-07 | 乐山师范学院 | Cross-age face image recognition method based on disentanglement representation learning |
| US11256090B2 (en) | 2015-03-05 | 2022-02-22 | Magic Leap, Inc. | Systems and methods for augmented reality |
| CN114155567A (en)* | 2020-09-08 | 2022-03-08 | 京东方科技集团股份有限公司 | Target detection method and device, storage medium and electronic device |
| US11315214B2 (en) | 2017-03-17 | 2022-04-26 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual con tent using same |
| US11361585B2 (en)* | 2018-06-11 | 2022-06-14 | Zkteco Usa Llc | Method and system for face recognition via deep learning |
| US11379948B2 (en) | 2018-07-23 | 2022-07-05 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
| US11410269B2 (en) | 2017-03-17 | 2022-08-09 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
| US11423626B2 (en) | 2017-03-17 | 2022-08-23 | Magic Leap, Inc. | Mixed reality system with multi-source virtual content compositing and method of generating virtual content using same |
| US11429183B2 (en) | 2015-03-05 | 2022-08-30 | Magic Leap, Inc. | Systems and methods for augmented reality |
| US11536973B2 (en) | 2016-08-02 | 2022-12-27 | Magic Leap, Inc. | Fixed-distance virtual and augmented reality systems and methods |
| US11711668B2 (en) | 2017-01-23 | 2023-07-25 | Magic Leap, Inc. | Localization determination for mixed reality systems |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110103695A1 (en)* | 2009-11-04 | 2011-05-05 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| CN102663413A (en)* | 2012-03-09 | 2012-09-12 | 中盾信安科技(江苏)有限公司 | Multi-gesture and cross-age oriented face image authentication method |
| CN103902964A (en)* | 2012-12-31 | 2014-07-02 | 深圳先进技术研究院 | Face recognition method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110103695A1 (en)* | 2009-11-04 | 2011-05-05 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| CN102663413A (en)* | 2012-03-09 | 2012-09-12 | 中盾信安科技(江苏)有限公司 | Multi-gesture and cross-age oriented face image authentication method |
| CN103902964A (en)* | 2012-12-31 | 2014-07-02 | 深圳先进技术研究院 | Face recognition method |
| Title |
|---|
| BOR-CHUN CHEN ETAL: "Cross-Age Reference Coding for Age-Invariant Face Recognition and Retrieval", 《EUROPEAN CONFERENCE ON COMPUTER VISION》* |
| W. ZHAO ETAL: "Face Recognition: A Literature Survey", 《ACM COMPUTING SURVEYS》* |
| YANIV TAIGMAN ETAL: "DeepFace: Closing the Gap to Human-Level Performance in Face Verification", 《2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》* |
| 付晓峰: "基于二元模式的人脸识别与表情识别研究", 《中国博士学位论文全文数据库 信息科技辑》* |
| 张雯 等: "基于局部二值模式和深度学习的人脸识别", 《计算机应用》* |
| 黄非非: "基于LBP的人脸识别研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12386417B2 (en) | 2015-03-05 | 2025-08-12 | Magic Leap, Inc. | Systems and methods for augmented reality |
| US11429183B2 (en) | 2015-03-05 | 2022-08-30 | Magic Leap, Inc. | Systems and methods for augmented reality |
| US11619988B2 (en) | 2015-03-05 | 2023-04-04 | Magic Leap, Inc. | Systems and methods for augmented reality |
| US11256090B2 (en) | 2015-03-05 | 2022-02-22 | Magic Leap, Inc. | Systems and methods for augmented reality |
| CN108604383A (en)* | 2015-12-04 | 2018-09-28 | 奇跃公司 | Reposition system and method |
| US11288832B2 (en) | 2015-12-04 | 2022-03-29 | Magic Leap, Inc. | Relocalization systems and methods |
| CN105550658A (en)* | 2015-12-24 | 2016-05-04 | 蔡叶荷 | Face comparison method based on high-dimensional LBP (Local Binary Patterns) and convolutional neural network feature fusion |
| CN105844227A (en)* | 2016-03-21 | 2016-08-10 | 湖南君士德赛科技发展有限公司 | Driver identity authentication method for school bus safety |
| CN105956572A (en)* | 2016-05-15 | 2016-09-21 | 北京工业大学 | In vivo face detection method based on convolutional neural network |
| CN106022380A (en)* | 2016-05-25 | 2016-10-12 | 中国科学院自动化研究所 | Individual identity identification method based on deep learning |
| CN106022287A (en)* | 2016-05-27 | 2016-10-12 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Over-age face verification method based on deep learning and dictionary representation |
| CN107480575B (en)* | 2016-06-07 | 2020-09-22 | 深圳市商汤科技有限公司 | Model training method, trans-age face recognition method and corresponding device |
| CN107480575A (en)* | 2016-06-07 | 2017-12-15 | 深圳市商汤科技有限公司 | The training method of model, across age face identification method and corresponding device |
| CN106096535B (en)* | 2016-06-07 | 2020-10-23 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | A face verification method based on bilinear joint CNN |
| CN106096535A (en)* | 2016-06-07 | 2016-11-09 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | A kind of face verification method based on bilinearity associating CNN |
| CN106227718A (en)* | 2016-07-18 | 2016-12-14 | 中国民航大学 | Land based on CNN sky call semantic consistency method of calibration |
| CN106203533A (en)* | 2016-07-26 | 2016-12-07 | 厦门大学 | The degree of depth based on combined training study face verification method |
| CN106203533B (en)* | 2016-07-26 | 2019-09-20 | 厦门大学 | Deep learning face verification method based on hybrid training |
| US11536973B2 (en) | 2016-08-02 | 2022-12-27 | Magic Leap, Inc. | Fixed-distance virtual and augmented reality systems and methods |
| CN106326868A (en)* | 2016-08-26 | 2017-01-11 | 江苏华通晟云科技有限公司 | Face identification method based on cosine similarity measure learning |
| CN106650573A (en)* | 2016-09-13 | 2017-05-10 | 华南理工大学 | Cross-age face verification method and system |
| CN106650573B (en)* | 2016-09-13 | 2019-07-16 | 华南理工大学 | A cross-age face verification method and system |
| CN107871099A (en)* | 2016-09-23 | 2018-04-03 | 北京眼神科技有限公司 | Face detection method and apparatus |
| CN106485235A (en)* | 2016-10-24 | 2017-03-08 | 厦门美图之家科技有限公司 | A kind of convolutional neural networks generation method, age recognition methods and relevant apparatus |
| CN106485235B (en)* | 2016-10-24 | 2019-05-03 | 厦门美图之家科技有限公司 | A kind of convolutional neural networks generation method, age recognition methods and relevant apparatus |
| CN106557743A (en)* | 2016-10-26 | 2017-04-05 | 桂林电子科技大学 | A kind of face characteristic extraction system and method based on FECNN |
| CN106557743B (en)* | 2016-10-26 | 2020-04-24 | 桂林电子科技大学 | Facial feature extraction system and method based on FECNN |
| CN107053168A (en)* | 2016-12-09 | 2017-08-18 | 南京理工大学 | A kind of target identification method and hot line robot based on deep learning network |
| CN106650650B (en)* | 2016-12-14 | 2020-04-24 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Cross-age face recognition method |
| CN106650650A (en)* | 2016-12-14 | 2017-05-10 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Cross-age face recognition method |
| CN106599926A (en)* | 2016-12-20 | 2017-04-26 | 上海寒武纪信息科技有限公司 | Expression picture pushing method and system |
| CN106709448A (en)* | 2016-12-21 | 2017-05-24 | 华南理工大学 | Face convolutional neural network-based traditional Chinese medicine constitution recognition method |
| CN108229269A (en)* | 2016-12-31 | 2018-06-29 | 深圳市商汤科技有限公司 | Method for detecting human face, device and electronic equipment |
| US11182591B2 (en) | 2016-12-31 | 2021-11-23 | Shenzhen Sensetime Technology Co, Ltd | Methods and apparatuses for detecting face, and electronic devices |
| US11711668B2 (en) | 2017-01-23 | 2023-07-25 | Magic Leap, Inc. | Localization determination for mixed reality systems |
| US11315214B2 (en) | 2017-03-17 | 2022-04-26 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual con tent using same |
| US11978175B2 (en) | 2017-03-17 | 2024-05-07 | Magic Leap, Inc. | Mixed reality system with color virtual content warping and method of generating virtual content using same |
| US11410269B2 (en) | 2017-03-17 | 2022-08-09 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
| US11423626B2 (en) | 2017-03-17 | 2022-08-23 | Magic Leap, Inc. | Mixed reality system with multi-source virtual content compositing and method of generating virtual content using same |
| CN106980834A (en)* | 2017-03-24 | 2017-07-25 | 中山大学 | A kind of automatic classification method of facial image |
| CN107341463B (en)* | 2017-06-28 | 2020-06-05 | 苏州飞搜科技有限公司 | Face feature recognition method combining image quality analysis and metric learning |
| CN107341463A (en)* | 2017-06-28 | 2017-11-10 | 北京飞搜科技有限公司 | A kind of face characteristic recognition methods of combination image quality analysis and metric learning |
| CN107622225A (en)* | 2017-07-27 | 2018-01-23 | 成都信息工程大学 | Face Recognition Method Based on Independent Component Analysis Network |
| CN107622225B (en)* | 2017-07-27 | 2020-04-03 | 成都信息工程大学 | Face recognition method based on independent component analysis network |
| CN107507286A (en)* | 2017-08-02 | 2017-12-22 | 五邑大学 | A kind of bi-mode biology feature based on face and handwritten signature is registered system |
| CN107590449A (en)* | 2017-08-31 | 2018-01-16 | 电子科技大学 | A kind of gesture detecting method based on weighted feature spectrum fusion |
| CN107590478A (en)* | 2017-09-26 | 2018-01-16 | 四川长虹电器股份有限公司 | A kind of age estimation method based on deep learning |
| CN107818314A (en)* | 2017-11-22 | 2018-03-20 | 北京达佳互联信息技术有限公司 | Face image processing method, device and server |
| WO2019223254A1 (en)* | 2018-05-21 | 2019-11-28 | 北京亮亮视野科技有限公司 | Construction method for multi-scale lightweight face detection model and face detection method based on model |
| US11361585B2 (en)* | 2018-06-11 | 2022-06-14 | Zkteco Usa Llc | Method and system for face recognition via deep learning |
| CN108960412A (en)* | 2018-06-29 | 2018-12-07 | 北京京东尚科信息技术有限公司 | Image-recognizing method, device and computer readable storage medium |
| US11379948B2 (en) | 2018-07-23 | 2022-07-05 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
| US12190468B2 (en) | 2018-07-23 | 2025-01-07 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
| US11790482B2 (en) | 2018-07-23 | 2023-10-17 | Magic Leap, Inc. | Mixed reality system with virtual content warping and method of generating virtual content using same |
| CN109255289B (en)* | 2018-07-27 | 2021-10-26 | 电子科技大学 | Cross-aging face recognition method based on unified generation model |
| CN109255289A (en)* | 2018-07-27 | 2019-01-22 | 电子科技大学 | A kind of across aging face identification method generating model based on unified formula |
| CN109344774A (en)* | 2018-10-08 | 2019-02-15 | 国网经济技术研究院有限公司 | Thermal power station target identification method in remote sensing image |
| CN109815864A (en)* | 2019-01-11 | 2019-05-28 | 浙江工业大学 | A transfer learning-based age recognition method for face images |
| CN109766998A (en)* | 2019-01-14 | 2019-05-17 | 温岭市非普电气有限公司 | A Feature Pyramid Network Structure Based on MobileNet Acceleration |
| CN109886121B (en)* | 2019-01-23 | 2021-04-06 | 浙江大学 | Human face key point positioning method for shielding robustness |
| CN109886121A (en)* | 2019-01-23 | 2019-06-14 | 浙江大学 | An Occlusion Robust Face Keypoint Localization Method |
| CN110322509A (en)* | 2019-06-26 | 2019-10-11 | 重庆邮电大学 | Object localization method, system and computer equipment based on level Class Activation figure |
| CN111881721A (en)* | 2020-06-10 | 2020-11-03 | 广东芯盾微电子科技有限公司 | Cross-age face recognition method, system, device and storage medium |
| CN111881721B (en)* | 2020-06-10 | 2021-03-30 | 广东芯盾微电子科技有限公司 | Cross-age face recognition method, system, device and storage medium |
| CN111710424A (en)* | 2020-06-19 | 2020-09-25 | 浙江新芮信息科技有限公司 | Catering personnel health monitoring method, device and computer-readable storage medium |
| WO2022052785A1 (en)* | 2020-09-08 | 2022-03-17 | 京东方科技集团股份有限公司 | Target detection method and apparatus, and storage medium and electronic device |
| CN114155567A (en)* | 2020-09-08 | 2022-03-08 | 京东方科技集团股份有限公司 | Target detection method and device, storage medium and electronic device |
| CN112287965A (en)* | 2020-09-21 | 2021-01-29 | 卓尔智联(武汉)研究院有限公司 | Image quality detection model training method and device and computer equipment |
| CN112766157A (en)* | 2021-01-20 | 2021-05-07 | 乐山师范学院 | Cross-age face image recognition method based on disentanglement representation learning |
| Publication number | Publication date |
|---|---|
| CN104866829B (en) | 2019-02-19 |
| Publication | Publication Date | Title |
|---|---|---|
| CN104866829A (en) | Cross-age face verify method based on characteristic learning | |
| US11195051B2 (en) | Method for person re-identification based on deep model with multi-loss fusion training strategy | |
| CN106096538B (en) | Face identification method and device based on sequencing neural network model | |
| CN111325115B (en) | Adversarial cross-modal person re-identification method and system with triple constraint loss | |
| Zhang et al. | Improving multiview face detection with multi-task deep convolutional neural networks | |
| CN105956560B (en) | A kind of model recognizing method based on the multiple dimensioned depth convolution feature of pondization | |
| CN109063666A (en) | The lightweight face identification method and system of convolution are separated based on depth | |
| CN103279768B (en) | A kind of video face identification method based on incremental learning face piecemeal visual characteristic | |
| CN102938065B (en) | Face feature extraction method and face identification method based on large-scale image data | |
| CN109255289B (en) | Cross-aging face recognition method based on unified generation model | |
| CN106650806A (en) | Cooperative type deep network model method for pedestrian detection | |
| CN108427921A (en) | A kind of face identification method based on convolutional neural networks | |
| CN103020602B (en) | Based on the face identification method of neural network | |
| CN106778796A (en) | Human motion recognition method and system based on hybrid cooperative model training | |
| CN109344856B (en) | Offline signature identification method based on multilayer discriminant feature learning | |
| CN108921019A (en) | A kind of gait recognition method based on GEI and TripletLoss-DenseNet | |
| CN107220598B (en) | Iris image classification method based on deep learning features and Fisher Vector coding model | |
| CN108520213A (en) | A face beauty prediction method based on multi-scale depth | |
| CN107330412A (en) | A kind of face age estimation method based on depth rarefaction representation | |
| CN106407958A (en) | Double-layer-cascade-based facial feature detection method | |
| CN106096517A (en) | A kind of face identification method based on low-rank matrix Yu eigenface | |
| CN107818299A (en) | Face recognition algorithms based on fusion HOG features and depth belief network | |
| CN110738071A (en) | face algorithm model training method based on deep learning and transfer learning | |
| CN114155572B (en) | Facial expression recognition method and system | |
| CN110880010A (en) | Visual SLAM closed loop detection algorithm based on convolutional neural network |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |