技术领域technical field
本发明涉及人脸面部图像处理技术领域，具体而言涉及一种基于特征约束的有序卷积神经网络人脸年龄估计方法。The present invention relates to the technical field of human face image processing, in particular to a method for estimating the age of a human face in an ordered convolutional neural network based on feature constraints.
背景技术Background technique
随着科学技术的发展，图像处理相关的应用越来越贴近人们的生活，而人脸属性作为每个人类个体独特的生物特征，由于包含了丰富的个人信息，使得在安防、智慧商业、人机交互等领域中广泛应用。人脸属性特征不仅可以用来进行身份识别，还可以得到性别、种族、情绪和年龄等信息。借助年龄信息，可以对现有的社会研究和商业应用带来巨大的便利和影响，例如人口统计分析，商业用户管理，视觉监控，甚至是老龄化进程。因此年龄估计研究具有重要的科学意义和应用价值。With the development of science and technology, image processing-related applications are getting closer to people's lives, and face attributes, as the unique biological characteristics of each human individual, contain a wealth of personal information, making it a It is widely used in computer interaction and other fields. Facial attribute features can not only be used for identification, but also information such as gender, race, emotion and age can be obtained. With the help of age information, it can bring great convenience and impact to existing social research and business applications, such as demographic analysis, business user management, visual monitoring, and even the aging process. Therefore, age estimation research has important scientific significance and application value.
然而，由于不同个体之间存在着生活环境、健康状况和种族差异等信息上的不同，相同个体因为妆容、饰品和表情的差异也会造成估计面部特征表现的年龄信息的不准确。而且大多数人脸年龄数据库都是在限制性的环境下拍摄的，而现实场景中拍摄的照片，可能会因人脸姿态变化、拍摄光照、部分面部遮挡等影响，造成模型的泛化性能很弱。同时，在生物学上，从儿童到老年时期面部轮廓变化和皮肤纹理的变化形成的面部衰老效应，这种性质使由老化模式形成的随机过程通常是非平稳的。而且，回归问题的非平稳内核学习通常很困难，因为它很容易导致训练过程的过拟合。这种情况便会造成我们在一个封闭有限的数据集上进行训练，但在开放的场景中模型的泛化性能很差。However, due to differences in information such as living environment, health status, and ethnic differences among different individuals, differences in makeup, accessories, and expressions of the same individual will also cause inaccurate age information for estimating facial features. Moreover, most face age databases are taken in restrictive environments, and photos taken in real scenes may be affected by changes in face poses, shooting lighting, and partial facial occlusions, resulting in poor generalization performance of the model. weak. At the same time, biologically, facial aging effects from childhood to old age are formed by changes in facial contours and changes in skin texture. This property makes the stochastic process formed by aging patterns usually non-stationary. Moreover, non-stationary kernel learning for regression problems is often difficult because it can easily lead to overfitting in the training process. This situation will cause us to train on a closed and limited data set, but the generalization performance of the model in the open scene is poor.
由于面部老化过程是一个非平稳过程，因此我们可以采用的一个可靠信息就是年龄标签之间的相对顺序以及它们的确切值。很多时候我们人类自己仅从面部特征上去预测另一个人的年龄时，也无法准确得出真实年龄，而更多的是给出一个年龄范围。而且，不同的人对同一张人脸可能会有不同的猜测。Since the face aging process is a non-stationary process, one piece of reliable information we can use is the relative order between the age labels and their exact values. Many times when we human beings only predict the age of another person from facial features, we cannot accurately obtain the real age, but give an age range more. Moreover, different people may have different guesses about the same face.
发明内容Contents of the invention
本发明目的在于提供一种基于特征约束的有序卷积神经网络人脸年龄估计方法，能够使有序二元多分类年龄模型在区分类间关系的过程中，维护每个年龄段的聚类中心，不仅能扩大类别之间的距离，还使同一类别特征更加聚集，这样便可以尽量减小类别在被错分的时候落入非邻近类别的情况，可以有效提升人脸年龄分类的准确率，实现高精度的年龄估计。The purpose of the present invention is to provide a method for estimating face age based on an ordered convolutional neural network based on feature constraints, which can enable the ordered binary multi-classification age model to maintain the clustering of each age group in the process of distinguishing the relationship between classes The center can not only expand the distance between categories, but also make the features of the same category more clustered, so that it can minimize the situation that categories fall into non-adjacent categories when they are misclassified, and can effectively improve the accuracy of face age classification , to achieve high-precision age estimation.
为达成上述目的，结合图1，本发明提出一种基于特征约束的有序卷积神经网络人脸年龄估计方法，所述方法包括：In order to achieve the above-mentioned purpose, in conjunction with Fig. 1, the present invention proposes a face age estimation method based on an ordered convolutional neural network based on feature constraints, the method comprising:
S1：对人脸图像进行特征表示，提取人脸表观年龄特征向量，利用中心损失函数对每一年龄类别的人脸表观年龄特征向量进行中心约束；S1: Perform feature representation on the face image, extract the face apparent age feature vector, and use the center loss function to perform center constraints on the face apparent age feature vector of each age category;
S2：将受约束的人脸表观年龄特征向量与一系列有序二元分类模块进行连接，创建有序二元多分类年龄模型，所述有序二元多分类年龄模型用于将输入的人脸特征向量与各个年龄类别进行年龄大小比对，输出比对结果，创建过程包括以下步骤：S2: Connect the constrained face apparent age feature vector with a series of ordered binary classification modules to create an ordered binary multi-classification age model, which is used to combine the input The face feature vector is compared with each age category, and the comparison result is output. The creation process includes the following steps:
设网络输入xi在特定的空间xi∈X中表示,并且每一个输入图像xi都对应着存在结果空间yi∈Y＝{r1,r1,...,rk}，其中有序类别rk＞rk-1＞...＞r1；Let the network input xi be represented in a specific space xi ∈ X, and each input image xi corresponds to the result space yi ∈ Y = {r1 ,r1 ,...,rk }, where Ordered classes rk >rk-1 >... >r1 ;
对于每个年龄类别r∈{r1,r2,...,rk}，采用训练样本中给定的年龄标签yi与每一个rk进行年龄大小上的比较，从而转化为一组k个二元标签yki∈{0,1}，用于表示第i个样本yi的秩是否大于rk，基于该样本训练k个二元分类器预测每个样本xi的年龄类别yi；For each age category r∈{r1 ,r2 ,...,rk }, using training samples The age label yi given in is compared with each rk in terms of age, thus transforming into a set of k binary labels yki ∈ {0,1}, used to represent the rank of the i-th sample yi Whether it is greater than rk , train k binary classifiers based on this sample to predict the age category yi of each samplexi ;
S3：将待预测的人脸图像导入有序二元多分类年龄模型，经过有序二元多分类年龄模型的前向计算输出一组K对二元置信度数值，结合公式计算模型预测的人脸年龄，其中fk(x′)∈{0,1}是样本x′在网络模型中的第k个二元分类器的分类结果。S3: Import the face image to be predicted into the ordered binary multi-classification age model, and output a set of K pairs of binary confidence values through the forward calculation of the ordered binary multi-classification age model, combined with the formula Calculate the face age predicted by the model, where fk (x′)∈{0,1} is the classification result of the kth binary classifier for sample x′ in the network model.
进一步的实施例中，步骤S1中，所述利用中心损失函数对每一年龄类别的人脸表观年龄特征向量进行中心约束包括以下步骤：In a further embodiment, in step S1, the use of the center loss function to carry out center constraints on the face apparent age feature vectors of each age category includes the following steps:
S11：对每一最小批次训练样本按照每个年龄类别进行特征质心计算；S11: Perform feature centroid calculation for each minimum batch of training samples according to each age category;
S12：采用下述特征约束损失函数对人脸表观年龄特征向量进行约束处理，在保持不同类别的特征可分离的同时，最小化同类别样本特征与其中心距离：S12: Use the following feature constraint loss function to constrain the apparent age feature vector of the face, while keeping the features of different categories separable, minimize the distance between the same category sample features and their centers:
其中，Cyi∈Rd表示深部特征的类中心。Among them, Cyi ∈ Rd represents the deep feature class center.
进一步的实施例中，通过对每个类别特征取平均以获取每个年龄类别的特征质心。In a further embodiment, the feature centroid of each age category is obtained by averaging the features of each category.
进一步的实施例中，采用下述更新方程以更新xi的Lcenter的梯度和Cyi：In a further embodiment, the following update equation is used to update the gradient of Lcenter ofxi and Cyi :
其中，标量用于控制类别中心更新的学习率，如果内部条件满足，则δ(·)＝1，如果内部条件不满足，则δ(·)＝0。where the scalar Learning rate for controlling category center updates, If the internal condition is satisfied, δ(·)=1, and if the internal condition is not satisfied, δ(·)=0.
进一步的实施例中，所述人脸图像采用RGB三通道人脸图像。In a further embodiment, the human face image adopts an RGB three-channel human face image.
进一步的实施例中，所述方法还包括：In a further embodiment, the method also includes:
所述有序二元多分类年龄模型的有序神经网络输出层的损失函数为：The loss function of the ordered neural network output layer of the ordered binary multi-classification age model is:
总的损失函数为：The overall loss function is:
其中，o表示第i个图像的第t个任务的输出，表示第t个任务的第i个图像的权重，wt表示第t个任务的参数；如果内部条件为真，则1{·}为1，否则为0，σ1,σ2是有序二元多分类年龄模型中根据有序二元多分类年龄模型训练进行调整的自适应参数。where o represents the output of the t-th task for the i-th image, Represents the weight of the i-th image of the t-th task, wt represents the parameters of the t-th task; if the internal condition is true, then 1{ } is 1, otherwise it is 0, σ1 , σ2 are ordered two Adaptive parameters in the meta-multiclass age model that are adjusted based on the training of the ordered binary multi-class age model.
进一步的实施例中，所述σ1,σ2的调整过程包括以下步骤：In a further embodiment, the adjustment process of σ1 and σ2 includes the following steps:
在有序二元多分类年龄模型分支前的共享层分离出一组向量，经过连接层和激活函数后输出两个与有序二元多分类年龄模型特征紧密联系的权重变量σ1,σ2。A set of vectors is separated in the shared layer before the branch of the ordered binary multi-classification age model, and two weight variables σ1 , σ2 that are closely related to the characteristics of the ordered binary multi-classification age model are output after the connection layer and the activation function .
进一步的实施例中，所述方法还包括：In a further embodiment, the method also includes:
对输入的人脸图像进行预处理。Preprocess the input face image.
进一步的实施例中，所述预处理包括以下步骤：In a further embodiment, the preprocessing includes the following steps:
采用Mtcnn模型去除掉不包含完整人脸图像的图片；Use the Mtcnn model to remove pictures that do not contain complete face images;
从余下的人脸图像中裁剪出人脸框，进行人脸关键点对齐；Cut out the face frame from the remaining face images, and align the key points of the face;
在训练模型前对图像数据进行了随机裁剪、随机旋转，将图片调整为统一尺寸，根据下述公式对图片像素值进行归一化处理：Before training the model, the image data is randomly cropped and rotated, the image is adjusted to a uniform size, and the pixel values of the image are normalized according to the following formula:
Xpix＝(Xpix-128)/128Xpix = (Xpix -128)/128
其中，Xpix是网络输入的人脸图片像素值。Among them, Xpix is the pixel value of the face image input by the network.
相对于从人脸图片去猜测真实年龄，我们更擅长从两张人脸图片中区分出哪一张年龄更大哪一张年龄更小，将年龄预测的有序回归问题转化成一系列有序二元分类的比较问题。本发明正是基于前述原理，对人脸年龄进行估计。本发明在有序多分类的神经网络中对中间层特征向量采用具有中心约束的损失函数进行特征约束，使得有序二元多分类年龄模型在区分类间关系的过程中，维护每个年龄段的聚类中心，这样不仅能扩大类别之间的距离，还能使同一类别特征更加聚集，这样便可以尽量减小类别在被错分的时候落入非邻近类别的情况。我们采用这种新的网络结构可以有效提升人脸年龄分类的准确率Compared with guessing the real age from face pictures, we are better at distinguishing which one is older and which one is younger from two face pictures, transforming the ordered regression problem of age prediction into a series of ordered two A comparison problem for metaclassification. The present invention estimates the face age based on the foregoing principles. In the neural network of ordered multi-classification, the present invention uses a loss function with central constraints to perform feature constraints on the middle layer feature vector, so that the ordered binary multi-classification age model maintains each age group in the process of distinguishing the relationship between classes The clustering center, which can not only expand the distance between categories, but also make the features of the same category more clustered, so that it can minimize the situation that categories fall into non-adjacent categories when they are misclassified. Our new network structure can effectively improve the accuracy of face age classification
以上本发明的技术方案，与现有相比，其显著的有益效果在于：Above technical scheme of the present invention, compared with existing, its remarkable beneficial effect is:
(1)通过多粒度级联森林网络模型的多粒度扫描层进行特征提取保证了网络具有良好的泛化性能，充分提取了云图中的云特征，使得网络模型取得的云图分类结果更加精确。利用多粒度扫描来增强数据的特征表示，并利用级联网络中不同类型的森林来确保特征的多样性。本发明不需要反向误差迭代训练，在准确率提高的前提下，在同等硬件条件下样本的训练测试速度和样本分类速度都取得了很大的提高。(1) The feature extraction through the multi-granularity scanning layer of the multi-granularity cascaded forest network model ensures that the network has good generalization performance, fully extracts the cloud features in the cloud image, and makes the cloud image classification results obtained by the network model more accurate. Multi-grained scanning is utilized to enhance the feature representation of the data, and different types of forests in a cascaded network are utilized to ensure feature diversity. The present invention does not require reverse error iterative training, and under the premise of improving the accuracy rate, the training and testing speed of samples and the speed of sample classification are greatly improved under the same hardware conditions.
(2)在有序多分类的神经网络中对中间层特征向量进行特征约束，使得有序二元多分类年龄模型在区分类间关系的过程中，维护每个年龄段的聚类中心，这样不仅能扩大类别之间的距离，还能使同一类别特征更加聚集，这样便可以尽量减小类别在被错分的时候落入非邻近类别的情况。(2) In the ordered multi-classification neural network, the feature vector of the middle layer is constrained, so that the ordered binary multi-classification age model maintains the cluster center of each age group in the process of distinguishing the relationship between classes, so that It can not only expand the distance between categories, but also make the features of the same category more clustered, so that it can minimize the situation that categories fall into non-adjacent categories when they are misclassified.
(3)为了多个不同任务的损失函数处在同一尺度上进行训练，本发明在网络结构中引入了可以根据网络特征进行动态调整的超参数，使得不同损失函数之间处于平衡状态。采用这种方法可以有效提升人脸年龄分类的准确率，而且相比之前方法在相同的场景下具有更好的泛化效果。(3) In order to train the loss functions of multiple different tasks on the same scale, the present invention introduces hyperparameters that can be dynamically adjusted according to network characteristics in the network structure, so that the different loss functions are in a balanced state. Using this method can effectively improve the accuracy of face age classification, and has a better generalization effect in the same scene than the previous method.
应当理解，前述构思以及在下面更加详细地描述的额外构思的所有组合只要在这样的构思不相互矛盾的情况下都可以被视为本公开的发明主题的一部分。另外，所要求保护的主题的所有组合都被视为本公开的发明主题的一部分。It should be understood that all combinations of the foregoing concepts, as well as additional concepts described in more detail below, may be considered part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent. Additionally, all combinations of claimed subject matter are considered a part of the inventive subject matter of this disclosure.
结合附图从下面的描述中可以更加全面地理解本发明教导的前述和其他方面、实施例和特征。本发明的其他附加方面例如示例性实施方式的特征和/或有益效果将在下面的描述中显见，或通过根据本发明教导的具体实施方式的实践中得知。The foregoing and other aspects, embodiments and features of the present teachings can be more fully understood from the following description when taken in conjunction with the accompanying drawings. Other additional aspects of the invention, such as the features and/or advantages of the exemplary embodiments, will be apparent from the description below, or learned by practice of specific embodiments in accordance with the teachings of the invention.
附图说明Description of drawings
附图不意在按比例绘制。在附图中，在各个图中示出的每个相同或近似相同的组成部分可以用相同的标号表示。为了清晰起见，在每个图中，并非每个组成部分均被标记。现在，将通过例子并参考附图来描述本发明的各个方面的实施例，其中：The figures are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like reference numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of the various aspects of the invention will now be described by way of example with reference to the accompanying drawings, in which:
图1是本发明的基于特征约束的有序卷积神经网络人脸年龄估计方法的流程图。Fig. 1 is a flow chart of the face age estimation method based on the ordered convolutional neural network of the present invention.
图2是本发明的有序二元多分类年龄模型的结构示意图。Fig. 2 is a structural schematic diagram of the ordered binary multi-classification age model of the present invention.
具体实施方式Detailed ways
为了更了解本发明的技术内容，特举具体实施例并配合所附图式说明如下。In order to better understand the technical content of the present invention, specific embodiments are given together with the attached drawings for description as follows.
结合图1、图2，本实例的基于特征约束的有序卷积神经网络人脸年龄估计方法，包括以下步骤：Combined with Figure 1 and Figure 2, the face age estimation method based on the ordered convolutional neural network based on feature constraints in this example includes the following steps:
步骤一、数据预处理Step 1. Data preprocessing
由于现有的人脸图片数据集中人脸部分在整张图片上所占的比例很小，而且面部之外的图像信息包含了大量不同场景，如衣服穿着，肢体动作和背景环境等，这些干扰信息都会影响模型学习面部年龄，并且考虑到人脸年龄的模型训练对数据的敏感性，为了避免人脸之外的非可控环境对模型训练的干扰，本发明提出对数据集进行清洗和数据增强处理，具体的：Due to the fact that the proportion of the face in the entire picture in the existing face picture data set is very small, and the image information other than the face contains a large number of different scenes, such as clothing, body movements and background environment, etc., these interferences Information will affect the model to learn the age of the face, and considering the sensitivity of the model training of the age of the face to the data, in order to avoid the interference of the uncontrollable environment other than the face on the model training, the present invention proposes to clean the data set and Enhanced processing, specifically:
采用Mtcnn模型去除掉那些没有人脸的图片，同时裁剪出人脸框，进行人脸关键点对齐。然后在训练模型前对图像数据进行了随机裁剪、随机旋转(比如旋转角度±20°)，并将图片调整为统一像素尺寸，如144x144。而且为了使模型快速收敛，统一对数据像素值进行了归一化，公式如下：Use the Mtcnn model to remove those pictures without faces, and cut out the face frame at the same time to align the key points of the face. Then, before training the model, the image data is randomly cropped and rotated (for example, the rotation angle is ±20°), and the image is adjusted to a uniform pixel size, such as 144x144. And in order to make the model converge quickly, the data pixel values are uniformly normalized, the formula is as follows:
Xpix＝(Xpix-128)/128Xpix = (Xpix -128)/128
其中，Xpix是网络输入的人脸图片像素值。Among them, Xpix is the pixel value of the face image input by the network.
步骤二、卷积神经网络提取的人脸特征进行约束Step 2. The face features extracted by the convolutional neural network are constrained
设定特征提取层网络模型的基本结构由四个卷积模块组成，卷积模块第一层采用卷积核为[3x3]，步长为1，输出特征层为64的卷积层，然后连接BatchNorrmal层进行特征归一化，然后采用relu作为激活函数；第二层与第一层结构相似，采用卷积核为[3x3]，步长为1，输出特征层为96，然后连接BatchNorrmal层进行特征归一化，采用relu作为激活函数，最后连接池化窗口为[2x2]步长为2的最大池化层。后面连接的卷积模块结构相同，只是第一个卷积模块输出特征层为64，第二个卷积模块输出特征层为96，第三个卷积模块输出特征层为128，第四个卷积模块输出特征层为256。最后将特征展开拉成一维连接全连接层，节点设为128，然后经过dropout层和relu激活层后输出人脸面部特征向量，然后采用中心损失函数对每一类别人脸表观年龄特征向量进行中心约束，使得有序二元多分类年龄模型在区分类间关系的过程中，维护每个年龄段的聚类中心，这样不仅能扩大类别之间的距离，还能使同一类别特征更加聚集，这样便可以尽量减小类别在被错分的时候落入非邻近类别的情况。Set the basic structure of the feature extraction layer network model to consist of four convolution modules. The first layer of the convolution module uses a convolution kernel of [3x3], a step size of 1, and an output feature layer of 64 convolution layers, and then connects The BatchNorrmal layer performs feature normalization, and then uses relu as the activation function; the second layer is similar to the first layer in structure, using a convolution kernel of [3x3], a step size of 1, and an output feature layer of 96, and then connects the BatchNorrmal layer for Feature normalization, using relu as the activation function, and finally connect the pooling window to a [2x2] maximum pooling layer with a step size of 2. The convolution modules connected later have the same structure, except that the output feature layer of the first convolution module is 64, the output feature layer of the second convolution module is 96, the output feature layer of the third convolution module is 128, and the fourth convolution module The output feature layer of the product module is 256. Finally, the feature is expanded into a one-dimensional fully connected layer with nodes set to 128, and then the face feature vector is output after the dropout layer and the relu activation layer, and then the center loss function is used to perform the apparent age feature vector of each category. The center constraint makes the ordered binary multi-category age model maintain the cluster center of each age group in the process of distinguishing the relationship between categories, which can not only expand the distance between categories, but also make the features of the same category more clustered, This minimizes cases where classes fall into non-adjacent classes when they are misclassified.
在保持不同类别的特征可分离的同时，最小化同类别样本特征与其中心距离，例如采用下述特征约束损失函数：While keeping the features of different categories separable, minimize the distance between the sample features of the same category and their centers, for example, using the following feature constraint loss function:
其中，Cyi∈Rd表示深部特征的yith类中心，本发明对每一最小批次训练样本按照每个类别进行特征质心计算(例如通过对每个类别特征取平均以获取特征质心)，同时，采用标量来控制类别中心更新的学习率，以避免存在误标样本对样本中心造成大的更新扰动。Wherein, Cyi ∈ Rd represents the yith class center of the deep feature, and the present invention calculates the feature centroid for each minimum batch of training samples according to each category (for example, by averaging each category feature to obtain the feature centroid), At the same time, using the scalar To control the learning rate of the update of the category center to avoid large update disturbances caused by mislabeled samples to the sample center.
关于xi的Lcenter的梯度和Cyi的更新方程如下计算：The gradient of Lcenter with respect toxi and the update equation of Cyi are calculated as follows:
其中，如果内部条件yi＝j满足，则δ(·)＝1，如果内部条件yi＝j不满足，则δ(·)＝0，Wherein, if the internal condition yi =j is satisfied, then δ(·)=1, if the internal condition yi =j is not satisfied, then δ(·)=0,
在确保特征约束损失函数进行样本特征类内约束的同时，将受约束的特征向量作为输入与有序二元多分类网络进行联合监督训练。While ensuring that the feature-constrained loss function performs intra-class constraints on sample features, the constrained feature vectors are used as input for joint supervised training with an ordered binary multi-classification network.
步骤三、特征约束的有序卷积神经网络模型训练Step 3. Training of ordered convolutional neural network model with feature constraints
对进行中心约束后的特征向量与一系列有序二元分类模块进行连接，假设网络输入xi在特定的空间xi∈X中表示,并且每一个输入图像xi都对应着存在结果空间yi∈Y＝{r1,r1,...,rk}，其中有序类别rk＞rk-1＞...＞r1。给定训练样本则序列回归模型将寻找一组图像到有序类别的映射关系h(·):X→Y，从而使得成本风险函数R(X×Y)最小化。对于每个年龄类别rk∈{r1,r2,...,rk}，采用训练样本中给定的年龄标签yi与每一个rk进行年龄大小上的比较，从而转化为一组k个二元标签(0-1形式)yki∈{0,1}，表示第i个样本yi的秩是否大于rk，然后基于该样本上训练k个二元分类器预测每个样本xi的年龄类别yi。Connect the feature vector after the central constraint with a series of ordered binary classification modules, assuming that the network input xi is represented in a specific space xi ∈ X, and each input image xi corresponds to the existence of the result space yi ∈ Y={r1 ,r1 ,...,rk }, where the ordered category rk >rk-1 >...>r1 . given training samples Then the sequential regression model will look for the mapping relationship h( ):X→Y from a set of images to ordered categories, so as to minimize the cost-risk function R(X×Y). For each age category rk ∈{r1 ,r2 ,...,rk }, use the age label yi given in the training sample to compare the age with each rk , thus transforming into a Set k binary labels (0-1 form) yki ∈ {0,1}, indicating whether the rank of the i-th sample yi is greater than rk , and then train k binary classifiers based on this sample to predict each Age category yi of sample x i.
对于每个年龄类别rk∈{r1,r2,...,rk}，采用训练样本中给定的年龄标签yi与每一个rk进行年龄大小上的比较，从而转化为一组k个二元标签(0-1形式)yki∈{0,1}，表示第i个样本yi的秩是否大于rk，然后基于该样本上训练k个二元分类器预测每个样本xi的年龄类别yi，例如：For each age category rk ∈{r1 ,r2 ,...,rk }, use the age label yi given in the training sample to compare the age with each rk , thus transforming into a Set k binary labels (0-1 form) yki ∈ {0,1}, indicating whether the rank of the i-th sample yi is greater than rk , and then train k binary classifiers based on this sample to predict each The age category yi of sample xi , for example:
这种方法可以端到端的解决序数回归问题，以便能够自动从人脸图像中学习更好的特征，避免直接设计手工特征所带来的先验认知所带来的偏差。在最后的二元多分类层之前，每个训练样本xi都共享相同的网络结构，这样可以保持二元有序类别之间的内在联系。This method can solve the ordinal regression problem end-to-end, so that better features can be automatically learned from face images, avoiding the bias caused by prior cognition caused by directly designing manual features. Before the final binary multi-classification layer, each training samplexi shares the same network structure, which preserves the intrinsic connection between binary ordered categories.
步骤四、人脸表观年龄模型预测Step 4. Face Apparent Age Model Prediction
待预测的人脸图像传入训练后模型，经过模型的前向计算最后输出一组K对二元置信度数值，采用计算模型预测的人脸年龄，其中fk(x′)∈{0,1}是样本x′在网络模型中的第k个二元分类器的分类结果。The face image to be predicted is passed into the trained model, and a set of K pairs of binary confidence values are finally output after the forward calculation of the model, using Calculate the face age predicted by the model, where fk (x′)∈{0,1} is the classification result of the kth binary classifier for sample x′ in the network model.
考虑的两组损失函数的训练任务不同，可能会导致两组损失函数梯度更新时不在同一度量超平面上，这样会造成其中一个损失函数站主导地位，而另一个损失函数的迭代更新往往被淹没掉。为了避免出现这种多任务损失函数之类不平衡的情况，我们在网络中添加了两个可学习的权重参数变量σ1,σ2。不同于人工设置的独立固定的参数，手动调整两组损失函数。我们的方法是在网络分支前的共享层分离出一组向量，在经过一些网络连接层和激活函数后输出两个与网络特征紧密联系的权重变量σ1,σ2。前述权重参数变量的生成方式的优点包括：消除人工设置带来的先验认知上带来的局限和片面性，而且使得权重变量与网络特征紧密相关，可以能够通过网络特征学习更好地寻找到两组损失函数之间的内在联系，使得两种损失函数处在同一尺度上进行训练。The training tasks of the two sets of loss functions considered are different, which may cause the gradient update of the two sets of loss functions to not be on the same metric hyperplane, which will cause one of the loss functions to dominate, while the iterative update of the other loss function is often submerged Lose. In order to avoid such unbalanced multi-task loss functions, we add two learnable weight parameter variables σ1 , σ2 to the network. Instead of manually setting independent fixed parameters, two sets of loss functions are manually tuned. Our method is to separate a group of vectors in the shared layer before the network branch, and output two weight variables σ1 , σ2 closely related to the network features after passing through some network connection layers and activation functions. The advantages of the generation method of the aforementioned weight parameter variables include: eliminating the limitations and one-sidedness brought about by the prior cognition brought about by manual settings, and making the weight variables closely related to the network features, which can be better found through network feature learning. The intrinsic connection between the two sets of loss functions allows the two loss functions to be trained on the same scale.
下面对特征约束的有序卷积神经网络损失函数的设计过程进行说明。The following describes the design process of the feature-constrained ordered convolutional neural network loss function.
优选的，本发明所提及的人脸图像为RGB三通道人脸图像而不是采用灰度图，因为面部色彩也是判断人脸的年龄属性的因素之一。在网络结构上采用resnet网络前半部分与有序卷积神经网络联接，resnet网络输出在经过全连接层得到一层特征向量，采用特征向量约束函数对特征层进行约束使得同类特征逐渐向其质心靠拢。之后，网络分出K个输出层，其中每个输出层包含2个神经元进行对应的二元分类任务，第k个任务是预测第i个面部图像的年龄是否大于等级rk。Preferably, the human face image mentioned in the present invention is an RGB three-channel human face image instead of a grayscale image, because facial color is also one of the factors for judging the age attribute of a human face. In terms of network structure, the first half of the resnet network is connected with the ordered convolutional neural network. The output of the resnet network is obtained through a fully connected layer to obtain a layer of feature vectors. The feature vector constraint function is used to constrain the feature layer so that similar features gradually move closer to their centroids. . Afterwards, the network divides into K output layers, each of which contains 2 neurons for the corresponding binary classification task, and the kth task is to predict whether the age of the i-th facial image is greater than the grade rk .
对于具有K个输出的神经网络，每个输出对应不同的任务。所有T＝K个输出共享相同的网络参数，但是具有不同的类别标签。同时，依据训练样本的数据分布，设置λt表示为第t个有序类别分类比较时的重要性系数，则有序神经网络输出层的损失函数可写为：For a neural network with K outputs, each output corresponds to a different task. All T=K outputs share the same network parameters, but have different class labels. At the same time, according to the data distribution of the training samples, λt is set to represent the importance coefficient of thet -th ordered category classification comparison, then the loss function of the output layer of the ordered neural network can be written as:
这样有序卷积神经网络中总的损失函数公式可以表示为：In this way, the total loss function formula in the ordered convolutional neural network can be expressed as:
其中，o表示第i个图像的第t个任务的输出，表示第t个任务的第i个图像的权重，wt表示第t个任务的参数。如果内部条件为真，则1{·}为1，否则为0，σ1,σ2是网络模型中根据网络训练进行调整的自适应参数。where o represents the output of the t-th task for the i-th image, Indicates the weight of the i-th image of the t-th task, and wt indicates the parameters of the t-th task. If the internal condition is true, 1{ } is 1, otherwise it is 0, σ1 , σ2 are adaptive parameters in the network model that are adjusted according to network training.
本发明通过有序多分类的神经网络中对中间层特征向量进行特征约束，使得有序二元多分类年龄模型在区分类间关系的过程中，维护每个年龄段的聚类中心，这样不仅能扩大类别之间的距离，还能使同一类别特征更加聚集，这样便可以尽量减小类别在被错分的时候落入非邻近类别的情况。同时，为了多个不同任务的损失函数处在同一尺度上进行训练，在网络结构中引入了可以根据网络特征进行动态调整的超参数，使得不同损失函数之间处于平衡状态。实验结果表明，本发明采用这种方法可以有效提升人脸年龄分类的准确率，在不同场景数据集上达到了比之前方法更好的效果，更适合后续的人脸属性研究工作和应用。The present invention implements feature constraints on the middle layer feature vector in the neural network of ordered multi-classification, so that the ordered binary multi-classification age model can maintain the clustering center of each age group in the process of distinguishing the relationship between classes, so that not only It can expand the distance between categories and make the features of the same category more clustered, so that it can minimize the situation that categories fall into non-adjacent categories when they are misclassified. At the same time, in order to train the loss functions of multiple different tasks on the same scale, hyperparameters that can be dynamically adjusted according to network characteristics are introduced into the network structure, so that the different loss functions are in a balanced state. Experimental results show that the method adopted by the present invention can effectively improve the accuracy of face age classification, achieve better results than previous methods on different scene data sets, and is more suitable for subsequent research work and application of face attributes.
在本公开中参照附图来描述本发明的各方面，附图中示出了许多说明的实施例。本公开的实施例不必定义在包括本发明的所有方面。应当理解，上面介绍的多种构思和实施例，以及下面更加详细地描述的那些构思和实施方式可以以很多方式中任意一种来实施，这是因为本发明所公开的构思和实施例并不限于任何实施方式。另外，本发明公开的一些方面可以单独采用，或者与本发明公开的其他方面的任何适当组合来采用。Aspects of the invention are described in this disclosure with reference to the accompanying drawings, which show a number of illustrated embodiments. Embodiments of the present disclosure are not necessarily defined to include all aspects of the present invention. It should be appreciated that the various concepts and embodiments described above, as well as those described in more detail below, can be implemented in any of numerous ways, since the concepts and embodiments disclosed herein are not limited to any implementation. Additionally, some aspects of the present disclosure may be employed alone or in any suitable combination with other aspects of the present disclosure.
虽然本发明已以较佳实施例揭露如上，然其并非用以限定本发明。本发明所属技术领域中具有通常知识者，在不脱离本发明的精神和范围内，当可作各种的更动与润饰。因此，本发明的保护范围当视权利要求书所界定者为准。Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910645607.8ACN110378280A (en) | 2019-07-17 | 2019-07-17 | Orderly convolutional neural networks face age estimation method based on feature constraint |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910645607.8ACN110378280A (en) | 2019-07-17 | 2019-07-17 | Orderly convolutional neural networks face age estimation method based on feature constraint |
| Publication Number | Publication Date |
|---|---|
| CN110378280Atrue CN110378280A (en) | 2019-10-25 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910645607.8APendingCN110378280A (en) | 2019-07-17 | 2019-07-17 | Orderly convolutional neural networks face age estimation method based on feature constraint |
| Country | Link |
|---|---|
| CN (1) | CN110378280A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274882A (en)* | 2020-01-11 | 2020-06-12 | 上海悠络客电子科技股份有限公司 | Automatic estimation method for human face age based on weak supervision |
| CN112163526A (en)* | 2020-09-29 | 2021-01-01 | 重庆紫光华山智安科技有限公司 | Method and device for identifying age based on face information and electronic equipment |
| CN114219977A (en)* | 2021-11-10 | 2022-03-22 | 中国工商银行股份有限公司 | Age estimation method, system, electronic device and storage medium |
| CN114495208A (en)* | 2022-01-05 | 2022-05-13 | 电子科技大学长三角研究院(衢州) | An Age Estimation Method Based on Restricted Loss and Deep Regression Forest |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975916A (en)* | 2016-04-28 | 2016-09-28 | 西安电子科技大学 | Age estimation method based on multi-output convolution neural network and ordered regression |
| CN109101938A (en)* | 2018-08-23 | 2018-12-28 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of multi-tag age estimation method based on convolutional neural networks |
| CN109255289A (en)* | 2018-07-27 | 2019-01-22 | 电子科技大学 | A kind of across aging face identification method generating model based on unified formula |
| CN109902757A (en)* | 2019-03-08 | 2019-06-18 | 山东领能电子科技有限公司 | One kind being based on the improved faceform's training method of Center Loss |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105975916A (en)* | 2016-04-28 | 2016-09-28 | 西安电子科技大学 | Age estimation method based on multi-output convolution neural network and ordered regression |
| CN109255289A (en)* | 2018-07-27 | 2019-01-22 | 电子科技大学 | A kind of across aging face identification method generating model based on unified formula |
| CN109101938A (en)* | 2018-08-23 | 2018-12-28 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of multi-tag age estimation method based on convolutional neural networks |
| CN109902757A (en)* | 2019-03-08 | 2019-06-18 | 山东领能电子科技有限公司 | One kind being based on the improved faceform's training method of Center Loss |
| Title |
|---|
| ALEX KENDALL 等: "Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics", 《CVPR 2018》* |
| YANDONG WEN 等: "A Discriminative Feature Learning Approach for Deep Face Recognition", 《SPRINGER INTERNATIONAL PUBLISHING AG 2016》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274882A (en)* | 2020-01-11 | 2020-06-12 | 上海悠络客电子科技股份有限公司 | Automatic estimation method for human face age based on weak supervision |
| CN112163526A (en)* | 2020-09-29 | 2021-01-01 | 重庆紫光华山智安科技有限公司 | Method and device for identifying age based on face information and electronic equipment |
| CN112163526B (en)* | 2020-09-29 | 2022-10-21 | 重庆紫光华山智安科技有限公司 | Method and device for identifying age based on face information and electronic equipment |
| CN114219977A (en)* | 2021-11-10 | 2022-03-22 | 中国工商银行股份有限公司 | Age estimation method, system, electronic device and storage medium |
| CN114495208A (en)* | 2022-01-05 | 2022-05-13 | 电子科技大学长三角研究院(衢州) | An Age Estimation Method Based on Restricted Loss and Deep Regression Forest |
| Publication | Publication Date | Title |
|---|---|---|
| Xia et al. | Multi-stage feature constraints learning for age estimation | |
| KR102433971B1 (en) | Systems and Methods for Facial Expression Recognition and Annotation | |
| Huang et al. | Building deep networks on grassmann manifolds | |
| Tahir et al. | An open-ended continual learning for food recognition using class incremental extreme learning machines | |
| CN104992142B (en) | A kind of pedestrian recognition method being combined based on deep learning and attribute study | |
| US20210027048A1 (en) | Human face image classification method and apparatus, and server | |
| CN110378280A (en) | Orderly convolutional neural networks face age estimation method based on feature constraint | |
| Ansar et al. | Robust hand gesture tracking and recognition for healthcare via Recurrent neural network | |
| CN106295694B (en) | Face recognition method for iterative re-constrained group sparse representation classification | |
| CN106355170A (en) | Photo classifying method and device | |
| CN112464865A (en) | Facial expression recognition method based on pixel and geometric mixed features | |
| CN109815826A (en) | The generation method and device of face character model | |
| Zhu et al. | Discriminative feature adaptation for cross-domain facial expression recognition | |
| Jagadeesh et al. | Facial expression recognition of online learners from real-time videos using a novel deep learning model | |
| Wang et al. | Robust CSI-based human activity recognition with augment few shot learning | |
| CN115082963B (en) | Human attribute recognition model training and human attribute recognition method and related device | |
| CN114170659A (en) | A Facial Emotion Recognition Method Based on Attention Mechanism | |
| US20240249547A1 (en) | Pedestrian attribute recognition method based on a pedestrian attribute recognition system and method for training the same | |
| CN116110089A (en) | A Facial Expression Recognition Method Based on Deep Adaptive Metric Learning | |
| Singla et al. | Age and gender detection using Deep Learning | |
| CN109508640A (en) | Crowd emotion analysis method and device and storage medium | |
| Maheshwari et al. | Performance analysis of mango leaf disease using machine learning technique | |
| Wang et al. | Zero-shot learning based on deep weighted attribute prediction | |
| Arnaud et al. | Tree-gated deep mixture-of-experts for pose-robust face alignment | |
| CN108009512A (en) | A kind of recognition methods again of the personage based on convolutional neural networks feature learning |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20191025 | |
| RJ01 | Rejection of invention patent application after publication |