CN114529685A - Three-dimensional style face generation method, device, equipment and storage medium - Google Patents
Three-dimensional style face generation method, device, equipment and storage mediumDownload PDFInfo
- Publication number
- CN114529685A CN114529685ACN202210156774.8ACN202210156774ACN114529685ACN 114529685 ACN114529685 ACN 114529685ACN 202210156774 ACN202210156774 ACN 202210156774ACN 114529685 ACN114529685 ACN 114529685A
- Authority
- CN
- China
- Prior art keywords
- face
- dimensional
- pca
- style
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Computer Graphics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Architecture (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Hardware Design (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及三维人脸重建技术领域，尤其涉及一种三维风格人脸的生成方法、装置、设备及存储介质。The present invention relates to the technical field of three-dimensional face reconstruction, and in particular, to a method, device, device and storage medium for generating a three-dimensional style face.
背景技术Background technique
目前，直播应用通过构建虚拟互动空间和用户对应的三维虚拟形象，使得主播和观看用户可以使用三维虚拟形象在三维虚拟空间中进行互动。At present, by constructing a virtual interactive space and a three-dimensional virtual image corresponding to the user, the live broadcast application enables the anchor and the viewing user to use the three-dimensional virtual image to interact in the three-dimensional virtual space.
三维虚拟形象可分为三维真实人脸和三维风格人脸，三维真实人脸的目标是贴近用户原本的形象，三维风格人脸的目标是将用户原本的形象卡通化一些。现有技术中，根据单张图片生成三维真实人脸的技术相对成熟，但生成三维风格人脸更多是依靠用户“手捏”。由于三维风格人脸可捏的维度太多，用户不好控制，因此很难做到三维风格人脸与用户相像。The 3D virtual image can be divided into 3D real face and 3D style face. The goal of the 3D real face is to be close to the original image of the user, and the goal of the 3D style face is to cartoonize the user's original image. In the prior art, the technology of generating a three-dimensional real face from a single image is relatively mature, but the generation of a three-dimensional style face is more dependent on the user's "hand pinching". Since there are too many pinchable dimensions for a 3D-style face, it is difficult for the user to control it, so it is difficult to make the 3D-style face resemble the user.
发明内容SUMMARY OF THE INVENTION
本发明提供了一种三维风格人脸的生成方法、装置、设备及存储介质，以解决用户“手捏”的三维风格人脸与用户人脸相似度低的问题。The invention provides a method, device, equipment and storage medium for generating a three-dimensional style face, so as to solve the problem of low similarity between a user's "hand pinched" three-dimensional style face and the user's face.
根据本发明的一方面，提供了一种三维风格人脸的生成方法，包括：According to an aspect of the present invention, a method for generating a three-dimensional style human face is provided, comprising:
获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；Obtain a 2D face image marked with face key points, and construct a 3D real face that exactly matches the real face in the 2D face image according to the face key points;
建立标准三维风格人脸的主成分分析(Principal Component Analysis，PCA)模型与三维真实人脸的PCA模型之间的映射关系；Establish the mapping relationship between the Principal Component Analysis (PCA) model of the standard 3D style face and the PCA model of the 3D real face;
根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸。The standard 3D style face is adjusted according to the mapping relationship, and the 3D style face corresponding to the 2D face image is obtained.
可选的，建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系，包括：Optionally, establish a mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face, including:
建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的PCA基映射关系以及幅度映射关系。Establish the PCA base mapping relationship and the amplitude mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face.
可选的，建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的PCA基映射关系以及幅度映射关系，包括：Optionally, establish a PCA base mapping relationship and an amplitude mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face, including:
在标准三维风格人脸的PCA模型中，获取与目标人脸形状维度对应的第一PCA基以及第一PCA基的幅度值；In the PCA model of the standard three-dimensional style face, obtain the first PCA basis corresponding to the shape dimension of the target face and the amplitude value of the first PCA basis;
在三维真实人脸的PCA模型中，查询与目标人脸形状维度对应的第二PCA基以及第二PCA基的幅度值；In the PCA model of the three-dimensional real face, query the second PCA basis corresponding to the shape dimension of the target face and the amplitude value of the second PCA basis;
建立第一PCA基以及第一PCA基的幅度值与第二PCA基以及第二PCA基的幅度值之间的映射关系。Establish a mapping relationship between the first PCA basis and the amplitude values of the first PCA basis and the second PCA basis and the amplitude values of the second PCA basis.
可选的，确定标准三维风格人脸的第一PCA基的幅度值，包括：Optionally, determine the amplitude value of the first PCA basis of the standard 3D style face, including:
在标准三维风格人脸不畸形的情况下，获取第一PCA基的坐标最小值和坐标最大值；In the case that the standard 3D style face is not deformed, obtain the minimum and maximum coordinates of the first PCA basis;
将坐标最小值和坐标最大值的均值作为第一PCA基的幅度值。The average value of the coordinate minimum value and the coordinate maximum value is taken as the amplitude value of the first PCA basis.
可选的，根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸，包括：Optionally, adjust the standard 3D style face according to the mapping relationship to obtain a 3D style face corresponding to the 2D face image, including:
将标准三维风格人脸中已建立映射关系的每个PCA基的人脸形状系数，更新为三维真实人脸中的对应PCA基的人脸形状系数；Update the face shape coefficient of each PCA base for which the mapping relationship has been established in the standard 3D style face to the face shape coefficient of the corresponding PCA base in the 3D real face;
将标准三维风格人脸中已建立映射关系的每个PCA基的幅度值，更新为三维真实人脸中的对应PCA基的幅度值；Update the amplitude value of each PCA basis for which the mapping relationship has been established in the standard 3D style face to the amplitude value of the corresponding PCA basis in the 3D real face;
将更新后的标准三维风格人脸作为与二维人脸图像对应的三维风格人脸。The updated standard 3D style face is taken as the 3D style face corresponding to the 2D face image.
可选的，在构建标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系之前，还包括：Optionally, before constructing the mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face, the method further includes:
获取标准三维风格人脸以及三维风格人脸样本集；Obtain standard 3D style faces and 3D style face sample sets;
根据公式S(α)＝μs+Us*diag(σs)*α，对三维风格人脸样本集进行主成分分析构建不同维度的PCA基，组成PCA模型；According to the formula S(α)=μs +Us *diag(σs )*α, perform principal component analysis on the three-dimensional style face sample set to construct PCA bases of different dimensions to form a PCA model;
其中，S(α)为PCA基，μs为所有三维风格人脸样本的平均值，σs为每个三维风格人脸样本与平均值的方差，Us为对每个三维风格人脸样本与平均值的差值做主成分分析得到的特征向量，diag(σs)为特征向量Us对应的权重，α为与特征向量Us对应的特征值。Among them, S(α) is the PCA basis, μs is the average value of all 3D style face samples, σs is the variance between each 3D style face sample and the average value, Us is the average value of each 3D style face sample The eigenvector obtained by principal component analysis of the difference from the mean value, diag(σs ) is the weight corresponding to the eigenvector Us , and α is the eigenvalue corresponding to the eigenvector Us .
可选的，获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸，包括：Optionally, obtain a two-dimensional face image marked with face key points, and construct a three-dimensional real face that completely matches the real face in the two-dimensional face image according to the face key points, including:
获取标注有人脸关键点的二维人脸图像，并基于巴塞尔人脸模型(Basel FaceModel，BFM)拓扑构建三维人脸模型；Obtain 2D face images marked with face key points, and construct 3D face models based on Basel FaceModel (BFM) topology;
建立三维人脸模型中的人脸关键点与二维人脸图像中的人脸关键点间的映射关系；Establish the mapping relationship between the face key points in the 3D face model and the face key points in the 2D face image;
调整三维人脸模型的模型参数，直至三维人脸模型中的人脸关键点的投影点与二维人脸图像中的人脸关键点的欧式距离最小；Adjust the model parameters of the 3D face model until the Euclidean distance between the projection points of the face key points in the 3D face model and the face key points in the 2D face image is the smallest;
将调整后的三维人脸模型作为与二维人脸图像中的真实人脸完全匹配的三维真实人脸。The adjusted 3D face model is regarded as a 3D real face that exactly matches the real face in the 2D face image.
根据本发明的另一方面，提供了一种三维风格人脸的生成装置，包括：According to another aspect of the present invention, a device for generating a three-dimensional style human face is provided, comprising:
三维真实人脸构建模块，用于执行获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；The 3D real face building module is used for obtaining a 2D face image marked with face key points, and constructing a 3D real face that completely matches the real face in the 2D face image according to the face key points;
映射关系建立模块，用于执行建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系；The mapping relationship establishment module is used for establishing the mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face;
三维风格人脸生成模块，用于执行根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸。The 3D style face generation module is used to perform adjustment of the standard 3D style face according to the mapping relationship to obtain a 3D style face corresponding to the 2D face image.
根据本发明的另一方面，提供了一种电子设备，电子设备包括：According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
至少一个处理器；以及at least one processor; and
与至少一个处理器通信连接的存储器；其中，a memory communicatively coupled to the at least one processor; wherein,
存储器存储有可被至少一个处理器执行的计算机程序，计算机程序被至少一个处理器执行，以使至少一个处理器能够执行本发明任一实施例的三维风格人脸的生成方法。The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the method for generating a three-dimensional style face according to any embodiment of the present invention.
根据本发明的另一方面，提供了一种计算机可读存储介质，计算机可读存储介质存储有计算机指令，计算机指令用于使处理器执行时实现本发明任一实施例的三维风格人脸的生成方法。According to another aspect of the present invention, there is provided a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and the computer instructions are used to enable a processor to realize the three-dimensional style face of any embodiment of the present invention when executed. Generate method.
本发明实施例的技术方案，通过获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系；根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸，解决了用户“手捏”的三维风格人脸与用户人脸相似度低的问题，取到了基于单张图片，快速生成保留用户人脸特征信息的三维风格人脸的有益效果。The technical solution of the embodiment of the present invention is to obtain a two-dimensional face image marked with key points of the face, and construct a three-dimensional real face that completely matches the real face in the two-dimensional face image according to the key points of the face; establish a standard The mapping relationship between the PCA model of the 3D style face and the PCA model of the 3D real face; the standard 3D style face is adjusted according to the mapping relationship to obtain the 3D style face corresponding to the 2D face image, which solves the problem for users. The problem of low similarity between the "hand pinched" 3D-style face and the user's face is obtained, and the beneficial effect of quickly generating a 3D-style face that retains the user's face feature information based on a single image is obtained.
应当理解，本部分所描述的内容并非旨在标识本发明的实施例的关键或重要特征，也不用于限制本发明的范围。本发明的其它特征将通过以下的说明书而变得容易理解。It should be understood that the content described in this section is not intended to identify key or critical features of the embodiments of the invention, nor is it intended to limit the scope of the invention. Other features of the present invention will become readily understood from the following description.
附图说明Description of drawings
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.
图1是根据本发明实施例一提供的一种三维风格人脸的生成方法的流程图；1 is a flowchart of a method for generating a three-dimensional style human face according to Embodiment 1 of the present invention;
图2是根据本发明实施例一所适用的一种三维风格人脸的生成示意图；FIG. 2 is a schematic diagram of generating a three-dimensional style face according to Embodiment 1 of the present invention;
图3是根据本发明实施例一所适用的一种PCA模型的示意图；3 is a schematic diagram of a PCA model to which Embodiment 1 of the present invention is applicable;
图4是根据本发明实施例二提供的一种三维风格人脸的生成方法的流程图；4 is a flowchart of a method for generating a three-dimensional style human face according to Embodiment 2 of the present invention;
图5是根据本发明实施例三提供的一种三维风格人脸的生成装置的结构示意图；5 is a schematic structural diagram of an apparatus for generating a three-dimensional style human face according to Embodiment 3 of the present invention;
图6是实现本发明实施例的三维风格人脸的生成方法的电子设备的结构示意图。FIG. 6 is a schematic structural diagram of an electronic device implementing the method for generating a three-dimensional style face according to an embodiment of the present invention.
具体实施方式Detailed ways
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“目标”、“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "object", "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order. or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.
实施例一Example 1
图1是根据本发明实施例一提供的一种三维风格人脸的生成方法的流程图，本实施例可适用于基于单张图片，快速生成与该图片中的用户人脸的脸型相似的三维风格人脸的情况，该方法可以由三维风格人脸的生成装置来执行，该三维风格人脸的生成装置可以采用硬件和/或软件的形式实现，该三维风格人脸的生成装置可配置于电子设备中。如图1所示，该方法包括：1 is a flowchart of a method for generating a 3D-style face according to Embodiment 1 of the present invention. This embodiment can be applied to quickly generate a 3D-style face based on a single picture that is similar to the face shape of the user’s face in the picture. In the case of a style face, the method can be performed by a device for generating a three-dimensional style face, which can be implemented in the form of hardware and/or software, and the device for generating a three-dimensional style face can be configured in in electronic equipment. As shown in Figure 1, the method includes:
S110、获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸。S110: Acquire a two-dimensional face image marked with face key points, and construct a three-dimensional real face that completely matches the real face in the two-dimensional face image according to the face key points.
其中，二维人脸图像是指包含用户正面人脸的图片，人脸关键点是人脸面部各个部位的重要特征点，通常是轮廓点与角点。如图2所示，在左侧第一列图像中的眉毛、眼睛、鼻子、嘴巴、脸部轮廓等部位标记的特征点即为人脸关键点。本实施例中，获取到用户输入的包括正面人脸的图片之后，如果图片中没有标注人脸关键点，则先采用人脸关键点检测技术，在该图片中定位出人脸关键点。The two-dimensional face image refers to a picture containing the frontal face of the user, and the key points of the face are important feature points of various parts of the face, usually contour points and corner points. As shown in Figure 2, the feature points marked on the eyebrows, eyes, nose, mouth, face contours and other parts in the first column of images on the left are face key points. In this embodiment, after obtaining the picture including the frontal face input by the user, if there is no face key point marked in the picture, the face key point detection technology is firstly used to locate the face key point in the picture.
本实施例中，在获取到二维人脸图像中的人脸关键点后，可以利用参数拟合方法，基于BFM拓扑建立三维人脸关键点与二维人脸关键点间的映射关系，然后通过优化三维人脸的参数，使得最终三维人脸关键点的投影点与二维人脸关键点完全对齐，从而得到与二维人脸图像中的用户真实人脸完全匹配的三维真实人脸，如图2中左侧第二列图像所示。通过生成与用户真实人脸完全对齐的三维真实人脸，使得三维真实人脸中保留有用户真实人脸的脸型信息。In this embodiment, after obtaining the face key points in the two-dimensional face image, the parameter fitting method can be used to establish the mapping relationship between the three-dimensional face key points and the two-dimensional face key points based on the BFM topology, and then By optimizing the parameters of the 3D face, the projection points of the final 3D face key points are completely aligned with the 2D face key points, so as to obtain a 3D real face that completely matches the real face of the user in the 2D face image. As shown in the second column of images from the left in Figure 2. By generating a three-dimensional real face that is completely aligned with the user's real face, the face shape information of the user's real face is retained in the three-dimensional real face.
其中，BFM是在PCA的基础上进行存储的人脸数据库，BFM拓扑是基于BFM人脸数据库建立的三维人脸模型拓扑。PCA的主要目的是用较少的变量去解释原来数据中的大部分变异，选出比原始特征值个数少的变量，构建能解释最终输出的新变量，即为主成分。Among them, BFM is a face database stored on the basis of PCA, and BFM topology is a three-dimensional face model topology established based on the BFM face database. The main purpose of PCA is to use fewer variables to explain most of the variation in the original data, select variables with fewer eigenvalues than the original, and construct new variables that can explain the final output, namely principal components.
S120、建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系。S120. Establish a mapping relationship between the PCA model of the standard three-dimensional style face and the PCA model of the three-dimensional real face.
其中，标准三维风格人脸可以理解为通用的三维风格人脸，通常对标准三维风格人脸进行人脸特征调整后，才能生成与具体的用户真实人脸匹配的三维风格人脸。风格人脸可以是将真实人脸卡通化的虚拟人脸，也可以是其他风格的虚拟人脸。Among them, the standard 3D style face can be understood as a general 3D style face. Usually, the face feature adjustment of the standard 3D style face can be used to generate a 3D style face that matches the real face of a specific user. The style face can be a virtual face that cartoonizes a real face, or it can be a virtual face of other styles.
本实施例中，基于S110构建出的三维真实人脸与三维风格人脸有很大不同，三维风格人脸的面部五官通常会卡通化，整体脸型的长宽比也与真实人脸有较大不同。为了能将三维真实人脸转换为三维风格人脸，且在三维风格人脸中保留三维真实人脸的脸型信息，可以先准备一个标准的三维风格人脸，构建如图3所示三维风格人脸的PCA模型，然后基于标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系，将三维真实人脸的人脸形状信息转移到标准的三维风格人脸中。In this embodiment, the 3D real face constructed based on S110 is very different from the 3D style face. The facial features of the 3D style face are usually cartoonized, and the aspect ratio of the overall face shape is also larger than that of the real face. different. In order to convert a 3D real face into a 3D style face and retain the face shape information of the 3D real face in the 3D style face, a standard 3D style face can be prepared first, and a 3D style face as shown in Figure 3 can be constructed. Then, based on the mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face, the face shape information of the 3D real face is transferred to the standard 3D style face.
其中，建立映射关系使用的PCA模型主要是指人脸形状模型，不考虑表情模型，这是因为表情模型依赖于表情驱动，不需要重建人脸表情。Among them, the PCA model used to establish the mapping relationship mainly refers to the face shape model, and does not consider the expression model, because the expression model depends on the expression drive and does not need to reconstruct the facial expression.
可选的，建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系，可以包括：建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的PCA基映射关系以及幅度映射关系。Optionally, establishing a mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face may include: establishing a mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face. PCA basis mapping relationship and amplitude mapping relationship.
本实施例中，映射关系可以理解为两个高维正交空间的每个坐标轴上的坐标值对应的幅度映射关系，即映射关系包括PCA基之间的映射关系和PCA基的幅度之间的映射关系。这是因为，在找到标准三维风格人脸和三维真实人脸之间的对应维度的PCA基后，不同的PCA基在同样的维度变化系数下的幅度是不一样的，因此，还需要确定基变化的幅度映射关系，使得后续三维真实人脸转换到三维风格人脸时可以兼顾PCA基的方向和幅度。In this embodiment, the mapping relationship can be understood as the amplitude mapping relationship corresponding to the coordinate value on each coordinate axis of the two high-dimensional orthogonal spaces, that is, the mapping relationship includes the mapping relationship between the PCA bases and the mapping between the amplitudes of the PCA bases relation. This is because, after finding the PCA basis of the corresponding dimension between the standard 3D style face and the 3D real face, the amplitudes of different PCA basis under the same dimensional change coefficient are different. Therefore, it is necessary to determine the basis The changing amplitude mapping relationship makes it possible to take into account the direction and amplitude of the PCA basis when the subsequent 3D real face is converted to a 3D style face.
可选的，建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的PCA基映射关系以及幅度映射关系，可以包括：在标准三维风格人脸的PCA模型中，获取与目标人脸形状维度对应的第一PCA基以及第一PCA基的幅度值；在三维真实人脸的PCA模型中，查询与目标人脸形状维度对应的第二PCA基以及第二PCA基的幅度值；建立第一PCA基以及第一PCA基的幅度值与第二PCA基以及第二PCA基的幅度值之间的映射关系。Optionally, establishing a PCA base mapping relationship and an amplitude mapping relationship between the PCA model of the standard 3D-style face and the PCA model of the 3D real face may include: in the PCA model of the standard 3D-style face, obtaining and target The first PCA basis and the amplitude value of the first PCA basis corresponding to the face shape dimension; in the PCA model of the three-dimensional real face, query the second PCA basis and the amplitude value of the second PCA basis corresponding to the target face shape dimension ; Establish a mapping relationship between the first PCA basis and the amplitude value of the first PCA basis and the second PCA basis and the amplitude value of the second PCA basis.
本实施例中，由于三维真实人脸的PCA基的维度较大，而三维风格人脸的PCA基的维度较小，因此不可能每个PCA基的维度都有映射关系，并且要将真实人脸的脸型信息转移到三维风格人脸，只需要针对部分人脸形状维度的PCA基建立映射关系即可，例如人脸胖瘦维度，下巴长短维度，眼睛间距维度等。可以在标准三维风格人脸和三维真实人脸中，基于PCA基的正交性，分别改变各PCA基的幅度值观察人脸各维度的变化，找到待建立映射关系的人脸形状维度，进而为两种三维人脸在该维度下的PCA基以及PCA基的幅度建立映射关系。In this embodiment, since the dimension of the PCA base of the 3D real face is relatively large, and the dimension of the PCA base of the 3D style face is relatively small, it is impossible for each dimension of the PCA base to have a mapping relationship, and it is impossible to map the dimensions of the real face. To transfer the face shape information of the face to the 3D style face, it is only necessary to establish a mapping relationship for the PCA basis of some face shape dimensions, such as the face fat and thin dimension, the chin length dimension, the eye distance dimension, etc. In the standard 3D style face and the 3D real face, based on the orthogonality of the PCA basis, the amplitude value of each PCA basis can be changed to observe the changes of each dimension of the face, and the shape dimension of the face to be established in the mapping relationship can be found, and then A mapping relationship is established for the PCA basis and the amplitude of the PCA basis of the two 3D faces in this dimension.
可选的，确定标准三维风格人脸的第一PCA基的幅度值，可以包括：在标准三维风格人脸不畸形的情况下，获取第一PCA基的坐标最小值和坐标最大值；将坐标最小值和坐标最大值的均值作为第一PCA基的幅度值。Optionally, determining the amplitude value of the first PCA basis of the standard three-dimensional style face may include: in the case that the standard three-dimensional style face is not deformed, obtaining the coordinate minimum and coordinate maximum value of the first PCA basis; The mean value of the minimum value and the coordinate maximum value is taken as the amplitude value of the first PCA basis.
本实施例中，在确定第一PCA基的幅度值时，可以将以标准三维风格人脸不畸形为标准对PCA基的坐标值进行调整，得到第一PCA基的坐标最大值和坐标最小值，进而将坐标最大值和坐标最小值的均值作为第一PCA基的幅度值。相应的，采用该方式计算三维真实人脸中的第二PCA基的幅度值。其中，将PCA基的坐标最大值和坐标最小值的均值作为PCA基的幅度值只是一种可选方式，也可以根据需求设置其他的幅度值计算方式。In this embodiment, when determining the amplitude value of the first PCA base, the coordinate value of the PCA base may be adjusted based on the standard three-dimensional face not deformed to obtain the maximum coordinate value and the minimum coordinate value of the first PCA base. , and then take the mean value of the coordinate maximum value and the coordinate minimum value as the amplitude value of the first PCA basis. Correspondingly, the amplitude value of the second PCA basis in the three-dimensional real face is calculated in this way. It is only an optional way to use the mean value of the maximum coordinate value of the PCA base and the minimum coordinate value of the PCA base as the amplitude value of the PCA base, and other calculation methods of the amplitude value can also be set according to requirements.
S130、根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸。S130. Adjust the standard three-dimensional style face according to the mapping relationship to obtain a three-dimensional style face corresponding to the two-dimensional face image.
本实施例中，根据映射关系，按照三维真实人脸中的脸型信息对标准风格人脸中的部分人脸形状维度进行调整，使得最终得到的三维风格人脸可以保留更多的用户真实人脸的脸型信息，保留较少的纹理信息，如图2中右侧第一列图像所示。其中，图2中右侧第一列图像中的眉毛应该与用户真实人脸的眉毛相似，图中未表示出来。In this embodiment, according to the mapping relationship, according to the face shape information in the three-dimensional real face, some face shape dimensions in the standard style face are adjusted, so that the finally obtained three-dimensional style face can retain more real faces of the user face shape information, and retain less texture information, as shown in the first column on the right in Figure 2. Among them, the eyebrows in the first column of images on the right in Figure 2 should be similar to the eyebrows of the user's real face, which are not shown in the figure.
可选的，根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸，可以包括：将标准三维风格人脸中已建立映射关系的每个PCA基的人脸形状系数，更新为三维真实人脸中的对应PCA基的人脸形状系数；将标准三维风格人脸中已建立映射关系的每个PCA基的幅度值，更新为三维真实人脸中的对应PCA基的幅度值；将更新后的标准三维风格人脸作为与二维人脸图像对应的三维风格人脸。Optionally, the standard three-dimensional style face is adjusted according to the mapping relationship to obtain a three-dimensional style face corresponding to the two-dimensional face image, which may include: converting each PCA basis for which the mapping relationship has been established in the standard three-dimensional style face. The face shape coefficient is updated to the face shape coefficient corresponding to the PCA basis in the 3D real face; the amplitude value of each PCA basis for which the mapping relationship has been established in the standard 3D style face is updated to the 3D real face. The amplitude value corresponding to the PCA basis; the updated standard 3D style face is regarded as the 3D style face corresponding to the 2D face image.
示例性的，以人脸胖瘦维度为例，获取三维真实人脸中人脸胖瘦维度对应的第一PCA基以及形状系数，查询映射关系确定三维真实人脸中该第一PCA基的幅度值，以及标准三维风格人脸中的人脸胖瘦维度对应的第二PCA基，使用第一PCA基的形状系数替换第二PCA基的形状系数，并且使用第一PCA基的幅度值替换第二PCA基的幅度值。Exemplarily, taking the face fat and thin dimension as an example, obtain the first PCA basis and shape coefficient corresponding to the face fat and thin dimension in the three-dimensional real face, and query the mapping relationship to determine the magnitude of the first PCA basis in the three-dimensional real face. value, and the second PCA basis corresponding to the face fat and thin dimensions in the standard 3D style face, the shape coefficient of the first PCA basis is used to replace the shape coefficient of the second PCA basis, and the amplitude value of the first PCA basis is used to replace the first PCA basis. The magnitude of the two PCA bases.
本发明实施例的技术方案，通过获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系；根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸，解决了用户“手捏”的三维风格人脸与用户人脸相似度低的问题，取到了基于单张图片，快速生成保留用户人脸特征信息的三维风格人脸的有益效果。The technical solution of the embodiment of the present invention is to obtain a two-dimensional face image marked with key points of the face, and construct a three-dimensional real face that completely matches the real face in the two-dimensional face image according to the key points of the face; establish a standard The mapping relationship between the PCA model of the 3D style face and the PCA model of the 3D real face; the standard 3D style face is adjusted according to the mapping relationship to obtain the 3D style face corresponding to the 2D face image, which solves the problem for users. The problem of low similarity between the "hand pinched" 3D-style face and the user's face is obtained, and the beneficial effect of quickly generating a 3D-style face that retains the user's face feature information based on a single image is obtained.
实施例二Embodiment 2
图4是根据本发明实施例二提供的一种三维风格人脸的生成方法的流程图，本实施例在上述实施例的基础上，进一步提供了获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸的具体步骤，以及构建标准三维风格人脸的PCA模型的具体步骤。如图4所示，该方法包括：FIG. 4 is a flowchart of a method for generating a 3D-style face according to Embodiment 2 of the present invention. On the basis of the above-mentioned embodiments, this embodiment further provides a method for obtaining a 2D face image marked with key points of a human face. , and the specific steps of constructing a 3D real face that completely matches the real face in the 2D face image according to the key points of the face, and the specific steps of constructing a PCA model of a standard 3D style face. As shown in Figure 4, the method includes:
S210、获取标注有人脸关键点的二维人脸图像，并基于BFM拓扑构建三维人脸模型。S210 , acquiring a two-dimensional face image marked with key points of a face, and constructing a three-dimensional face model based on the BFM topology.
本实施例中，在获取到二维人脸图像中的人脸关键点后，基于BFM拓扑，采用三维人脸重建公式：
其中，三维人脸重建公式其中各参数的含义如下：s为缩放参数，用于将三维人脸模型缩放到与二维人脸图像中的用户人脸的尺度对齐,是一个未知量。P为正交投影矩阵，是一个3*3的单位矩阵。R为旋转矩阵，是一个3*3的单位正交矩阵，是一个未知量。
PCA三维建模会构建出形状维度以及表情维度的PCA基，形状维度表示为m，取值为199，表情维度表示为n，取值为29。αi为形状维度的PCA基，其维度是199*65536*3，65536是指三维人脸模型的点云个数，3是指XYZ三个坐标轴，i为维度自变量。si为形状系数，包括199个未知数。βi为表情维度的PCA基，它的维度是29*65536*3。ei为表情系数，包括29个未知数。t2d为偏移矩阵，用于将三维人脸模型与二维人脸图像对齐，是一个未知量。The PCA 3D modeling will construct the PCA basis of the shape dimension and the expression dimension. The shape dimension is expressed as m, and the value is 199. αi is the PCA basis of the shape dimension, and its dimension is 199*65536*3, 65536 refers to the number of point clouds of the three-dimensional face model, 3 refers to the three coordinate axes of XYZ, and i is the dimension independent variable. si is the shape coefficient, including 199 unknowns. βi is the PCA basis of the expression dimension, and its dimension is 29*65536*3. ei is the expression coefficient, including 29 unknowns. t2d is the offset matrix, which is used to align the 3D face model with the 2D face image and is an unknown quantity.
S220、建立三维人脸模型中的人脸关键点与二维人脸图像中的人脸关键点间的映射关系。S220. Establish a mapping relationship between the face key points in the three-dimensional face model and the face key points in the two-dimensional face image.
本实施例中，在建立三维人脸模型之后，可以根据二维人脸图像中的各人脸关键点，在三维人脸模型中选择对应位置的人脸关键点，并建立相应的关键点映射关系。In this embodiment, after the three-dimensional face model is established, the face key points at the corresponding positions can be selected in the three-dimensional face model according to each face key point in the two-dimensional face image, and the corresponding key point mapping can be established. relation.
S230、调整三维人脸模型的模型参数，直至三维人脸模型中的人脸关键点的投影点与二维人脸图像中的人脸关键点的欧式距离最小。S230. Adjust the model parameters of the three-dimensional face model until the Euclidean distance between the projection points of the face key points in the three-dimensional face model and the face key points in the two-dimensional face image is the smallest.
本实施例中，确定三维人脸模型中的人脸关键点的投影点，然后通过最小化投影点与二维人脸图像中的人脸关键点的欧式距离，来优化三维人脸模型中的参数s、R、si、ei以及t2d，实现最终的人脸重建。In this embodiment, the projection points of the key points of the face in the three-dimensional face model are determined, and then the Euclidean distance between the projection points and the key points of the face in the two-dimensional face image is minimized to optimize the projection points in the three-dimensional face model. Parameters s, R, si , ei and t2d to achieve the final face reconstruction.
S240、将调整后的三维人脸模型作为与二维人脸图像中的真实人脸完全匹配的三维真实人脸。S240. Use the adjusted three-dimensional face model as a three-dimensional real face that completely matches the real face in the two-dimensional face image.
S250、构建标准三维风格人脸的PCA模型。S250 , constructing a PCA model of a standard three-dimensional style face.
本实施例中，为了能将三维真实人脸转换为保留用户人脸的脸型信息的三维风格人脸，可以从网上下载一个标准的三维风格人脸，使用多个基于美术生成的具有不同脸型的三维风格人脸，构建如图3所示的标准三维风格人脸的PCA模型，便于后续基于标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系，将用户脸型信息转移到三维风格人脸中。In this embodiment, in order to convert a 3D real face into a 3D style face that retains the face shape information of the user's face, a standard 3D style face can be downloaded from the Internet, and a plurality of art-generated faces with different face shapes are used. 3D style face, build the PCA model of the standard 3D style face as shown in Figure 3, which is convenient for the subsequent mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face, and the user face information Transfer to a 3D style face.
可选的，获取标准三维风格人脸以及三维风格人脸样本集；根据公式S(α)＝μs+Us*diag(σs)*α，对三维风格人脸样本集进行主成分分析构建不同维度的PCA基，组成PCA模型；其中，S(α)为PCA基，μs为所有三维风格人脸样本的平均值，σs为每个三维风格人脸样本与平均值的方差，Us为对每个三维风格人脸样本与平均值的差值做主成分分析得到的特征向量，diag(σs)为特征向量Us对应的权重，α为与特征向量Us对应的特征值。Optionally, obtain a standard 3D style face and a 3D style face sample set; perform principal component analysis on the 3D style face sample set according to the formula S(α)=μs +Us *diag(σs )*α Construct PCA bases of different dimensions to form a PCA model; where S(α) is the PCA base, μs is the average value of all 3D style face samples, σs is the variance between each 3D style face sample and the average value, Us is the feature vector obtained by principal component analysis of the difference between each 3D style face sample and the average value, diag(σs ) is the weight corresponding to the feature vector Us , α is the feature value corresponding to the feature vector Us .
S260、建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系。S260. Establish a mapping relationship between the PCA model of the standard three-dimensional style face and the PCA model of the three-dimensional real face.
S270、根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸。S270. Adjust the standard three-dimensional style face according to the mapping relationship to obtain a three-dimensional style face corresponding to the two-dimensional face image.
需要说明的是，本实施例通过单张图片即可构建带有用户人脸特征信息的三维风格人脸，可保留更多的用户脸型信息，纹理可公用美术提供的资源，眉毛、口红、眼睛等都可调。It should be noted that, in this embodiment, a single image can be used to construct a three-dimensional style face with user face feature information, more user face shape information can be retained, and textures can share the resources provided by art, eyebrows, lipstick, and eyes. etc. are adjustable.
本发明实施例的技术方案，通过获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系；根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸，解决了依靠用户“手捏”的三维风格人脸与用户人脸相似度低的问题，取到了基于单张图片，快速生成保留用户人脸特征信息的三维风格人脸的有益效果。The technical solution of the embodiment of the present invention is to obtain a two-dimensional face image marked with key points of the face, and construct a three-dimensional real face that completely matches the real face in the two-dimensional face image according to the key points of the face; establish a standard The mapping relationship between the PCA model of the 3D style face and the PCA model of the 3D real face; the standard 3D style face is adjusted according to the mapping relationship, and the 3D style face corresponding to the 2D face image is obtained. The problem of low similarity between the user's "hand pinched" 3D-style face and the user's face has the beneficial effect of quickly generating a 3D-style face that retains the user's face feature information based on a single image.
实施例三Embodiment 3
图5是根据本发明实施例三提供的一种三维风格人脸的生成装置的结构示意图。本实施例可适用于基于单张图片，快速生成与该图片中的用户人脸的脸型相似的三维风格人脸的情况，该三维风格人脸的生成装置可以采用硬件和/或软件的形式实现，该三维风格人脸的生成装置可配置于电子设备中。如图5所示，该装置包括：FIG. 5 is a schematic structural diagram of an apparatus for generating a three-dimensional style face according to Embodiment 3 of the present invention. This embodiment can be applied to the situation of quickly generating a 3D style face similar to the face shape of the user's face in the picture based on a single picture, and the device for generating the 3D style face can be implemented in the form of hardware and/or software , the device for generating a three-dimensional style face can be configured in an electronic device. As shown in Figure 5, the device includes:
三维真实人脸构建模块510，用于执行获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；A three-dimensional real
映射关系建立模块520，用于执行建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系；The mapping
三维风格人脸生成模块530，用于执行根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸。The 3D style
本发明实施例的技术方案，通过获取标注有人脸关键点的二维人脸图像，并根据人脸关键点构建与二维人脸图像中的真实人脸完全匹配的三维真实人脸；建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系；根据映射关系对标准三维风格人脸进行调整，得到与二维人脸图像对应的三维风格人脸，解决了依靠用户“手捏”的三维风格人脸与用户人脸相似度低的问题，取到了基于单张图片，快速生成保留用户人脸特征信息的三维风格人脸的有益效果。The technical solution of the embodiment of the present invention is to obtain a two-dimensional face image marked with key points of the face, and construct a three-dimensional real face that completely matches the real face in the two-dimensional face image according to the key points of the face; establish a standard The mapping relationship between the PCA model of the 3D style face and the PCA model of the 3D real face; the standard 3D style face is adjusted according to the mapping relationship, and the 3D style face corresponding to the 2D face image is obtained. The problem of low similarity between the user's "hand pinched" 3D-style face and the user's face has the beneficial effect of quickly generating a 3D-style face that retains the user's face feature information based on a single image.
可选的，映射关系建立模块520，用于执行：建立标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的PCA基映射关系以及幅度映射关系。Optionally, the mapping
可选的，映射关系建立模块520，包括：Optionally, the mapping
第一获取模块，用于执行在标准三维风格人脸的PCA模型中，获取与目标人脸形状维度对应的第一PCA基以及第一PCA基的幅度值；The first acquisition module is used to obtain the first PCA basis corresponding to the shape dimension of the target face and the amplitude value of the first PCA basis in the PCA model of the standard three-dimensional style face;
第二获取模块，用于执行在三维真实人脸的PCA模型中，查询与目标人脸形状维度对应的第二PCA基以及第二PCA基的幅度值；The second acquisition module is used to query the second PCA basis corresponding to the shape dimension of the target face and the amplitude value of the second PCA basis in the PCA model of the three-dimensional real face;
建立模块，用于执行建立第一PCA基以及第一PCA基的幅度值与第二PCA基以及第二PCA基的幅度值之间的映射关系。The establishing module is configured to execute establishing the first PCA basis and the mapping relationship between the amplitude values of the first PCA basis and the second PCA basis and the amplitude values of the second PCA basis.
可选的，第一获取模块，用于执行：Optionally, the first acquisition module is used to execute:
在标准三维风格人脸不畸形的情况下，获取第一PCA基的坐标最小值和坐标最大值；In the case that the standard 3D style face is not deformed, obtain the minimum and maximum coordinates of the first PCA basis;
将坐标最小值和坐标最大值的均值作为第一PCA基的幅度值。The average value of the coordinate minimum value and the coordinate maximum value is taken as the amplitude value of the first PCA basis.
可选的，三维风格人脸生成模块530，用于执行：Optionally, the three-dimensional style
将标准三维风格人脸中已建立映射关系的每个PCA基的人脸形状系数，更新为三维真实人脸中的对应PCA基的人脸形状系数；Update the face shape coefficient of each PCA base for which the mapping relationship has been established in the standard 3D style face to the face shape coefficient of the corresponding PCA base in the 3D real face;
将标准三维风格人脸中已建立映射关系的每个PCA基的幅度值，更新为三维真实人脸中的对应PCA基的幅度值；Update the amplitude value of each PCA basis for which the mapping relationship has been established in the standard 3D style face to the amplitude value of the corresponding PCA basis in the 3D real face;
将更新后的标准三维风格人脸作为与二维人脸图像对应的三维风格人脸。The updated standard 3D style face is taken as the 3D style face corresponding to the 2D face image.
可选的，还包括PCA模型建立模块，用于执行：Optionally, a PCA model building module is also included to perform:
在构建标准三维风格人脸的PCA模型与三维真实人脸的PCA模型之间的映射关系之前，获取标准三维风格人脸以及三维风格人脸样本集；Before constructing the mapping relationship between the PCA model of the standard 3D style face and the PCA model of the 3D real face, obtain the standard 3D style face and a sample set of 3D style faces;
根据公式S(α)＝μs+Us*diag(σs)*α，对三维风格人脸样本集进行主成分分析构建不同维度的PCA基，组成PCA模型；According to the formula S(α)=μs +Us *diag(σs )*α, perform principal component analysis on the three-dimensional style face sample set to construct PCA bases of different dimensions to form a PCA model;
其中，S(α)为PCA基，μs为所有三维风格人脸样本的平均值，σs为每个三维风格人脸样本与平均值的方差，Us为对每个三维风格人脸样本与平均值的差值做主成分分析得到的特征向量，diag(σs)为特征向量Us对应的权重，α为与特征向量Us对应的特征值。Among them, S(α) is the PCA basis, μs is the average value of all 3D style face samples, σs is the variance between each 3D style face sample and the average value, Us is the average value of each 3D style face sample The eigenvector obtained by principal component analysis of the difference from the mean value, diag(σs ) is the weight corresponding to the eigenvector Us , and α is the eigenvalue corresponding to the eigenvector Us .
可选的，三维真实人脸构建模块510，用于执行：Optionally, the three-dimensional real
获取标注有人脸关键点的二维人脸图像，并基于巴塞尔人脸模型BFM拓扑构建三维人脸模型；Obtain 2D face images marked with face key points, and build 3D face models based on Basel face model BFM topology;
建立三维人脸模型中的人脸关键点与二维人脸图像中的人脸关键点间的映射关系；Establish the mapping relationship between the face key points in the 3D face model and the face key points in the 2D face image;
调整三维人脸模型的模型参数，直至三维人脸模型中的人脸关键点的投影点与二维人脸图像中的人脸关键点的欧式距离最小；Adjust the model parameters of the 3D face model until the Euclidean distance between the projection points of the face key points in the 3D face model and the face key points in the 2D face image is the smallest;
将调整后的三维人脸模型作为与二维人脸图像中的真实人脸完全匹配的三维真实人脸。The adjusted 3D face model is regarded as a 3D real face that exactly matches the real face in the 2D face image.
本发明实施例所提供的三维风格人脸的生成装置可执行本发明任意实施例所提供的三维风格人脸的生成方法，具备执行方法相应的功能模块和有益效果。The apparatus for generating a 3D style face provided by the embodiment of the present invention can execute the method for generating a 3D style face provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
实施例四Embodiment 4
图6示出了可以用来实施本发明的实施例的电子设备10的结构示意图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备(如头盔、眼镜、手表等)和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本发明的实现。FIG. 6 shows a schematic structural diagram of an
如图6所示，电子设备10包括至少一个处理器11，以及与至少一个处理器11通信连接的存储器，如只读存储器(ROM)12、随机访问存储器(RAM)13等，其中，存储器存储有可被至少一个处理器执行的计算机程序，处理器11可以根据存储在只读存储器(ROM)12中的计算机程序或者从存储单元18加载到随机访问存储器(RAM)13中的计算机程序，来执行各种适当的动作和处理。在RAM 13中，还可存储电子设备10操作所需的各种程序和数据。处理器11、ROM 12以及RAM 13通过总线14彼此相连。输入/输出(I/O)接口15也连接至总线14。As shown in FIG. 6, the
电子设备10中的多个部件连接至I/O接口15，包括：输入单元16，例如键盘、鼠标等；输出单元17，例如各种类型的显示器、扬声器等；存储单元18，例如磁盘、光盘等；以及通信单元19，例如网卡、调制解调器、无线通信收发机等。通信单元19允许电子设备10通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Various components in the
处理器11可以是各种具有处理和计算能力的通用和/或专用处理组件。处理器11的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的处理器、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。处理器11执行上文所描述的各个方法和处理，例如三维风格人脸的生成方法。The
在一些实施例中，三维风格人脸的生成方法可被实现为计算机程序，其被有形地包含于计算机可读存储介质，例如存储单元18。在一些实施例中，计算机程序的部分或者全部可以经由ROM 12和/或通信单元19而被载入和/或安装到电子设备10上。当计算机程序加载到RAM 13并由处理器11执行时，可以执行上文描述的三维风格人脸的生成方法的一个或多个步骤。备选地，在其他实施例中，处理器11可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行三维风格人脸的生成方法。In some embodiments, the method of generating a three-dimensional style face may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.
用于实施本发明的方法的计算机程序可以采用一个或多个编程语言的任何组合来编写。这些计算机程序可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器，使得计算机程序当由处理器执行时使流程图和/或框图中所规定的功能/操作被实施。计算机程序可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/operations specified in the flowcharts and/or block diagrams to be carried out. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
在本发明的上下文中，计算机可读存储介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的计算机程序。计算机可读存储介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。备选地，计算机可读存储介质可以是机器可读信号介质。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in connection with the instruction execution system, apparatus or device. Computer-readable storage media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
为了提供与用户的交互，可以在电子设备上实施此处描述的系统和技术，该电子设备具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给电子设备。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having a display device (eg, a CRT (cathode ray tube) or an LCD (liquid crystal display)) for displaying information to the user monitor); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)、区块链网络和互联网。The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the Internet.
计算系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与VPS服务中，存在的管理难度大，业务扩展性弱的缺陷。A computing system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the traditional physical host and VPS services, which are difficult to manage and weak in business scalability. defect.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发明中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本发明的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present invention can be performed in parallel, sequentially or in different orders, and as long as the desired results of the technical solutions of the present invention can be achieved, no limitation is imposed herein.
上述具体实施方式，并不构成对本发明保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本发明的精神和原则之内所作的修改、等同替换和改进等，均应包含在本发明保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210156774.8ACN114529685A (en) | 2022-02-21 | 2022-02-21 | Three-dimensional style face generation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210156774.8ACN114529685A (en) | 2022-02-21 | 2022-02-21 | Three-dimensional style face generation method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114529685Atrue CN114529685A (en) | 2022-05-24 |
Family
ID=81624477
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210156774.8APendingCN114529685A (en) | 2022-02-21 | 2022-02-21 | Three-dimensional style face generation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114529685A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115240254A (en)* | 2022-07-22 | 2022-10-25 | 佛山虎牙虎信科技有限公司 | Method, device and electronic device for generating cartoon face |
| CN116309009A (en)* | 2022-09-08 | 2023-06-23 | 佛山虎牙虎信科技有限公司 | Stylized face generation method and device and image processing equipment |
| CN117593447A (en)* | 2023-04-25 | 2024-02-23 | 上海任意门科技有限公司 | Three-dimensional face construction method, system, device and medium based on 2D key points |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104756156A (en)* | 2013-03-14 | 2015-07-01 | 梦工厂动画公司 | Compressing data representing computer animated hair |
| CN106447785A (en)* | 2016-09-30 | 2017-02-22 | 北京奇虎科技有限公司 | Method for driving virtual character and device thereof |
| CN108062791A (en)* | 2018-01-12 | 2018-05-22 | 北京奇虎科技有限公司 | A kind of method and apparatus for rebuilding human face three-dimensional model |
| CN108447017A (en)* | 2018-05-31 | 2018-08-24 | Oppo广东移动通信有限公司 | Face virtual face-lifting method and device |
| CN112258382A (en)* | 2020-10-23 | 2021-01-22 | 北京中科深智科技有限公司 | Face style transfer method and system based on image-to-image |
| CN113409454A (en)* | 2021-07-14 | 2021-09-17 | 北京百度网讯科技有限公司 | Face image processing method and device, electronic equipment and storage medium |
- 2022
- 2022-02-21CNCN202210156774.8Apatent/CN114529685A/enactivePending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104756156A (en)* | 2013-03-14 | 2015-07-01 | 梦工厂动画公司 | Compressing data representing computer animated hair |
| CN106447785A (en)* | 2016-09-30 | 2017-02-22 | 北京奇虎科技有限公司 | Method for driving virtual character and device thereof |
| CN108062791A (en)* | 2018-01-12 | 2018-05-22 | 北京奇虎科技有限公司 | A kind of method and apparatus for rebuilding human face three-dimensional model |
| CN108447017A (en)* | 2018-05-31 | 2018-08-24 | Oppo广东移动通信有限公司 | Face virtual face-lifting method and device |
| CN112258382A (en)* | 2020-10-23 | 2021-01-22 | 北京中科深智科技有限公司 | Face style transfer method and system based on image-to-image |
| CN113409454A (en)* | 2021-07-14 | 2021-09-17 | 北京百度网讯科技有限公司 | Face image processing method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 高翔等: "3DMM与GAN结合的实时人脸表情迁移方法", 计算机软件与应用, 30 April 2020 (2020-04-30)* |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115240254A (en)* | 2022-07-22 | 2022-10-25 | 佛山虎牙虎信科技有限公司 | Method, device and electronic device for generating cartoon face |
| CN116309009A (en)* | 2022-09-08 | 2023-06-23 | 佛山虎牙虎信科技有限公司 | Stylized face generation method and device and image processing equipment |
| CN117593447A (en)* | 2023-04-25 | 2024-02-23 | 上海任意门科技有限公司 | Three-dimensional face construction method, system, device and medium based on 2D key points |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113643412B (en) | Virtual image generation method, device, electronic device and storage medium | |
| CN114529685A (en) | Three-dimensional style face generation method, device, equipment and storage medium | |
| CN114842123B (en) | Three-dimensional face reconstruction model training and three-dimensional face image generation method and device | |
| CN111695471A (en) | Virtual image generation method, device, equipment and storage medium | |
| CN113052962B (en) | Model training method, information output method, device, equipment and storage medium | |
| CN113963110B (en) | Texture map generation method and device, electronic equipment and storage medium | |
| CN116051729B (en) | Three-dimensional content generation method, device and electronic device | |
| CN108399383A (en) | Expression moving method, device storage medium and program | |
| CN115147265B (en) | Avatar generation method, apparatus, electronic device, and storage medium | |
| CN115345980A (en) | Method and device for generating personalized texture maps | |
| CN114723888B (en) | Three-dimensional hair model generation method, device, equipment, storage medium and product | |
| CN114549710A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN115049799A (en) | Method and device for generating 3D model and virtual image | |
| CN116524165B (en) | Migration method, device, equipment and storage medium for three-dimensional expression model | |
| CN112581573A (en) | Avatar driving method, apparatus, device, medium, and program product | |
| CN113808249A (en) | Image processing method, apparatus, device and computer storage medium | |
| CN114792355A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN115359171B (en) | Virtual image processing method and device, electronic equipment and storage medium | |
| CN114202597B (en) | Image processing method and apparatus, device, medium and product | |
| CN115147681A (en) | Method and device for training clothing generation model and method and device for generating clothing image | |
| CN116188640B (en) | Three-dimensional virtual image generation method, device, equipment and medium | |
| CN114078184B (en) | Data processing method, apparatus, electronic device and medium | |
| CN115906987A (en) | Deep learning model training method, virtual image driving method and device | |
| CN116468849A (en) | Three-dimensional virtual image generation method, device, equipment and storage medium | |
| CN114820908A (en) | Virtual image generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20220524 |