CN112085835A - Three-dimensional cartoon face generation method and device, electronic equipment and storage medium - Google Patents

Abstract

The application relates to the field of computers, in particular to the technical field of vision, and discloses a three-dimensional cartoon face generation method, a device, electronic equipment and a storage medium, wherein the three-dimensional cartoon face which is high in precision and highly similar to a real face can be generated based on a real face image, and the method comprises the following steps: generating a corresponding human face-like three-dimensional point cloud model based on the human face image; determining a target vertex corresponding to each vertex in the three-dimensional cartoon face model in the human-like three-dimensional point cloud model according to a vertex index relationship, wherein the vertex index relationship comprises a corresponding relationship between the vertex in the human-like three-dimensional point cloud model and the vertex in the three-dimensional cartoon face model; determining coordinates corresponding to each vertex in the three-dimensional cartoon face model respectively based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model respectively; and fitting to obtain the three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

Description

Three-dimensional cartoon face generation method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of computers, and in particular, to a method and an apparatus for generating a three-dimensional cartoon face, an electronic device, and a storage medium.

Background

In recent years, cartoon products attract people of all ages in the society, and with the development and common application of three-dimensional virtual environment technology, three-dimensional cartoon images are more and more widely applied in the fields of animation movies and televisions, network games, virtual communities, auxiliary teaching and the like. At present, the method for automatically generating the three-dimensional cartoon face mainly comprises the steps of extracting face features from a face image by a computer, and carrying out exaggeration on the face features to automatically generate a three-dimensional cartoon image of the face, wherein the generated three-dimensional cartoon image has low precision and is greatly different from a real face.

Disclosure of Invention

The embodiment of the application provides a three-dimensional cartoon face generation method and device, electronic equipment and a storage medium, and the three-dimensional cartoon face which is high in precision and is highly similar to a real face can be generated based on a real face image.

On one hand, an embodiment of the present application provides a method for generating a three-dimensional cartoon face, including:

generating a corresponding human face-like three-dimensional point cloud model based on the human face image;

determining the coordinates of each vertex in the three-dimensional cartoon face model based on the coordinates of each vertex in the face-like three-dimensional point cloud model;

and fitting to obtain a three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

In one aspect, an embodiment of the present application provides a three-dimensional cartoon face generation apparatus, including:

the human face-like three-dimensional point cloud model generating module is used for generating a corresponding human face-like three-dimensional point cloud model based on the human face image;

the three-dimensional cartoon face model generating module is used for determining a target vertex corresponding to each vertex in the three-dimensional cartoon face model in the face-like three-dimensional point cloud model according to a vertex index relationship, and determining coordinates corresponding to each vertex based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model, wherein the vertex index relationship comprises a corresponding relationship between the vertex in the face-like three-dimensional point cloud model and the vertex in the three-dimensional cartoon face model;

and the fitting output module is used for fitting to obtain the three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

Optionally, the fitting output module is specifically configured to:

fitting to obtain cartoon expression base coefficients respectively corresponding to the cartoon expression bases based on the coordinates of each vertex in the three-dimensional cartoon face model, wherein each cartoon expression base corresponds to a three-dimensional cartoon face template with one expression;

and carrying out weighted summation on each cartoon expression base based on the cartoon expression base coefficient respectively corresponding to each cartoon expression base so as to obtain the three-dimensional cartoon face corresponding to the face image.

Optionally, the human-face-like three-dimensional point cloud model and the three-dimensional cartoon human face model both include a plurality of feature areas, and each feature area includes at least one vertex; the fitting output module is specifically configured to:

for each feature area, fitting to obtain local cartoon expression base coefficients corresponding to the cartoon expression bases respectively based on coordinates of vertexes in each feature area in the three-dimensional cartoon face model;

and for each cartoon expression base, fusing the local cartoon expression base coefficients of each cartoon expression base corresponding to the plurality of characteristic areas respectively to obtain the cartoon expression base coefficient corresponding to each cartoon expression base.

Optionally, the three-dimensional cartoon face model generation module is specifically configured to:

and updating the coordinates of each vertex in the three-dimensional cartoon face model into the coordinates of the corresponding target vertex.

Optionally, the apparatus further includes an index determining module, configured to determine the vertex index relationship by:

carrying out point cloud preliminary alignment on the human face-like three-dimensional point cloud model and the three-dimensional cartoon human face model;

attaching the vertex of the three-dimensional cartoon face model to the vertex of the similar face three-dimensional point cloud model to obtain an updated three-dimensional cartoon face model;

and aiming at each vertex in the updated three-dimensional cartoon face model, screening at least one vertex from the similar face three-dimensional point cloud model according to the distance value between each vertex in the similar face three-dimensional point cloud model and each vertex, and establishing a corresponding relation between each vertex and the screened at least one vertex.

Optionally, the human-face-like three-dimensional point cloud model and the three-dimensional cartoon human face model both include a plurality of feature areas, and each feature area includes at least one vertex; the index determining module is specifically configured to:

and aiming at each characteristic region, performing non-rigid registration on the vertex in each characteristic region in the three-dimensional cartoon face model and the vertex in each characteristic region in the human face-like three-dimensional point cloud model so as to attach the vertex in the three-dimensional cartoon face model to the human face-like three-dimensional point cloud model.

Optionally, the index determining module is specifically configured to:

and according to the distance value between each vertex and each vertex in the face-like three-dimensional point cloud model, screening the vertex closest to each vertex from the face-like three-dimensional point cloud model, and establishing a corresponding relation between each vertex and the closest vertex.

Optionally, the apparatus further comprises a posture adjustment module configured to:

determining a pose scaling parameter between the human face-like three-dimensional point cloud model and the three-dimensional cartoon human face model;

and adjusting the pose and the size of the three-dimensional cartoon face based on the pose scaling parameter.

Optionally, the face-like three-dimensional point cloud model generation module is specifically configured to:

carrying out face recognition on a face image containing a target face to obtain feature points of each feature part in the target face;

acquiring three-dimensional feature points for representing each feature part in the target face based on the feature points of each feature part in the target face;

and obtaining a human face-like three-dimensional point cloud model according to the obtained three-dimensional feature points of each feature part in the target human face.

Optionally, the face-like three-dimensional point cloud model generation module is specifically configured to:

fitting to obtain corresponding expression coefficients and shape coefficients based on the obtained three-dimensional feature points of each feature part in the target face and a human-like deformable template in a human-like deformable template library;

and obtaining a human face-like three-dimensional point cloud model corresponding to the human face image based on the expression coefficient and the shape coefficient obtained by fitting and the human face-like deformable template library.

In one aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods when executing the computer program.

In one aspect, an embodiment of the present application provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of any of the above-described methods.

In one aspect, an embodiment of the present application provides a computer program product or a computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in any of the various alternative implementations of control of TCP transmission performance described above.

According to the three-dimensional cartoon face generation method, the three-dimensional cartoon face generation device, the electronic equipment and the storage medium, the high-precision face-like three-dimensional point cloud model which is highly similar to a real face is reconstructed based on the face image, then the reconstruction of the face three-dimensional cartoon face model is carried out based on the topological structure of the face three-dimensional point cloud model, the precision of the three-dimensional cartoon face obtained through final fitting is improved, the detail characteristics of more real faces can be reserved, the generated three-dimensional cartoon face is highly similar to the real face in the image, and the discrimination and the recognition degree between the three-dimensional cartoon faces generated based on different face images are improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic view of an application scene of a three-dimensional cartoon face generation method provided in an embodiment of the present application;

fig. 2 is a schematic flow chart of a three-dimensional cartoon face generation method according to an embodiment of the present application;

FIG. 3A illustrates a portion of face-like deformable templates in a bilinear face-like deformable template library according to an embodiment of the present application;

fig. 3B is a schematic diagram of a face-like three-dimensional point cloud model according to an embodiment of the present application;

fig. 4 is a schematic flow chart of generating a corresponding human-face-like three-dimensional point cloud model based on a human face image according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating a process of establishing a vertex index relationship according to an embodiment of the present application;

fig. 6 is an effect diagram of a human-like three-dimensional point cloud model and a three-dimensional cartoon human face model after point cloud preliminary alignment according to an embodiment of the present application;

fig. 7 is a schematic diagram of vertex distribution of feature areas in a human-like three-dimensional point cloud model and a three-dimensional cartoon human face model according to an embodiment of the present application;

fig. 8A is a schematic diagram of a process of generating a three-dimensional cartoon face based on a face image according to an embodiment of the present application;

fig. 8B is a schematic diagram of a three-dimensional model generated in a process of generating a three-dimensional cartoon face based on a face image according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a three-dimensional cartoon face generation device according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

For convenience of understanding, terms referred to in the embodiments of the present application are explained below:

artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Computer Vision technology (CV) Computer Vision is a science for researching how to make a machine "see", and further refers to that a camera and a Computer are used to replace human eyes to perform machine Vision such as identification, tracking and measurement on a target, and further image processing is performed, so that the Computer processing becomes an image more suitable for human eyes to observe or transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technologies, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also include common biometric technologies such as face recognition and fingerprint recognition.

3 DMM: the three-dimensional face Model is a 3d Morph Model, is a three-dimensional face deformation statistical Model, is a universal three-dimensional face Model, expresses a face by using fixed points, and is used for solving the problem of recovering a three-dimensional shape from a two-dimensional face image. The core idea of 3DMM is that faces can be matched one by one in a three-dimensional space and can be obtained by the weighted linear addition of orthogonal bases of many other faces, that is, each three-dimensional face can be represented in a base vector space composed of a plurality of face-like deformable templates in a 3DMM database, and the solution of a model of any three-dimensional face is practically equivalent to the solution of coefficients of each base vector.

A human-face-like three-dimensional point cloud model: the embodiment of the application refers to a three-dimensional model similar to a real face, generally refers to a three-dimensional face model with a large number of vertexes, and a three-dimensional point cloud model with high resolution and approximate to the real face can be reconstructed according to the face model.

The three-dimensional cartoon face model comprises: the three-dimensional cartoon face model is generally a three-dimensional model with a small number of vertexes, and the three-dimensional cartoon face model can be used for reconstructing the anthropomorphic three-dimensional cartoon face.

Non-rigid registration: english is called Non-edge Registration, also called Non-edge ICP, or NRICP for short, and is used in the fields of remote sensing data analysis, computer vision, image processing and the like. In the field of image processing, two or more images acquired at different times, different sensors (imaging devices) or under different conditions (weather, illuminance, camera position and angle, etc.) are matched and superimposed. In the field of computer vision, two or more three-dimensional point cloud models are matched and superposed. The open source algorithm is known as "Optimal Step nonlinear ICP Algorithms for Surface Registration".

Any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.

In a specific practice process, the automatic generation of the three-dimensional cartoon face mainly comprises the steps of extracting face features from a face image by a computer, and carrying out exaggeration on the face features to automatically generate a three-dimensional cartoon image of the face, wherein the generated three-dimensional cartoon image has low precision and cannot accurately show detail features of the face, so that the error between an expression base and a real face is large and is not vivid enough.

Therefore, the embodiment of the application provides a method for generating a three-dimensional cartoon face, which comprises the following specific processes: generating a corresponding human face-like three-dimensional point cloud model based on the human face image; determining the coordinates of each vertex in the three-dimensional cartoon face model based on the coordinates of each vertex in the similar face three-dimensional point cloud model; and fitting to obtain the three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model. The method comprises the steps of firstly constructing a high-precision human face-like three-dimensional point cloud model based on a human face image to obtain coordinates of each vertex in the human face-like three-dimensional point cloud model corresponding to a real human face, then determining the coordinates of each vertex in the three-dimensional cartoon human face model based on the coordinates of each vertex in the human face-like three-dimensional point cloud model to enable the three-dimensional cartoon human face model to be more fit with the real human face, and finally fitting to obtain the three-dimensional cartoon human face based on the coordinates of each vertex in the three-dimensional cartoon human face model. Compared with a mode of generating the three-dimensional cartoon face directly based on the face image, the three-dimensional cartoon face model is constructed based on the high-precision face-like three-dimensional point cloud model, the precision of the three-dimensional cartoon face obtained through final fitting can be improved, more detailed characteristics such as the expression and the shape of the real face can be reserved, the similarity between the generated three-dimensional cartoon face and the real face in the image is higher, and the discrimination and the recognition degree between the three-dimensional cartoon faces generated based on different face images are improved.

After introducing the design concept of the embodiment of the present application, some simple descriptions are provided below for application scenarios to which the technical solution of the embodiment of the present application can be applied, and it should be noted that the application scenarios described below are only used for describing the embodiment of the present application and are not limited. In specific implementation, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.

Reference is made to fig. 1, which is a schematic view of an application scenario of the three-dimensional cartoon face generation method provided in the embodiment of the present application. The application scenario includes aterminal device 101 and aserver 102. Theterminal device 101 and theserver 102 are connected via a wireless or wired network, and theterminal device 101 includes, but is not limited to, a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a smart television, and other electronic devices. Theserver 102 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like.

Theterminal device 101 may acquire a face image of a user through a camera and send the face image to theserver 102, or may send a face image shot in advance to theserver 102. A 3d media motion model (DMM) model library is built in theserver 102, and the 3D media motion model library comprises a similar human face deformable template and a cartoon human face deformable template. Theserver 102 generates a corresponding human-face-like three-dimensional point cloud model based on the human face image and the human-face-like deformable templates in the 3d MM model library, namely, coordinates of each vertex in the human-face-like three-dimensional point cloud model corresponding to the human face image can be determined, then coordinates of each vertex in the three-dimensional cartoon human face model are determined based on the coordinates of each vertex in the human-face-like three-dimensional point cloud model, finally, a three-dimensional cartoon human face corresponding to the human face image is obtained through fitting based on the coordinates of each vertex in the three-dimensional cartoon human face model and the plurality of cartoon human-face deformable templates in the 3D MM model library, and the three-dimensional cartoon human face obtained through fitting is. Theterminal device 101 renders the three-dimensional cartoon face through the rendering engine and displays the three-dimensional cartoon face to the user.

The three-dimensional cartoon face generation method provided by the embodiment of the application can be applied to various application scenes suitable for virtual images, such as news broadcasting, weather forecast, virtual assistants, intelligent customer service, game explanation, game scenes, product release meetings and the like, is used for constructing the three-dimensional cartoon image which is the same as the face of a user, and meets the more personalized requirements of the user. Taking a virtual assistant scene as an example, a user can send a picture of the user or a picture of a friend to a server, the server can generate a three-dimensional cartoon face similar to the face in the picture, and the three-dimensional cartoon face is used as a virtual assistant to increase the intimacy brought to the user by the virtual assistant. Taking a game scene as an example, a three-dimensional cartoon face similar to a user can be generated based on a photo of the user, so that each user in the game can have a unique three-dimensional cartoon image with identification, and the individual requirements of the user are met.

Of course, the method provided in the embodiment of the present application is not limited to be used in the application scenario shown in fig. 1, and may also be used in other possible application scenarios, and the embodiment of the present application is not limited. The functions that can be implemented by each device in the application scenario shown in fig. 1 will be described in the following method embodiments, and will not be described in detail herein.

To further illustrate the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the accompanying drawings and the detailed description. Although the embodiments of the present application provide the method operation steps as shown in the following embodiments or figures, more or less operation steps may be included in the method based on the conventional or non-inventive labor. In steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by the embodiments of the present application.

The scheme provided by the embodiment of the application relates to the technologies of artificial intelligence, such as computer vision, three-dimensional object reconstruction and the like, and is specifically explained by the following embodiments in combination with the application scene shown in fig. 1:

referring to fig. 2, an embodiment of the present application provides a three-dimensional cartoon face generation method, which can be applied to a terminal device or a server, and includes the following steps:

s201, generating a corresponding human face-like three-dimensional point cloud model based on the human face image.

The face image is a two-dimensional face photo, the face in the face image can be in any posture and any expression, and only the face is required to be clearly visible, so that reliable face features can be detected. In practical application, the face image can be a front face picture shot under the condition of good light, so that a three-dimensional image reconstructed by the two-dimensional image can be guaranteed to be more like a target face and has more authenticity.

The human-face-like three-dimensional point cloud model is a three-dimensional human face point cloud model highly similar to a real human face, the real human face is restored through point cloud formed by a large number of vertexes, the more the vertexes are, the higher the precision of the three-dimensional human face point cloud model is, for example, the human-face-like three-dimensional point cloud model shown in fig. 3B approximately comprises 2 ten thousand vertexes. Each vertex in the human face-like three-dimensional point cloud model has a unique number, human faces with different shapes and expressions can be simulated by adjusting the coordinates of the vertices, and once the coordinates of all the vertices in the human face-like three-dimensional point cloud model are determined, the shapes and the expressions of the three-dimensional human faces can be determined. The whole human face three-dimensional point cloud model can be divided into a plurality of characteristic regions, such as an eye region, a nose region, a mouth region, a jaw region and the like, each characteristic region comprises a plurality of vertexes, five sense organs in different shapes and five sense organs corresponding to different expressions are simulated based on the vertexes in each characteristic region, for example, the vertexes of the eye region are used for simulating the shape of eyes, and the vertexes of the nose region are used for simulating the shape of a nose.

In specific implementation, a human-face-like three-dimensional point cloud model corresponding to the human face image can be generated based on a human-face-like deformable template in a 3DMM model library. The human face-like deformable template library comprises an expression base and a shape base, and can be in a linear or bilinear form, fig. 3A shows a typical bilinear human face-like deformable template library, the human face-like deformable template library is obtained based on massive human face statistics, each line in the human face-like deformable template library represents a human face shape (namely a shape base), the shape base describes inherent appearance characteristics of different human faces, such as a long face, a square face, a round face, a high nose bridge, a red-phoenix eye and the like, and m shape bases exist, so that m lines exist. Each column in the human-like face deformable template library represents a human face expression (namely an expression base), the expression base describes the state of the same human face under different expressions, such as surprise, smile, laugh and the like, and n expression bases exist so that n columns exist. Based on the similar-face deformable template library shown in fig. 3A, faces with arbitrary expressions and arbitrary shapes can be represented parametrically by using the similar-face deformable template library, and the specific representation mode is as formula (1):

M＝exp×Cr×id (1)

the method comprises the steps of obtaining a face-like deformable template library, determining the size of Cr, and obtaining a face-like three-dimensional point cloud model corresponding to the face, wherein the Cr refers to the face-like deformable template library, the dimension of Cr is n multiplied by k multiplied by M, wherein k is the number of vertexes contained in a face-like deformable template, n is the number of expression bases contained in Cr, M is the number of shape bases contained in Cr, exp is an expression coefficient, the dimension is 1 multiplied by n, id is a shape coefficient, the dimension is M multiplied by 1, and M. Wherein, the expression coefficient is used for controlling the expression of the generated human face, and the shape coefficient is used for controlling the long-phase of the generated human face. Based on the formula (1), it can be seen that each face deformable template in the face-like deformable template library can be fused based on the expression coefficient exp and the shape coefficient id, so as to obtain a face-like three-dimensional point cloud model corresponding to any one face when any one expression is made.

In specific implementation, the expression coefficient and the shape coefficient corresponding to the face image can be determined based on an artificial intelligence mode. For example, a face image is input into a trained neural network model, the neural network model can extract deep features of the face image, and obtain an expression coefficient and a shape coefficient based on the extracted deep features, and then, a high-precision human-face-like three-dimensional point cloud model corresponding to the face image can be obtained based on the expression coefficient and the shape coefficient output by the neural network model and a human-face-like deformable template in a 3d mm model library, as shown in fig. 3B.

S202, determining a target vertex corresponding to each vertex in the three-dimensional cartoon face model in the human-like three-dimensional point cloud model according to a vertex index relation, wherein the vertex index relation comprises a corresponding relation between the vertex in the human-like three-dimensional point cloud model and the vertex in the three-dimensional cartoon face model.

The three-dimensional cartoon face model is similar to a similar face three-dimensional point cloud model, the point cloud formed by a large number of vertexes simulates three-dimensional cartoon faces with various expressions and shapes, and the more the vertexes are, the higher the precision of the three-dimensional cartoon face model is. In specific implementation, the number of vertexes of the human-face-like three-dimensional point cloud model is generally larger than that of the three-dimensional cartoon human face model, a real human face is reconstructed through the high-precision human-face-like three-dimensional point cloud model, and then the high-precision three-dimensional cartoon human face model is obtained based on the high-precision human-face-like three-dimensional point cloud model, so that the precision of the finally generated three-dimensional cartoon human face is improved. Similarly, the whole three-dimensional cartoon face model can also be divided into a plurality of characteristic areas, the dividing mode and the number of the characteristic areas are consistent with the human-face-like three-dimensional point cloud model, such as an eye area, a nose area, a mouth area, a chin area and the like, and each characteristic area comprises a plurality of vertexes.

The vertex index relationship comprises the corresponding relationship between the vertexes in the human-face-like three-dimensional point cloud model and the vertexes in the three-dimensional cartoon human face model, and the vertex index relationship can be predetermined. Specifically, one vertex in the three-dimensional cartoon face model corresponds to one vertex in the similar face three-dimensional point cloud model, for example, a vertex a in the similar face three-dimensional point cloud model corresponds to a vertex a' in the three-dimensional cartoon face model; alternatively, because the number of vertices of the face-like three-dimensional point cloud model is greater than the number of vertices in the three-dimensional cartoon face model, one vertex in the three-dimensional cartoon face model may correspond to multiple vertices in the face-like three-dimensional point cloud model at the same time, for example, vertex B, C, D in the face-like three-dimensional point cloud model corresponds to vertex B' in the three-dimensional cartoon face model. Based on the predetermined vertex index relationship, the coordinates of each vertex of the three-dimensional cartoon face model corresponding to the face can be rapidly determined, and the processing efficiency is improved.

S203, determining coordinates corresponding to each vertex in the three-dimensional cartoon face model based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model.

When the three-dimensional cartoon face model is implemented specifically, when one vertex in the three-dimensional cartoon face model corresponds to one vertex in the similar face three-dimensional point cloud model, the coordinates of each vertex in the three-dimensional cartoon face model can be directly updated to the coordinates of the corresponding target vertex. For example, the vertex a in the face-like three-dimensional point cloud model corresponds to the vertex a' in the three-dimensional cartoon face model, if the coordinates of the vertex a are (x)_A,y_A,z_A) Then the coordinates of vertex A' are updated to (x)_A,y_A,z_A)。

When the method is concretely implemented, when one vertex in the three-dimensional cartoon face model can simultaneously correspond to a plurality of vertexes in the human face-like three-dimensional point cloud model, the plurality of vertexes can be determined as target vertexes corresponding to the vertexes in the three-dimensional cartoon face model, and the coordinates of the vertexes in the three-dimensional cartoon face model are determined based on the coordinates of the plurality of target vertexes. Specifically, the coordinates of the multiple target vertices may be weighted and summed, and the weighted and summed result is used as the coordinates of the vertex in the three-dimensional cartoon face model. The target vertexes corresponding to the same vertex in the three-dimensional cartoon face model are vertexes with relatively close spatial positions, and the coordinate precision of each vertex in the three-dimensional cartoon face model can be improved by obtaining the coordinate of each vertex in the three-dimensional cartoon face model through weighted summation of the coordinates of the target vertexes.

And S204, fitting to obtain a three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

In specific implementation, the 3DMM model library further includes a cartoon face deformable template library, the cartoon face deformable template library includes a plurality of cartoon expression bases, each cartoon expression base corresponds to a three-dimensional cartoon face template of one expression, the cartoon expression base coefficients corresponding to each cartoon expression base can be adjusted, and the plurality of cartoon expression bases are weighted and summed based on the cartoon expression base coefficients to obtain different three-dimensional cartoon faces, for example, the cartoon face deformable template library includes 69 different cartoon expression bases, and then different three-dimensional cartoon faces can be linearly expressed by combining 69 different cartoon expression base coefficients, and a specific formula is as follows:

wherein, b_i(i-1, …, u) represents different cartoon expression bases in the cartoon face deformable template library, p_iIs b_iAnd (4) corresponding cartoon expression base coefficients, wherein u is the number of cartoon expression bases in the cartoon face deformable template library, and T is the three-dimensional cartoon face corresponding to the face. Based on the above formula (2), it can be seen that the expression base coefficient p can be based on cartoon expression_iAnd fusing all three-dimensional cartoon face templates in the cartoon face deformable template library to obtain a three-dimensional cartoon face corresponding to any one face.

In specific implementation, step S204 specifically includes: fitting to obtain cartoon expression base coefficients respectively corresponding to the cartoon expression bases based on the coordinates of each vertex in the three-dimensional cartoon face model, wherein each cartoon expression base corresponds to a three-dimensional cartoon face template with one expression; and carrying out weighted summation on each cartoon expression base based on the cartoon expression base coefficient respectively corresponding to each cartoon expression base so as to obtain the three-dimensional cartoon face corresponding to the face image.

Specifically, the cartoon expression base coefficients corresponding to the cartoon expression bases in the cartoon face deformable template library can be obtained through fitting according to the following formula:

wherein A is_ktIs the coordinate of the vertex in the three-dimensional cartoon face model, b_i(i＝1, …, u) represents different cartoon expression bases in a cartoon face deformable template library, p_iIs b_iThe corresponding cartoon expression base coefficients, alpha and beta, are preset weighting coefficients respectively, and alpha and beta can be set according to actual requirements without limitation. The method comprises the steps of finding out a proper cartoon expression base coefficient through the fitting formula (3), and obtaining a three-dimensional cartoon face based on the formula (2), so that the three-dimensional cartoon face obtained based on the cartoon expression base coefficient and the cartoon expression base approaches to a three-dimensional cartoon face model, and is further more fit with a real face in a face image.

According to the three-dimensional cartoon face generation method provided by the embodiment of the application, the high-precision face-like three-dimensional point cloud model which is highly similar to the real face is reconstructed based on the face image, then the reconstruction of the face three-dimensional cartoon face model is carried out based on the topological structure of the face three-dimensional point cloud model, the precision of the three-dimensional cartoon face obtained through final fitting is improved, and the detail characteristics of more real faces can be reserved, so that the generated three-dimensional cartoon face is highly similar to the real face in the image, and the discrimination and the recognition degree between the three-dimensional cartoon faces generated based on different face images are improved.

Referring to fig. 4, in specific implementation, a corresponding human-face-like three-dimensional point cloud model may be generated based on a human face image in the following manner:

s401, carrying out face recognition on the face image containing the target face to obtain the feature points of each feature part in the target face.

Specifically, the face recognition SDK provided by the Tencent optimal graph can be used to perform face recognition and facial feature positioning on the face image, and feature points used for representing each feature part in the target face are extracted. The features available in the face recognition technology are generally classified into visual features, pixel statistical features, face image transformation coefficient features, face image algebraic features and the like, the face recognition technology has a feature extraction function, and the face feature extraction is to extract some features of a face, so that feature points of each feature part in the face can be obtained. The characteristic points in the embodiments of the present application include, but are not limited to, including: the characteristic points are used for representing characteristic parts such as face contour, eye contour, nose, lips and the like.

S402, obtaining three-dimensional feature points for representing each feature part in the target face based on the feature points of each feature part in the target face.

After the feature points for representing each feature part in the target face are obtained, since three-dimensional reconstruction is performed, 3D fitting processing needs to be performed on each feature point to obtain a three-dimensional feature point corresponding to the feature point, which is equivalent to adding a depth value to a two-dimensional image. The resulting three-dimensional feature points can be represented as: (x, y, z), wherein x represents the abscissa value of the pixel point corresponding to the three-dimensional feature point; y is an abscissa value of the ordinate corresponding to the three-dimensional feature point; z is expressed as a depth value of the three-dimensional feature point. Wherein x and y are the same as x and y values of the feature points obtained based on the two-dimensional image.

Specifically, feature points of each feature part in the target face obtained based on the face recognition technology are subjected to 3D fitting processing according to a standard face model and a constraint relationship between the feature points to obtain three-dimensional feature points corresponding to the target face. For example, 83 feature points are obtained based on a face recognition technology, and 1000 three-dimensional feature points on the target face can be calculated after 3D fitting is performed according to a standard face model and a constraint relationship between the 83 feature points.

And S403, obtaining a face-like three-dimensional point cloud model according to the obtained three-dimensional feature points of each feature part in the target face.

During specific implementation, the corresponding expression coefficients and shape coefficients can be obtained through fitting based on the obtained three-dimensional feature points of the feature parts in the target face and the similar face deformable template, and the similar face three-dimensional point cloud model corresponding to the face image is obtained based on the expression coefficients and shape coefficients obtained through fitting and the similar face deformable template library. Specifically, the formula A can be used_TPerforming three-dimensional fitting reconstruction on each characteristic part, wherein A is multiplied by Cr multiplied by id_TObtaining the expression coefficient and the shape coefficient corresponding to each characteristic part for the three-dimensional characteristic points of each characteristic part; fusing each characteristic partRespectively obtaining corresponding expression coefficients exp of the target face; fusing the shape coefficients id corresponding to each characteristic part to obtain the shape coefficient corresponding to the target face; and finally, obtaining a human face-like three-dimensional point cloud model with the expression of the target human face in the human face image based on the formula (1).

In practical application, a vertex index relationship between the human-face-like three-dimensional point cloud model and the three-dimensional cartoon human face model can be established in advance, and once the vertex index relationship is determined, the human-face-like three-dimensional point cloud model of a user can be directly and quickly converted into a corresponding three-dimensional cartoon human face model according to the vertex index relationship subsequently aiming at the human face image of any user.

Referring to fig. 5, it shows a flowchart of a method for establishing a vertex index relationship, which specifically includes the following steps:

s501, performing point cloud preliminary alignment on the similar-face three-dimensional point cloud model and the three-dimensional cartoon face model.

The purpose of the point cloud initial alignment is to convert the vertexes of the human face-like three-dimensional point cloud model and the vertexes of the three-dimensional cartoon human face model into the same coordinate system, and the two vertexes are not originally in the same coordinate system and are converted into the same coordinate system so as to facilitate subsequent calculation.

Specifically, the face-like three-dimensional point cloud model and the three-dimensional cartoon face model can be preliminarily aligned according to the feature area to which the vertexes in the face-like three-dimensional point cloud model belong and the feature area to which the vertexes in the three-dimensional cartoon face model belong, for example, the eye area in the face-like three-dimensional point cloud model is aligned with the eye area in the three-dimensional cartoon face model, and the nose area in the face three-dimensional point cloud model is aligned with the nose area in the three-dimensional cartoon face model.

Specifically, the spatial position conversion relationship between the vertexes of the three-dimensional point cloud model of the human face and the vertexes of the three-dimensional cartoon human face model can be represented by the following formula:

A_kt＝s×(R|T)A_h

wherein A is_ktIs the vertex of the three-dimensional cartoon face model, A_hIs face-like three-dimensionalThe vertexes of the point cloud model, s, R, T, are the scaling, rotation matrix and translation matrix to be calculated, i.e. the pose scaling parameters.

It should be noted that, because the vertices of the face-like three-dimensional point cloud model and the vertices of the three-dimensional cartoon face model are not the same, the vertices in the two models cannot be completely aligned, i.e., completely coincide in space. Illustratively, as shown in fig. 6, which shows an effect diagram after the point cloud preliminary alignment of the human-face-like three-dimensionalpoint cloud model 61 and the three-dimensional cartoonhuman face model 62, it can be seen that although the human-face-like three-dimensionalpoint cloud model 61 and the three-dimensional cartoonhuman face model 62 are not completely overlapped, the two models have been transformed to approximately aligned positions.

And S502, attaching the vertex of the three-dimensional cartoon face model to the vertex of the human-like three-dimensional point cloud model to obtain an updated three-dimensional cartoon face model.

The human face-like three-dimensional point cloud model is a three-dimensional point cloud model with a large number of vertexes, and the three-dimensional cartoon human face model is a three-dimensional point cloud model with a small number of vertexes. The three-dimensional cartoon face model obtained in step S502 is a three-dimensional point cloud model of a target face in a long-length image of the face.

In specific implementation, a Non-rigid Registration (NRICP) mode may be adopted to attach the vertices of the three-dimensional cartoon face model to the vertices of the human-like three-dimensional point cloud model, and in the above process, the coordinates of each vertex in the three-dimensional cartoon face model may change. After one-time non-rigid registration is performed, the point cloud alignment and the non-rigid registration in the steps S501 and S502 can be performed again by using the coordinates of the vertices of the newly generated three-dimensional cartoon face model, and after iteration is performed for several times, a three-dimensional cartoon face model satisfying the topological structure of the 3d dm library is obtained.

It should be noted that, in the above iterative process, the calculation may be performed based on only a number of key vertices, and not all the vertices participate in the calculation, so as to improve the calculation efficiency. However, the more vertexes participating in calculation, the higher similarity between the finally generated three-dimensional cartoon face model and the similar face three-dimensional point cloud model, and in practical application, balance needs to be made between the two models.

Further, in order to improve the accuracy of the vertex coordinates of the obtained three-dimensional cartoon face model, registration may be performed in regions, that is: and aiming at each characteristic region, performing non-rigid registration on the vertex in each characteristic region in the three-dimensional cartoon face model and the vertex in each characteristic region in the human-face-like three-dimensional point cloud model so as to attach the vertex in the three-dimensional cartoon face model to the human-face-like three-dimensional point cloud model. Taking the eye area as an example, non-rigid registration is carried out on the vertexes in the eye area in the three-dimensional cartoon face model and the vertexes in the eye area in the human-face-like three-dimensional point cloud model, so that the vertexes in the eye area in the three-dimensional cartoon face model are attached to the eye area of the human-face-like three-dimensional point cloud model.

S503, aiming at each vertex in the updated three-dimensional cartoon face model, at least one vertex is screened out from the similar face three-dimensional point cloud model according to the distance value between each vertex and the vertex in the similar face three-dimensional point cloud model, and a corresponding relation is established between the vertex and the screened at least one vertex.

For convenience of description, the vertex in the updated three-dimensional cartoon face model is subsequently used as a first vertex, and the vertex of the face-like three-dimensional point cloud model is used as a second vertex.

In specific implementation, aiming at each vertex in the updated three-dimensional cartoon face model, the vertex closest to each vertex can be screened from the similar face three-dimensional point cloud model according to the distance value between each vertex and each vertex in the similar face three-dimensional point cloud model, and the corresponding relation is established between each vertex and the closest vertex. For example, for a first vertex in the updated three-dimensional cartoon face model, distance values between the first vertex and a plurality of second vertices located around the first vertex can be calculated, a minimum distance value is screened from the plurality of distance values, the second vertex corresponding to the minimum distance value is a vertex closest to the first vertex, and a corresponding relationship is established for the first vertex and the corresponding closest vertex. Specifically, a second vertex closest to the first vertex can be determined from the face-like three-dimensional point cloud model through nearest neighbor point search.

In specific implementation, for each vertex in the updated three-dimensional cartoon face model, the first N distance values can be selected according to the distance values between each vertex and each vertex in the face-like three-dimensional point cloud model and the sequence of the distance values from small to large, and the corresponding relation is established for each vertex and the vertex corresponding to the selected first N distance values. Wherein, N can be set according to the actual application scene. For example, for a first vertex in the updated three-dimensional cartoon face model, distance values between the first vertex and a plurality of second vertices located around the first vertex can be calculated, the first N distance values are selected according to the order of the distance values from small to large, N second vertices corresponding to the first N distance values are determined, and a corresponding relationship is established for the first vertex and the N second vertices.

Further, the vertex index relationship can be established by regions. For example, a vertex a in the human-face-like three-dimensional point cloud model corresponds to a vertex a 'in the three-dimensional cartoon human face model, and a vertex B in the eye region in the human-face-like three-dimensional point cloud model corresponds to a vertex B' in the eye region in the three-dimensional cartoon human face model. Alternatively, one vertex in a certain feature region in the three-dimensional cartoon face model may correspond to multiple vertices in the corresponding feature region in the similar face three-dimensional point cloud model at the same time, for example, multiple vertices in an eye region in the similar face three-dimensional point cloud model correspond to a certain vertex in an eye region in the three-dimensional cartoon face model, and a vertex B in a region in the similar face three-dimensional point cloud model corresponds to a vertex B' in an eye region in the three-dimensional cartoon face model. For this reason, S503 specifically includes: aiming at each first vertex in the updated three-dimensional cartoon face model, at least one vertex is screened out from the candidate vertices according to the distance value between the candidate vertices and the first vertex in the human-like three-dimensional point cloud model, and a corresponding relation is established between the first vertex and the screened out at least one vertex, wherein the candidate vertices are vertices in a target characteristic region in the human-like three-dimensional point cloud model, and the target characteristic region is a characteristic region formed by the first vertex.

Referring to fig. 7, the first row of 4 images respectively shows the vertices of the eyes, nose, mouth and other areas in the human-face-like three-dimensional point cloud model, and the second row of 4 images respectively shows the vertices of the eyes, nose, mouth and other areas in the three-dimensional cartoon face model. The human face three-dimensional cartoon human face model and the human face three-dimensional point cloud model can be pre-aligned; then aiming at each characteristic region, carrying out non-rigid registration on the top point in each characteristic region in the three-dimensional cartoon face model and the top point in each characteristic region in the human face-like three-dimensional point cloud model, and matching each top point of each characteristic region in the three-dimensional cartoon face model to the corresponding characteristic region on the human face-like three-dimensional point cloud model after a plurality of iterations of alignment and non-rigid registration so as to accurately attach the three-dimensional cartoon face model to the human face-like three-dimensional point cloud model; and finally, aiming at each vertex in each characteristic area in the three-dimensional cartoon face model, determining a nearest neighbor point which is nearest to the vertex from the vertex of the corresponding characteristic area in the human-face-like three-dimensional point cloud model through nearest neighbor point search, and updating the coordinates of the vertex in the three-dimensional cartoon face model into the coordinates of the nearest neighbor point. By the method, the corresponding relation between the human face three-dimensional cartoon human face model and the vertexes of the eyes, the nose, the mouth, other areas and the like of the human face three-dimensional point cloud model can be obtained, and the vertex index relation of each characteristic area can be obtained.

On the basis of any one of the above embodiments, when the cartoon expression basis coefficients are fitted based on the three-dimensional cartoon face model, fitting can be performed in regions to improve fitting accuracy. Specifically, for each feature area, fitting to obtain local cartoon expression base coefficients corresponding to the cartoon expression bases respectively based on coordinates of vertexes in each feature area in the three-dimensional cartoon face model; and for each cartoon expression base, fusing local cartoon expression base coefficients of each cartoon expression base corresponding to the plurality of characteristic areas respectively to obtain the cartoon expression base coefficient corresponding to each cartoon expression base.

Taking the eye area as an example, the local cartoon expression basis coefficient of each cartoon expression basis corresponding to the eye area can be obtained by fitting the following formula:

wherein A is_kt,eyeThe coordinates of the vertex of the eye region in the three-dimensional cartoon face model, and the meaning of other parameters refer to formula (3).

Respectively fitting the above formulas to obtain local cartoon expression basis coefficients of each feature region in the three-dimensional cartoon face model, wherein the local cartoon expression basis coefficients of the eye regions are P₁＝(p₁，p₂，…p_u) The local cartoon expression base coefficient of the nose area is P₂＝(p′₁，p′₂，…p′_u) The local cartoon expression base coefficient of the mouth area is P₃＝(p″₁，p″₂，…p″_u) The local cartoon expression base coefficient of the chin area is P₄＝(p″′₁，p″′₂，…p″′_u) Then, carrying out weighted summation or averaging on the local cartoon expression base coefficients of all the characteristic areas to obtain the cartoon expression base coefficient corresponding to each cartoon expression base in the three-dimensional cartoon face model, such as the cartoon expression base coefficient

Wherein x is the number of characteristic areas contained in the three-dimensional cartoon face model and the cartoon expression base coefficient P is used₀Multiplying by C_ktAnd a more vivid three-dimensional cartoon face can be obtained. In addition, the user can adjust the cartoon expression base coefficient P₀And further adjusting the generated three-dimensional cartoon face.

On the basis of any one of the above embodiments, the method for generating the three-dimensional cartoon face according to the embodiment of the application further includes the following steps: determining a pose scaling parameter between the human face-like three-dimensional point cloud model and the three-dimensional cartoon human face model; and adjusting the pose and the size of the three-dimensional cartoon face based on the pose scaling parameter.

In specific implementation, the human face three-dimensional cartoon human face model and the human face three-dimensional point cloud model can be aligned firstly based on the predetermined vertex index relationship, for example, the alignment precision is improved, and the alignment can be performed based on the predetermined vertex index relationship; and then, calculating a pose scaling parameter between the two human face three-dimensional cartoon face models and the similar human face three-dimensional Point cloud model directly in a least square mode, and further optimizing a position conversion relation between the two models through an ICP (Iterative Closest Point algorithm), namely optimizing the pose scaling parameter. Specifically, the pose scaling parameter may be calculated by the following formula:

A_kt＝s×(R|T)A_h

wherein A is_ktIs the vertex of the three-dimensional cartoon face model, A_hThe vertex of the human face-like three-dimensional point cloud model is shown, and s, R and T are scaling, rotation matrix and translation matrix to be calculated, namely pose scaling parameters. And adjusting the final displayed size, angle, position and the like of the three-dimensional cartoon face through the attitude scaling parameter to ensure that the displayed three-dimensional cartoon face is consistent with the height of the face image.

Referring to fig. 8A, first, based on a real face image a and a similar face deformable template library in the 3DMM model library, a similar face point cloud model is reconstructed (corresponding to step S201), so as to obtain a similar face three-dimensional point cloud model B highly similar to a face in the face image a, as shown in fig. 8B. Then, the similar-face three-dimensional point cloud model B and the three-dimensional cartoon model are aligned, non-rigidly registered and the like, and coordinates of each vertex of the three-dimensional cartoon model are determined based on a vertex index relationship between the similar-face three-dimensional model and the three-dimensional cartoon model obtained in advance and coordinates of each vertex in the similar-face three-dimensional point cloud model B, so that a three-dimensional cartoon model c is obtained (corresponding to step S202), as shown in fig. 8B. And for each characteristic region, fitting to obtain local cartoon expression base coefficients corresponding to each cartoon expression base based on the coordinates of the vertex in each characteristic region in the three-dimensional cartoon face model c, and respectively calculating the cartoon expression base coefficients of the regions such as eyes, nose, mouth, chin and the like. And for each cartoon expression base, fusing local cartoon expression base coefficients of each cartoon expression base corresponding to the areas such as eyes, nose, mouth, chin and the like to obtain the cartoon expression base coefficient corresponding to each cartoon expression base. And obtaining a three-dimensional cartoon face d based on the cartoon expression base coefficient and the cartoon expression base, wherein the three-dimensional cartoon face d meets the topological structure of the 3DMM library and can be driven as shown in FIG. 8B. Then, inputting the three-dimensional cartoon face d and the specified cartoon face texture map into a rendering engine, rendering the three-dimensional cartoon face with texture, and presenting the three-dimensional cartoon face to a user.

In practical application, three-dimensional cartoon faces of users under different expressions can be generated by adjusting the cartoon expression base coefficients, then texture rendering is carried out through a rendering engine, the three-dimensional cartoon faces with textures of the users under different expressions can be obtained, the three-dimensional cartoon faces under various expressions are continuously played, and coherent action expressions such as opening mouth, closing mouth, blinking and the like can be obtained.

Taking a virtual assistant as an example, a corresponding drivable three-dimensional cartoon face is generated based on a face image of a user, in the interaction process of the user and the virtual assistant, to-be-played voice information corresponding to the virtual assistant and an expression sequence corresponding to the voice information are determined, the expression sequence comprises at least one expression, wherein a mapping relation between the voice information and the expression sequence can be stored in advance, and after the voice information to be played is determined, the expression sequence corresponding to the voice information can be found according to the mapping relation. For example, the mapping relationship between each pronunciation and the expression may be stored, after the voice information to be played is determined, a pronunciation sequence corresponding to the voice information may be determined, the pronunciation sequence includes at least one pronunciation arranged in sequence, and the expression corresponding to each pronunciation is obtained, that is, the expression sequence corresponding to the voice information may be obtained. And then, for each expression in the expression sequence, generating a three-dimensional cartoon face of the user under each expression according to an expression base coefficient corresponding to each expression, and performing texture rendering through a rendering engine to obtain the three-dimensional cartoon face with textures of the user under different expressions. And finally, in the process of playing the voice information, sequentially displaying the three-dimensional face model with the texture under each expression according to the sequence of each expression contained in the expression sequence. In the process of playing voice information, the three-dimensional face models under the expressions are sequentially displayed according to the sequence of the expressions contained in the expression sequence, so that a virtual assistant highly similar to a user is constructed based on the two-dimensional face image, matched expressions can be displayed according to the voice played in real time, and the voice playing method is more vivid.

As shown in fig. 9, based on the same inventive concept as the three-dimensional cartoon face generation method, the embodiment of the present application further provides a three-dimensional cartoon face generation apparatus 90, which includes a human-like three-dimensional point cloudmodel generation module 901, a three-dimensional cartoon facemodel generation module 902, and afitting output module 903.

A human face-like three-dimensional point cloudmodel generation module 901, configured to generate a corresponding human face-like three-dimensional point cloud model based on a human face image;

the three-dimensional cartoon facemodel generating module 902 is configured to determine, according to a vertex index relationship, a target vertex corresponding to each vertex in the three-dimensional cartoon face model in the face-like three-dimensional point cloud model, and determine, based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model, a coordinate corresponding to each vertex in the three-dimensional cartoon face model, where the vertex index relationship includes a correspondence relationship between a vertex in the face-like three-dimensional point cloud model and a vertex in the three-dimensional cartoon face model;

and thefitting output module 903 is used for fitting to obtain a three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

Optionally, the human-like three-dimensional point cloud model and the three-dimensional cartoon human face model both comprise a plurality of feature areas, and each feature area comprises at least one vertex; thefitting output module 903 is specifically configured to:

for each feature area, fitting to obtain local cartoon expression base coefficients corresponding to the cartoon expression bases respectively based on coordinates of vertexes in each feature area in the three-dimensional cartoon face model;

and for each cartoon expression base, fusing local cartoon expression base coefficients of each cartoon expression base corresponding to the plurality of characteristic areas respectively to obtain the cartoon expression base coefficient corresponding to each cartoon expression base.

Optionally, the three-dimensional cartoon facemodel generating module 902 is specifically configured to:

and updating the coordinates of each vertex in the three-dimensional cartoon face model into the coordinates of the corresponding target vertex.

Optionally, the three-dimensional cartoon face generating apparatus 90 further includes an index determining module, configured to determine the vertex index relationship by:

carrying out point cloud preliminary alignment on the human face-like three-dimensional point cloud model and the three-dimensional cartoon human face model;

attaching the vertex of the three-dimensional cartoon face model to the vertex of the similar face three-dimensional point cloud model to obtain an updated three-dimensional cartoon face model;

and aiming at each vertex in the updated three-dimensional cartoon face model, screening at least one vertex from the similar face three-dimensional point cloud model according to the distance value between each vertex and each vertex in the similar face three-dimensional point cloud model, and establishing a corresponding relation between each vertex and the screened at least one vertex.

Optionally, the human-like three-dimensional point cloud model and the three-dimensional cartoon human face model both comprise a plurality of feature areas, and each feature area comprises at least one vertex; an index determination module specifically configured to:

and aiming at each characteristic region, performing non-rigid registration on the vertex in each characteristic region in the three-dimensional cartoon face model and the vertex in each characteristic region in the human-face-like three-dimensional point cloud model so as to attach the vertex in the three-dimensional cartoon face model to the human-face-like three-dimensional point cloud model.

Optionally, the index determining module is specifically configured to:

and screening vertexes nearest to each vertex from the similar human face three-dimensional point cloud model according to the distance value between each vertex and each vertex in the similar human face three-dimensional point cloud model, and establishing a corresponding relation between each vertex and the nearest vertex.

Optionally, the three-dimensional cartoon face generation apparatus 90 further includes a pose adjustment module, configured to determine a pose scaling parameter between the face-like three-dimensional point cloud model and the three-dimensional cartoon face model; and adjusting the pose and the size of the three-dimensional cartoon face based on the pose scaling parameter.

Optionally, the human-face-like three-dimensional point cloudmodel generating module 901 is specifically configured to:

carrying out face recognition on a face image containing a target face to obtain feature points of each feature part in the target face;

acquiring three-dimensional feature points for representing each feature part in the target face based on the feature points of each feature part in the target face;

and obtaining a human face-like three-dimensional point cloud model according to the obtained three-dimensional feature points of each feature part in the target human face.

Optionally, the human-face-like three-dimensional point cloudmodel generating module 901 is specifically configured to:

fitting to obtain corresponding expression coefficients and shape coefficients based on the obtained three-dimensional feature points of each feature part in the target face and a human-like deformable template in a human-like deformable template library;

and obtaining a human face-like three-dimensional point cloud model corresponding to the human face image based on the expression coefficient and the shape coefficient obtained by fitting and a human face-like deformable template library.

The three-dimensional cartoon face generation device and the three-dimensional cartoon face generation method provided by the embodiment of the application adopt the same inventive concept, can obtain the same beneficial effects, and are not repeated herein.

Based on the same inventive concept as the three-dimensional cartoon face generation method, the embodiment of the present application further provides an electronic device, which may be specifically a desktop computer, a portable computer, a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a server, and the like. As shown in fig. 10, theelectronic device 100 may include aprocessor 1001 and amemory 1002.

TheProcessor 1001 may be a general-purpose Processor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic blocks disclosed in the embodiments of the present Application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

Memory 1002, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charged Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. The memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. Thememory 1002 in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.

The embodiment of the application provides a computer readable storage medium for storing computer program instructions for the electronic device, which contains a program for executing the three-dimensional cartoon face generation method.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; the computer storage media may be any available media or data storage device that can be accessed by a computer, including but not limited to: various media that can store program codes include a removable Memory device, a Random Access Memory (RAM), a magnetic Memory (e.g., a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical Memory (e.g., a CD, a DVD, a BD, an HVD, etc.), and a semiconductor Memory (e.g., a ROM, an EPROM, an EEPROM, a nonvolatile Memory (NAND FLASH), a Solid State Disk (SSD)).

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media that can store program codes include a removable Memory device, a Random Access Memory (RAM), a magnetic Memory (e.g., a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical Memory (e.g., a CD, a DVD, a BD, an HVD, etc.), and a semiconductor Memory (e.g., a ROM, an EPROM, an EEPROM, a nonvolatile Memory (NAND FLASH), a Solid State Disk (SSD)).

The above embodiments are only used to describe the technical solutions of the present application in detail, but the above embodiments are only used to help understanding the method of the embodiments of the present application, and should not be construed as limiting the embodiments of the present application. Modifications and substitutions that may be readily apparent to those skilled in the art are intended to be included within the scope of the embodiments of the present application.

Claims (13)

1. A three-dimensional cartoon face generation method is characterized by comprising the following steps:

generating a corresponding human face-like three-dimensional point cloud model based on the human face image;

determining a target vertex corresponding to each vertex in the three-dimensional cartoon face model in the similar face three-dimensional point cloud model according to a vertex index relationship, wherein the vertex index relationship comprises a corresponding relationship between the vertex in the similar face three-dimensional point cloud model and the vertex in the three-dimensional cartoon face model;

determining coordinates corresponding to each vertex in the three-dimensional cartoon face model respectively based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model respectively;

and fitting to obtain a three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

2. The method according to claim 1, wherein the fitting to obtain the three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model specifically comprises:

fitting to obtain cartoon expression base coefficients respectively corresponding to the cartoon expression bases based on the coordinates of each vertex in the three-dimensional cartoon face model, wherein each cartoon expression base corresponds to a three-dimensional cartoon face template with one expression;

and carrying out weighted summation on each cartoon expression base based on the cartoon expression base coefficient respectively corresponding to each cartoon expression base so as to obtain the three-dimensional cartoon face corresponding to the face image.

3. The method of claim 2, wherein the face-like three-dimensional point cloud model and the three-dimensional cartoon face model each comprise a plurality of feature areas, each feature area comprising at least one vertex;

the fitting to obtain the cartoon expression base coefficients respectively corresponding to the cartoon expression bases based on the coordinates of each vertex in the three-dimensional cartoon face model specifically comprises the following steps:

for each feature area, fitting to obtain local cartoon expression base coefficients corresponding to the cartoon expression bases respectively based on coordinates of vertexes in each feature area in the three-dimensional cartoon face model;

and for each cartoon expression base, fusing the local cartoon expression base coefficients of each cartoon expression base corresponding to the plurality of characteristic areas respectively to obtain the cartoon expression base coefficient corresponding to each cartoon expression base.

4. The method according to any one of claims 1 to 3, wherein the determining the coordinates corresponding to each vertex in the three-dimensional cartoon face model based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model comprises:

and updating the coordinates of each vertex in the three-dimensional cartoon face model into the coordinates of the corresponding target vertex.

5. The method of claim 4, wherein the vertex index relationship is determined by:

carrying out point cloud preliminary alignment on the human face-like three-dimensional point cloud model and the three-dimensional cartoon human face model;

attaching the vertex of the three-dimensional cartoon face model to the vertex of the similar face three-dimensional point cloud model to obtain an updated three-dimensional cartoon face model;

and aiming at each vertex in the updated three-dimensional cartoon face model, screening at least one vertex from the similar face three-dimensional point cloud model according to the distance value between each vertex in the similar face three-dimensional point cloud model and each vertex, and establishing a corresponding relation between each vertex and the screened at least one vertex.

6. The method of claim 5, wherein the face-like three-dimensional point cloud model and the three-dimensional cartoon face model each comprise a plurality of feature areas, each feature area comprising at least one vertex; the step of attaching the vertex of the three-dimensional cartoon face model to the vertex of the similar face three-dimensional point cloud model to obtain the updated three-dimensional cartoon face model specifically comprises the following steps:

and aiming at each characteristic region, performing non-rigid registration on the vertex in each characteristic region in the three-dimensional cartoon face model and the vertex in each characteristic region in the human face-like three-dimensional point cloud model so as to attach the vertex in the three-dimensional cartoon face model to the human face-like three-dimensional point cloud model.

7. The method according to claim 5, wherein the step of screening at least one vertex from the face-like three-dimensional point cloud model according to the distance value between each vertex and each vertex in the face-like three-dimensional point cloud model, and establishing a corresponding relationship between each vertex and the screened at least one vertex comprises the steps of:

and according to the distance value between each vertex and each vertex in the face-like three-dimensional point cloud model, screening the vertex closest to each vertex from the face-like three-dimensional point cloud model, and establishing a corresponding relation between each vertex and the closest vertex.

8. The method according to any one of claims 1 to 3, further comprising:

determining a pose scaling parameter between the human face-like three-dimensional point cloud model and the three-dimensional cartoon human face model;

and adjusting the pose and the size of the three-dimensional cartoon face based on the pose scaling parameter.

9. The method according to any one of claims 1 to 3, wherein the generating of the corresponding human-face-like three-dimensional point cloud model based on the human face image specifically comprises:

carrying out face recognition on a face image containing a target face to obtain feature points of each feature part in the target face;

acquiring three-dimensional feature points for representing each feature part in the target face based on the feature points of each feature part in the target face;

and obtaining a human face-like three-dimensional point cloud model according to the obtained three-dimensional feature points of each feature part in the target human face.

10. The method according to claim 9, wherein the obtaining a face-like three-dimensional point cloud model according to the obtained three-dimensional feature points of each feature part in the target face specifically comprises:

fitting to obtain corresponding expression coefficients and shape coefficients based on the obtained three-dimensional feature points of each feature part in the target face and a human-like deformable template in a human-like deformable template library;

and obtaining a human face-like three-dimensional point cloud model corresponding to the human face image based on the expression coefficient and the shape coefficient obtained by fitting and the human face-like deformable template library.

11. A three-dimensional cartoon face generation device is characterized by comprising:

the human face-like three-dimensional point cloud model generating module is used for generating a corresponding human face-like three-dimensional point cloud model based on the human face image;

the three-dimensional cartoon face model generating module is used for determining a target vertex corresponding to each vertex in the three-dimensional cartoon face model in the face-like three-dimensional point cloud model according to a vertex index relationship, and determining coordinates corresponding to each vertex based on the target vertex corresponding to each vertex in the three-dimensional cartoon face model, wherein the vertex index relationship comprises a corresponding relationship between the vertex in the face-like three-dimensional point cloud model and the vertex in the three-dimensional cartoon face model;

and the fitting output module is used for fitting to obtain the three-dimensional cartoon face corresponding to the face image based on the coordinates of each vertex in the three-dimensional cartoon face model.

12. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 10 are implemented by the processor when executing the computer program.

13. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 10.

Images