技术领域technical field
本公开涉及图像处理领域，尤其涉及人脸光照迁移方法及装置。The present disclosure relates to the field of image processing, and in particular to a method and device for face illumination migration.
背景技术Background technique
近年来，高真实感的人脸光照迁移技术引起了计算机图形学及图像处理研究人员的广泛关注。人脸图像的光照迁移，是指给定一张目标人脸图像和一张参照人脸图像，将参考人脸图像上的光照效果迁移到目标人脸图像上。人脸图像的光照迁移技术可广泛应用于电影特效制作、游戏娱乐、虚拟现实和美颜相机、人脸识别等领域。In recent years, high-realistic facial light migration technology has attracted extensive attention from researchers in computer graphics and image processing. The illumination transfer of the face image refers to the transfer of the illumination effect on the reference face image to the target face image given a target face image and a reference face image. The light migration technology of face image can be widely used in the fields of film special effects production, game entertainment, virtual reality and beauty camera, face recognition and so on.
相关技术中，人脸图像的光照迁移方法包括：基于商图像的方法和基于3D可形变模型的方法。In related technologies, methods for light migration of face images include methods based on quotient images and methods based on 3D deformable models.
发明内容Contents of the invention
为克服相关技术中存在的问题，本公开实施例提供一种人脸光照迁移方法及装置。所述技术方案如下：In order to overcome the problems existing in the related technologies, embodiments of the present disclosure provide a method and device for face light migration. Described technical scheme is as follows:
根据本公开实施例的第一方面，提供一种人脸光照迁移方法，包括：According to the first aspect of an embodiment of the present disclosure, a method for face illumination migration is provided, including:
根据目标人脸图像对参照人脸图像进行形变，得到与所述目标人脸图像对齐的形变后的参照人脸图像；Deforming the reference face image according to the target face image to obtain a deformed reference face image aligned with the target face image;
分别对所述目标人脸图像和所述形变后的参照人脸图像进行分解，得到所述形变后的参照人脸图像的结构图、所述目标人脸图像的色彩图、结构图及细节图；Decomposing the target face image and the deformed reference face image respectively to obtain a structure diagram of the deformed reference face image, a color map, a structure diagram and a detail map of the target face image ;
以所述目标人脸图像的结构图为导向图，对所述形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；Taking the structural diagram of the target human face image as a guiding diagram, performing guided filtering on the structural diagram of the deformed reference human face image to obtain the structural diagram of the resulting human face image;
将所述结果人脸图像的结构图与所述目标人脸图像的细节图及色彩图进行合成，得到所述结果人脸图像。Synthesizing the structure diagram of the resultant face image with the detail map and color map of the target face image to obtain the resultant face image.
在一个实施例中，所述根据目标人脸图像对参照人脸图像进行形变，得到与所述目标人脸图像对齐的形变后的参照人脸图像，包括：In one embodiment, the deforming the reference face image according to the target face image to obtain the deformed reference face image aligned with the target face image includes:
分别对目标人脸图像和参照人脸图像进行人脸关键点定位，得到所述目标人脸图像的人脸关键点的位置、及所述参照人脸图像的人脸关键点的位置；Carrying out face key point location to target face image and reference face image respectively, obtain the position of the face key point of described target face image, and the position of the face key point of described reference face image;
根据所述目标人脸图像的人脸关键点的位置、及所述参照人脸图像的人脸关键点的位置，对所述参照人脸图像进行形变，得到与所述目标人脸图像对齐的形变后的参照人脸图像。According to the positions of the key points of the face of the target face image and the positions of the key points of the face of the reference face image, the reference face image is deformed to obtain a face image aligned with the target face image. The deformed reference face image.
在一个实施例中，所述分别对所述目标人脸图像和所述形变后的参照人脸图像进行分解，得到所述形变后的参照人脸图像的结构图、所述目标人脸图像的色彩图、结构图及细节图，包括：In one embodiment, the target face image and the deformed reference face image are respectively decomposed to obtain the structure diagram of the deformed reference face image and the structure diagram of the target face image. Color map, structure map and detail map, including:
将所述目标人脸图像和所述形变后的参照人脸图像从红绿蓝RGB色彩空间转换到能够将图像的亮度和色彩分开的色彩空间，得到所述目标人脸图像的亮度图和色彩图、及所述形变后的参照人脸图像的亮度图和色彩图；Converting the target human face image and the deformed reference human face image from the red, green and blue RGB color space to a color space that can separate the brightness and color of the image to obtain the brightness map and color of the target human face image graph, and the brightness graph and the color graph of the reference face image after the deformation;
将所述目标人脸图像的亮度图进行滤波，得到所述目标人脸图像的结构图和细节图，及将所述形变后的参照人脸图像的亮度图进行滤波，得到所述形变后的参照人脸图像的结构图和细节图。Filtering the luminance map of the target face image to obtain the structure map and detail map of the target face image, and filtering the luminance map of the deformed reference face image to obtain the deformed Refer to the structure diagram and detail diagram of the face image.
在一个实施例中，所述将所述结果人脸图像的结构图与所述目标人脸图像的细节图及色彩图进行合成，得到所述结果人脸图像，包括：In one embodiment, the said result human face image is obtained by synthesizing the structure diagram of said result human face image with the detail map and color map of said target human face image, comprising:
将所述结果人脸图像的结构图和所述目标人脸图像的细节图进行合成，得到所述结果人脸图像的亮度图；Synthesizing the structure diagram of the result human face image and the detail diagram of the target human face image to obtain the brightness map of the result human face image;
将所述结果人脸图像的亮度图和所述目标人脸图像的色彩图进行合成，将合成的图像从能够将图像的亮度和色彩分开的色彩空间转换到RGB色彩空间，得到所述结果人脸图像。Synthesize the brightness map of the result human face image and the color map of the target human face image, convert the composite image from a color space that can separate the brightness and color of the image to RGB color space, and obtain the result human face image face image.
在一个实施例中，所述将所述目标人脸图像的亮度图进行滤波，得到所述目标人脸图像的结构图和细节图，及将所述形变后的参照人脸图像的亮度图进行滤波，得到所述形变后的参照人脸图像的结构图和细节图，包括：In one embodiment, the luminance map of the target face image is filtered to obtain the structure map and detail map of the target face image, and the luminance map of the deformed reference face image is processed Filter to obtain the structure diagram and detail diagram of the deformed reference face image, including:
将所述目标人脸图像的亮度图进行加权最小二乘法WLS滤波或双边滤波，得到所述目标人脸图像的结构图和细节图；Carrying out weighted least squares method WLS filtering or bilateral filtering to the brightness map of the target human face image to obtain the structure diagram and detail diagram of the target human face image;
将所述形变后的参照人脸图像的亮度图进行WLS滤波或双边滤波，得到所述形变后的参照人脸图像的结构图和细节图。WLS filtering or bilateral filtering is performed on the brightness map of the deformed reference face image to obtain the structure map and detail map of the deformed reference face image.
根据本公开实施例的第二方面，提供一种人脸光照迁移装置，包括：According to the second aspect of the embodiments of the present disclosure, there is provided a face illumination transfer device, including:
人脸对齐模块，用于根据目标人脸图像对参照人脸图像进行形变，得到与所述目标人脸图像对齐的形变后的参照人脸图像；A face alignment module, configured to deform the reference face image according to the target face image, to obtain a deformed reference face image aligned with the target face image;
分解模块，用于分别对所述目标人脸图像和所述形变后的参照人脸图像进行分解，得到所述形变后的参照人脸图像的结构图、所述目标人脸图像的色彩图、结构图及细节图；A decomposition module, configured to decompose the target face image and the deformed reference face image respectively, to obtain the structure diagram of the deformed reference face image, the color map of the target face image, Structural and detail drawings;
导向滤波模块，用于以所述目标人脸图像的结构图为导向图，对所述形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；A guided filtering module, configured to use the structural diagram of the target face image as a guided diagram to perform guided filtering on the deformed structural diagram of the reference human face image to obtain the structural diagram of the resulting human face image;
合成模块，用于将所述结果人脸图像的结构图与所述目标人脸图像的细节图及色彩图进行合成，得到所述结果人脸图像。The synthesis module is used for synthesizing the structure diagram of the result face image with the detail map and color map of the target face image to obtain the result face image.
在一个实施例中，所述人脸对齐模块，包括：In one embodiment, the face alignment module includes:
定位子模块，用于分别对目标人脸图像和参照人脸图像进行人脸关键点定位，得到所述目标人脸图像的人脸关键点的位置、及所述参照人脸图像的人脸关键点的位置；The positioning sub-module is used to locate the key points of the face of the target face image and the reference face image respectively, and obtain the position of the key points of the face of the target face image and the key points of the face of the reference face image. point position;
形变子模块，用于根据所述目标人脸图像的人脸关键点的位置、及所述参照人脸图像的人脸关键点的位置，对所述参照人脸图像进行形变，得到与所述目标人脸图像对齐的形变后的参照人脸图像。The deformation sub-module is used to deform the reference face image according to the position of the key point of the face of the target face image and the position of the key point of the face of the reference face image, so as to obtain the The deformed reference face image to which the target face image is aligned.
在一个实施例中，所述分解模块，包括：In one embodiment, the decomposition module includes:
色彩空间转换子模块，用于将所述目标人脸图像和所述形变后的参照人脸图像从红绿蓝RGB色彩空间转换到能够将图像的亮度和色彩分开的色彩空间，得到所述目标人脸图像的亮度图和色彩图、及所述形变后的参照人脸图像的亮度图和色彩图；The color space conversion sub-module is used to convert the target face image and the deformed reference face image from the red, green and blue RGB color space to a color space that can separate the brightness and color of the image to obtain the target face image. A brightness map and a color map of a face image, and a brightness map and a color map of the deformed reference face image;
滤波子模块，用于将所述目标人脸图像的亮度图进行滤波，得到所述目标人脸图像的结构图和细节图，及将所述形变后的参照人脸图像的亮度图进行滤波，得到所述形变后的参照人脸图像的结构图和细节图。The filtering submodule is used to filter the luminance map of the target face image to obtain the structure map and detail map of the target face image, and filter the luminance map of the deformed reference face image, A structure diagram and a detail diagram of the deformed reference face image are obtained.
在一个实施例中，所述合成模块，包括：In one embodiment, the synthesis module includes:
第一合成子模块，用于将所述结果人脸图像的结构图和所述目标人脸图像的细节图进行合成，得到所述结果人脸图像的亮度图；The first synthesizing sub-module is used for synthesizing the structure diagram of the resultant face image and the detail map of the target human face image to obtain the brightness map of the resultant face image;
第二合成子模块，用于将所述结果人脸图像的亮度图和所述目标人脸图像的色彩图进行合成，将合成的图像从能够将图像的亮度和色彩分开的色彩空间转换到RGB色彩空间，得到所述结果人脸图像。The second synthesis sub-module is used to synthesize the brightness map of the result face image and the color map of the target face image, and convert the synthesized image from a color space capable of separating the brightness and color of the image to RGB color space to obtain the result face image.
在一个实施例中，所述滤波子模块：将所述目标人脸图像的亮度图进行加权最小二乘法WLS滤波或双边滤波，得到所述目标人脸图像的结构图和细节图；及将所述形变后的参照人脸图像的亮度图进行WLS滤波或双边滤波，得到所述形变后的参照人脸图像的结构图和细节图。In one embodiment, the filtering sub-module: performs weighted least squares WLS filtering or bilateral filtering on the brightness map of the target human face image to obtain the structure map and detail map of the target human face image; and WLS filtering or bilateral filtering is performed on the luminance map of the deformed reference face image to obtain the structure map and detail map of the deformed reference face image.
根据本公开实施例的第三方面，提供一种人脸光照迁移装置，包括：According to a third aspect of the embodiments of the present disclosure, there is provided a face illumination transfer device, including:
处理器；processor;
用于存储处理器可执行指令的存储器；memory for storing processor-executable instructions;
其中，所述处理器被配置为：Wherein, the processor is configured as:
根据目标人脸图像对参照人脸图像进行形变，得到与所述目标人脸图像对齐的形变后的参照人脸图像；Deforming the reference face image according to the target face image to obtain a deformed reference face image aligned with the target face image;
分别对所述目标人脸图像和所述形变后的参照人脸图像进行分解，得到所述形变后的参照人脸图像的结构图、所述目标人脸图像的色彩图、结构图及细节图；Decomposing the target face image and the deformed reference face image respectively to obtain a structure diagram of the deformed reference face image, a color map, a structure diagram and a detail map of the target face image ;
以所述目标人脸图像的结构图为导向图，对所述形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；Taking the structural diagram of the target human face image as a guiding diagram, performing guided filtering on the structural diagram of the deformed reference human face image to obtain the structural diagram of the resulting human face image;
将所述结果人脸图像的结构图与所述目标人脸图像的细节图及色彩图进行合成，得到所述结果人脸图像。Synthesizing the structure diagram of the resultant face image with the detail map and color map of the target face image to obtain the resultant face image.
根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机指令，该指令被处理器执行时实现上述第一方面所述方法的步骤。According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the above-mentioned first aspect are implemented.
本公开的实施例提供的技术方案可以包括以下有益效果：该技术方案通过对目标人脸图像和参照人脸图像进行分解，基于分解得到的色彩图、结构图及细节图将参照人脸图像的光照效果迁移到目标人脸上，能够保持目标人脸的脸部结构和色彩信息；该方法只需要一张参照人脸图像，实用性强；无需事先建立3D人脸模型库，实际应用方便，解决了相关技术存在的方案实用性低及实际应用不便的问题，能够提高用户体验。The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the technical solution decomposes the target face image and the reference face image, based on the color map, structure map and detail map obtained by the decomposition, the reference face image The illumination effect is migrated to the target face, which can maintain the facial structure and color information of the target face; this method only needs a reference face image, which is practical; it does not need to establish a 3D face model library in advance, which is convenient for practical application. The problems of low solution practicability and inconvenient practical application existing in related technologies are solved, and user experience can be improved.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.
附图说明Description of drawings
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.
图1是根据一示例性实施例示出的人脸光照迁移方法的流程图。Fig. 1 is a flow chart of a method for face illumination migration according to an exemplary embodiment.
图2是根据一示例性实施例示出的人脸光照迁移方法的流程图。Fig. 2 is a flow chart of a method for face illumination migration according to an exemplary embodiment.
图3是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 3 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
图4是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 4 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
图5是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 5 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
图6是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 6 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
图7是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 7 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
图8是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 8 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
图9是根据一示例性实施例示出的人脸光照迁移装置的框图。Fig. 9 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment.
具体实施方式detailed description
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.
相关技术中，人脸图像的光照迁移方法包括：基于商图像的方法和基于3D可形变模型的方法。然而，基于商图像的方法需要两张参照人脸图像，一张具有和目标人脸相似的光照，另一张具有用户想要的光照效果，这就在一定程度上降低了方案的实用性；基于3D可形变模型的方法则需要事先建立3D人脸模型库，这就给实际应用造成不便，影响用户体验。In related technologies, methods for light migration of face images include methods based on quotient images and methods based on 3D deformable models. However, the method based on the quotient image requires two reference face images, one with the lighting similar to the target face, and the other with the lighting effect that the user wants, which reduces the practicability of the scheme to a certain extent; The method based on a 3D deformable model needs to establish a 3D face model library in advance, which causes inconvenience to practical applications and affects user experience.
为了解决上述问题，本公开实施例提供了一种人脸光照迁移方法，该方法包括：根据目标人脸图像对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像；分别对目标人脸图像和形变后的参照人脸图像进行分解，得到形变后的参照人脸图像的结构图、目标人脸图像的色彩图、结构图及细节图；以目标人脸图像的结构图为导向图，对形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；将结果人脸图像的结构图与目标人脸图像的细节图及色彩图进行合成，得到结果人脸图像。In order to solve the above problems, an embodiment of the present disclosure provides a face illumination transfer method, the method includes: deforming the reference face image according to the target face image, and obtaining the deformed reference face aligned with the target face image image; decompose the target face image and the deformed reference face image respectively to obtain the structure diagram of the deformed reference face image, the color map, structure diagram and detail map of the target face image; The structure graph of the result face image is a guide graph, and the structure graph of the deformed reference face image is guided to filter to obtain the structure graph of the result face image; the structure graph of the result face image is combined with the detail map and color map of the target face image Synthesize to obtain the result face image.
本公开实施例提供的人脸光照迁移方法，通过对目标人脸图像和参照人脸图像进行分解，基于分解得到的色彩图、结构图及细节图将参照人脸图像的光照效果迁移到目标人脸上，能够保持目标人脸的脸部结构和色彩信息；该方法只需要一张参照人脸图像，实用性强；无需事先建立3D人脸模型库，实际应用方便，解决了相关技术存在的方案实用性低及实际应用不便的问题，能够提高用户体验。In the face illumination transfer method provided by the embodiments of the present disclosure, by decomposing the target face image and the reference face image, the illumination effect of the reference face image is transferred to the target face image based on the decomposed color map, structure map and detail map. The face can maintain the facial structure and color information of the target face; this method only needs a reference face image, which is practical; it does not need to establish a 3D face model library in advance, which is convenient for practical application and solves the problems existing in related technologies. The low practicality of the solution and the inconvenience of practical application can improve the user experience.
基于上述分析，提出以下各具体实施例。Based on the above analysis, the following specific embodiments are proposed.
图1是根据一示例性实施例示出的一种人脸光照迁移方法的流程图，该方法的执行主体可以为终端，例如智能手机、平板电脑、台式机、笔记本电脑等；如图1所示，该方法包括以下步骤101-104：Fig. 1 is a flow chart of a method for face illumination migration shown according to an exemplary embodiment, the execution subject of the method may be a terminal, such as a smart phone, a tablet computer, a desktop computer, a notebook computer, etc.; as shown in Fig. 1 , the method includes the following steps 101-104:
在步骤101中，根据目标人脸图像对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像；In step 101, deforming the reference face image according to the target face image to obtain a deformed reference face image aligned with the target face image;
示例的，在需要将参考人脸图像上的光照效果迁移到目标人脸图像的场景中，首先，获取目标人脸图像及参照人脸图像；然后，根据目标人脸图像对参照人脸图像进行形变，得到形变后的参照人脸图像，形变后的参照人脸图像与目标人脸图像对齐，也即，形变后的参照人脸图像中各个人脸器官的坐标分别与目标人脸图像中对应器官的坐标相同。For example, in the scene where the lighting effect on the reference face image needs to be migrated to the target face image, first, the target face image and the reference face image are acquired; then, the reference face image is processed according to the target face image Deformation, to obtain the deformed reference face image, the deformed reference face image is aligned with the target face image, that is, the coordinates of each face organ in the deformed reference face image correspond to the target face image respectively The coordinates of the organs are the same.
示例的，根据目标人脸图像对参照人脸图像进行形变的实现步骤可以包括：分别对目标人脸图像和参照人脸图像进行人脸关键点定位，得到目标人脸图像的人脸关键点的位置及参照人脸图像的人脸关键点的位置；根据目标人脸图像的人脸关键点的位置及参照人脸图像的人脸关键点的位置，对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像。形变方法可以包括：移动最小二乘法(MLS，moving least squares)变换或三角网格变换。人脸关键点定位算法可以包括主动外观模型(AAM，activeappearance models)、有监督的梯度下降方法(SDM，supervised descent method)或卷积神经网络(CNN，convolutional neural networks)等。For example, the implementation step of deforming the reference face image according to the target face image may include: respectively performing face key point positioning on the target face image and the reference face image, and obtaining the face key points of the target face image. position and the position of the key point of the face of the reference face image; according to the position of the key point of the face of the target face image and the position of the key point of the face of the reference face image, the reference face image is deformed to obtain the The deformed reference face image for face image alignment. The deformation method may include: moving least squares (MLS, moving least squares) transformation or triangular grid transformation. Face key point positioning algorithms can include active appearance models (AAM, active appearance models), supervised gradient descent methods (SDM, supervised descent method) or convolutional neural networks (CNN, convolutional neural networks), etc.
在步骤102中，分别对目标人脸图像和形变后的参照人脸图像进行分解，得到形变后的参照人脸图像的结构图、目标人脸图像的色彩图、结构图及细节图；In step 102, the target face image and the deformed reference face image are decomposed respectively to obtain a structure diagram of the deformed reference face image, a color map, a structure diagram and a detail map of the target face image;
示例的，图像的结构图可以包括图像的大尺度信息；图像的细节图可以包括图像的纹理、毛发等细节信息。For example, the structure map of the image may include large-scale information of the image; the detail map of the image may include detailed information such as texture and hair of the image.
示例的，对目标人脸图像和形变后的参照人脸图像进行分解的实现步骤可以包括：分别对目标人脸图像和形变后的参照人脸图像进行色彩空间转换，将目标人脸图像和形变后的参照人脸图像从当前的红绿蓝(RGB)色彩空间转换到亮度颜色(Lab)色彩空间，得到目标人脸图像的亮度图和色彩图、及形变后的参照人脸图像的亮度图和色彩图；其中，Lab色彩空间中，L通道代表亮度图，a通道和b通道为色彩图。通过将图像(即目标人脸图像和形变后的参照人脸图像)的亮度和色彩进行分离，接下来只对亮度图进行处理，从而保证后续的处理只影响图像的亮度，而不会改变图像的色彩。然后，将目标人脸图像的亮度图进行滤波，得到目标人脸图像的结构图和细节图，及将形变后的参照人脸图像的亮度图进行滤波，得到形变后的参照人脸图像的结构图和细节图。例如，将目标人脸图像的亮度图进行WLS滤波或双边滤波，得到目标人脸图像的结构图和细节图；将形变后的参照人脸图像的亮度图进行WLS滤波或双边滤波，得到形变后的参照人脸图像的结构图和细节图。需要说明的是，为了得到目标人脸图像的亮度图和色彩图，除了可以将目标人脸图像和形变后的参照人脸图像从RGB色彩空间转换到Lab色彩空间，也可以将目标人脸图像和形变后的参照人脸图像从RGB色彩空间转换到任意能够将图像的亮度和色彩分开的色彩空间，例如，YUV色彩空间、或HSV色彩空间，本公开并不对此进行限定。For example, the implementation step of decomposing the target face image and the deformed reference face image may include: performing color space conversion on the target face image and the deformed reference face image respectively, and converting the target face image and the deformed The final reference face image is converted from the current red, green and blue (RGB) color space to the brightness color (Lab) color space, and the brightness map and color map of the target face image and the brightness map of the deformed reference face image are obtained and a color map; among them, in the Lab color space, the L channel represents the brightness map, and the a channel and the b channel are the color map. By separating the brightness and color of the image (that is, the target face image and the deformed reference face image), only the brightness map is processed next, so as to ensure that the subsequent processing only affects the brightness of the image without changing the image color. Then, filter the luminance map of the target face image to obtain the structure map and detail map of the target face image, and filter the luminance map of the deformed reference face image to obtain the structure of the deformed reference face image diagrams and details. For example, WLS filtering or bilateral filtering is performed on the luminance map of the target face image to obtain the structure map and detail map of the target face image; WLS filtering or bilateral filtering is performed on the luminance map of the deformed reference face image to obtain the deformed The structure diagram and detail diagram of the reference face image. It should be noted that, in order to obtain the brightness map and color map of the target face image, in addition to converting the target face image and the deformed reference face image from the RGB color space to the Lab color space, the target face image can also be and transforming the deformed reference face image from the RGB color space to any color space capable of separating the brightness and color of the image, for example, the YUV color space or the HSV color space, which is not limited in this disclosure.
在步骤103中，以目标人脸图像的结构图为导向图，对形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；In step 103, take the structural diagram of the target human face image as the guiding diagram, carry out guided filtering to the structural diagram of the reference human face image after deformation, obtain the structural diagram of the result human face image;
在步骤104中，将结果人脸图像的结构图与目标人脸图像的细节图及色彩图进行合成，得到结果人脸图像。In step 104, the structure map of the resultant face image is synthesized with the detail map and color map of the target face image to obtain a resultant face image.
示例的，将结果人脸图像的结构图与目标人脸图像的细节图及色彩图进行合成的实现步骤可以包括：将结果人脸图像的结构图和目标人脸图像的细节图进行合成，得到结果人脸图像的亮度图；将结果人脸图像的亮度图和目标人脸图像的色彩图进行合成，将合成的图像从能够将图像的亮度和色彩分开的色彩空间，例如Lab色彩空间，转换到RGB色彩空间，得到结果人脸图像。For example, the implementation step of synthesizing the structure diagram of the result face image with the detail map and the color map of the target face image may include: synthesizing the structure diagram of the result face image and the detail map of the target face image to obtain The brightness map of the result face image; the brightness map of the result face image and the color map of the target face image are synthesized, and the composite image is converted from a color space that can separate the brightness and color of the image, such as the Lab color space. to RGB color space to get the result face image.
本公开的实施例提供的技术方案，通过对目标人脸图像和参照人脸图像进行分解，基于分解得到的色彩图、结构图及细节图将参照人脸图像的光照效果迁移到目标人脸上，能够保持目标人脸的脸部结构和色彩信息；该方法只需要一张参照人脸图像，实用性强；无需事先建立3D人脸模型库，实际应用方便，解决了相关技术存在的方案实用性低及实际应用不便的问题，能够提高用户体验。In the technical solution provided by the embodiments of the present disclosure, by decomposing the target face image and the reference face image, the illumination effect of the reference face image is transferred to the target face based on the decomposed color map, structure map and detail map , which can maintain the facial structure and color information of the target face; this method only needs a reference face image, which is practical; it does not need to establish a 3D face model library in advance, it is convenient for practical application, and solves the practical problems of related technologies It solves the problems of low performance and inconvenient practical application, and can improve user experience.
图2是根据一示例性实施例示出的一种人脸光照迁移方法的流程图。如图2所示，在图1所示实施例的基础上，本公开涉及的人脸光照迁移方法可以包括以下步骤201-208：Fig. 2 is a flow chart showing a method for face illumination migration according to an exemplary embodiment. As shown in FIG. 2, on the basis of the embodiment shown in FIG. 1, the face illumination migration method involved in the present disclosure may include the following steps 201-208:
在步骤201中，获取目标人脸图像及参照人脸图像；In step 201, the target face image and the reference face image are obtained;
示例的，目标人脸图像记为Itarget、及参照人脸图像记为Ireference。For example, the target face image is denoted as Itarget , and the reference face image is denoted as Ireference .
在步骤202中，分别对目标人脸图像和参照人脸图像进行人脸关键点定位，得到目标人脸图像的人脸关键点的位置、及参照人脸图像的人脸关键点的位置；In step 202, target face image and reference face image are carried out face key point location respectively, obtain the position of the face key point of target face image and the position of the face key point of reference face image;
示例的，人脸关键点定位可以采用AAM、SDM或CNN算法。For example, AAM, SDM or CNN algorithms may be used for face key point location.
在步骤203中，根据目标人脸图像的人脸关键点的位置、及参照人脸图像的人脸关键点的位置，对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像；In step 203, according to the position of the key point of the face of the target face image and the position of the key point of the face of the reference face image, the reference face image is deformed to obtain the deformed face image aligned with the target face image Refer to the face image;
示例的，形变可以使用MLS变换或三角网格变换。人脸对齐就是通过对参照人脸图像进行形变，从而将参照人脸图像中的各个人脸器官(眼睛、鼻子等)的坐标，调整至与目标人脸图像中对应人脸器官的坐标相同。Exemplarily, the deformation can use MLS transformation or triangular mesh transformation. Face alignment is to adjust the coordinates of each face organ (eyes, nose, etc.) in the reference face image to be the same as the coordinates of the corresponding face organs in the target face image by deforming the reference face image.
在步骤204中，将目标人脸图像和形变后的参照人脸图像从红绿蓝RGB色彩空间转换到能够将图像的亮度和色彩分开的色彩空间，例如亮度颜色Lab色彩空间，得到目标人脸图像的亮度图和色彩图、及形变后的参照人脸图像的亮度图和色彩图；In step 204, the target face image and the deformed reference face image are converted from the red, green, blue RGB color space to a color space that can separate the brightness and color of the image, such as the brightness color Lab color space, to obtain the target face The brightness map and color map of the image, and the brightness map and color map of the deformed reference face image;
示例的，记目标人脸图像的亮度图和色彩图分别为Ltarget和Ctarget，形变后的参照人脸图像的亮度图和色彩图分为Lreference和Creference。For example, the luminance map and the color map of the target face image are respectively Ltarget and Ctarget , and the luminance map and color map of the deformed reference face image are divided into Lreference and Creference .
在步骤205中，将目标人脸图像的亮度图进行WLS滤波或双边滤波，得到目标人脸图像的结构图和细节图；将形变后的参照人脸图像的亮度图进行WLS滤波或双边滤波，得到形变后的参照人脸图像的结构图和细节图。In step 205, WLS filtering or bilateral filtering is performed on the luminance map of the target face image to obtain a structure map and a detail map of the target face image; WLS filtering or bilateral filtering is performed on the luminance map of the deformed reference face image, The structure diagram and detail diagram of the deformed reference face image are obtained.
示例的，目标人脸图像的结构图Starget和细节图Dtarget，其中，Dtarget＝Ltarget-Starget。形变后的参照人脸图像的结构图Sreference和细节图Dreference，其中，Dreference＝Lreference-Sreference。Exemplarily, the structure graph Starget and the detail graph Dtarget of the target face image, wherein, Dtarget =Ltarget -Starget . The deformed structure graph Sreference and the detail graph Dreference of the reference face image, wherein, Dreference =Lreference −Sreference .
在步骤206中，以目标人脸图像的结构图为导向图，对形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；In step 206, with the structural diagram of the target human face image as the guiding diagram, the structural diagram of the reference human face image after deformation is subjected to guided filtering to obtain the structural diagram of the resultant human face image;
示例的，结果人脸图像的结构图Sresult。For example, the structure diagram Sresult of the result face image.
在步骤207中，将结果人脸图像的结构图和目标人脸图像的细节图进行合成，得到结果人脸图像的亮度图；In step 207, the structure diagram of the result human face image and the detail diagram of the target human face image are synthesized to obtain the brightness map of the result human face image;
示例的，结果人脸图像的亮度图Lresult＝Sresult+Dtarget。For example, the brightness map of the result face image Lresult =Sresult +Dtarget .
在步骤208中，将结果人脸图像的亮度图和目标人脸图像的色彩图进行合成，将合成的图像从能够将图像的亮度和色彩分开的色彩空间，例如Lab色彩空间，转换到RGB色彩空间，得到结果人脸图像。In step 208, the brightness map of the result face image and the color map of the target face image are synthesized, and the synthesized image is converted to RGB color from a color space that can separate the brightness and color of the image, such as the Lab color space space to get the result face image.
示例的，结果人脸图像记为Iresult。For example, the result face image is recorded as Iresult .
本公开的实施例提供的技术方案，基于对人脸图像进行分解来完成光照的迁移，只需要一张参照人脸图像，能够在得到的结果人脸图像上较好地保持目标人脸的脸部结构和色彩信息。The technical solution provided by the embodiments of the present disclosure is based on decomposing the face image to complete the migration of illumination. Only one reference face image is needed, and the face of the target face can be better preserved in the obtained result face image. Internal structure and color information.
下述为本公开装置实施例，可以用于执行本公开方法实施例。The following are device embodiments of the present disclosure, which can be used to implement the method embodiments of the present disclosure.
图3是根据一示例性实施例示出的一种人脸光照迁移装置的框图，该装置可以采用各种方式来实施，例如在终端中实施装置的全部组件，或者，在终端侧以耦合的方式实施装置中的组件；该装置可以通过软件、硬件或者两者的结合实现上述本公开涉及的方法，如图3所示，该人脸光照迁移装置包括：人脸对齐模块301、分解模块302、导向滤波模块303及合成模块304，其中：Fig. 3 is a block diagram of a device for face illumination migration according to an exemplary embodiment. The device can be implemented in various ways, for example, all components of the device are implemented in the terminal, or in a coupled manner on the terminal side Components in the implementation device; the device can implement the above-mentioned method involved in the present disclosure through software, hardware, or a combination of the two. As shown in FIG. 3 , the face illumination migration device includes: a face alignment module 301, a decomposition module 302, Guide filtering module 303 and synthesis module 304, wherein:
人脸对齐模块301被配置为根据目标人脸图像对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像；The face alignment module 301 is configured to deform the reference face image according to the target face image to obtain a deformed reference face image aligned with the target face image;
分解模块302被配置为分别对目标人脸图像和形变后的参照人脸图像进行分解，得到形变后的参照人脸图像的结构图、目标人脸图像的色彩图、结构图及细节图；The decomposition module 302 is configured to decompose the target face image and the deformed reference face image respectively, to obtain the structure diagram of the deformed reference face image, the color map, the structure diagram and the detail diagram of the target face image;
导向滤波模块303被配置为以目标人脸图像的结构图为导向图，对形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；The guided filtering module 303 is configured to take the structural diagram of the target face image as the guided diagram, and perform guided filtering on the deformed structural diagram of the reference human face image to obtain the structural diagram of the resulting human face image;
合成模块304被配置为将结果人脸图像的结构图与目标人脸图像的细节图及色彩图进行合成，得到结果人脸图像。The synthesizing module 304 is configured to synthesize the structure map of the resultant face image with the detail map and color map of the target face image to obtain the resultant face image.
本公开实施例提供的装置能够用于执行图1所示实施例的技术方案，其执行方式和有益效果类似，此处不再赘述。The device provided by the embodiment of the present disclosure can be used to execute the technical solution of the embodiment shown in FIG. 1 , and its execution mode and beneficial effect are similar, and details are not repeated here.
在一种可能的实施方式中，如图4所示，图3示出的人脸光照迁移装置还可以包括把人脸对齐模块301配置成包括：定位子模块401及形变子模块402，其中：In a possible implementation, as shown in FIG. 4 , the face illumination transfer device shown in FIG. 3 may also include configuring the face alignment module 301 to include: a positioning submodule 401 and a deformation submodule 402, wherein:
定位子模块401被配置为分别对目标人脸图像和参照人脸图像进行人脸关键点定位，得到目标人脸图像的人脸关键点的位置、及参照人脸图像的人脸关键点的位置；The localization sub-module 401 is configured to locate key points of the face on the target face image and the reference face image respectively, to obtain the positions of the key points of the face of the target face image and the positions of the key points of the face of the reference face image ;
形变子模块402被配置为根据目标人脸图像的人脸关键点的位置、及参照人脸图像的人脸关键点的位置，对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像。The deformation sub-module 402 is configured to deform the reference face image according to the positions of the key points of the face of the target face image and the positions of the key points of the face of the reference face image to obtain a deformation aligned with the target face image The subsequent reference face image.
在一种可能的实施方式中，如图5所示，图3示出的人脸光照迁移装置还可以包括把分解模块302配置成包括：色彩空间转换子模块501及滤波子模块502，其中：In a possible implementation, as shown in FIG. 5 , the face illumination migration device shown in FIG. 3 may also include configuring the decomposition module 302 to include: a color space conversion submodule 501 and a filtering submodule 502, wherein:
色彩空间转换子模块501被配置为将目标人脸图像和形变后的参照人脸图像从红绿蓝RGB色彩空间转换到能够将图像的亮度和色彩分开的色彩空间，例如亮度颜色Lab色彩空间，得到目标人脸图像的亮度图和色彩图、及形变后的参照人脸图像的亮度图和色彩图；The color space conversion sub-module 501 is configured to convert the target face image and the deformed reference face image from the red, green, blue RGB color space to a color space that can separate the brightness and color of the image, such as the brightness color Lab color space, Obtain the brightness map and color map of the target face image, and the brightness map and color map of the deformed reference face image;
滤波子模块502被配置为将目标人脸图像的亮度图进行滤波，得到目标人脸图像的结构图和细节图，及将形变后的参照人脸图像的亮度图进行滤波，得到形变后的参照人脸图像的结构图和细节图。The filtering sub-module 502 is configured to filter the luminance map of the target face image to obtain the structure map and detail map of the target face image, and filter the luminance map of the deformed reference face image to obtain the deformed reference face image. Structural and detail maps of face images.
在一种可能的实施方式中，如图6所示，图3示出的人脸光照迁移装置还可以包括把合成模块304配置成包括：第一合成子模块601及第二合成子模块602，其中：In a possible implementation manner, as shown in FIG. 6 , the face illumination transfer device shown in FIG. 3 may further include configuring the synthesis module 304 to include: a first synthesis sub-module 601 and a second synthesis sub-module 602, in:
第一合成子模块601被配置为将结果人脸图像的结构图和目标人脸图像的细节图进行合成，得到结果人脸图像的亮度图；The first synthesizing sub-module 601 is configured to synthesize the structural diagram of the resulting human face image and the detail diagram of the target human facial image to obtain a brightness map of the resulting human facial image;
第二合成子模块602被配置为将结果人脸图像的亮度图和目标人脸图像的色彩图进行合成，将合成的图像从能够将图像的亮度和色彩分开的色彩空间，例如Lab色彩空间，转换到RGB色彩空间，得到结果人脸图像。The second synthesis sub-module 602 is configured to synthesize the brightness map of the result face image and the color map of the target face image, and convert the synthesized image from a color space that can separate the brightness and color of the image, such as the Lab color space, Convert to RGB color space to get the result face image.
在一种可能的实施方式中，滤波子模块502被配置为：将目标人脸图像的亮度图进行加权最小二乘法WLS滤波或双边滤波，得到目标人脸图像的结构图和细节图；及将形变后的参照人脸图像的亮度图进行WLS滤波或双边滤波，得到形变后的参照人脸图像的结构图和细节图。In a possible implementation manner, the filtering submodule 502 is configured to: perform weighted least squares WLS filtering or bilateral filtering on the brightness image of the target human face image to obtain the structure diagram and detail diagram of the target human face image; and WLS filtering or bilateral filtering is performed on the luminance map of the deformed reference face image to obtain the structure map and detail map of the deformed reference face image.
图7是根据一示例性实施例示出的一种人脸光照迁移装置700的框图，人脸光照迁移装置700适用于终端或服务器，人脸光照迁移装置700包括：Fig. 7 is a block diagram showing a face illumination migration device 700 according to an exemplary embodiment. The face illumination migration device 700 is suitable for a terminal or a server, and the face illumination migration device 700 includes:
处理器701；Processor 701;
用于存储处理器可执行指令的存储器702；memory 702 for storing processor-executable instructions;
其中，处理器701被配置为：Wherein, the processor 701 is configured as:
根据目标人脸图像对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像；deforming the reference face image according to the target face image to obtain a deformed reference face image aligned with the target face image;
分别对目标人脸图像和形变后的参照人脸图像进行分解，得到形变后的参照人脸图像的结构图、目标人脸图像的色彩图、结构图及细节图；Decomposing the target face image and the deformed reference face image respectively to obtain the structure diagram of the deformed reference face image, the color map, the structure diagram and the detail diagram of the target face image;
以目标人脸图像的结构图为导向图，对形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；Taking the structure graph of the target face image as the guide graph, performing guided filtering on the deformed reference face image structure graph to obtain the result face image structure graph;
将结果人脸图像的结构图与目标人脸图像的细节图及色彩图进行合成，得到结果人脸图像。Synthesize the structure map of the result face image with the detail map and color map of the target face image to obtain the result face image.
在一个实施例中，上述处理器701还可被配置为：In one embodiment, the above-mentioned processor 701 may also be configured to:
分别对目标人脸图像和参照人脸图像进行人脸关键点定位，得到目标人脸图像的人脸关键点的位置、及参照人脸图像的人脸关键点的位置；Carry out face key point location to target face image and reference face image respectively, obtain the position of the face key point of target face image, and the position of the face key point of reference face image;
根据目标人脸图像的人脸关键点的位置、及参照人脸图像的人脸关键点的位置，对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像。According to the positions of the key points of the face of the target face image and the positions of the key points of the face of the reference face image, the reference face image is deformed to obtain a deformed reference face image aligned with the target face image.
在一个实施例中，上述处理器701还可被配置为：In one embodiment, the above-mentioned processor 701 may also be configured to:
将目标人脸图像和形变后的参照人脸图像从红绿蓝RGB色彩空间转换到能够将图像的亮度和色彩分开的色彩空间，例如亮度颜色Lab色彩空间，得到目标人脸图像的亮度图和色彩图、及形变后的参照人脸图像的亮度图和色彩图；Convert the target face image and the deformed reference face image from the red, green, blue RGB color space to a color space that can separate the brightness and color of the image, such as the brightness color Lab color space, to obtain the brightness map and Color map, and brightness map and color map of the deformed reference face image;
将目标人脸图像的亮度图进行滤波，得到目标人脸图像的结构图和细节图，及将形变后的参照人脸图像的亮度图进行滤波，得到形变后的参照人脸图像的结构图和细节图。Filter the luminance map of the target face image to obtain the structure map and detail map of the target face image, and filter the luminance map of the deformed reference face image to obtain the structure map and detail map of the deformed reference face image Detail image.
在一个实施例中，上述处理器701还可被配置为：In one embodiment, the above-mentioned processor 701 may also be configured to:
将结果人脸图像的结构图和目标人脸图像的细节图进行合成，得到结果人脸图像的亮度图；Synthesizing the structure diagram of the resultant face image and the detail map of the target face image to obtain a brightness map of the resultant face image;
将结果人脸图像的亮度图和目标人脸图像的色彩图进行合成，将合成的图像从能够将图像的亮度和色彩分开的色彩空间，例如Lab色彩空间转换到RGB色彩空间，得到结果人脸图像。Synthesize the brightness map of the result face image and the color map of the target face image, convert the composite image from a color space that can separate the brightness and color of the image, such as the Lab color space, into the RGB color space, and obtain the result face image.
在一个实施例中，上述处理器701还可被配置为：In one embodiment, the above-mentioned processor 701 may also be configured to:
将目标人脸图像的亮度图进行加权最小二乘法WLS滤波或双边滤波，得到目标人脸图像的结构图和细节图；Perform weighted least squares WLS filtering or bilateral filtering on the brightness map of the target face image to obtain the structure map and detail map of the target face image;
将形变后的参照人脸图像的亮度图进行WLS滤波或双边滤波，得到形变后的参照人脸图像的结构图和细节图。WLS filtering or bilateral filtering is performed on the luminance map of the deformed reference face image to obtain the structure map and detail map of the deformed reference face image.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
图8是根据一示例性实施例示出的一种人脸光照迁移装置的框图；人脸光照迁移装置800适用于终端；人脸光照迁移装置800可以包括以下一个或多个组件：处理组件802，存储器804，电源组件806，多媒体组件808，音频组件810，输入/输出(I/O)的接口812，传感器组件814，以及通信组件816。Fig. 8 is a block diagram of a face illumination migration device according to an exemplary embodiment; the face illumination migration device 800 is suitable for a terminal; the face illumination migration device 800 may include one or more of the following components: a processing component 802, Memory 804 , power supply component 806 , multimedia component 808 , audio component 810 , input/output (I/O) interface 812 , sensor component 814 , and communication component 816 .
处理组件802通常控制人脸光照迁移装置800的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件802可以包括一个或多个处理器820来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件802可以包括一个或多个模块，便于处理组件802和其他组件之间的交互。例如，处理组件802可以包括多媒体模块，以方便多媒体组件808和处理组件802之间的交互。The processing component 802 generally controls the overall operations of the facial lighting migration apparatus 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .
存储器804被配置为存储各种类型的数据以支持在人脸光照迁移装置800的操作。这些数据的示例包括用于在人脸光照迁移装置800上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器804可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。The memory 804 is configured to store various types of data to support the operation of the face illumination migration apparatus 800 . Examples of these data include instructions for any application or method operating on the face lighting migration apparatus 800 , contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
电源组件806为人脸光照迁移装置800的各种组件提供电力。电源组件806可以包括电源管理系统，一个或多个电源，及其他与为人脸光照迁移装置800生成、管理和分配电力相关联的组件。The power supply component 806 provides power for various components of the face light migration device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for face lighting migration apparatus 800 .
多媒体组件808包括在人脸光照迁移装置800和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件808包括一个前置摄像头和/或后置摄像头。当人脸光照迁移装置800处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。The multimedia component 808 includes a screen that provides an output interface between the face lighting migration device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or a swipe action, but also detect duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the face illumination transfer device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
音频组件810被配置为输出和/或输入音频信号。例如，音频组件810包括一个麦克风(MIC)，当人脸光照迁移装置800处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器804或经由通信组件816发送。在一些实施例中，音频组件810还包括一个扬声器，用于输出音频信号。The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive an external audio signal when the face light transfer device 800 is in an operation mode, such as a calling mode, a recording mode and a speech recognition mode. Received audio signals may be further stored in memory 804 or sent via communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
I/O接口812为处理组件802和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
传感器组件814包括一个或多个传感器，用于为人脸光照迁移装置800提供各个方面的状态评估。例如，传感器组件814可以检测到人脸光照迁移装置800的打开/关闭状态，组件的相对定位，例如组件为人脸光照迁移装置800的显示器和小键盘，传感器组件814还可以检测人脸光照迁移装置800或人脸光照迁移装置800一个组件的位置改变，用户与人脸光照迁移装置800接触的存在或不存在，人脸光照迁移装置800方位或加速/减速和人脸光照迁移装置800的温度变化。传感器组件814可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件814还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件814还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。The sensor component 814 includes one or more sensors for providing state assessment of various aspects of the face illumination migration apparatus 800 . For example, the sensor component 814 can detect the open/closed state of the human face light transfer device 800, the relative positioning of the components, for example, the components are the display and the keypad of the human face light transfer device 800, and the sensor component 814 can also detect the human face light transfer device 800. 800 or a change in the position of a component of the face light transfer device 800, the presence or absence of contact between the user and the face light transfer device 800, the orientation or acceleration/deceleration of the face light transfer device 800 and the temperature change of the face light transfer device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
通信组件816被配置为便于人脸光照迁移装置800和其他设备之间有线或无线方式的通信。人脸光照迁移装置800可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件816经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，通信组件816还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。The communication component 816 is configured to facilitate wired or wireless communication between the face illumination migration apparatus 800 and other devices. The face illumination migration device 800 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
在示例性实施例中，人脸光照迁移装置800可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子组件实现，用于执行上述方法。In an exemplary embodiment, the face light migration apparatus 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs) , a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor or other electronic component implementations for performing the above methods.
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器804，上述指令可由人脸光照迁移装置800的处理器820执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions. The above instructions can be executed by the processor 820 of the face illumination migration apparatus 800 to complete the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
图9是根据一示例性实施例示出的一种人脸光照迁移装置的框图。例如，人脸光照迁移装置900可以被提供为一服务器。人脸光照迁移装置900包括处理组件902，其进一步包括一个或多个处理器，以及由存储器903所代表的存储器资源，用于存储可由处理组件902的执行的指令，例如应用程序。存储器903中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件902被配置为执行指令，以执行上述方法。Fig. 9 is a block diagram of an apparatus for face illumination migration according to an exemplary embodiment. For example, the face illumination migration apparatus 900 may be provided as a server. The face illumination migration apparatus 900 includes a processing component 902, which further includes one or more processors, and a memory resource represented by a memory 903 for storing instructions executable by the processing component 902, such as application programs. The application program stored in memory 903 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 902 is configured to execute instructions to perform the above method.
人脸光照迁移装置900还可以包括一个电源组件906被配置为执行人脸光照迁移装置900的电源管理，一个有线或无线网络接口905被配置为将人脸光照迁移装置900连接到网络，和一个输入输出(I/O)接口908。人脸光照迁移装置900可以操作基于存储在存储器903的操作系统，例如Windows ServerTM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTM或类似。The people's face lighting migration device 900 can also include a power supply component 906 configured to perform power management of the people's face lighting migration device 900, a wired or wireless network interface 905 configured to connect the people's face lighting migration device 900 to the network, and a Input Output (I/O) interface 908 . The face illumination migration apparatus 900 can operate based on the operating system stored in the memory 903, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.
一种非临时性计算机可读存储介质，当存储介质中的指令由人脸光照迁移装置800或人脸光照迁移装置900的处理器执行时，使得人脸光照迁移装置800或人脸光照迁移装置900能够执行如下人脸光照迁移方法，方法包括：A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the human face illumination migration device 800 or the human face illumination migration device 900, the human face illumination migration device 800 or the human face illumination migration device The 900 can perform the following face light migration methods, including:
根据目标人脸图像对参照人脸图像进行形变，得到与目标人脸图像对齐的形变后的参照人脸图像；deforming the reference face image according to the target face image to obtain a deformed reference face image aligned with the target face image;
分别对目标人脸图像和形变后的参照人脸图像进行分解，得到形变后的参照人脸图像的结构图、目标人脸图像的色彩图、结构图及细节图；Decomposing the target face image and the deformed reference face image respectively to obtain the structure diagram of the deformed reference face image, the color map, the structure diagram and the detail diagram of the target face image;
以目标人脸图像的结构图为导向图，对形变后的参照人脸图像的结构图进行导向滤波，得到结果人脸图像的结构图；Taking the structure graph of the target face image as the guide graph, performing guided filtering on the deformed reference face image structure graph to obtain the result face image structure graph;
将结果人脸图像的结构图与目标人脸图像的细节图及色彩图进行合成，得到结果人脸图像。Synthesize the structure map of the result face image with the detail map and color map of the target face image to obtain the result face image.
本领域技术人员在考虑说明书及实践这里公开的公开后，将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, and these modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It should be understood that the present disclosure is not limited to the precise constructions which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710751867.4ACN107527334A (en) | 2017-08-28 | 2017-08-28 | Human face light moving method and device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710751867.4ACN107527334A (en) | 2017-08-28 | 2017-08-28 | Human face light moving method and device |
| Publication Number | Publication Date |
|---|---|
| CN107527334Atrue CN107527334A (en) | 2017-12-29 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710751867.4APendingCN107527334A (en) | 2017-08-28 | 2017-08-28 | Human face light moving method and device |
| Country | Link |
|---|---|
| CN (1) | CN107527334A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109949216A (en)* | 2019-04-19 | 2019-06-28 | 中共中央办公厅电子科技学院(北京电子科技学院) | A kind of complicated dressing moving method based on face parsing and illumination migration |
| CN111091492A (en)* | 2019-12-23 | 2020-05-01 | 韶鼎人工智能科技有限公司 | Face image illumination migration method based on convolutional neural network |
| CN113743284A (en)* | 2021-08-30 | 2021-12-03 | 杭州海康威视数字技术股份有限公司 | Image recognition method, device, device, camera and access control device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360513A (en)* | 2011-09-30 | 2012-02-22 | 北京航空航天大学 | Object illumination moving method based on gradient operation |
| CN102509346A (en)* | 2011-09-30 | 2012-06-20 | 北京航空航天大学 | Object illumination migration method based on edge retaining |
| CN102881011A (en)* | 2012-08-31 | 2013-01-16 | 北京航空航天大学 | Region-segmentation-based portrait illumination transfer method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102360513A (en)* | 2011-09-30 | 2012-02-22 | 北京航空航天大学 | Object illumination moving method based on gradient operation |
| CN102509346A (en)* | 2011-09-30 | 2012-06-20 | 北京航空航天大学 | Object illumination migration method based on edge retaining |
| CN102881011A (en)* | 2012-08-31 | 2013-01-16 | 北京航空航天大学 | Region-segmentation-based portrait illumination transfer method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109949216A (en)* | 2019-04-19 | 2019-06-28 | 中共中央办公厅电子科技学院(北京电子科技学院) | A kind of complicated dressing moving method based on face parsing and illumination migration |
| CN109949216B (en)* | 2019-04-19 | 2022-12-02 | 中共中央办公厅电子科技学院(北京电子科技学院) | Complex makeup transfer method based on facial analysis and illumination transfer |
| CN111091492A (en)* | 2019-12-23 | 2020-05-01 | 韶鼎人工智能科技有限公司 | Face image illumination migration method based on convolutional neural network |
| CN113743284A (en)* | 2021-08-30 | 2021-12-03 | 杭州海康威视数字技术股份有限公司 | Image recognition method, device, device, camera and access control device |
| Publication | Publication Date | Title |
|---|---|---|
| CN107622472A (en) | Face makeup transfer method and device | |
| CN109495688B (en) | Photographing preview method of electronic device, graphical user interface and electronic device | |
| US12254544B2 (en) | Image-text fusion method and apparatus, and electronic device | |
| CN106339680B (en) | Face key independent positioning method and device | |
| US20210256672A1 (en) | Method, electronic device and storage medium for processing image | |
| CN105825486B (en) | The method and device of U.S. face processing | |
| WO2020093837A1 (en) | Method for detecting key points in human skeleton, apparatus, electronic device, and storage medium | |
| WO2020088280A1 (en) | Image style transfer method and system | |
| WO2020007241A1 (en) | Image processing method and apparatus, electronic device, and computer-readable storage medium | |
| JP2016531362A (en) | Skin color adjustment method, skin color adjustment device, program, and recording medium | |
| CN106570110A (en) | De-overlapping processing method and apparatus of image | |
| CN107958449A (en) | A kind of image combining method and device | |
| CN108876732A (en) | Face U.S. face method and device | |
| WO2022179025A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| CN107958439A (en) | Image processing method and device | |
| CN107705245A (en) | Image processing method and device | |
| WO2022110837A1 (en) | Image processing method and device | |
| CN114387445A (en) | Object key point identification method and device, electronic equipment and storage medium | |
| CN113570581A (en) | Image processing method and device, electronic device and storage medium | |
| CN107967459A (en) | convolution processing method, device and storage medium | |
| WO2023010912A9 (en) | Image processing method and electronic device | |
| CN107527334A (en) | Human face light moving method and device | |
| WO2020114097A1 (en) | Boundary box determining method and apparatus, electronic device, and storage medium | |
| JP2025513070A (en) | IMAGE PROCESSING METHOD, APPARATUS, ELECTRONIC DEVICE, AND STORAGE MEDIUM | |
| WO2022193573A1 (en) | Facial fusion method and apparatus |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20171229 |