技术领域Technical field
本公开涉及计算机技术领域，尤其涉及增强/虚拟现实、图像处理等人工智能技术领域，具体涉及一种图像处理方法、装置、电子设备、计算机可读存储介质和计算机程序产品。The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology such as augmented/virtual reality and image processing, and specifically to an image processing method, device, electronic equipment, computer-readable storage medium and computer program product.
背景技术Background technique
人工智能是研究使计算机来模拟人的某些思维过程和智能行为(如学习、推理、思考、规划等)的学科，既有硬件层面的技术也有软件层面的技术。人工智能硬件技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理等技术：人工智能软件技术主要包括计算机视觉技术、语音识别技术、自然语言处理技术以及机器学习/深度学习、大数据处理技术、知识图谱技术等几大方向。Artificial intelligence is the study of using computers to simulate certain human thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level technology and software-level technology. Artificial intelligence hardware technology generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, etc. Artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning/depth Learning, big data processing technology, knowledge graph technology and other major directions.
基于人工智能的图像处理技术，已经渗透到各个领域。其中，随着元宇宙的兴起，基于人工智能的图像处理技术，根据图像，生成具有多重人类特征(例如，外貌特征，交互能力等)的虚拟形象，由于其高度拟人化，受到人们的广泛关注。Image processing technology based on artificial intelligence has penetrated into various fields. Among them, with the rise of the metaverse, image processing technology based on artificial intelligence generates virtual images with multiple human characteristics (such as appearance characteristics, interaction capabilities, etc.) based on images. Due to its high degree of anthropomorphism, it has received widespread attention. .
在此部分中描述的方法不一定是之前已经设想到或采用的方法。除非另有指明，否则不应假定此部分中描述的任何方法仅因其包括在此部分中就被认为是现有技术。类似地，除非另有指明，否则此部分中提及的问题不应认为在任何现有技术中已被公认。The approaches described in this section are not necessarily those that have been previously envisioned or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is prior art merely by virtue of its inclusion in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to be recognized in any prior art.
发明内容Contents of the invention
本公开提供了一种图像处理方法、装置、电子设备、计算机可读存储介质和计算机程序产品。The present disclosure provides an image processing method, device, electronic equipment, computer-readable storage medium and computer program product.
根据本公开的一方面，提供了一种图像处理方法，包括：获得与目标对象相应的虚拟形象，所述虚拟形象包括与所述目标对象上的多个目标对象特征点分别对应的多个虚拟形象特征点，所述多个虚拟形象特征点之间的相对空间位置关系指示所述虚拟形象对应于所述目标对象；基于包含所述目标对象的第一图像，确定所述虚拟形象的多个控制参数；以及基于所述多个控制参数，调整所述虚拟形象的多个虚拟对象特征，以与所述目标对象的与所述第一图像相关的多个目标对象特征对应。According to an aspect of the present disclosure, an image processing method is provided, including: obtaining a virtual image corresponding to a target object, the virtual image including a plurality of virtual images respectively corresponding to a plurality of target object feature points on the target object. Image feature points, the relative spatial position relationship between the plurality of virtual image feature points indicates that the virtual image corresponds to the target object; based on the first image containing the target object, determine a plurality of the virtual image Control parameters; and based on the plurality of control parameters, adjust a plurality of virtual object characteristics of the avatar to correspond to a plurality of target object characteristics of the target object related to the first image.
根据本公开的另一方面，提供了一种图像处理装置，包括：第一获取单元，被配置用于获得与目标对象相应的虚拟形象，所述虚拟形象包括与所述目标对象上的多个目标对象特征点分别对应的多个虚拟形象特征点，所述多个虚拟形象特征点之间的相对空间位置关系指示所述虚拟形象对应于所述目标对象；第一确定单元，被配置用于基于包含所述目标对象的第一图像，确定所述虚拟形象的多个控制参数；以及调整单元，被配置用于基于所述多个控制参数，调整所述虚拟形象的多个虚拟对象特征，以与所述目标对象的与所述第一图像相关的多个目标对象特征对应。According to another aspect of the present disclosure, an image processing device is provided, including: a first acquisition unit configured to obtain a virtual image corresponding to a target object, where the virtual image includes a plurality of images corresponding to the target object. A plurality of virtual image feature points respectively corresponding to the target object feature points, and the relative spatial position relationship between the plurality of virtual image feature points indicates that the virtual image corresponds to the target object; a first determination unit configured to determining a plurality of control parameters of the virtual image based on a first image containing the target object; and an adjustment unit configured to adjust a plurality of virtual object characteristics of the virtual image based on the plurality of control parameters, Corresponding to a plurality of target object features of the target object related to the first image.
根据本公开的另一方面，提供了一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器实现根据上述的方法。According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores information that can be executed by the at least one processor. instructions, which are executed by the at least one processor, so that the at least one processor implements the method according to the above.
根据本公开的另一方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机实现根据上述的方法。According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to implement the method according to the above.
根据本公开的另一方面，提供了一种计算机程序产品包括计算机程序，其中，所述计算机程序在被处理器执行时实现根据上述的方法。According to another aspect of the present disclosure, there is provided a computer program product including a computer program, wherein the computer program implements the method according to the above when executed by a processor.
根据本公开的一个或多个实施例，通过包含目标对象的第一图像，调整目标对象对应的虚拟形象的虚拟对象特征，使虚拟形象的虚拟对象特征与第一图像中的目标对象的目标对象特征对应，可以实现根据图像调整目标对象对应的虚拟形象。当图像来自实时捕获的目标对象的目标对象特征发生连续变化的视频帧时，实现根据目标对象的目标对象特征的变化实时调整虚拟形象的虚拟对象特征，实现虚拟形象与目标对象相似的同时，随目标对象变化。According to one or more embodiments of the present disclosure, by including the first image of the target object, the virtual object characteristics of the avatar corresponding to the target object are adjusted so that the virtual object characteristics of the avatar are consistent with the target object of the target object in the first image. Feature correspondence can adjust the virtual image corresponding to the target object based on the image. When the image comes from a video frame in which the target object characteristics of the target object are continuously changed in real-time capture, the virtual object characteristics of the avatar can be adjusted in real time according to the changes in the target object characteristics of the target object, so that the virtual image can be similar to the target object at the same time. Target objects change.
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.
附图说明Description of the drawings
附图示例性地示出了实施例并且构成说明书的一部分，与说明书的文字描述一起用于讲解实施例的示例性实施方式。所示出的实施例仅出于例示的目的，并不限制权利要求的范围。在所有附图中，相同的附图标记指代类似但不一定相同的要素。The drawings illustrate exemplary embodiments and constitute a part of the specification, and together with the written description, serve to explain exemplary implementations of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numbers refer to similar, but not necessarily identical, elements.
图1示出了根据本公开的实施例的可以在其中实施本文描述的各种方法的示例性系统的示意图；1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented in accordance with embodiments of the present disclosure;
图2示出了根据本公开的实施例的图像处理方法的流程图；Figure 2 shows a flowchart of an image processing method according to an embodiment of the present disclosure;
图3示出了根据本公开的实施例的图像处理方法中目标图像中获得与目标对象相应的虚拟形象的过程的流程图；3 shows a flow chart of a process of obtaining an avatar corresponding to a target object from a target image in an image processing method according to an embodiment of the present disclosure;
图4示出了根据本公开的实施例的图像处理方法中确定虚拟形象的多个控制参数的过程的流程图；4 shows a flowchart of a process of determining multiple control parameters of an avatar in an image processing method according to an embodiment of the present disclosure;
图5示出了根据本公开的实施例的图像处理方法中训练用以获得确定虚拟形象的多个控制参数的神经网络模型的示意图；5 shows a schematic diagram of a neural network model trained to obtain multiple control parameters for determining a virtual image in an image processing method according to an embodiment of the present disclosure;
图6示出了根据本公开的实施例的图像处理方法中确定虚拟形象的多个控制参数的过程的流程图；6 shows a flowchart of a process of determining multiple control parameters of an avatar in an image processing method according to an embodiment of the present disclosure;
图7示出了根据本公开的实施例的图像处理方法中基于更新的虚拟形象获得第二图像的过程的流程图；7 shows a flowchart of a process of obtaining a second image based on an updated virtual image in an image processing method according to an embodiment of the present disclosure;
图8示出了根据本公开的实施例的图像处理装置的结构框图；以及8 shows a structural block diagram of an image processing device according to an embodiment of the present disclosure; and
图9示出了能够用于实现本公开的实施例的示例性电子设备的结构框图。9 shows a structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.
具体实施方式Detailed ways
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
在本公开中，除非另有说明，否则使用术语“第一”、“第二”等来描述各种要素不意图限定这些要素的位置关系、时序关系或重要性关系，这种术语只是用于将一个元件与另一元件区分开。在一些示例中，第一要素和第二要素可以指向该要素的同一实例，而在某些情况下，基于上下文的描述，它们也可以指代不同实例。In this disclosure, unless otherwise stated, the use of the terms “first”, “second”, etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements. Such terms are only used for Distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on contextual description, they may refer to different instances.
在本公开中对各种所述示例的描述中所使用的术语只是为了描述特定示例的目的，而并非旨在进行限制。除非上下文另外明确地表明，如果不特意限定要素的数量，则该要素可以是一个也可以是多个。此外，本公开中所使用的术语“和/或”涵盖所列出的项目中的任何一个以及全部可能的组合方式。The terminology used in the description of the various described examples in this disclosure is for the purpose of describing the particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. Furthermore, the term "and/or" as used in this disclosure encompasses any and all possible combinations of the listed items.
下面将结合附图详细描述本公开的实施例。Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
图1示出了根据本公开的实施例可以将本文描述的各种方法和装置在其中实施的示例性系统100的示意图。参考图1，该系统100包括一个或多个客户端设备101、102、103、104、105和106、服务器120以及将一个或多个客户端设备耦接到服务器120的一个或多个通信网络110。客户端设备101、102、103、104、105和106可以被配置为执行一个或多个应用程序。Figure 1 shows a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to FIG. 1 , the system 100 includes one or more client devices 101 , 102 , 103 , 104 , 105 , and 106 , a server 120 , and one or more communication networks coupling the one or more client devices to the server 120 110. Client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
在本公开的实施例中，服务器120可以运行使得能够执行图像处理方法的一个或多个服务或软件应用。In embodiments of the present disclosure, the server 120 may run one or more services or software applications that enable execution of image processing methods.
在某些实施例中，服务器120还可以提供可以包括非虚拟环境和虚拟环境的其他服务或软件应用。在某些实施例中，这些服务可以作为基于web的服务或云服务提供，例如在软件即服务(SaaS)模型下提供给客户端设备101、102、103、104、105和/或106的用户。In some embodiments, server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as under a Software as a Service (SaaS) model to users of client devices 101, 102, 103, 104, 105, and/or 106 .
在图1所示的配置中，服务器120可以包括实现由服务器120执行的功能的一个或多个组件。这些组件可以包括可由一个或多个处理器执行的软件组件、硬件组件或其组合。操作客户端设备101、102、103、104、105和/或106的用户可以依次利用一个或多个客户端应用程序来与服务器120进行交互以利用这些组件提供的服务。应当理解，各种不同的系统配置是可能的，其可以与系统100不同。因此，图1是用于实施本文所描述的各种方法的系统的一个示例，并且不旨在进行限制。In the configuration shown in FIG. 1 , server 120 may include one or more components that implement the functions performed by server 120 . These components may include software components, hardware components, or combinations thereof that are executable by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and/or 106 may, in turn, utilize one or more client applications to interact with server 120 to utilize services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100 . Accordingly, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
用户可以使用客户端设备101、102、103、104、105和/或106来观看包含虚拟形象的图像。客户端设备可以提供使客户端设备的用户能够与客户端设备进行交互的接口。客户端设备还可以经由该接口向用户输出信息。尽管图1仅描绘了六种客户端设备，但是本领域技术人员将能够理解，本公开可以支持任何数量的客户端设备。Users may use client devices 101, 102, 103, 104, 105, and/or 106 to view images containing avatars. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device can also output information to the user via the interface. Although FIG. 1 depicts only six client devices, those skilled in the art will understand that the present disclosure can support any number of client devices.
客户端设备101、102、103、104、105和/或106可以包括各种类型的计算机设备，例如便携式手持设备、通用计算机(诸如个人计算机和膝上型计算机)、工作站计算机、可穿戴设备、智能屏设备、自助服务终端设备、服务机器人、游戏系统、瘦客户端、各种消息收发设备、传感器或其他感测设备等。这些计算机设备可以运行各种类型和版本的软件应用程序和操作系统，例如MICROSOFT Windows、APPLE iOS、类UNIX操作系统、Linux或类Linux操作系统(例如GOOGLE Chrome OS)；或包括各种移动操作系统，例如MICROSOFT WindowsMobile OS、iOS、Windows Phone、Android。便携式手持设备可以包括蜂窝电话、智能电话、平板电脑、个人数字助理(PDA)等。可穿戴设备可以包括头戴式显示器(诸如智能眼镜)和其他设备。游戏系统可以包括各种手持式游戏设备、支持互联网的游戏设备等。客户端设备能够执行各种不同的应用程序，例如各种与Internet相关的应用程序、通信应用程序(例如电子邮件应用程序)、短消息服务(SMS)应用程序，并且可以使用各种通信协议。Client devices 101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, Smart screen equipment, self-service terminal equipment, service robots, game systems, thin clients, various messaging equipment, sensors or other sensing equipment, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems , such as MICROSOFT WindowsMobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), and the like. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, Internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (such as email applications), Short Message Service (SMS) applications, and can use various communication protocols.
网络110可以是本领域技术人员熟知的任何类型的网络，其可以使用多种可用协议中的任何一种(包括但不限于TCP/IP、SNA、IPX等)来支持数据通信。仅作为示例，一个或多个网络110可以是局域网(LAN)、基于以太网的网络、令牌环、广域网(WAN)、因特网、虚拟网络、虚拟专用网络(VPN)、内部网、外部网、公共交换电话网(PSTN)、红外网络、无线网络(例如蓝牙、WIFI)和/或这些和/或其他网络的任意组合。Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols (including, but not limited to, TCP/IP, SNA, IPX, etc.). By way of example only, one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, Public Switched Telephone Network (PSTN), infrared network, wireless network (e.g. Bluetooth, WIFI) and/or any combination of these and/or other networks.
服务器120可以包括一个或多个通用计算机、专用服务器计算机(例如PC(个人计算机)服务器、UNIX服务器、中端服务器)、刀片式服务器、大型计算机、服务器群集或任何其他适当的布置和/或组合。服务器120可以包括运行虚拟操作系统的一个或多个虚拟机，或者涉及虚拟化的其他计算架构(例如可以被虚拟化以维护服务器的虚拟存储设备的逻辑存储设备的一个或多个灵活池)。在各种实施例中，服务器120可以运行提供下文所描述的功能的一个或多个服务或软件应用。Server 120 may include one or more general purpose computers, special purpose server computers (eg, PC (Personal Computer) servers, UNIX servers, midrange servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination . Server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (eg, one or more flexible pools of logical storage devices that may be virtualized to maintain the server's virtual storage devices). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.
服务器120中的计算单元可以运行包括上述任何操作系统以及任何商业上可用的服务器操作系统的一个或多个操作系统。服务器120还可以运行各种附加服务器应用程序和/或中间层应用程序中的任何一个，包括HTTP服务器、FTP服务器、CGI服务器、JAVA服务器、数据库服务器等。Computing units in server 120 may run one or more operating systems, including any of the operating systems described above, as well as any commercially available server operating system. Server 120 may also run any of a variety of additional server applications and/or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.
在一些实施方式中，服务器120可以包括一个或多个应用程序，以分析和合并从客户端设备101、102、103、104、105和106的用户接收的数据馈送和/或事件更新。服务器120还可以包括一个或多个应用程序，以经由客户端设备101、102、103、104、105和106的一个或多个显示设备来显示数据馈送和/或实时事件。In some implementations, server 120 may include one or more applications to analyze and incorporate data feeds and/or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of client devices 101 , 102 , 103 , 104 , 105 , and 106 .
在一些实施方式中，服务器120可以为分布式系统的服务器，或者是结合了区块链的服务器。服务器120也可以是云服务器，或者是带人工智能技术的智能云计算服务器或智能云主机。云服务器是云计算服务体系中的一项主机产品，以解决传统物理主机与虚拟专用服务器(VPS，Virtual Private Server)服务中存在的管理难度大、业务扩展性弱的缺陷。In some implementations, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. Cloud server is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical host and virtual private server (VPS) services.
系统100还可以包括一个或多个数据库130。在某些实施例中，这些数据库可以用于存储数据和其他信息。例如，数据库130中的一个或多个可用于存储诸如音频文件和对象文件的信息。数据存储库130可以驻留在各种位置。例如，由服务器120使用的数据存储库可以在服务器120本地，或者可以远离服务器120且可以经由基于网络或专用的连接与服务器120通信。数据存储库130可以是不同的类型。在某些实施例中，由服务器120使用的数据存储库可以是数据库，例如关系数据库。这些数据库中的一个或多个可以响应于命令而存储、更新和检索到数据库以及来自数据库的数据。System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of databases 130 may be used to store information such as audio files and object files. Data repository 130 may reside in various locations. For example, a data repository used by server 120 may be local to server 120, or may be remote from server 120 and may communicate with server 120 via a network-based or dedicated connection. Data repository 130 may be of different types. In some embodiments, the data repository used by server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to commands.
在某些实施例中，数据库130中的一个或多个还可以由应用程序使用来存储应用程序数据。由应用程序使用的数据库可以是不同类型的数据库，例如键值存储库，对象存储库或由文件系统支持的常规存储库。In some embodiments, one or more of databases 130 may also be used by applications to store application data. The database used by the application can be different types of databases such as key-value repositories, object repositories or regular repositories backed by a file system.
图1的系统100可以以各种方式配置和操作，以使得能够应用根据本公开所描述的各种方法和装置。The system 100 of Figure 1 may be configured and operated in various ways to enable the application of the various methods and apparatus described in accordance with the present disclosure.
参看图2，根据本公开的一些实施例的一种图像处理方法200包括：Referring to Figure 2, an image processing method 200 according to some embodiments of the present disclosure includes:
步骤S210：获得与目标对象相应的虚拟形象；Step S210: Obtain the virtual image corresponding to the target object;
步骤S220：基于包含所述目标对象的第一图像，确定所述虚拟形象的多个控制参数；以及Step S220: Determine a plurality of control parameters of the virtual image based on the first image containing the target object; and
步骤S230：基于所述多个控制参数，调整所述虚拟形象的多个虚拟对象特征。Step S230: Adjust multiple virtual object characteristics of the avatar based on the multiple control parameters.
其中，在步骤S210中，所述虚拟形象包括与所述目标对象上的多个目标对象特征点分别对应的多个虚拟形象特征点，所述多个虚拟形象特征点之间的相对空间位置关系指示所述虚拟形象对应于所述目标对象；以及在步骤S230中，所述调整所述虚拟形象的多个虚拟对象特征经调整后与目标对象的与所述第一图像相关的多个目标对象特征对应。Wherein, in step S210, the virtual image includes a plurality of virtual image feature points respectively corresponding to a plurality of target object feature points on the target object, and the relative spatial position relationship between the multiple virtual image feature points is Indicate that the virtual image corresponds to the target object; and in step S230, the adjusting a plurality of virtual object features of the virtual image is adjusted to a plurality of target objects of the target object related to the first image. Feature correspondence.
通过包含目标对象的第一图像，调整目标对象对应的虚拟形象的虚拟对象特征，使虚拟形象的虚拟对象特征与第一图像中的目标对象的目标对象特征对应，可以实现根据图像调整目标对象对应的虚拟形象。当图像来自实时捕获的目标对象的目标对象特征发生连续变化的视频帧时，实现根据目标对象的目标对象特征的变化实时调整虚拟形象的虚拟对象特征，实现虚拟形象与目标对象相似的同时，随目标对象变化。By adjusting the virtual object characteristics of the virtual image corresponding to the target object by including the first image of the target object, so that the virtual object characteristics of the virtual image correspond to the target object characteristics of the target object in the first image, the target object correspondence can be adjusted according to the image. virtual image. When the image comes from a video frame in which the target object characteristics of the target object are continuously changed in real-time capture, the virtual object characteristics of the avatar can be adjusted in real time according to the changes in the target object characteristics of the target object, so that the virtual image can be similar to the target object at the same time. Target objects change.
在相关技术中，根据直播视频帧中的人脸图像确定个性化表情系数，以应用到用来代替主播自身形象进行虚拟直播的个性化虚拟形象上，从而驱动个性化虚拟形象的表情。其中，根据直播视频帧中的人脸图像确定个性化表情系数的方法是首先根据标准表情基组确定与人脸图像对应的标准表情系数，再根据标准表情系数确定与之对应的个性化表情系数。这种方法难以复用，对于不同的虚拟形象，往往具有不同的标准表情基组，使标准表情基组绑定的标准表情系数难以迁移。In related technologies, personalized expression coefficients are determined based on face images in live video frames and applied to the personalized virtual image used to replace the anchor's own image for virtual live broadcast, thereby driving the expression of the personalized virtual image. Among them, the method of determining the personalized expression coefficient based on the face image in the live video frame is to first determine the standard expression coefficient corresponding to the face image based on the standard expression basis set, and then determine the corresponding personalized expression coefficient based on the standard expression coefficient. . This method is difficult to reuse. Different virtual images often have different standard expression base sets, making it difficult to migrate the standard expression coefficients bound to the standard expression base sets.
在根据本公开的实施例中，可以针对不同的目标对象，通过获得与之相应的虚拟形象，并通过包含相应的目标对象的图像，获得虚拟形象的控制参数，使虚拟形象的虚拟对象特征与图像中的与之相应的目标对象的目标对象特征相对应，实现针对不同的虚拟形象可以进行与之相应的调整，使对虚拟形象的调整更加灵活。同时，由于虚拟形象和目标对象对应，其上的多个虚拟形象特征点的相对空间位置关系指示其对应于目标对象，通过根据本公开的实施例，进一步使虚拟形象的虚拟对象特征能够被调整为与目标对象在图像中的目标对象特征(例如，颜色、空间位姿、光照纹理等)更加一致或相似，使虚拟形象和目标对象相似度更高。In embodiments according to the present disclosure, the control parameters of the virtual image can be obtained for different target objects by obtaining corresponding virtual images, and by including images of the corresponding target objects, so that the virtual object characteristics of the virtual image are consistent with The target object characteristics of the corresponding target object in the image correspond to each other, so that different virtual images can be adjusted accordingly, making the adjustment of the virtual image more flexible. At the same time, since the virtual image corresponds to the target object, the relative spatial position relationship of the multiple virtual image feature points thereon indicates that it corresponds to the target object. Through embodiments according to the present disclosure, the virtual object characteristics of the virtual image can be further adjusted. In order to be more consistent or similar to the target object's characteristics (for example, color, spatial pose, lighting texture, etc.) in the image, the similarity between the virtual image and the target object is higher.
在一些实施例中，目标对象包括人脸，虚拟形象包括与人脸对应的人脸模型。In some embodiments, the target object includes a human face, and the avatar includes a facial model corresponding to the human face.
在一些实施例中，目标对象还包括动物的脸、人的身体、动物的身体、植物、车辆等可以随时间发生变化或移动的任何对象，在此并不限定。In some embodiments, target objects also include animal faces, human bodies, animal bodies, plants, vehicles, and other objects that can change or move over time, which are not limited here.
在一些实施例中，虚拟形象可以包括根据目标对象在现实空间中的坐标位置计算的位于虚拟空间中的多个坐标位置，虚拟形象上的多个虚拟形象特征点在虚拟空间中对应的多个坐标位置与目标对象上的多个目标对象特征点在现实空间中对应的多个坐标位置分别对应。In some embodiments, the virtual image may include multiple coordinate positions in the virtual space calculated based on the coordinate position of the target object in the real space. Multiple virtual image feature points on the virtual image correspond to multiple coordinate positions in the virtual space. The coordinate positions correspond to multiple coordinate positions corresponding to multiple target object feature points on the target object in real space.
在一些实施例中，虚拟形象可以是基于目标对象的特征(例如，年龄、性别、发型等)从多个虚拟模型中获得的与之特征对应的虚拟模型。In some embodiments, the avatar may be a virtual model corresponding to the characteristics of the target object obtained from multiple virtual models based on the characteristics of the target object (eg, age, gender, hairstyle, etc.).
在一些实施例中，如图3所示，获得与目标对象相应的虚拟形象包括：In some embodiments, as shown in Figure 3, obtaining an avatar corresponding to the target object includes:
步骤S310：获取包含目标对象的目标图像；Step S310: Obtain the target image containing the target object;
步骤S320：对所述目标图像进行人脸对齐处理，以获得标准图像；以及Step S320: Perform face alignment processing on the target image to obtain a standard image; and
步骤S330：基于所述标准图像，生成所述虚拟形象。Step S330: Generate the virtual image based on the standard image.
通过基于对包含目标对象的目标图像进行人脸对齐处理后获得的标准图像，生成虚拟形象，使生成的虚拟形象与目标对象相似度更高，使后续基于该虚拟形象进行调整而获得的调整后的虚拟形象与目标对象相似度更高。By generating an avatar based on a standard image obtained by face alignment processing of a target image containing the target object, the generated avatar is more similar to the target object, and the adjusted image obtained by subsequent adjustments based on the avatar is The virtual image is more similar to the target object.
在一些实施例中，在步骤S310中，目标图像例如可以是用户输入的证件照、人脸自拍照或者从任意包含人的人脸的照片中获得的图像，在此并不限定。In some embodiments, in step S310, the target image may be, for example, an ID photo input by the user, a face selfie, or an image obtained from any photo containing a person's face, which is not limited here.
在一些实施例中，在步骤S320中，通过对目标图像中的人脸进行72个关键点检测，分别获得72个关键点坐标，基于72个关键点坐标调整图像中的人脸，使人脸为正脸，并且基于72个关键点获得包含人脸检测框的，基于人脸检测框获得该标准图像，其中该标准图像包括大于该人脸检测框所包围的人脸区域。In some embodiments, in step S320, 72 key point coordinates are obtained by detecting 72 key points on the human face in the target image, and the human face in the image is adjusted based on the 72 key point coordinates so that the human face is a frontal face, and a face detection frame containing a face is obtained based on 72 key points, and the standard image is obtained based on the face detection frame, where the standard image includes a face area larger than that surrounded by the face detection frame.
在一些实施例中，在步骤S330中，通过将目标图像输入至虚拟形象生成模型，生成该虚拟形象。虚拟形象生成模型，例如可以包括生成对抗网络或者循环生成网络等。In some embodiments, in step S330, the avatar is generated by inputting the target image into the avatar generation model. The avatar generation model may include, for example, a generative adversarial network or a recurrent generation network.
在一些实施例中，获得与目标对象对应的虚拟形象的过程中，还可以基于用户从多个风格类型中确定的目标风格类型，获得具有目标风格类型的虚拟形象。In some embodiments, in the process of obtaining the avatar corresponding to the target object, an avatar with the target style type may also be obtained based on the target style type determined by the user from multiple style types.
在一些示例中，多个风格类型包括：古风类型、漫画类型和玩偶、阿凡达(Avatar)风格、迪士尼风格等等。In some examples, the multiple style types include: antique style, comic style and doll, Avatar style, Disney style, etc.
在一些实施例中，还可以根据目标用户从多个属性特征中确定的目标属性特征，获得具有目标属性特征的虚拟形象。In some embodiments, an avatar with target attribute characteristics can also be obtained based on the target attribute characteristics determined by the target user from multiple attribute characteristics.
在一些示例中，多个属性特征可以是眼镜类型、头饰类型等等，在此不作限制。In some examples, the multiple attribute features may be glasses type, headgear type, etc., which are not limited here.
在一些实施例中，所述多个目标对象特征包括与所述第一图像的图像特征相关的至少一个目标对象特征，所述多个控制参数包括至少一个渲染参数，其中，如图4所示，基于包含所述目标对象的第一图像，确定所述虚拟形象的多个控制参数包括：In some embodiments, the plurality of target object features include at least one target object feature related to image features of the first image, and the plurality of control parameters include at least one rendering parameter, wherein, as shown in FIG. 4 , based on the first image containing the target object, determining a plurality of control parameters of the virtual image includes:
步骤S410：提取所述第一图像的图像特征；以及Step S410: Extract image features of the first image; and
步骤S420：基于所提取的图像特征，获得所述至少一个渲染参数。Step S420: Obtain the at least one rendering parameter based on the extracted image features.
其中，基于所述多个控制参数，调整所述虚拟形象的多个虚拟对象特征包括：基于所述至少一个渲染参数渲染所述多个虚拟形象特征点，使所述多个虚拟形象特征点的颜色特征与所述第一图像中的所述多个目标对象特征点的颜色特征对应。Wherein, adjusting multiple virtual object features of the virtual image based on the multiple control parameters includes: rendering the multiple virtual image feature points based on the at least one rendering parameter, so that the multiple virtual image feature points are The color features correspond to color features of the plurality of target object feature points in the first image.
通过至少一个渲染参数调整虚拟形象的多个虚拟对象特征，使虚拟形象上的多个虚拟形象特征点与第一图像中的多个目标对象特征点的颜色特征分别对应，实现虚拟形象和目标对象在像素级别的一对一对应，提升虚拟形象与目标对象的相似性。The plurality of virtual object features of the virtual image are adjusted by at least one rendering parameter, so that the plurality of virtual image feature points on the virtual image correspond to the color features of the plurality of target object feature points in the first image, thereby realizing the virtual image and the target object. One-to-one correspondence at the pixel level improves the similarity between the virtual image and the target object.
在一些实施例中，通过经训练的神经网络模型，提取该图像特征，并且基于该图像特征获得至少一个渲染参数。In some embodiments, the image features are extracted through a trained neural network model, and at least one rendering parameter is obtained based on the image features.
具体的，如图5示出了神经网络模型的训练过程，其中，神经网络模型500采用可微分渲染器510辅助训练，在训练过程中，首先，将包含训练目标对象的训练图像501A输入至神经网络模型500并输出渲染参数502A、502B、503C；接着，将渲染参数502A、502B、503C与训练图像501A中的训练目标对象对应的虚拟形象503a，进一步输入至可微分渲染器510，输出经调整的虚拟形象504a’，同时虚拟形象504a’与采集训练图像501A的过程中基于训练目标对象获得的标注数据501B进行对比，获得对抗损失，以调整神经网络模型500的参数。其中，在获得对抗损失的过程中，通过将标注数据和经调整的虚拟形象504a’计算RGB空间的一对一dense损失值，从而达到像素级精炼的效果。Specifically, Figure 5 shows the training process of the neural network model. The neural network model 500 uses the differentiable renderer 510 to assist in training. During the training process, first, the training image 501A containing the training target object is input to the neural network model. The network model 500 also outputs the rendering parameters 502A, 502B, and 503C; then, the rendering parameters 502A, 502B, and 503C and the virtual image 503a corresponding to the training target object in the training image 501A are further input to the differentiable renderer 510, and the adjusted output At the same time, the virtual image 504a' is compared with the annotation data 501B obtained based on the training target object in the process of collecting the training image 501A to obtain the adversarial loss to adjust the parameters of the neural network model 500. Among them, in the process of obtaining the adversarial loss, the one-to-one dense loss value in the RGB space is calculated by combining the annotation data and the adjusted virtual image 504a', thereby achieving a pixel-level refining effect.
在一些实施例中，所述至少一个目标对象特征包括：目标对象纹理特征、目标对象形状特征和目标对象表情特征，所述至少一个渲染参数包括纹理参数、形状参数和表情参数，所述所提取的图像特征包括与纹理特征、形状特征和几何空间特征。In some embodiments, the at least one target object feature includes: a target object texture feature, a target object shape feature, and a target object expression feature; the at least one rendering parameter includes a texture parameter, a shape parameter, and an expression parameter; and the extracted The image features include texture features, shape features and geometric space features.
通过获得包括纹理参数、形状参数和表情参数的至少一个渲染参数，使基于纹理参数、形状参数和表情参数渲染的虚拟形象，在人脸的表情特征、纹理特征和形状特征等与图像中的人脸相似，使基于上述至少一个渲染参数渲染后的虚拟形象和目标对象相似度更高。By obtaining at least one rendering parameter including a texture parameter, a shape parameter, and an expression parameter, the virtual image rendered based on the texture parameter, the shape parameter, and the expression parameter is made to match the facial expression features, texture features, and shape features of the person in the image. Face similarity makes the virtual image rendered based on at least one of the above rendering parameters more similar to the target object.
在一些示例中，目标对象纹理特征包括人脸的皱纹、肤色、以及光照纹理等。In some examples, the target object texture features include facial wrinkles, skin color, lighting texture, etc.
在一些示例中，目标对象形状特征包括五官(眼睛、鼻子和嘴巴)的尺寸等。In some examples, the target object shape characteristics include the size of facial features (eyes, nose, and mouth), etc.
在一些示例中，目标对象的表情特征包括与五官(眼睛、鼻子和嘴巴)的尺寸相关的特征。In some examples, the target object's expression features include features related to the size of the facial features (eyes, nose, and mouth).
在一些示例中，纹理特征包括光照、目标对象的肤色等特征。In some examples, texture features include lighting, skin color of the target object, and other features.
在一些示例中，形状特征包括目标对象的脸型、五官(眼睛、鼻子和嘴巴)尺寸等特征。In some examples, the shape features include features such as face shape, facial features (eyes, nose, and mouth) size of the target object.
在一些示例中，几何空间特征包括目标对象的五官(眼睛、鼻子和嘴巴)相对位置等。In some examples, the geometric spatial features include the relative positions of the target object's facial features (eyes, nose, and mouth), etc.
在一些实施例中，所述多个目标对象特征包括所述第一图像对应的相机坐标系相关的目标对象位姿特征，所述多个控制参数包括旋转平移参数，所述基于包含所述目标对象的第一图像，如图6所示、基于包含所述目标对象的第一图像，确定所述虚拟形象的多个控制参数包括：In some embodiments, the plurality of target object characteristics include target object pose characteristics related to the camera coordinate system corresponding to the first image, the plurality of control parameters include rotation and translation parameters, and the plurality of control parameters include the target object based on The first image of the object, as shown in Figure 6, based on the first image containing the target object, determining a plurality of control parameters of the virtual image includes:
步骤S610：提取所述第一图像对应的相机坐标系和所述目标对象的所述目标对象位姿特征；以及Step S610: Extract the camera coordinate system corresponding to the first image and the target object pose characteristics of the target object; and
步骤S620：基于所述相机坐标系和所述目标对象位姿特征，确定所述旋转平移参数。Step S620: Determine the rotation and translation parameters based on the camera coordinate system and the pose characteristics of the target object.
基于所述多个控制参数，调整所述虚拟形象的多个虚拟对象特征包括：基于所述旋转平移参数，调整所述多个虚拟形象特征点的空间位置。Based on the plurality of control parameters, adjusting the plurality of virtual object features of the virtual image includes: adjusting the spatial positions of the plurality of virtual image feature points based on the rotation and translation parameters.
通过获得用于调整虚拟形象的虚拟对象位姿特征的旋转平移参数，通过调整虚拟形象特征点的空间位置(例如空间位置坐标)，进一步调整虚拟形象的虚拟对象位姿特征与目标对象的目标对象位姿特征对应，使调整后的虚拟形象与第一图像中的目标对象在姿态上与保持一致，进一步提升虚拟形象与第一图像中的目标对象的相似性。By obtaining the rotation and translation parameters used to adjust the virtual object pose characteristics of the avatar, and by adjusting the spatial position (such as spatial position coordinates) of the virtual image feature points, the virtual object pose characteristics of the avatar and the target object of the target object are further adjusted. The posture feature correspondence ensures that the adjusted virtual image is consistent in posture with the target object in the first image, further improving the similarity between the virtual image and the target object in the first image.
例如，第一图像中的人脸为侧脸，经过旋转平移参数调整后的虚拟形象也转向与该侧脸相对的一侧。For example, the human face in the first image is a side face, and the virtual image adjusted by the rotation and translation parameters also turns to the side opposite to the side face.
在一些实施例中，通过采用上述参考图5介绍的经训练的神经网络模型，执行步骤S610和步骤S620，以获得该旋转平移参数。In some embodiments, step S610 and step S620 are performed by using the trained neural network model introduced above with reference to FIG. 5 to obtain the rotation and translation parameters.
在一些实施例中，进一步基于经过多个控制参数调整后的虚拟形象，输出相应的图像。In some embodiments, corresponding images are output based on the virtual image adjusted by multiple control parameters.
在一些实施例中，如图7所示、图像处理方法还包括：In some embodiments, as shown in Figure 7, the image processing method further includes:
步骤S710：基于经调整的虚拟形象，获得第二图像；以及Step S710: Obtain a second image based on the adjusted virtual image; and
步骤S720：输出所述第二图像。Step S720: Output the second image.
通过基于经调整的虚拟形象获得第二图像，并输出第二图像，实现根据第一图像实时驱动虚拟形象的变化，并且将虚拟形象的变化可视化。By obtaining a second image based on the adjusted virtual image and outputting the second image, real-time driving of changes in the virtual image based on the first image is achieved, and changes in the virtual image are visualized.
在一些实施例中，根据旋转平移参数对虚拟形象进行位姿调整，通过渲染器将至少一个渲染参数渲染到虚拟形象上，获得经调整的虚拟形象。In some embodiments, the pose of the virtual image is adjusted according to the rotation and translation parameters, and the renderer renders at least one rendering parameter to the virtual image to obtain the adjusted virtual image.
在一些实施例中，基于虚拟相机获得第二图像。In some embodiments, the second image is obtained based on a virtual camera.
根据本公开的另一方面，还提供一种图像处理装置，参看图8、装置800包括：第一获取单元810，被配置用于获得与目标对象相应的虚拟形象，其中，所述虚拟形象包括与所述目标对象上的多个目标对象特征点分别对应的多个虚拟形象特征点，所述多个虚拟形象特征点之间的相对空间位置关系指示所述虚拟形象对应于所述目标对象；第一确定单元820，被配置用于基于包含所述目标对象的第一图像，确定所述虚拟形象的多个控制参数，以及调整单元，被配置用于基于所述多个控制参数，调整所述虚拟形象的多个虚拟对象特征，以与所述目标对象的与所述第一图像相关的多个目标对象特征对应。According to another aspect of the present disclosure, an image processing device is also provided. Referring to Figure 8, the device 800 includes: a first acquisition unit 810 configured to obtain a virtual image corresponding to the target object, wherein the virtual image includes A plurality of virtual image feature points respectively corresponding to a plurality of target object feature points on the target object, and the relative spatial position relationship between the plurality of virtual image feature points indicates that the virtual image corresponds to the target object; The first determining unit 820 is configured to determine a plurality of control parameters of the virtual image based on the first image containing the target object, and the adjusting unit is configured to adjust the control parameters based on the plurality of control parameters. A plurality of virtual object characteristics of the virtual image are configured to correspond to a plurality of target object characteristics of the target object related to the first image.
在一些实施例中，所述第一获取单元810包括：目标图像获取单元，被配置用于获取包含目标对象的目标图像；标准图像获取单元，被配置用于对所述目标图像进行人脸对齐处理，以获得标准图像；以及虚拟形象生成单元，被配置用于基于所述标准图像，生成所述虚拟形象。In some embodiments, the first acquisition unit 810 includes: a target image acquisition unit configured to acquire a target image containing a target object; a standard image acquisition unit configured to perform face alignment on the target image processing to obtain a standard image; and an avatar generation unit configured to generate the avatar based on the standard image.
在一些实施例中，所述多个目标对象特征包括与所述第一图像的图像特征相关的至少一个目标对象特征，所述多个控制参数包括至少一个渲染参数，所述第一确定单元820包括：第一特征提取单元，被配置用于提取所述第一图像的图像特征；以及第一确定子单元，被配置用于基于所提取的图像特征，获得所述至少一个渲染参数，并且其中，所述调整单元被配置用于基于所述至少一个渲染参数，渲染所述多个虚拟形象特征点，使所述多个虚拟形象特征点的颜色特征与所述第一图像中的所述多个目标对象特征点的颜色特征对应。In some embodiments, the plurality of target object features include at least one target object feature related to image features of the first image, the plurality of control parameters include at least one rendering parameter, and the first determining unit 820 including: a first feature extraction unit configured to extract image features of the first image; and a first determination subunit configured to obtain the at least one rendering parameter based on the extracted image features, and wherein , the adjustment unit is configured to render the plurality of virtual image feature points based on the at least one rendering parameter, so that the color characteristics of the plurality of virtual image feature points are consistent with the plurality of virtual image feature points in the first image. Correspondence of color features of feature points of target objects.
在一些实施例中，所述至少一个目标对象特征包括：纹理特征、形状特征和表情特征，所述至少一个渲染参数包括纹理参数、形状参数和表情参数，以及所述所提取的图像特征包括与纹理特征、形状特征和几何空间特征。In some embodiments, the at least one target object feature includes: texture features, shape features and expression features, the at least one rendering parameter includes texture parameters, shape parameters and expression parameters, and the extracted image features include and Texture features, shape features and geometric space features.
在一些实施例中，所述多个目标对象特征包括所述第一图像对应的相机坐标系与目标对象位姿特征，所述多个控制参数包括旋转平移参数，所述第一确定单元820包括：第二特征提取单元，被配置用于提取所述第一图像对应的相机坐标系和所述目标对象的所述目标对象位姿特征；以及第二确定子单元，被配置用于基于所述相机坐标系和所述目标对象位姿特征，确定所述旋转平移参数；并且其中，所述调整单元被配置用于基于所述旋转平移参数，调整所述多个虚拟形象特征点的空间位置。In some embodiments, the plurality of target object characteristics include the camera coordinate system corresponding to the first image and the target object pose characteristics, the plurality of control parameters include rotation and translation parameters, and the first determination unit 820 includes : a second feature extraction unit configured to extract the camera coordinate system corresponding to the first image and the target object pose feature of the target object; and a second determination subunit configured to extract based on the The camera coordinate system and the pose characteristics of the target object determine the rotation and translation parameters; and wherein the adjustment unit is configured to adjust the spatial positions of the plurality of virtual image feature points based on the rotation and translation parameters.
在一些实施例中，装置800还包括：第二图像生成单元，被配置用于基于经调整的虚拟形象，获得第二图像，以及输出单元，被配置用于输出所述第二图像。In some embodiments, the apparatus 800 further includes a second image generation unit configured to obtain a second image based on the adjusted virtual image, and an output unit configured to output the second image.
根据本公开的另一方面，还提供一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中所述存储器存储有计算机程序，所述计算机程序在被所述至少一个处理器执行时实现根据上述的方法。According to another aspect of the present disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, and the computer program is The at least one processor implements the method according to the above when executed.
根据本公开的另一方面，还提供一种存储有计算机程序的非瞬时计算机可读存储介质，其中，所述计算机程序在被处理器执行时实现根据上述的方法。According to another aspect of the present disclosure, there is also provided a non-transitory computer-readable storage medium storing a computer program, wherein the computer program implements the method according to the above when executed by a processor.
根据本公开的另一方面，还提供了一种计算机程序产品，包括计算机程序，其中，所述计算机程序在被处理器执行时实现根据上述的方法。According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the computer program implements the method according to the above when executed by a processor.
参考图9，现将描述可以作为本公开的服务器或客户端的电子设备900的结构框图，其是可以应用于本公开的各方面的硬件设备的示例。电子设备旨在表示各种形式的数字电子的计算机设备，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。Referring to FIG. 9 , a structural block diagram of an electronic device 900 that may serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that may be applied to aspects of the present disclosure. Electronic devices are intended to refer to various forms of digital electronic computing equipment, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.
如图9所示，电子设备900包括计算单元901，其可以根据存储在只读存储器(ROM)902中的计算机程序或者从存储单元908加载到随机访问存储器(RAM)903中的计算机程序，来执行各种适当的动作和处理。在RAM903中，还可存储电子设备900操作所需的各种程序和数据。计算单元901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(I/O)接口905也连接至总线904。As shown in FIG. 9 , the electronic device 900 includes a computing unit 901 that can perform calculations according to a computer program stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903 . Perform various appropriate actions and processing. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. Computing unit 901, ROM 902 and RAM 903 are connected to each other via bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
电子设备900中的多个部件连接至I/O接口905，包括：输入单元906、输出单元907、存储单元908以及通信单元909。输入单元906可以是能向电子设备900输入信息的任何类型的设备，输入单元906可以接收输入的数字或字符信息，以及产生与电子设备的用户设置和/或功能控制有关的键信号输入，并且可以包括但不限于鼠标、键盘、触摸屏、轨迹板、轨迹球、操作杆、麦克风和/或遥控器。输出单元907可以是能呈现信息的任何类型的设备，并且可以包括但不限于显示器、扬声器、对象/音频输出终端、振动器和/或打印机。存储单元908可以包括但不限于磁盘、光盘。通信单元909允许电子设备900通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据，并且可以包括但不限于调制解调器、网卡、红外通信设备、无线通信收发机和/或芯片组，例如蓝牙TM设备、802.11设备、WiFi设备、WiMax设备、蜂窝通信设备和/或类似物。Multiple components in the electronic device 900 are connected to the I/O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 may be any type of device capable of inputting information to the electronic device 900, the input unit 906 may receive input numeric or character information, and generate key signal input related to user settings and/or function control of the electronic device, and This may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and/or remote control. The output unit 907 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, an object/audio output terminal, a vibrator, and/or a printer. The storage unit 908 may include, but is not limited to, magnetic disks and optical disks. The communication unit 909 allows the electronic device 900 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and/or a chip Groups such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communications devices, and/or the like.
计算单元901可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元901的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元901执行上文所描述的各个方法和处理，例如方法200。例如，在一些实施例中，方法200可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元908。在一些实施例中，计算机程序的部分或者全部可以经由ROM 902和/或通信单元909而被载入和/或安装到电子设备900上。当计算机程序加载到RAM 903并由计算单元901执行时，可以执行上文描述的方法200的一个或多个步骤。备选地，在其他实施例中，计算单元901可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行方法200。Computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto electronic device 900 via ROM 902 and/or communication unit 909 . When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, computing unit 901 may be configured to perform method 200 in any other suitable manner (eg, by means of firmware).
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or a combination thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，也可以为分布式系统的服务器，或者是结合了区块链的服务器。Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, a distributed system server, or a server combined with a blockchain.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本公开中记载的各步骤可以并行地执行、也可以顺序地或以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution disclosed in the present disclosure can be achieved, there is no limitation here.
虽然已经参照附图描述了本公开的实施例或示例，但应理解，上述的方法、系统和设备仅仅是示例性的实施例或示例，本发明的范围并不由这些实施例或示例限制，而是仅由授权后的权利要求书及其等同范围来限定。实施例或示例中的各种要素可以被省略或者可由其等同要素替代。此外，可以通过不同于本公开中描述的次序来执行各步骤。进一步地，可以以各种方式组合实施例或示例中的各种要素。重要的是随着技术的演进，在此描述的很多要素可以由本公开之后出现的等同要素进行替换。Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are only exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples. It is limited only by the granted claims and their equivalent scope. Various elements in the embodiments or examples may be omitted or replaced by equivalent elements thereof. Furthermore, the steps may be performed in a different order than described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many elements described herein may be replaced by equivalent elements appearing after this disclosure.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111435489.1ACN114119935B (en) | 2021-11-29 | 2021-11-29 | Image processing methods and devices |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111435489.1ACN114119935B (en) | 2021-11-29 | 2021-11-29 | Image processing methods and devices |
| Publication Number | Publication Date |
|---|---|
| CN114119935A CN114119935A (en) | 2022-03-01 |
| CN114119935Btrue CN114119935B (en) | 2023-10-03 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111435489.1AActiveCN114119935B (en) | 2021-11-29 | 2021-11-29 | Image processing methods and devices |
| Country | Link |
|---|---|
| CN (1) | CN114119935B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115345981B (en)* | 2022-10-19 | 2023-03-24 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN118097082B (en)* | 2024-04-26 | 2024-07-30 | 腾讯科技（深圳）有限公司 | Virtual object image generation method, device, computer equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109584151A (en)* | 2018-11-30 | 2019-04-05 | 腾讯科技（深圳）有限公司 | Method for beautifying faces, device, terminal and storage medium |
| CN110796721A (en)* | 2019-10-31 | 2020-02-14 | 北京字节跳动网络技术有限公司 | Color rendering method and device of virtual image, terminal and storage medium |
| CN110827379A (en)* | 2019-10-31 | 2020-02-21 | 北京字节跳动网络技术有限公司 | Virtual image generation method, device, terminal and storage medium |
| CN111695471A (en)* | 2020-06-02 | 2020-09-22 | 北京百度网讯科技有限公司 | Virtual image generation method, device, equipment and storage medium |
| CN111694429A (en)* | 2020-06-08 | 2020-09-22 | 北京百度网讯科技有限公司 | Virtual object driving method and device, electronic equipment and readable storage |
| CN112132979A (en)* | 2020-09-29 | 2020-12-25 | 支付宝(杭州)信息技术有限公司 | Virtual resource selection method, device and equipment |
| CN113112580A (en)* | 2021-04-20 | 2021-07-13 | 北京字跳网络技术有限公司 | Method, device, equipment and medium for generating virtual image |
| CN113240778A (en)* | 2021-04-26 | 2021-08-10 | 北京百度网讯科技有限公司 | Virtual image generation method and device, electronic equipment and storage medium |
| WO2021208648A1 (en)* | 2020-04-17 | 2021-10-21 | Oppo广东移动通信有限公司 | Virtual object adjusting method and apparatus, storage medium and augmented reality device |
| CN113569614A (en)* | 2021-02-23 | 2021-10-29 | 腾讯科技（深圳）有限公司 | Virtual image generation method, device, device and storage medium |
| CN113643412A (en)* | 2021-07-14 | 2021-11-12 | 北京百度网讯科技有限公司 | Virtual image generation method and device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10592780B2 (en)* | 2018-03-30 | 2020-03-17 | White Raven Ltd. | Neural network training system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109584151A (en)* | 2018-11-30 | 2019-04-05 | 腾讯科技（深圳）有限公司 | Method for beautifying faces, device, terminal and storage medium |
| CN110796721A (en)* | 2019-10-31 | 2020-02-14 | 北京字节跳动网络技术有限公司 | Color rendering method and device of virtual image, terminal and storage medium |
| CN110827379A (en)* | 2019-10-31 | 2020-02-21 | 北京字节跳动网络技术有限公司 | Virtual image generation method, device, terminal and storage medium |
| WO2021208648A1 (en)* | 2020-04-17 | 2021-10-21 | Oppo广东移动通信有限公司 | Virtual object adjusting method and apparatus, storage medium and augmented reality device |
| CN111695471A (en)* | 2020-06-02 | 2020-09-22 | 北京百度网讯科技有限公司 | Virtual image generation method, device, equipment and storage medium |
| CN111694429A (en)* | 2020-06-08 | 2020-09-22 | 北京百度网讯科技有限公司 | Virtual object driving method and device, electronic equipment and readable storage |
| CN112132979A (en)* | 2020-09-29 | 2020-12-25 | 支付宝(杭州)信息技术有限公司 | Virtual resource selection method, device and equipment |
| CN113569614A (en)* | 2021-02-23 | 2021-10-29 | 腾讯科技（深圳）有限公司 | Virtual image generation method, device, device and storage medium |
| CN113112580A (en)* | 2021-04-20 | 2021-07-13 | 北京字跳网络技术有限公司 | Method, device, equipment and medium for generating virtual image |
| CN113240778A (en)* | 2021-04-26 | 2021-08-10 | 北京百度网讯科技有限公司 | Virtual image generation method and device, electronic equipment and storage medium |
| CN113643412A (en)* | 2021-07-14 | 2021-11-12 | 北京百度网讯科技有限公司 | Virtual image generation method and device, electronic equipment and storage medium |
| Publication number | Publication date |
|---|---|
| CN114119935A (en) | 2022-03-01 |
| Publication | Publication Date | Title |
|---|---|---|
| CN116051729B (en) | Three-dimensional content generation method, device and electronic device | |
| CN115511779B (en) | Image detection method, device, electronic equipment and storage medium | |
| CN116311519B (en) | Action recognition method, model training method and device | |
| CN114119935B (en) | Image processing methods and devices | |
| CN115661375B (en) | Three-dimensional hair style generation method and device, electronic equipment and storage medium | |
| CN114120412B (en) | Image processing method and device | |
| CN115953512A (en) | Expression generation method, neural network training method, device, equipment and medium | |
| CN115601555A (en) | Image processing method and apparatus, device and medium | |
| CN116030185A (en) | Three-dimensional hair generation method and model training method | |
| CN116245998B (en) | Rendering map generation method and device, model training method and device | |
| CN117274575B (en) | Target detection model training method, target detection method, device and equipment | |
| CN115345981B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN115578451B (en) | Image processing method, training method and device of image processing model | |
| CN114120448B (en) | Image processing method and device | |
| CN117274370A (en) | Three-dimensional pose determining method, three-dimensional pose determining device, electronic equipment and medium | |
| CN116824014A (en) | Data generation method, device, electronic device, and medium for virtual image | |
| CN115393514A (en) | Training method of three-dimensional reconstruction model, three-dimensional reconstruction method, device and equipment | |
| CN115564992A (en) | Image classification method and training method of image classification model | |
| CN115578501A (en) | Image processing method, device, electronic device and storage medium | |
| CN115131562A (en) | Three-dimensional scene segmentation method, model training method, device and electronic device | |
| CN115423827B (en) | Image processing method, device, electronic device and storage medium | |
| CN116385641B (en) | Image processing methods and devices, electronic equipment and storage media | |
| CN116030191B (en) | Method, device, equipment and medium for displaying virtual object | |
| CN117218499B (en) | Facial expression capture model training method, facial expression driving method and device | |
| CN115359194B (en) | Image processing method, device, electronic device and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |