CN112987932B - Human-computer interaction and control method and device based on virtual image - Google Patents
Human-computer interaction and control method and device based on virtual imageDownload PDFInfo
- Publication number
- CN112987932B CN112987932BCN202110316646.0ACN202110316646ACN112987932BCN 112987932 BCN112987932 BCN 112987932BCN 202110316646 ACN202110316646 ACN 202110316646ACN 112987932 BCN112987932 BCN 112987932B
- Authority
- CN
- China
- Prior art keywords
- user
- avatar
- virtual image
- exclusive
- slider
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B15/00—Systems controlled by a computer
- G05B15/02—Systems controlled by a computer electric
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/26—Pc applications
- G05B2219/2642—Domotique, domestic, home control, automation, smart house
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Automation & Control Theory (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本公开涉及虚拟现实领域，尤其涉及人机交互、人工智能、深度学习、物联网和语音技术等领域。具体地，涉及一种基于虚拟形象的人机交互方法及装置、一种基于虚拟形象的控制方法及装置、另一种基于虚拟形象的人机交互方法及装置、一种电子设备、一种存储有计算机指令的非瞬时计算机可读存储介质和一种计算机程序产品。The present disclosure relates to the field of virtual reality, and in particular to fields such as human-computer interaction, artificial intelligence, deep learning, Internet of Things, and voice technology. Specifically, it relates to an avatar-based human-computer interaction method and device, an avatar-based control method and device, another avatar-based human-computer interaction method and device, an electronic device, a storage A non-transitory computer readable storage medium having computer instructions and a computer program product.
背景技术Background technique
在未来的增强现实系统中，虚拟形象将是人机交互的主要承载方式。In the future augmented reality system, avatar will be the main carrier of human-computer interaction.
当前市面上的虚拟形象生成App，通常会要求用户上传一张照片，而后根据照片中的人像自动生成一个初级的虚拟形象。但要实现最终的虚拟形象，还需要用户使用App中的捏脸功能，手动对初级的虚拟形象进行捏脸操作。The avatar generation apps currently on the market usually require users to upload a photo, and then automatically generate a primary avatar based on the portrait in the photo. But to realize the final avatar, users still need to use the face pinching function in the app to manually pinch the face of the primary avatar.
发明内容Contents of the invention
本公开提供了一种基于虚拟形象的人机交互、控制方法、装置、设备、存储介质以及计算机程序产品。The present disclosure provides a virtual image-based human-computer interaction, a control method, a device, a device, a storage medium, and a computer program product.
根据本公开的一方面，提供了一种基于虚拟形象的人机交互方法，包括：在智能设备上显示虚拟形象；控制所述虚拟形象与用户进行交流互动；以及在所述虚拟形象与所述用户交流互动的过程中，控制所述虚拟形象对所述用户进行消费推荐。According to one aspect of the present disclosure, a human-computer interaction method based on an avatar is provided, including: displaying the avatar on a smart device; controlling the avatar to communicate and interact with the user; In the process of user communication and interaction, the avatar is controlled to make consumption recommendations to the user.
根据本公开的另一方面，提供了一种基于虚拟形象的控制方法，包括：远程控制显示在智能设备上的虚拟形象与用户进行交流互动；以及在所述虚拟形象与所述用户交流互动的过程中，向所述智能设备下发针对所述用户的营销策略，以便所述虚拟形象基于所述营销策略，对所述用户进行消费推荐。According to another aspect of the present disclosure, there is provided an avatar-based control method, including: remotely controlling the avatar displayed on the smart device to communicate and interact with the user; and when the avatar communicates and interacts with the user In the process, a marketing strategy for the user is delivered to the smart device, so that the avatar can recommend consumption to the user based on the marketing strategy.
根据本公开的另一方面，提供了另一种基于虚拟形象的人机交互方法，包括：在特定场所中设置的特定互动设备上显示虚拟形象；以及在用户在所述特定场所活动的过程中，控制所述虚拟形象与所述用户进行交流互动。According to another aspect of the present disclosure, another avatar-based human-computer interaction method is provided, including: displaying the avatar on a specific interactive device set in a specific place; and during the user's activities in the specific place , controlling the avatar to communicate and interact with the user.
根据本公开的另一方面，提供了一种基于虚拟形象的人机交互装置，包括：第一显示模块，用于在智能设备上显示虚拟形象；第一控制模块，用于控制所述虚拟形象与用户进行交流互动；以及第二控制模块，用于在所述虚拟形象与所述用户交流互动的过程中，控制所述虚拟形象对所述用户进行消费推荐。According to another aspect of the present disclosure, a human-computer interaction device based on an avatar is provided, including: a first display module, configured to display an avatar on a smart device; a first control module, configured to control the avatar Communicating and interacting with the user; and a second control module, configured to control the avatar to recommend consumption to the user during the process of communicating and interacting between the avatar and the user.
根据本公开的另一方面，提供了一种基于虚拟形象的控制装置，包括：第三控制模块，用于远程控制显示在智能设备上的虚拟形象与用户进行交流互动；以及第一发送模块，用于在所述虚拟形象与所述用户交流互动的过程中，向所述智能设备下发针对所述用户的营销策略，以便所述虚拟形象基于所述营销策略，对所述用户进行消费推荐。According to another aspect of the present disclosure, an avatar-based control device is provided, including: a third control module, configured to remotely control the avatar displayed on the smart device to interact with the user; and a first sending module, In the process of communicating and interacting between the avatar and the user, issue a marketing strategy for the user to the smart device, so that the avatar can make consumption recommendations for the user based on the marketing strategy .
根据本公开的另一方面，提供了另一种基于虚拟形象的人机交互装置，包括：第二显示模块，用于在特定场所中设置的特定互动设备上显示虚拟形象；以及第四控制模块，用于在用户在所述特定场所活动的过程中，控制所述虚拟形象与所述用户进行交流互动。According to another aspect of the present disclosure, another avatar-based human-computer interaction device is provided, including: a second display module, configured to display the avatar on a specific interactive device set in a specific place; and a fourth control module , used to control the avatar to communicate and interact with the user during the user's activities in the specific place.
根据本公开的另一方面，提供了一种电子设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行本公开实施例所述的方法。According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores Instructions to be executed, the instructions are executed by the at least one processor, so that the at least one processor can execute the method described in the embodiments of the present disclosure.
根据本公开的另一方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行根据本公开实施例所述的方法。According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to the embodiments of the present disclosure.
根据本公开的另一方面，提供了一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据本公开实施例所述的方法。According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, the computer program implements the method according to the embodiments of the present disclosure when executed by a processor.
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.
附图说明Description of drawings
附图用于更好地理解本方案，不构成对本公开的限定。其中：The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:
图1A示例性示出了适于本公开实施例的虚拟形象的生成方法及装置的系统架构；FIG. 1A exemplarily shows the system architecture of the method and device for generating an avatar suitable for an embodiment of the present disclosure;
图1B示例性示出了可以实现本公开实施例的虚拟形象的生成方法及装置的场景图；FIG. 1B exemplarily shows a scene diagram of a method and device for generating an avatar that can implement an embodiment of the present disclosure;
图1C示例性示出了根据本公开实施例的基于虚拟形象的人机交互方法的流程图；FIG. 1C exemplarily shows a flow chart of a human-computer interaction method based on an avatar according to an embodiment of the present disclosure;
图1D示例性示出了根据本公开实施例的基于虚拟形象的控制方法的流程图；FIG. 1D exemplarily shows a flowchart of an avatar-based control method according to an embodiment of the present disclosure;
图1E示例性示出了根据本公开另一实施例的基于虚拟形象的人机交互方法的流程图；FIG. 1E exemplarily shows a flow chart of an avatar-based human-computer interaction method according to another embodiment of the present disclosure;
图2示例性示出了根据本公开实施例的虚拟形象的生成方法的流程图；FIG. 2 exemplarily shows a flow chart of a method for generating an avatar according to an embodiment of the present disclosure;
图3示例性示出了根据本公开实施例的语义变换的示意图；Fig. 3 exemplarily shows a schematic diagram of semantic transformation according to an embodiment of the present disclosure;
图4A～图4D示例性示出了根据本公开实施例的虚拟形象滑块的示意图；4A-4D exemplarily show schematic diagrams of an avatar slider according to an embodiment of the present disclosure;
图5示例性示出了根据本公开实施例的已完成骨骼与蒙皮绑定的参考虚拟形象的示意图；Fig. 5 exemplarily shows a schematic diagram of a reference avatar that has completed binding of bones and skin according to an embodiment of the present disclosure;
图6示例性示出了根据本公开实施例的生成虚拟形象滑块的原理图；Fig. 6 exemplarily shows a schematic diagram of generating an avatar slider according to an embodiment of the present disclosure;
图7示例性示出了根据本公开实施例的生成虚拟形象的原理图；Fig. 7 exemplarily shows a principle diagram of generating an avatar according to an embodiment of the present disclosure;
图8A示例性示出了根据本公开实施例的基于虚拟形象的人机交互装置的框图；Fig. 8A schematically shows a block diagram of an avatar-based human-computer interaction device according to an embodiment of the present disclosure;
图8B示例性示出了根据本公开实施例的基于虚拟形象的控制装置的框图；Fig. 8B exemplarily shows a block diagram of an avatar-based control device according to an embodiment of the present disclosure;
图8C示例性示出了根据本公开另一实施例的基于虚拟形象的人机交互装置的框图；以及Fig. 8C exemplarily shows a block diagram of an avatar-based human-computer interaction device according to another embodiment of the present disclosure; and
图9示例性示出了用来实现本公开实施例的虚拟形象的生成及装置的电子设备的框图。Fig. 9 exemplarily shows a block diagram of an electronic device used to realize the generation and apparatus of the avatar according to the embodiment of the present disclosure.
具体实施方式Detailed ways
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
当前市面上的虚拟形象生成App，通常会要求用户上传一张照片，而后根据照片中的人像自动生成一个初级的虚拟形象。但要实现最终的虚拟形象，还需要用户使用App中的捏脸功能，手动对初级的虚拟形象进行捏脸操作。The avatar generation apps currently on the market usually require users to upload a photo, and then automatically generate a primary avatar based on the portrait in the photo. But to realize the final avatar, users still need to use the face pinching function in the app to manually pinch the face of the primary avatar.
这种先自动生成后手动定制的传统解决思路，虽然最终能够给出一个虚拟形象的结果，但是用户往往很难获取到令自己满意的理想虚拟形象。究其原因，通过这种传统的解决思路，用户无法进行高效的虚拟形象个性化定制。This traditional solution, which is automatically generated first and then manually customized, can eventually give a virtual image result, but it is often difficult for users to obtain an ideal virtual image that satisfies them. The reason is that with this traditional solution, users cannot carry out highly efficient personalized customization of avatars.
示例性的，用户想要生成一个高鼻梁、大眼睛、瘦下巴的虚拟形象。按照上述传统方式，首先用户极有可能无法找到具有类似形象特征的真人照片。其次即使能够找到具有类似形象特征的真人照片，使用上述捏脸功能，用户需要逐个五官去寻找，比如需要在鼻型、眼型、脸型等的基底中逐一寻找。一般App对单一五官都会提供数十种形状供用户选择，因此仅仅挑选各个五官的形状就需要花费十几甚至几十分钟，很多用户很有可能没有耐心花时间去挑选令自己满意的五官形状。Exemplarily, the user wants to generate an avatar with a high nose bridge, big eyes, and a thin chin. According to the above-mentioned traditional method, at first the user is very likely to be unable to find a photo of a real person with similar image features. Secondly, even if it is possible to find real-life photos with similar image characteristics, using the above-mentioned face pinching function, users need to search for each of the five senses, such as nose shape, eye shape, face shape, etc., one by one. Generally, apps will provide dozens of shapes for users to choose from a single facial feature, so it will take more than ten or even tens of minutes just to choose the shape of each facial feature. Many users may not have the patience to take the time to choose the facial features that satisfy them.
此外，这种传统的虚拟形象定制方案，不便于用户使用，甚至会给用户造成极大的挫败感和体验伤害。In addition, this traditional avatar customization solution is not convenient for users to use, and may even cause great frustration and experience damage to users.
本公开提出的基于语言描述的虚拟形象智能生成方案可以解决上述技术问题，实现高效的虚拟形象个性化定制。以下将结合具体实施例详细阐述本公开。The language description-based intelligent virtual image generation scheme proposed in the present disclosure can solve the above-mentioned technical problems and realize efficient personalized customization of virtual images. The present disclosure will be described in detail below in conjunction with specific embodiments.
适于本公开实施例的虚拟形象的生成方法和装置的系统架构介绍如下。The system architecture of the method and device for generating an avatar suitable for the embodiments of the present disclosure is introduced as follows.
图1A示例性示出了适于本公开实施例的虚拟形象的生成方法和装置的系统架构。需要注意的是，图1A所示仅为可以应用本公开实施例的系统架构的示例，以帮助本领域技术人员理解本公开的技术内容，但并不意味着本公开实施例不可以用于其他环境或场景。Fig. 1A exemplarily shows the system architecture of the method and device for generating an avatar suitable for the embodiments of the present disclosure. It should be noted that FIG. 1A is only an example of the system architecture to which the embodiments of the present disclosure can be applied, so as to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other environment or scene.
如图1A所示，系统架构100可以包括：终端设备101。需要说明的是，终端设备101可以是客户端或者服务器，本公开在此不做限定。As shown in FIG. 1A , the
具体地，可以先通过终端设备101或者其他设备生成一个初始的虚拟形象作为参考虚拟形象。如果用户想要定制满足个性化需求的理想型虚拟形象，比如“高鼻梁、大眼睛、瘦下巴”是理想型，则用户可以通过语言描述该理想型虚拟形象。终端设备101获取到用户的语言描述后，可以基于该语言描述，提取对应的语义特征，进而基于提取的语义特征，生成该理想型虚拟形象。Specifically, an initial avatar may be firstly generated by the
在本公开实施例中，根据用户的语言描述，终端设备101能够准确识别到用户需要的虚拟形象的语义化特征信息，而后智能生成虚拟形象。该方案一方面能够提升个性化虚拟形象的生产效率，给用户更加便捷的使用体验；另一方面也能够提升虚拟形象定制的准确度，给用户输出理想型虚拟形象。In the embodiment of the present disclosure, according to the user's language description, the
应该理解，图1A中的终端设备的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备。It should be understood that the number of terminal devices in FIG. 1A is only illustrative. There can be any number of terminal devices according to implementation requirements.
适于本公开实施例的虚拟形象的生成方法和装置的应用场景介绍如下。The application scenarios of the virtual image generation method and device suitable for the embodiments of the present disclosure are introduced as follows.
应该理解，目前，线下消费引导方式还是以博主们探店分享为主，但如果消费者不去主动翻阅博主们分享的内容，则很难做到有效推荐。It should be understood that at present, the offline consumption guidance method is still based on bloggers visiting stores and sharing, but if consumers do not take the initiative to read the content shared by bloggers, it will be difficult to make effective recommendations.
还应该理解，个性化的虚拟形象通常与真人有相似性，不同虚拟形象之间又有区分度，通过虚拟形象的卡通外观还能提升亲和力。这都能帮助提升用户对虚拟形象的主观认可度。因而创建虚拟形象、实现基于虚拟形象的人机交互、以及在基于虚拟形象进行人机交互的过程中由虚拟形象适时进行消费推荐和消费陪伴，可以有效提高消费推荐的转化率，同时可以提高用户体验度。It should also be understood that personalized avatars are usually similar to real people, and there are distinctions between different avatars. The cartoon appearance of avatars can also enhance affinity. This can help improve users' subjective recognition of virtual images. Therefore, the creation of virtual images, the realization of human-computer interaction based on virtual images, and the timely consumption recommendation and consumption companionship by virtual images in the process of human-computer interaction based on virtual images can effectively improve the conversion rate of consumption recommendations, and at the same time improve the user experience. experience.
示例性的，在本公开实施例中，既可以基于虚拟形象在居家场景下对用户进行线下消费引导，又可以基于虚拟形象在线下消费场景下对用户进行消费陪伴。Exemplarily, in the embodiments of the present disclosure, it is possible to guide the user to consume offline based on the avatar in the home scene, and to accompany the user in the offline consumption scene based on the avatar.
如图1B所示，用户可以通过手机客户端创建自己的专属虚拟形象，并将其上传给云端，再由云端将其下发至与用户关联的智能设备、以及用户光顾的商超等公共场所中部署的交互设备(如商场交互设备、超市交互设备、餐馆交互设备等)，以供用户进行人机交互时使用。As shown in Figure 1B, users can create their own exclusive avatar through the mobile phone client, upload it to the cloud, and then the cloud will send it to the smart device associated with the user, as well as public places such as supermarkets that the user patronizes. The interactive devices (such as interactive devices in shopping malls, supermarkets, restaurants, etc.) deployed in the Internet are used by users for human-computer interaction.
需要说明的是，在本公开实施例中，智能设备包括带有显示屏的物联网IOT设备，比如冰箱、电视机、智能镜面、智能玻璃，等等。虚拟形象被下发至IOT设备上之后，可以在IOT设备被触发时显示出来，作为智能语音对话系统的前端承载，绘声绘色地与用户进行功能化的或无意义的对话交流。在对话交流过程中，虚拟形象可以根据云端下发的营销推荐指令，适时地对用户进行线下消费推荐。It should be noted that, in the embodiment of the present disclosure, the smart device includes an IOT device with a display screen, such as a refrigerator, a TV, a smart mirror, a smart glass, and the like. After the avatar is sent to the IOT device, it can be displayed when the IOT device is triggered, as the front-end bearer of the intelligent voice dialogue system, and it can communicate with the user in a functional or meaningless dialogue vividly. In the process of dialogue and communication, the avatar can timely recommend offline consumption to users according to the marketing recommendation instructions issued by the cloud.
此外，在本公开实施例中，商场互动设备也包括带有显示屏的、部署在商场中的能与用户交互的电子设备。示例性的，当用户光顾商场时，可以通过人脸识别技术确定用户身份，并通知云端下发相应的虚拟形象。比如用户A在某品牌服装店铺试穿衣服时，可以从云端下载用户A的专属虚拟形象，并在用户A附近的试装镜边缘显示其专属虚拟形象，使其与用户互动，如推荐用户试衣。再比如用户B在某个火锅店吃单人火锅时，可以在用户B的火锅边上立起来一个小显示屏，并在上面显示用户B的专属虚拟形象，使其与用户B进行无意义互动或游戏。In addition, in the embodiment of the present disclosure, the mall interaction device also includes an electronic device with a display screen that is deployed in a mall and can interact with users. Exemplarily, when a user visits a shopping mall, the identity of the user can be determined through face recognition technology, and the cloud can be notified to send a corresponding avatar. For example, when user A tries on clothes in a clothing store of a certain brand, he can download user A's exclusive avatar from the cloud, and display his exclusive avatar on the edge of the fitting mirror near user A so that he can interact with the user, such as recommending the user to try on clothes. Clothes. For another example, when user B eats single-person hot pot in a hot pot restaurant, a small display screen can be set up next to user B's hot pot, and user B's exclusive avatar can be displayed on it, so that he can interact with user B meaninglessly or games.
在本公开的一个实施例中，用户可以上传一张照片，而后由虚拟形象生成App根据照片中的人像自动生成一个初级的虚拟形象，然后再使用App中的捏脸功能，由用户手动对初级的虚拟形象进行捏脸操作来生成自己的专属虚拟形象。In one embodiment of the present disclosure, the user can upload a photo, and then the avatar generation App will automatically generate a primary avatar according to the portrait in the photo, and then use the pinching function in the App to manually create a primary avatar. pinch the face of your virtual image to generate your own exclusive virtual image.
在本公开的另一个实施例中，用户还可以基于语言描述的虚拟形象智能生成方案，生成自己的专属虚拟形象。In another embodiment of the present disclosure, the user can also generate his own exclusive avatar based on the intelligent generation scheme of the avatar described in the language.
此外，在本公开的另一个实施例中，云端除了接收手机客户端创建的虚拟形象并将其下发至与用户关联的智能设备、以及用户光顾的商超等公共场所中部署的交互设备之外，还可以通过分析用户大数据，适时给出营销策略，让智能家居设备上显示的虚拟形象去完成营销任务。比如某天下雨，天气很冷，用户好久没吃过火锅了，那么就可以让虚拟形象给用户深情地推荐一个火锅用餐地点。In addition, in another embodiment of the present disclosure, in addition to receiving the avatar created by the mobile phone client and sending it to the smart device associated with the user, and the interactive devices deployed in public places such as supermarkets visited by the user, the cloud In addition, by analyzing the big data of users, marketing strategies can be given in a timely manner, so that the avatars displayed on smart home devices can complete marketing tasks. For example, on a rainy day, the weather is very cold, and the user has not eaten hot pot for a long time, then the avatar can be asked to recommend a hot pot dining place to the user affectionately.
通过本公开实施例，使用与真人相似的虚拟形象提供消费推荐和陪伴式服务，可以使用户产生认同感。通过虚拟形象推荐，还可以将用户消费引流到线下。并且在真实消费场景中，用户的专属虚拟形象还可以陪伴在用户身边，帮助用户推荐商品、陪用户吃饭、聊天。如此可以实现线上服务与线下服务的打通。Through the embodiments of the present disclosure, using avatars similar to real people to provide consumption recommendations and accompanying services can make users feel a sense of identity. Through avatar recommendation, user consumption can also be diverted offline. And in real consumption scenarios, the user's exclusive virtual image can also accompany the user, helping the user recommend products, accompany the user to eat, and chat. In this way, online services and offline services can be connected.
本公开提供了一种基于虚拟形象的人机交互方法。The present disclosure provides a virtual image-based human-computer interaction method.
如图1C所示，该人机交互方法100C可以应用于居家场景，包括如下操作S110C～操作S130C。As shown in FIG. 1C , the human-
在操作S110C，在智能设备上显示虚拟形象。In operation S110C, the avatar is displayed on the smart device.
在操作S120C，控制虚拟形象与用户进行交流互动。In operation S120C, the avatar is controlled to communicate and interact with the user.
在操作S130C，在虚拟形象与用户交流互动的过程中，控制虚拟形象对用户进行消费推荐。In operation S130C, during the communication and interaction process between the avatar and the user, the avatar is controlled to recommend consumption to the user.
在本公开的一个实施例中，智能设备可以包括智能家居设备。示例性的，在居家场景下，可以在智能家居设备被触发后，在其显示屏上显示用户的专属虚拟形象。在本公开的其他实施例中，还可以在智能家居设备被触发后，在其显示屏上显示其他的不是用户专属的虚拟形象。In one embodiment of the present disclosure, the smart device may include a smart home device. Exemplarily, in a home scene, after the smart home device is triggered, the user's exclusive avatar can be displayed on its display screen. In other embodiments of the present disclosure, after the smart home device is triggered, other avatars not exclusive to the user may be displayed on its display screen.
虚拟形象显示在智能家居设备上，可以作为智能语音对话系统的前端承载，绘声绘色地与用户进行功能化的或无意义的对话交流、游戏互动等。在对话交流或游戏互动过程中，虚拟形象可以根据云端下发的营销推荐指令(包含营销策略)，适时地对用户进行线下消费推荐。比如某天下雨，天气很冷，用户好久没吃过火锅了，那么就可以让虚拟形象给用户深情地推荐一个火锅用餐地点。The avatar is displayed on the smart home device, which can be used as the front-end bearer of the intelligent voice dialogue system to carry out functional or meaningless dialogues and game interactions with users vividly and vividly. In the process of dialogue and game interaction, the avatar can timely recommend offline consumption to users according to the marketing recommendation instructions (including marketing strategies) issued by the cloud. For example, on a rainy day, the weather is very cold, and the user has not eaten hot pot for a long time, then the avatar can be asked to recommend a hot pot dining place to the user affectionately.
在居家场景中，用户通过与智能家居设备上的虚拟形象进行交流互动，可以建立对虚拟形象的信任感和认可度。由此让个性化虚拟形象成为用户的养成陪伴对象。这样云端直接对虚拟形象下达营销推荐指令，由虚拟形象完成消费引导的可能性比较大。In the home scene, users can establish a sense of trust and recognition for the avatar by interacting with the avatar on the smart home device. In this way, the personalized virtual image becomes the companion object of the user's development. In this way, the cloud directly issues marketing recommendation instructions to the avatar, and it is more likely that the avatar will complete consumption guidance.
在本公开实施例中，通过个性化虚拟形象的陪伴，可以建立用户对虚拟形象的信任感和认可度，从而实现以虚拟形象为交互载体的消费推荐，提升消费推荐的转化率。In the embodiment of the present disclosure, through the companionship of the personalized avatar, the user's sense of trust and recognition of the avatar can be established, so as to realize the consumption recommendation using the avatar as an interactive carrier and improve the conversion rate of the consumption recommendation.
作为一种可选的实施例，控制虚拟形象对用户进行消费推荐可以包括如下操作。As an optional embodiment, controlling the avatar to recommend consumption to the user may include the following operations.
获取云端针对该用户下发的营销策略。Obtain the marketing strategy issued by the cloud for the user.
控制虚拟形象基于该营销策略，对该用户进行消费推荐。Based on the marketing strategy, the control avatar makes consumption recommendations for the user.
在本公开的另一个实施例中，云端除了接收手机客户端创建的虚拟形象并将其下发至与用户关联的智能家居设备、以及用户光顾的商超等公共场所中部署的交互设备之外，还可以通过分析用户大数据，针对特定用户适时给出有针对性的营销策略，控制智能家居设备上显示的虚拟形象去完成营销任务。由此可以实现精准营销，为用户提供更合理、更精准、更满意的消费推荐。In another embodiment of the present disclosure, in addition to receiving the avatar created by the mobile phone client and sending it to the smart home device associated with the user and the interactive devices deployed in public places such as supermarkets visited by the user, the cloud , It is also possible to provide targeted marketing strategies for specific users in a timely manner by analyzing user big data, and control the virtual images displayed on smart home devices to complete marketing tasks. In this way, precise marketing can be realized, and more reasonable, accurate and satisfying consumption recommendations can be provided for users.
作为一种可选的实施例，在智能设备上显示虚拟形象可以包括：在智能设备上显示用户的专属虚拟形象。其中，专属虚拟形象是由云端下发给智能设备的。As an optional embodiment, displaying the avatar on the smart device may include: displaying the user's exclusive avatar on the smart device. Among them, the exclusive virtual image is sent to the smart device by the cloud.
示例性的，用户可以通过手机客户端创建自己的专属虚拟形象，并将其上传至云端，然后在用户触发其智能设备时，由云端将其创建的专属虚拟形象下发至其智能设备进行显示。Exemplarily, the user can create his own exclusive avatar through the mobile client and upload it to the cloud, and then when the user triggers his smart device, the cloud will send the created exclusive avatar to his smart device for display .
在本公开实施例中，使用用户的专属虚拟形象，在虚拟形象陪伴养成中，更容易取得用户的信任和认可，进而更容易由虚拟形象完成消费推荐。In the embodiment of the present disclosure, using the user's exclusive avatar, it is easier to obtain the trust and approval of the user during the avatar companion development, and it is easier to complete the consumption recommendation by the avatar.
本公开还提供了一种基于虚拟形象的控制方法。The present disclosure also provides an avatar-based control method.
如图1D所示，该基于虚拟形象的控制方法100D可以应用于云端等服务器端，包括如下操作S110D～操作S120D。As shown in FIG. 1D , the avatar-based
在操作S110D，远程控制显示在智能设备上的虚拟形象与用户进行交流互动。In operation S110D, the virtual image displayed on the smart device is remotely controlled to communicate and interact with the user.
在操作S120D，在虚拟形象与用户交流互动的过程中，向智能设备下发针对用户的营销策略，以便虚拟形象基于营销策略，对用户进行消费推荐。In operation S120D, during the process of communication and interaction between the avatar and the user, a marketing strategy for the user is delivered to the smart device, so that the avatar can recommend consumption to the user based on the marketing strategy.
在本公开的一个实施例中，以家居场景为例，智能设备包括智能家居设备。在智能家居设备被触发后，云端可以远程控制智能家居设备，如在其显示屏上显示用户的专属虚拟形象。在本公开的其他实施例中，在智能家居设备被触发后，云端还可以远程控制智能家居设备，如在其显示屏上显示其他的不是用户专属的虚拟形象。In an embodiment of the present disclosure, taking a home scene as an example, the smart devices include smart home devices. After the smart home device is triggered, the cloud can remotely control the smart home device, such as displaying the user's exclusive avatar on its display screen. In other embodiments of the present disclosure, after the smart home device is triggered, the cloud can also remotely control the smart home device, such as displaying other avatars not exclusive to the user on its display screen.
虚拟形象显示在智能家居设备上，可以作为智能语音对话系统的前端承载，绘声绘色地与用户进行功能化的或无意义的对话交流、游戏互动等。在对话交流或游戏互动过程中，虚拟形象可以根据云端下发的营销推荐指令(包含营销策略)，适时地对用户进行线下消费推荐。比如某天下雨，天气很冷，用户好久没吃过火锅了，那么就可以让虚拟形象给用户深情地推荐一个火锅用餐地点。The avatar is displayed on the smart home device, which can be used as the front-end bearer of the intelligent voice dialogue system to carry out functional or meaningless dialogues and game interactions with users vividly and vividly. In the process of dialogue and game interaction, the avatar can timely recommend offline consumption to users according to the marketing recommendation instructions (including marketing strategies) issued by the cloud. For example, on a rainy day, the weather is very cold, and the user has not eaten hot pot for a long time, then the avatar can be asked to recommend a hot pot dining place to the user affectionately.
在居家场景中，用户通过与智能家居设备上的虚拟形象进行交流互动，更容易建立对虚拟形象的信任，因而更容易认可虚拟形象给出的消费推荐，进而让容易让个性化虚拟形象成为用户的养成陪伴对象。这样云端直接对虚拟形象下达营销推荐指令，由虚拟形象完成消费引导的可能性比较大。In the home scene, it is easier for users to build trust in the avatar by interacting with the avatar on the smart home device, so it is easier to recognize the consumption recommendations given by the avatar, which in turn makes it easier for the personalized avatar to become a user. The cultivation of accompanying objects. In this way, the cloud directly issues marketing recommendation instructions to the avatar, and it is more likely that the avatar will complete consumption guidance.
在本公开实施例中，通过个性化虚拟形象的陪伴，可以增加用户对虚拟形象的信任感和认可度，从而实现以虚拟形象为交互载体的消费推荐，提升消费推荐的转化率。In the embodiment of the present disclosure, the user's sense of trust and recognition of the avatar can be increased through the companionship of the personalized avatar, so as to realize the consumption recommendation using the avatar as an interactive carrier and improve the conversion rate of the consumption recommendation.
作为一种可选的实施例，该方法还可以包括如下操作。As an optional embodiment, the method may further include the following operations.
获取用户提供的专属虚拟形象。Obtain an exclusive avatar provided by the user.
将专属虚拟形象下发给与该用户关联的智能设备，以便智能设备在面对用户时显示专属虚拟形象，并通过专属虚拟形象与用户进行人机交互。Send the exclusive virtual image to the smart device associated with the user, so that the smart device displays the exclusive virtual image when facing the user, and conducts human-computer interaction with the user through the exclusive virtual image.
示例性的，用户可以通过手机客户端创建自己的专属虚拟形象，并将其上传至云端，然后在用户触发其智能设备时，由云端将其创建的专属虚拟形象下发至与用户关联的智能设备进行显示，并由该虚拟形象与用户进行交流互动。Exemplarily, the user can create his own exclusive avatar through the mobile phone client and upload it to the cloud, and then when the user triggers his smart device, the cloud will send the created exclusive avatar to the smart device associated with the user. The device displays, and the avatar interacts with the user.
在本公开实施例中，使用用户的专属虚拟形象，在虚拟形象养成陪伴中，更容易取得用户的信任和认可，进而更容易由虚拟形象完成消费推荐。In the embodiment of the present disclosure, using the user's exclusive avatar, it is easier to obtain the user's trust and approval during the avatar development and companionship, and then it is easier to complete the consumption recommendation by the avatar.
作为一种可选的实施例，该方法还包括如下操作。As an optional embodiment, the method further includes the following operations.
获取用户的消费数据。Get the user's consumption data.
基于消费数据，生成针对用户的营销策略，以下发给智能设备。Based on the consumption data, a marketing strategy for the user is generated and sent to the smart device.
示例性的，云端可以获取由在商超等公共场所设置的交互设备、收银设备等上传的消费数据，并基于获取的消费数据进行大数据分析，从而生成针对用户个人或者某类用户群体的营销策略，并将生成的营销策略下发给与特定用户或者特定类别的用户关联的智能设备，以便由其上显示的虚拟形象适时对用户进行消费推荐。Exemplarily, the cloud can obtain consumption data uploaded by interactive devices and cash register devices installed in public places such as commercial supermarkets, and perform big data analysis based on the obtained consumption data, thereby generating marketing campaigns for individual users or certain user groups Strategy, and send the generated marketing strategy to the smart device associated with a specific user or a specific category of users, so that the avatar displayed on it can make timely consumption recommendations to the user.
通过本公开实施例，可以从用户的消费数据中学习到用户的消费习惯，然后制定符合用户本人消费习惯的营销策略，由此可以实现精准营销。Through the embodiments of the present disclosure, the user's consumption habits can be learned from the user's consumption data, and then a marketing strategy that conforms to the user's own consumption habits can be formulated, thereby realizing precise marketing.
本公开还提供了另一种基于虚拟形象的控制方法。The present disclosure also provides another avatar-based control method.
如图1E所示，该基于虚拟形象的人机交互方法100E可以应用于消费场景，包括如下操作S110E～操作S120E。As shown in FIG. 1E , the avatar-based human-
在操作S110E，在特定场所中设置的特定互动设备上显示虚拟形象。In operation S110E, the avatar is displayed on a specific interactive device set in a specific place.
在操作S120E，在用户在特定场所活动的过程中，控制虚拟形象与用户进行交流互动。In operation S120E, the avatar is controlled to communicate and interact with the user during the user's activities in a specific place.
需要说明的是，在本公开实施例中，特定场所可以包括商场、超市、餐馆、饭店等公共场所。特定互动设备可以包括带有显示屏的电子设备。It should be noted that, in the embodiment of the present disclosure, the specific place may include public places such as shopping malls, supermarkets, restaurants, and restaurants. Certain interactive devices may include electronic devices with display screens.
示例性的，当用户光顾商场时，可以通过人脸识别技术确定用户身份，并通知云端向用户附近的商场交互设备下发相应的虚拟形象，以便虚拟形象能够在用户消费过程中与用户交流互动，陪伴用户进行线下消费。Exemplarily, when a user visits a shopping mall, the identity of the user can be determined through face recognition technology, and the cloud can be notified to deliver a corresponding avatar to an interactive device in a shopping mall near the user, so that the avatar can communicate and interact with the user during the consumption process of the user , to accompany users in offline consumption.
通过本公开实施例，在消费场景中，可以使用户的专属虚拟形象出现在用户附近的商场互动设备上，以便陪伴用户的整个线下消费过程，给用户提供更好的消费体验。Through the embodiments of the present disclosure, in the consumption scene, the user's exclusive virtual image can be displayed on the shopping mall interactive device near the user, so as to accompany the user's entire offline consumption process and provide the user with a better consumption experience.
作为一种可选的实施例，在用户在特定场所活动的过程中，控制虚拟形象与用户进行交流互动，可以包括以下至少之一。As an optional embodiment, during the user's activities in a specific place, controlling the avatar to communicate and interact with the user may include at least one of the following.
在用户在商场或超市购物的过程中，控制虚拟形象与用户进行交流互动，以陪伴用户购物。During the shopping process of the user in the mall or supermarket, the virtual image is controlled to communicate and interact with the user, so as to accompany the user in shopping.
在用户在饭店或餐馆吃饭的过程中，控制虚拟形象与用户聊天或进行互动游戏，以陪伴用户用餐。When the user is eating in a restaurant or a restaurant, the avatar is controlled to chat with the user or play an interactive game, so as to accompany the user to eat.
在用户在休闲娱乐场所消费的过程中，控制虚拟形象与用户进行交流互动，以陪伴用户休闲娱乐。In the process of the user's consumption in leisure and entertainment places, the virtual image is controlled to communicate and interact with the user, so as to accompany the user in leisure and entertainment.
示例性的，比如用户A在某品牌服装店铺试穿衣服时，可以从云端下载用户A的专属虚拟形象，并在用户A附近的试装镜边缘显示其专属虚拟形象，使其与用户交流互动，如推荐用户试衣。再比如用户B在某个火锅店吃单人火锅时，可以在用户B的火锅边上立起来一个小显示屏，并在上面显示用户B的专属虚拟形象，使其与用户B进行无意义互动或游戏。Exemplarily, for example, when user A is trying on clothes in a clothing store of a certain brand, user A's exclusive avatar can be downloaded from the cloud, and its exclusive avatar can be displayed on the edge of the fitting mirror near user A, so that it can interact with the user , such as recommending a user to try on clothes. For another example, when user B eats single-person hot pot in a hot pot restaurant, a small display screen can be set up next to user B's hot pot, and user B's exclusive avatar can be displayed on it, so that he can interact with user B meaninglessly or games.
通过本公开实施例，使用与真人具有一定相似度的虚拟形象提供陪伴式消费服务，可以使用户产生认同感。并且在真实消费场景中，用户的专属虚拟形象陪伴在用户身边，可以帮助用户购买到用户满意的商品、陪用户吃饭、聊天等。如此可以实现线上服务与线下服务的打通。Through the embodiments of the present disclosure, using a virtual image that has a certain degree of resemblance to a real person to provide accompanying consumption services can make users feel a sense of identity. And in the real consumption scene, the user's exclusive avatar is with the user, which can help the user to buy the products that the user is satisfied with, eat with the user, chat with the user, etc. In this way, online services and offline services can be connected.
作为一种可选的实施例，该方法还可以包括如下操作。As an optional embodiment, the method may further include the following operations.
获取用户的消费数据。Get the user's consumption data.
将消费数据发送至云端，以便云端基于消费数据生成针对用户的营销策略。Send the consumption data to the cloud, so that the cloud can generate marketing strategies for users based on the consumption data.
示例性的，在商超等公共场所设置的交互设备、收银设备等可以采集用户的消费数据并上传给云端，进而由云端基于获取的消费数据进行大数据分析，从而生成针对用户个人或者某类用户群体的营销策略，以便云端将这些营销策略下发给对应的智能家居设备，由智能家居设备上显示的虚拟形象适时对用户进行消费推荐。Exemplarily, interactive devices and cashier devices set up in public places such as commercial supermarkets can collect user consumption data and upload it to the cloud, and then the cloud will perform big data analysis based on the acquired consumption data, so as to generate information targeted at the user or a certain category. Marketing strategies for user groups, so that the cloud can send these marketing strategies to the corresponding smart home devices, and the avatars displayed on the smart home devices will make timely consumption recommendations to users.
通过本公开实施例，可以采集并上传用户的消费数据，使得云端能够从用户的消费数据中学习到用户的消费习惯，然后生成符合用户本人消费习惯的营销策略，由此可以实现精准营销。Through the embodiments of the present disclosure, the user's consumption data can be collected and uploaded, so that the cloud can learn the user's consumption habits from the user's consumption data, and then generate a marketing strategy that conforms to the user's own consumption habits, thereby realizing precise marketing.
作为一种可选的实施例，该方法还可以包括如下操作。As an optional embodiment, the method may further include the following operations.
响应于用户进入特定场所，对用户进行人脸识别，以确定用户的身份。In response to the user entering a specific location, face recognition is performed on the user to determine the identity of the user.
基于用户的身份获取用户的专属虚拟形象。Obtain the user's exclusive avatar based on the user's identity.
其中，在特定场所中设置的特定互动设备上显示虚拟形象可以包括：在特定场所中设置的特定互动设备上显示专属虚拟形象。Wherein, displaying the avatar on the specific interactive device set in the specific place may include: displaying the exclusive avatar on the specific interactive device set in the specific place.
应该理解，个性化的虚拟形象通常与真人有相似性，不同虚拟形象之间又有区分度，通过虚拟形象的卡通外观还能提升亲和力。这都能帮助提升用户对虚拟形象的主观认可度。因而创建虚拟形象、实现基于虚拟形象的人机交互、以及在基于虚拟形象进行人机交互的过程中由虚拟形象适时进行消费推荐和消费陪伴，可以有效提高消费推荐的转化率，同时可以提高用户体验度。It should be understood that personalized avatars are usually similar to real people, and different avatars are differentiated, and the cartoon appearance of avatars can also enhance affinity. This can help improve users' subjective recognition of virtual images. Therefore, the creation of virtual images, the realization of human-computer interaction based on virtual images, and the timely consumption recommendation and consumption companionship by virtual images in the process of human-computer interaction based on virtual images can effectively improve the conversion rate of consumption recommendations, and at the same time improve the user experience. experience.
用户可以通过手机客户端创建自己的专属虚拟形象，并将其上传给云端，再由云端将其下发至与用户关联的智能家居设备、以及用户光顾的商超等公共场所中部署的交互设备(如商场交互设备、超市交互设备、餐馆交互设备等)，以供用户进行人机交互时使用。Users can create their own exclusive avatar through the mobile client, upload it to the cloud, and then the cloud will send it to the smart home devices associated with the user, as well as the interactive devices deployed in public places such as supermarkets that the user patronizes (such as interactive equipment in shopping malls, interactive equipment in supermarkets, interactive equipment in restaurants, etc.), for users to use when conducting human-computer interaction.
示例性的，当用户光顾商场时，可以通过人脸识别技术确定用户身份，并通知云端向用户附近的商场交互设备下发与用户身份关联的专属虚拟形象。比如用户A在某品牌服装店铺试穿衣服时，可以从云端下载用户A的专属虚拟形象，并在用户A附近的试装镜边缘显示其专属虚拟形象，使其与用户交流互动，如推荐用户试衣。再比如用户B在某个火锅店吃单人火锅时，可以在用户B的火锅边上立起来一个小显示屏，并在上面显示用户B的专属虚拟形象，使其与用户B进行无意义互动或游戏。Exemplarily, when a user visits a mall, the user's identity can be determined through face recognition technology, and the cloud can be notified to deliver a dedicated avatar associated with the user's identity to interactive devices in the mall near the user. For example, when user A tries on clothes in a clothing store of a certain brand, he can download user A's exclusive avatar from the cloud, and display his exclusive avatar on the edge of the fitting mirror near user A, so that he can communicate and interact with users, such as recommending users Dressing. For another example, when user B eats single-person hot pot in a hot pot restaurant, a small display screen can be set up next to user B's hot pot, and user B's exclusive avatar can be displayed on it, so that he can interact with user B meaninglessly or games.
通过本公开实施例，基于智能家居设备、商场互动设备和云端服务，实现了基于个性化虚拟形象的居家场景线上营销和线下消费场景陪伴的居家/商场联动营销，因而可以提升消费引导方式(或营销方式)，同时可以提高用户的消费体验度，比如能够使用户沉浸式接受个性化虚拟形象的陪伴式消费服务。Through the embodiments of the present disclosure, based on smart home equipment, shopping mall interactive equipment and cloud services, online marketing of home scenes based on personalized avatars and home/shopping mall linkage marketing accompanied by offline consumption scenes can be realized, thereby improving consumption guidance. (or marketing methods), and at the same time can improve the user's consumption experience, such as accompanying consumption services that enable users to accept personalized virtual images immersively.
根据本公开的实施例，本公开还提供了一种虚拟形象的生成方法。According to an embodiment of the present disclosure, the present disclosure also provides a method for generating an avatar.
图2示例性示出了根据本公开实施例的虚拟形象的生成方法的流程图。Fig. 2 exemplarily shows a flowchart of a method for generating an avatar according to an embodiment of the present disclosure.
如图2所示，方法200可以包括：操作S210～S230。As shown in FIG. 2 , the
操作S210，获取用户针对目标虚拟形象的语言描述。In operation S210, a user's language description for the target avatar is acquired.
操作S220，基于语言描述，提取对应的语义特征。Operation S220, extracting corresponding semantic features based on the language description.
操作S230，基于语义特征，生成目标虚拟形象。In operation S230, the target avatar is generated based on the semantic features.
需要说明的是，在操作S210，语言描述可以包括语音形式或文本形式的语言描述，本公开实施例在此不做限定。其中，对于语音形式的语言描述，在操作S210，可以通过自动语音识别ASR技术来捕捉用户对目标虚拟形象的语义需求。It should be noted that, in operation S210, the language description may include a language description in a speech form or a text form, which is not limited in this embodiment of the present disclosure. Wherein, for the language description in the voice form, in operation S210, automatic speech recognition (ASR) technology may be used to capture the user's semantic requirements for the target avatar.
示例性的，如果用户想要创建一个“高鼻梁、大眼睛、瘦下巴”的目标虚拟形象，则用户可以针对该目标虚拟形象输入以下语言描述“高鼻梁、大眼睛、瘦下巴”。由此，通过方法200提供的上述操作，可以获取语言描述“高鼻梁、大眼睛、瘦下巴”，并提取其中的语义特征如“高鼻梁、大眼睛、瘦下巴”。此外，在本公开实施例中，还可以预先任意创建一个虚拟形象作为参考虚拟形象。然后每次针对目标虚拟形象提取到对应的语义特征后，可以基于提取的语义特征，控制该参考虚拟形象变形，从而最终得到用户想要的目标虚拟形象。Exemplarily, if the user wants to create a target avatar with "high nose bridge, big eyes, and thin chin", the user can input the following language description "high nose bridge, big eyes, and thin chin" for the target avatar. Thus, through the above operations provided by
通过本公开实施例，只需要用户给出目标虚拟形象的语义描述，不需要用户额外执行手动定制操作，便可以基于用户的语义需求，对已有的参考虚拟形象进行语义化改造，从而实现高效率、高准确度的虚拟形象个性化定制。由此还可以提升用户体验度和对定制的虚拟形象的认可程度。Through the embodiments of the present disclosure, only the user needs to give the semantic description of the target avatar, and the existing reference avatar can be semantically transformed based on the user's semantic requirements without the need for the user to perform additional manual customization operations, so as to achieve high-quality High-efficiency, high-accuracy personalized customization of avatars. In this way, user experience and recognition of the customized avatar can also be improved.
此外，在本公开实施例，由于只需要用户给出目标虚拟形象的语义描述即可，不需要用户额外执行手动定制操作，因而还可以方便用户使用。In addition, in the embodiment of the present disclosure, since the user only needs to provide a semantic description of the target avatar, and does not require the user to perform additional manual customization operations, it is also convenient for the user to use.
作为一种可选的实施例，基于语义特征，生成目标虚拟形象，可以包括如下操作。As an optional embodiment, generating the target avatar based on semantic features may include the following operations.
获取参考虚拟形象。Get a reference avatar.
基于从用户的语义描述中提取的语义特征，控制参考虚拟形象形变，以生成目标虚拟形象。Based on the semantic features extracted from the user's semantic description, the deformation of the reference avatar is controlled to generate the target avatar.
在本公开实施例中，可以预先任意创建一个虚拟形象作为参考虚拟形象，在进行目标虚拟形象的个性化定制过程中，直接获取该参考虚拟形象，并利用从用户的语义描述中提取的语义特征，驱动该参考虚拟形象变形即可获得对应的目标虚拟形象。In the embodiment of the present disclosure, an avatar can be created arbitrarily in advance as a reference avatar, and in the process of personalizing the target avatar, the reference avatar is directly obtained, and the semantic features extracted from the user's semantic description are used , drive the reference virtual image to deform to obtain the corresponding target virtual image.
示例性的，可以先创建参考虚拟形象的虚拟形象模型，再针对该虚拟形象模型创建骨骼树，然后对骨骼树中的各骨骼节点进行骨骼蒙皮，从而将骨骼节点和对应的骨骼蒙皮网格节点关联起来，得到对应的参考虚拟形象。Exemplarily, an avatar model of a reference avatar may be created first, then a bone tree is created for the avatar model, and then bone skinning is performed on each bone node in the bone tree, so that the bone nodes and the corresponding bone skinning mesh grid nodes to get the corresponding reference avatar.
此外，基于从用户的语义描述中提取的语义特征控制参考虚拟形象形变可以包括：先获取带有该语义特征的虚拟形象滑块(以下简称滑块)，然后利用该滑块驱动参考虚拟形象的骨骼节点形变，从而带动参考虚拟形象的蒙皮节点随之变形，最终获得用户想要的目标虚拟形象，即理想虚拟形象。In addition, controlling the deformation of the reference avatar based on the semantic feature extracted from the user's semantic description may include: first obtaining the slider of the avatar with the semantic feature (hereinafter referred to as the slider), and then using the slider to drive the reference avatar. Skeleton nodes deform, thereby driving the skin nodes of the reference avatar to deform accordingly, and finally obtain the target avatar that the user wants, that is, the ideal avatar.
示例性的，如果用户想要创建一个“高鼻梁、大眼睛、瘦下巴”的目标虚拟形象，则可以基于该语义描述提取出如下语义特征“高鼻梁、大眼睛、瘦下巴”，然后获取“高鼻梁滑块”、“大眼睛滑块”、“瘦下巴滑块”3个滑块，并利用这3个滑块驱动预先创建的参考虚拟形象变形，从而最终获得满足“高鼻梁、大眼睛、瘦下巴”特征的理想虚拟形象。Exemplarily, if the user wants to create a target avatar with "high nose bridge, big eyes, and thin chin", the following semantic feature "high nose bridge, big eyes, and thin chin" can be extracted based on the semantic description, and then " High nose bridge slider", "big eyes slider" and "thin chin slider", and use these three sliders to drive the pre-created reference avatar to deform, so as to finally achieve the satisfaction of "high nose bridge, big eyes , thin chin” ideal avatar.
通过本公开实施例，通过运用人工智能算法进行个性化虚拟形象的定制，一方面可以提升个性化虚拟形象的生产效率，让用户体验上更加便捷；另一方面也可以提升个性化虚拟形象定制的准确度。Through the embodiments of the present disclosure, by using artificial intelligence algorithms to customize personalized virtual images, on the one hand, the production efficiency of personalized virtual images can be improved, making user experience more convenient; on the other hand, the efficiency of personalized virtual image customization can also be improved. Accuracy.
作为一种可选的实施例，基于从用户的语义描述中提取的语义特征，控制预先创建的参考虚拟形象形变，以生成目标虚拟形象，可以包括如下操作。As an optional embodiment, based on the semantic features extracted from the user's semantic description, controlling the deformation of a pre-created reference avatar to generate a target avatar may include the following operations.
将提取的语义特征转换为专业级语义特征。Transform the extracted semantic features into professional-level semantic features.
基于转换得到的专业级语义特征，控制该参考虚拟形象变形。The deformation of the reference avatar is controlled based on the converted professional-level semantic features.
应该理解，在实际使用中，不同的用户对相同或者类似形象的语言描述可能不尽相同。例如，对于“瘦下巴”，有的用户可能描述为“锥子脸”，有的用户可能描述为“瓜子脸”，甚至有的用户可能还有其他描述。并且，在实际应用中，用户很难给出“颧骨”、“下巴”级别的形状描述。更多地，用户会选择更加笼统的描述来刻画对虚拟形象的一种感觉，比如“像小姑娘”、“像老奶奶”、“阳光”、“帅气”等。It should be understood that in actual use, different users may have different language descriptions for the same or similar images. For example, for "thin chin", some users may describe it as "awl face", some users may describe it as "oval face", and some users may even have other descriptions. Moreover, in practical applications, it is difficult for users to give shape descriptions at the "cheekbone" and "chin" levels. More often, users will choose more general descriptions to describe a feeling for the avatar, such as "like a little girl", "like a grandma", "sunshine", "handsome" and so on.
因此，本公开实施例提出，先将从用户的语义描述中提取的语义特征统一转换为专业级语义特征，再基于转换后的语义特征控制参考虚拟形象变形，以获得最终的目标虚拟形象。Therefore, the embodiment of the present disclosure proposes to uniformly convert the semantic features extracted from the user's semantic description into professional-level semantic features, and then control the deformation of the reference avatar based on the converted semantic features to obtain the final target avatar.
示例性的，可以通过语义转换器提取的语义特征(即用户给出的笼统语义特征)转换为专业级语义特征。所谓专业级语义特征可以是从解剖学、生物学等角度进行语义描述中带有的语义特征。其中，在本公开实施例中，语义转换器可以是通过大量的数据收集和深度学习回归训练而实现的。Exemplarily, the semantic features extracted by the semantic converter (that is, the general semantic features given by the user) can be converted into professional-level semantic features. The so-called professional-level semantic features can be semantic features contained in semantic descriptions from the perspectives of anatomy and biology. Wherein, in the embodiment of the present disclosure, the semantic converter may be realized through a large amount of data collection and deep learning regression training.
如图3所示，在本公开实施例中，以脸型为例，从用户的语言描述中提取的语义关键字“瓜子脸”可以转换为如下专业级语义特征“低颧骨”和“窄下巴”；从用户的语言描述中提取的语义关键字“国字脸”可以转换为如下专业级语义特征“高颧骨”和“宽下巴”；从用户的语言描述中提取的语义关键字“萌脸”可以转换为如下专业级语义特征“大眼睛”和“圆脸”。As shown in Figure 3, in the embodiment of the present disclosure, taking the face shape as an example, the semantic keyword "oval face" extracted from the user's language description can be converted into the following professional-level semantic features "low cheekbones" and "narrow chin "; the semantic keyword "national character face" extracted from the user's language description can be converted into the following professional-level semantic features "high cheekbones" and "wide chin"; the semantic keyword "cute face" extracted from the user's language description " can be converted into the following professional-level semantic features "big eyes" and "round face".
此外，在本公开实施例中，由于用户的笼统语义描述可以被转换为对应的专业级语义描述，因此可以基于专业级语义创建驱动参考虚拟形象变形的滑块。如可以创建高颧骨滑块、低颧骨滑块、窄下巴滑块、大眼睛滑块、圆脸滑块，等等。如果用户输入的是“瓜子脸”，则可以将其转换为“低颧骨”和“窄下巴”两个专业级语义，后续直接利用“低颧骨滑块”和“窄下巴滑块”两个滑块驱动参考虚拟形象变形，最终实现“瓜子脸”的理想虚拟形象。In addition, in the embodiments of the present disclosure, since the user's general semantic description can be converted into a corresponding professional-level semantic description, a slider that drives the deformation of the reference avatar can be created based on the professional-level semantics. For example, you can create sliders for high cheekbones, sliders for low cheekbones, sliders for narrow chins, sliders for big eyes, sliders for round faces, and more. If the user enters "oval face", it can be converted into two professional-level semantics of "low cheekbones" and "narrow chin", and then directly use the "low cheekbones slider" and "narrow chin slider" A slider drives the deformation of the reference avatar, and finally realizes the ideal avatar of "oval face".
通过本公开实施例，即使实际应用中，用户输入笼统的语言描述，也可以通过语义转换获得对应的专业级的语言描述，进而精准地控制参考虚拟形象发生形变，从而最终得到用户想要的理想虚拟形象。Through the embodiments of the present disclosure, even if the user inputs a general language description in practical applications, the corresponding professional-level language description can be obtained through semantic conversion, and then the deformation of the reference avatar can be accurately controlled, so as to finally obtain the ideal user desired. Avatar.
作为一种可选的实施例，基于专业级语义特征，控制参考虚拟形象变形，可以包括如下操作。As an optional embodiment, controlling the deformation of the reference avatar based on professional-level semantic features may include the following operations.
基于专业级语义特征，确定至少一个滑块，每个滑块关联于对应的特定语义标签。Based on the professional-level semantic features, at least one slider is determined, and each slider is associated with a corresponding specific semantic label.
基于至少一个滑块，驱动用于支撑参考虚拟形象的骨骼树中的多个对应骨骼节点变形。A plurality of corresponding skeletal nodes in the skeletal tree for supporting the reference avatar are driven to deform based on the at least one slider.
基于多个对应骨骼节点的变形，驱动与多个对应骨骼节点关联的蒙皮网格节点变形。Drives the deformation of the skin mesh node associated with the plurality of corresponding bone nodes based on the deformation of the corresponding plurality of bone nodes.
具体地，在本公开实施例中，在将用户描述的笼统语义特征转换为专业级语义特征之后，可以提取转换得到的专业级语义特征中包含的至少一个关键词。然后找出包含该至少一个关键词的至少一个语义标签，然后再找出与该至少一个语义标签中的每个语义标签关联的滑块。最后利用找出的滑块驱动用于支撑参考虚拟形象的骨骼树中的多个对应骨骼节点变形，进而基于多个对应骨骼节点的变形，驱动与多个对应骨骼节点关联的蒙皮网格节点变形。Specifically, in the embodiment of the present disclosure, after converting the general semantic features described by the user into professional-level semantic features, at least one keyword included in the converted professional-level semantic features may be extracted. Then at least one semantic tag containing the at least one keyword is found, and then a slider associated with each semantic tag in the at least one semantic tag is found. Finally, use the found slider to drive the deformation of multiple corresponding bone nodes in the bone tree used to support the reference avatar, and then drive the skin mesh nodes associated with multiple corresponding bone nodes based on the deformation of the multiple corresponding bone nodes out of shape.
示例性的，如图4A～图4D依次表示“宽脸滑块”、“窄脸滑块”、“长脸滑块”和“短脸滑块”。如图5表示已完成骨骼与蒙皮绑定的参考虚拟形象。示例性的，在用户输入的笼统语言描述或者转换后的专业语言描述中包含“宽脸”特征的情况下，可以直接利用如图4A所示的“宽脸滑块”驱动如图5所示的参考虚拟形象变形，从而得到具有宽脸特征的目标虚拟形象。Exemplarily, as shown in Fig. 4A to Fig. 4D in sequence, "wide face slider", "narrow face slider", "long face slider" and "short face slider". Figure 5 shows the reference avatar that has completed the binding of bones and skin. Exemplarily, in the case that the general language description input by the user or the converted professional language description contains the "wide face" feature, the "wide face slider" as shown in Figure 4A can be directly used to drive the The reference avatar is deformed to obtain the target avatar with wide face features.
通过本公开实施例，采用带有语义信息的滑块驱动参考虚拟形象变形，可以提高目标虚拟形象的产出效率，同时可以提高所得目标虚拟形象的准确度。Through the embodiments of the present disclosure, using the slider with semantic information to drive the deformation of the reference avatar can improve the output efficiency of the target avatar and at the same time improve the accuracy of the obtained target avatar.
需要说明的是，为了实现虚拟形象的低成本变形，三维模型设计师通常会为人脸模型设计骨骼树，并在人脸蒙皮(骨骼蒙皮)的各蒙皮网格节点与骨骼树中的各个骨骼节点建立权重影响关系。之后，通过控制骨骼树中各骨骼节点的旋转、平移、缩放，就可以将各骨骼节点的变形传导到人脸蒙皮的各蒙皮网格节点上，实现各蒙皮网格节点的变形。It should be noted that, in order to realize the low-cost deformation of the avatar, the 3D model designer usually designs a bone tree for the face model, and connects each skin mesh node of the face skin (skeleton skin) with the bone tree Each bone node establishes weight influence relationship. Afterwards, by controlling the rotation, translation, and scaling of each bone node in the bone tree, the deformation of each bone node can be transmitted to each skin mesh node of the face skin to realize the deformation of each skin mesh node.
然而，这种骨骼树的设计是针对人脸的几何结构的，大部分骨骼节点并不具备如宽脸、高鼻梁等实际语义级意义。这使得设计师在完成骨骼蒙皮的设定工作后，还需要设计滑块，进而通过滑块来实现对骨骼树中各骨骼节点的批量操作，最终实现语义级表现能力。如通过宽脸滑块，就可以对左右太阳穴、左右颧骨、左右下颌角、左右额头等8个骨骼节点进行批量的联动调整设置。However, this skeletal tree is designed for the geometric structure of the face, and most of the skeletal nodes do not have actual semantic meanings such as wide face and high nose bridge. This makes the designer need to design the slider after completing the setting of the bone skin, and then use the slider to realize the batch operation of each bone node in the bone tree, and finally realize the semantic level performance. For example, through the wide face slider, you can perform batch linkage adjustment settings on the left and right temples, left and right cheekbones, left and right mandibular angles, and left and right foreheads.
但是，大量骨骼的联动设计不仅需要耗费设计师的人力成本，而且复杂的骨骼间关系常常导致设计出的语义级滑块的表现力并不好。However, the linkage design of a large number of bones not only requires the designer's labor cost, but also the complex relationship between bones often leads to the poor expressiveness of the designed semantic-level slider.
因此，本公开实施例提出了一种改进型的虚拟形象滑块设计方案。设计师在完成骨骼蒙皮设计后，即在完成骨骼与骨骼蒙皮绑定(又称骨骼与骨骼蒙皮关联)后，只需专注于设计与语义对应的形状模型即可，不需要继续设计对应的带有语义信息的虚拟形象滑块。这是因为，直接将设计师设计的由骨骼与骨骼蒙皮关联得到的参考虚拟形象和对应的形状模块输入“滑块设计系统”，即可自动输出带有语义信息的骨骼联动系数(也即滑块信息)，从而可以保证滑块的高质量设计。Therefore, the embodiment of the present disclosure proposes an improved avatar slider design solution. After the designer completes the bone skin design, that is, after the bone and bone skin binding (also known as the bone and bone skin association), the designer only needs to focus on designing the shape model corresponding to the semantics, and does not need to continue the design Corresponding avatar slider with semantic information. This is because, directly input the reference avatar and the corresponding shape module designed by the designer into the "slider design system", which can automatically output the bone linkage coefficient with semantic information (that is, Slider information) so that a high-quality design of the slider can be guaranteed.
作为一种可选的实施例，可以通过以下操作生成滑块。As an optional embodiment, the slider can be generated through the following operations.
获取与目标语义标签关联的形状模型。其中，目标语义标签与和滑块关联的特定语义标签相同。Get the shape model associated with the target semantic label. Wherein, the target semantic label is the same as the specific semantic label associated with the slider.
获取参考虚拟形象的骨骼蒙皮数据。Get the skeletal skinning data of the reference avatar.
基于骨骼蒙皮数据，对形状模型进行拟合，以获得对应的骨骼联动系数。Based on the bone skin data, the shape model is fitted to obtain the corresponding bone linkage coefficient.
基于骨骼联动系数，生成与目标语义标签关联的滑块。Based on the skeletal linkage coefficients, a slider associated with the target semantic label is generated.
其中，利用滑块驱动参考虚拟形象能够得到符合目标语义特征的虚拟形象，目标语义标签包含目标语义特征。Wherein, the avatar conforming to the target semantic feature can be obtained by using the slider to drive the reference avatar, and the target semantic label includes the target semantic feature.
示例性的，以脸型为例，将“宽脸模型”(与宽脸标签关联)和已完成骨骼与蒙皮绑定的参考虚拟形象输入骨骼拟合系数解算器，可以输出“宽脸骨骼联动系数”(即“宽脸滑块信息”)；将“窄脸模型”(与窄脸标签关联)和已完成骨骼与蒙皮绑定的参考虚拟形象输入骨骼拟合系数解算器，可以输出“窄脸骨骼联动系数”(即“窄脸滑块信息”)；将“长脸模型”(与长脸标签关联)和已完成骨骼与蒙皮绑定的参考虚拟形象输入骨骼拟合系数解算器，可以输出“长脸骨骼联动系数”(即“长脸滑块信息”)；将“短脸模型”(与短脸标签关联)和已完成骨骼与蒙皮绑定的参考虚拟形象输入骨骼拟合系数解算器，可以输出“短脸骨骼联动系数”(即“短脸滑块信息”)。其中，骨骼拟合系数解算器用于基于参考虚拟形象的骨骼蒙皮数据，对各形状模型进行拟合，从而得到对应的骨骼联动系数。在得到对应的骨骼联动系数后，将其与对应的语义标签关联，可以得到对应语义的滑块。例如，将“宽脸滑块信息”与“宽脸标签”关联，可以得到“宽脸滑块”。Exemplarily, taking the face shape as an example, the "wide face model" (associated with the wide face tag) and the reference avatar that has completed the binding of the skeleton and the skin are input into the bone fitting coefficient solver, and the "wide face skeleton Linkage coefficient" (that is, "wide face slider information"); input the "narrow face model" (associated with the narrow face tag) and the reference avatar that has completed the binding of bones and skin to the bone fitting coefficient solver, which can Output "narrow face bone linkage coefficient" (that is, "narrow face slider information"); input the "long face model" (associated with the long face label) and the reference avatar that has completed the binding of bones and skin to the bone fitting coefficient calculation It can output the "long face bone linkage coefficient" (that is, "long face slider information"); input the "short face model" (associated with the short face label) and the reference avatar that has completed the binding of bones and skin to bone fitting The coefficient solver can output "short face bone linkage coefficient" (that is, "short face slider information"). Wherein, the bone fitting coefficient solver is used to fit each shape model based on the bone skin data of the reference avatar, so as to obtain the corresponding bone linkage coefficient. After obtaining the corresponding skeletal linkage coefficient, associate it with the corresponding semantic label to obtain the corresponding semantic slider. For example, associating "wide face slider information" with "wide face label" can get "wide face slider".
需要说明的，在本公开的另一个实施例中，还可以将参考虚拟形象的骨骼蒙皮数据连同多个形状模型(每个形状模型对应于不同的语义标签)一同传入骨骼拟合系数解算器，从而自动获取到各语义标签对应的多个滑块，从而可以保证滑块的高效生产。It should be noted that in another embodiment of the present disclosure, the bone skin data of the reference avatar together with multiple shape models (each shape model corresponds to a different semantic label) can also be passed into the bone fitting coefficient solution Calculator, so as to automatically obtain multiple sliders corresponding to each semantic tag, so as to ensure efficient production of sliders.
示例性的，如图6所示，以脸型为例，将“宽脸模型”、“窄脸模型”、“长脸模型”和“短脸模型”连通已完成骨骼与蒙皮绑定的参考虚拟形象一同输入骨骼拟合系数解算器，从而自动输出“宽脸滑块”、“窄脸滑块”、“长脸滑块”和“短脸滑块”。Exemplarily, as shown in Figure 6, taking the face shape as an example, connect the "wide face model", "narrow face model", "long face model" and "short face model" to the reference virtual model that has completed the binding of bones and skin. The image is input to the bone fitting coefficient solver together, so as to automatically output "wide face slider", "narrow face slider", "long face slider" and "short face slider".
应该理解，相关技术中，语义化的滑块设计是多个骨骼节点在设计师的设计下联动实现的。具体地，骨骼一般具有平移、旋转、缩放三个自由度的变换能力，设计师会为骨骼影响到的蒙皮网格节点设置权重，在实际形变中，蒙皮网格节点根据骨骼变换数据和设置的对应权重，进行加权变形。It should be understood that in related technologies, the semantic slider design is realized through the linkage of multiple bone nodes under the designer's design. Specifically, bones generally have three degrees of freedom transformation capabilities of translation, rotation, and scaling. The designer will set weights for the skin mesh nodes affected by the bones. In actual deformation, the skin mesh nodes are transformed according to the bone transformation data and Set the corresponding weight to carry out weighted deformation.
但是，一个滑块通常能够影响多个骨骼节点，设计师通过设计滑块对多个骨骼节点的影响关系，使得滑块有了对应语义。如“低颧骨”滑块，“尖下巴”滑块等，但这些语义特征必须依靠多个骨骼节点的联动才能实现。However, a slider can usually affect multiple skeletal nodes. By designing the influence relationship of the slider to multiple skeletal nodes, the slider has corresponding semantics. Such as "low cheekbone" slider, "sharp chin" slider, etc., but these semantic features must rely on the linkage of multiple bone nodes to achieve.
而通过本公开实施例，设计师只需要专注于与语义标签关联的形状模型设计即可，后续可以通过骨骼系数拟合解算器对形状模型进行拟合，来实现滑块设计。即，本公开实施例，整合骨骼系数拟合能力，重新定义了滑块的生成方式、生产流程，全面减轻了设计师对于滑块设计的负担，让设计师摆脱了繁琐的多骨骼联动设计，能够更加专注于语义对应的形状模型的设计，因而可以从复杂的多骨骼节点联动设计中将设计师解放出来，从而提升数字资产的生产效率。However, through the embodiments of the present disclosure, the designer only needs to focus on the design of the shape model associated with the semantic label, and then the shape model can be fitted by the bone coefficient fitting solver to realize the slider design. That is, the embodiment of the present disclosure integrates the bone coefficient fitting ability, redefines the generation method and production process of the slider, and fully reduces the designer's burden on the slider design, freeing the designer from the cumbersome multi-skeleton linkage design. It can focus more on the design of the semantically corresponding shape model, thus freeing the designer from the complex multi-skeleton node linkage design, thereby improving the production efficiency of digital assets.
作为一种可选的实施例，可以通过以下操作创建参考虚拟形象。As an optional embodiment, the reference avatar may be created through the following operations.
针对参考虚拟形象，创建对应的骨骼树。Create a corresponding bone tree for the reference avatar.
基于骨骼树，将骨骼与骨骼蒙皮关联，以得到参考虚拟形象。Based on the skeleton tree, associate the skeleton with the skeleton skin to obtain the reference avatar.
示例性的，以人脸模型为例，设计师可以为人脸模型设计骨骼树，并将人脸蒙皮(骨骼蒙皮)与该骨骼树中的各骨骼节点进行关联，实现人脸蒙皮与各骨骼节点的绑定，从而得到对应的参考虚拟形象。Exemplarily, taking the human face model as an example, the designer can design a bone tree for the human face model, and associate the human face skin (skeleton skin) with each bone node in the bone tree, so as to realize the human face skin and Binding of each bone node to obtain the corresponding reference avatar.
作为一种可选的实施例，基于骨骼蒙皮数据，对形状模型进行拟合，以获得对应的骨骼联动系数，可以包括如下操作。As an optional embodiment, fitting the shape model based on the bone skin data to obtain the corresponding bone linkage coefficient may include the following operations.
基于骨骼蒙皮数据，从骨骼树的根骨骼节点到叶骨骼节点，对形状模型进行逐一迭代求解，以获得骨骼联动系数。Based on the bone skinning data, from the root bone node to the leaf bone node of the bone tree, the shape model is iteratively solved one by one to obtain the bone linkage coefficient.
其中，骨骼树是针对参考虚拟形象创建的。Among them, the bone tree is created for the reference avatar.
通过本公开实施例，可以通过一次自底向上的迭代算法获得骨骼联动系数，因而拟合计算效率更高。Through the embodiments of the present disclosure, the bone linkage coefficient can be obtained through a bottom-up iterative algorithm, so the fitting calculation efficiency is higher.
作为一种可选的实施例，从骨骼树的根骨骼节点到叶骨骼节点，对形状模型进行逐一迭代求解，可以包括：从骨骼树的根骨骼节点开始，逐级应用最小二乘法，分别对各骨骼节点的旋转、平移、缩放系数进行拟合，直到骨骼树中的所有叶骨骼节点的旋转、平移、缩放系数都解算完成为止。As an optional embodiment, iteratively solving the shape model one by one from the root bone node to the leaf bone node of the bone tree may include: starting from the root bone node of the bone tree, applying the least square method step by step, respectively The rotation, translation, and scaling coefficients of each bone node are fitted until the rotation, translation, and scaling coefficients of all leaf bone nodes in the bone tree are solved.
需要说明的是，在本公开实施例中，骨骼系数拟合解算器可以采用自底向上的解算策略。即，从骨骼树的根节点开始，逐级应用最小二乘法，分别对形状模型的各骨骼节点的旋转、平移、缩放系数进行拟合，直到骨骼树中的所有叶节点解算完成为止。It should be noted that, in the embodiment of the present disclosure, the bone coefficient fitting solver may adopt a bottom-up solving strategy. That is, starting from the root node of the skeleton tree, the least square method is applied step by step to fit the rotation, translation, and scaling coefficients of each skeleton node of the shape model, until all the leaf nodes in the skeleton tree are solved.
作为一种可选的实施例，基于骨骼蒙皮数据，对形状模型进行拟合，以获得对应的骨骼联动系数，可以包括：基于骨骼蒙皮数据，从骨骼树的根骨骼节点到叶骨骼节点，对形状模型进行多伦逐一迭代求解，以获得骨骼联动系数。其中，骨骼树是针对参考虚拟形象创建的。As an optional embodiment, based on the bone skin data, the shape model is fitted to obtain the corresponding bone linkage coefficient, which may include: based on the bone skin data, from the root bone node to the leaf bone node of the bone tree , to solve the shape model one by one iteratively to obtain the bone linkage coefficient. Among them, the bone tree is created for the reference avatar.
通过本公开实施例，也可以通过多伦自底向上的迭代算法获得骨骼联动系数，即，逐步将各级骨骼树节点的拟合系数解算出来，因而得到的拟合结果更加精确。Through the embodiments of the present disclosure, the bone linkage coefficient can also be obtained through the bottom-up iterative algorithm of Duolun, that is, the fitting coefficients of the skeleton tree nodes at all levels are gradually calculated, so that the obtained fitting result is more accurate.
此外，本公开实施例也可以支持其他类型的骨骼系数拟合解算算法，本公开实施例在此不再限定。In addition, the embodiments of the present disclosure may also support other types of bone coefficient fitting and solving algorithms, which are not limited in the embodiments of the present disclosure.
本公开实施例通过对相关算法的整合，以及对滑块智能生成系统输入输出的语义化定义，实现了虚拟形象滑块的智能生成。The embodiment of the present disclosure realizes the intelligent generation of the virtual image slider through the integration of related algorithms and the semantic definition of the input and output of the slider intelligent generation system.
作为一种可选的实施例，基于骨骼蒙皮数据，对形状模型进行拟合，以获得对应的骨骼联动系数，可以包括：将骨骼蒙皮数据和形状模型输入预先设定的骨骼系数拟合解算器，以便通过骨骼系数拟合解算器，对形状模型进行拟合，从而获得骨骼联动系数。As an optional embodiment, fitting the shape model based on the bone skin data to obtain the corresponding bone linkage coefficient may include: inputting the bone skin data and shape model into the preset bone coefficient fitting Solver, in order to fit the shape model through the bone coefficient fitting solver, so as to obtain the bone linkage coefficient.
通过本公开实施例，设计师只需要专注于与语义标签关联的形状模型设计即可，后续可以通过骨骼系数拟合解算器对形状模型进行拟合，来实现滑块设计。Through the embodiments of the present disclosure, the designer only needs to focus on the design of the shape model associated with the semantic label, and then the shape model can be fitted by the bone coefficient fitting solver to realize the slider design.
作为一种可选的实施例，该方法还可以包括：将骨骼蒙皮数据和生成的滑块存储在同一文件中。As an optional embodiment, the method may further include: storing the skeleton skinning data and the generated slider in the same file.
示例性的，在得到“宽脸滑块”后，如果将“宽脸滑块”和“参考虚拟形象”存储在同一文件中，则触发“宽脸虚拟形象”启动流程后，“宽脸滑块”可以自动驱动“参考虚拟形象”来生成“宽脸虚拟形象”。Exemplarily, after obtaining the "wide face slider", if the "wide face slider" and the "reference avatar" are stored in the same file, after the "wide face avatar" startup process is triggered, the "wide face slider" Block" can automatically drive the "reference avatar" to generate the "wide face avatar".
通过本公开实施例，将虚拟形象滑块与参考虚拟形象存储在同一文件中，启动虚拟形象时可以直接利用滑块驱动参考虚拟形象来快速输出用户描述的目标虚拟形象。Through the embodiments of the present disclosure, the avatar slider and the reference avatar are stored in the same file, and the slider can be directly used to drive the reference avatar to quickly output the target avatar described by the user when starting the avatar.
作为一种可选的实施例，基于语义特征，生成目标虚拟形象，可以包括如下操作。As an optional embodiment, generating the target avatar based on semantic features may include the following operations.
基于语义特征，确定至少一个语义标签。Based on the semantic features, at least one semantic label is determined.
基于语义标签，确定对应的至少一个配件模型和/或至少一个饰品模型。Based on the semantic tags, the corresponding at least one accessory model and/or at least one accessory model are determined.
在通过控制参考虚拟形象变形而得到的虚拟形象的基础上，增加至少一个配件模型和/或至少一个饰品模型，以得到目标虚拟形象。On the basis of the avatar obtained by controlling the deformation of the reference avatar, at least one accessory model and/or at least one accessory model are added to obtain the target avatar.
示例性的，如果用户想要创建一个“高鼻梁、大眼睛、瘦下巴、长头发、学生装、白球鞋”的目标虚拟形象，则可以先基于该语义描述提取出如下语义特征“高鼻梁、大眼睛、瘦下巴”。然后获取“高鼻梁滑块”、“大眼睛滑块”、“瘦下巴滑块”3个滑块，并利用这3个滑块驱动预先创建的参考虚拟形象变形，从而获得满足“高鼻梁、大眼睛、瘦下巴”特征的虚拟形象。同时，还可以基于该语义描述提取出如下语义特征“长头发、学生装、白球鞋”，并从发型配件数字资产库中获取“长头发模型”等配件模型，以及从包含服装、鞋、帽等的服饰数字资产库中获取“女学生装模型”和“白球鞋模型”等饰品模型，最后将“长头发模型”、“女学生装模型”和“白球鞋模型”都添加在前面已创建的具有“高鼻梁、大眼睛、瘦下巴”特征的虚拟形象上，最后输出的虚拟形象即为用户想要的目标虚拟形象。Exemplarily, if the user wants to create a target avatar of "high nose bridge, big eyes, thin chin, long hair, student uniform, white sneakers", the following semantic features can be extracted based on the semantic description "high nose bridge, big Eyes, thin chin". Then obtain the three sliders of "high nose bridge slider", "big eyes slider" and "thin chin slider", and use these three sliders to drive the deformation of the pre-created reference avatar, so as to obtain the "high nose bridge, An avatar with the characteristics of "big eyes and thin chin". At the same time, based on the semantic description, the following semantic features "long hair, student uniform, white sneakers" can be extracted, and accessory models such as "long hair model" can be obtained from the digital asset library of hair accessories, as well as accessories including clothing, shoes, hats, etc. Obtain accessories models such as "girl's school uniform model" and "white sneakers model" from the clothing digital asset library, and finally add "long hair model", "female school On the avatar with the characteristics of high nose bridge, big eyes, and thin chin, the final output avatar is the target avatar that the user wants.
应该理解，在本公开实施例中，发型、胡须配件数字资产库中可以包含各类男士胡须模型和各类男士发型模型，以及各类女士发型模型。也应该理解，服饰数字资产库中可以包含各类男性、女性的眼镜、衣服、鞋子、手表、手套、头饰、围巾等饰品模型。数字资产库中的每个模型都关联有唯一的语义标签，以便自动根据语义标签获取到对应的模型。It should be understood that, in the embodiment of the present disclosure, the hairstyle and beard accessories digital asset library may contain various types of men's beard models, various types of men's hairstyle models, and various types of women's hairstyle models. It should also be understood that the clothing digital asset library may contain various male and female glasses, clothes, shoes, watches, gloves, headwear, scarves and other accessories models. Each model in the digital asset library is associated with a unique semantic tag, so that the corresponding model can be automatically obtained according to the semantic tag.
示例性的，可以为数字资产库中每个模型定义不同的语义标签，如白球鞋、高跟鞋、小红裙、学生装、职业正装等。根据语义转换器输出的发型、服装等语义标签，可以在数字资产库中选出对应的模型，加入已生成的虚拟形象中。Exemplarily, different semantic tags can be defined for each model in the digital asset library, such as white sneakers, high heels, little red dress, student clothes, professional formal clothes, etc. According to the semantic tags such as hairstyle and clothing output by the semantic converter, the corresponding model can be selected from the digital asset library and added to the generated avatar.
如图7所示，个性化虚拟形象的定制过程可以如下：用户输入语音描述；通过ASR技术自动进行语音识别；提取语音描述中关键字；通过语义转换器对提取的关键字进行转换；从转换后的关键字中获取与滑块关联的滑块语义标签；基于滑块语义标签获取驱动骨骼变形的滑块；基于滑块获取对应的骨骼变形系数；基于骨骼变形系数驱动骨骼蒙皮联动变形；从转换后的关键字中获取与发型、服装等关联的模型语义标签；基于模型语义标签从数字资产库中获取对应的模型；将获取的模型添加在由骨骼蒙皮联动变形生成的虚拟形象，得到最终的目标虚拟形象。As shown in Figure 7, the customization process of a personalized avatar can be as follows: the user inputs a voice description; automatically recognizes the voice through ASR technology; extracts the keywords in the voice description; converts the extracted keywords through a semantic converter; Obtain the semantic label of the slider associated with the slider in the following keywords; obtain the slider that drives the bone deformation based on the semantic label of the slider; obtain the corresponding bone deformation coefficient based on the slider; drive the joint deformation of the bone skin based on the bone deformation coefficient; Obtain the model semantic tags associated with hairstyles, clothing, etc. from the converted keywords; obtain the corresponding model from the digital asset library based on the model semantic tags; add the obtained model to the avatar generated by the skeleton skin linkage deformation, Get the final target avatar.
通过本公开实施例，可以通过配饰美化和丰富目标虚拟形象，使得获得的目标虚拟形象更满足用户的理想型。Through the embodiments of the present disclosure, the target avatar can be beautified and enriched by accessories, so that the obtained target avatar can better meet the user's ideal type.
根据本公开的实施例，本公开还提供了一种基于虚拟形象的人机交互装置。According to an embodiment of the present disclosure, the present disclosure also provides an avatar-based human-computer interaction device.
图8A示例性示出了根据本公开实施例的基于虚拟形象的人机交互装置的框图。Fig. 8A schematically shows a block diagram of an avatar-based human-computer interaction device according to an embodiment of the present disclosure.
如图8A所示，基于虚拟形象的人机交互装置800A包括：第一显示模块810A、第一控制模块820A和第二控制模块830A。As shown in FIG. 8A , the avatar-based human-
第一显示模块810A，用于在智能家居设备上显示虚拟形象。The
第一控制模块820A，用于控制该虚拟形象与用户进行交流互动。The
第二控制模块830A，用于在该虚拟形象与该用户交流互动的过程中，控制该虚拟形象对该用户进行消费推荐。The
作为一种可选的实施例，该第一控制模块包括：获取单元，用于获取云端针对该用户下发的营销策略；以及控制单元，用于控制该虚拟形象基于该营销策略，对该用户进行消费推荐。As an optional embodiment, the first control module includes: an acquisition unit, configured to acquire a marketing strategy issued by the cloud for the user; and a control unit, configured to control the avatar to Make consumer recommendations.
作为一种可选的实施例，该第一显示模块还用于：在该智能家居设备上显示该用户的专属虚拟形象，其中，该专属虚拟形象是由云端下发给该智能家居设备的。As an optional embodiment, the first display module is further configured to: display the user's exclusive avatar on the smart home device, wherein the exclusive avatar is sent to the smart home device by the cloud.
根据本公开的实施例，本公开还提供了一种基于虚拟形象的控制装置。According to an embodiment of the present disclosure, the present disclosure also provides an avatar-based control device.
图8B示例性示出了根据本公开实施例的基于虚拟形象的控制装置的框图。Fig. 8B exemplarily shows a block diagram of an avatar-based control device according to an embodiment of the present disclosure.
如图8B所示，基于虚拟形象的控制装置800B包括：第三控制模块810B和第一发送模块820B。As shown in FIG. 8B , the avatar-based
第三控制模块810B，用于远程控制显示在智能家居设备上的虚拟形象与用户进行交流互动。The
第一发送模块820B，用于在该虚拟形象与该用户交流互动的过程中，向该智能家居设备下发针对该用户的营销策略，以便该虚拟形象基于该营销策略，对该用户进行消费推荐。The
作为一种可选的实施例，该装置还包括：第一获取模块，用于获取该用户提供的专属虚拟形象；以及第二发送模块，用于将该专属虚拟形象下发给与该用户关联的该智能家居设备，以便该智能家居设备在面对该用户时显示该专属虚拟形象，并通过该专属虚拟形象与该用户进行人机交互。As an optional embodiment, the device further includes: a first acquiring module, configured to acquire the exclusive avatar provided by the user; and a second sending module, configured to deliver the exclusive avatar to the user associated with the user. The smart home device, so that the smart home device displays the exclusive virtual image when facing the user, and conducts human-computer interaction with the user through the exclusive virtual image.
作为一种可选的实施例，该装置还包括：第二获取模块，用于获取该用户的消费数据；以及生成模块，用于基于该消费数据，生成针对该用户的该营销策略，以下发给该智能家居设备。As an optional embodiment, the device further includes: a second acquisition module, configured to acquire the user's consumption data; and a generation module, configured to generate the marketing strategy for the user based on the consumption data, as follows: to that smart home device.
根据本公开的实施例，本公开还提供了一种基于虚拟形象的人机交互装置。According to an embodiment of the present disclosure, the present disclosure also provides an avatar-based human-computer interaction device.
图8C示例性示出了根据本公开实施例的基于虚拟形象的人机交互装置的框图。Fig. 8C exemplarily shows a block diagram of an avatar-based human-computer interaction device according to an embodiment of the present disclosure.
如图8C所示，基于虚拟形象的人机交互装置800C包括：第二显示模块810C和第四控制模块820C。As shown in FIG. 8C , the avatar-based human-
第二显示模块810C，用于在特定场所中设置的特定互动设备上显示虚拟形象。The
第四控制模块820C，用于在用户在该特定场所活动的过程中，控制该虚拟形象与该用户进行交流互动。The
作为一种可选的实施例，该第四控制模块还用于执行以下至少之一：在该用户在商场或超市购物的过程中，控制该虚拟形象与该用户进行交流互动，以陪伴该用户购物；在该用户在饭店或餐馆吃饭的过程中，控制该虚拟形象与该用户聊天或进行互动游戏，以陪伴该用户用餐；在该用户在休闲娱乐场所消费的过程中，控制该虚拟形象与该用户进行交流互动，以陪伴该用户休闲娱乐。As an optional embodiment, the fourth control module is further configured to perform at least one of the following: when the user is shopping in a mall or a supermarket, control the avatar to interact with the user to accompany the user Shopping; when the user is eating in a restaurant or restaurant, control the virtual image to chat with the user or play an interactive game to accompany the user to eat; The user communicates and interacts to accompany the user for leisure and entertainment.
作为一种可选的实施例，该装置还包括：第三获取模块，用于获取用户的消费数据；以及第三发送模块，用于将该消费数据发送至云端，以便该云端基于该消费数据生成针对该用户的营销策略。As an optional embodiment, the device further includes: a third acquiring module, configured to acquire the user's consumption data; and a third sending module, configured to send the consumption data to the cloud, so that the cloud based on the consumption data Generate a marketing strategy for that user.
作为一种可选的实施例，该装置还包括：确定模块，用于响应于该用户进入该特定场所，对该用户进行人脸识别，以确定该用户的身份；以及第四获取模块，用于基于该用户的身份获取该用户的专属虚拟形象，其中，该第二显示模块还用于：在该特定场所中设置的该特定互动设备上显示该专属虚拟形象。As an optional embodiment, the device further includes: a determination module, configured to perform face recognition on the user in response to the user entering the specific place, so as to determine the identity of the user; and a fourth acquisition module, configured to Obtaining the exclusive virtual image of the user based on the identity of the user, wherein the second display module is further used for: displaying the exclusive virtual image on the specific interactive device set in the specific place.
应该理解，本公开装置部分的实施例与本公开方法部分的实施例对应相同或类似，所达到的技术效果也对应相同或类似，在此不再赘述。It should be understood that the embodiments of the device part of the present disclosure are correspondingly the same or similar to the embodiments of the method part of the present disclosure, and the achieved technical effects are also correspondingly the same or similar, and will not be repeated here.
根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
图9示出了可以用来实施本公开的实施例的示例电子设备900的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。FIG. 9 shows a schematic block diagram of an example
如图9所示，电子设备900包括计算单元901，其可以根据存储在只读存储器(ROM)902中的计算机程序或者从存储单元908加载到随机访问存储器(RAM)903中的计算机程序，来执行各种适当的动作和处理。在RAM 903中，还可存储电子设备900操作所需的各种程序和数据。计算单元901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(I/O)接口905也连接至总线904。As shown in FIG. 9 , an
电子设备900中的多个部件连接至I/O接口905，包括：输入单元906，例如键盘、鼠标等；输出单元907，例如各种类型的显示器、扬声器等；存储单元908，例如磁盘、光盘等；以及通信单元909，例如网卡、调制解调器、无线通信收发机等。通信单元909允许设备900通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Multiple components in the
计算单元901可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元901的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元901执行上文所描述的各个方法和处理，例如基于虚拟形象的人机交互、控制方法。例如，在一些实施例中，基于虚拟形象的人机交互、控制方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元908。在一些实施例中，计算机程序的部分或者全部可以经由ROM 902和/或通信单元909而被载入和/或安装到设备900上。当计算机程序加载到RAM 903并由计算单元901执行时，可以执行上文描述的基于虚拟形象的人机交互、控制方法的一个或多个步骤。备选地，在其他实施例中，计算单元901可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行基于虚拟形象的人机交互、控制方法。The
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor Can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide for interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-＝服务器关系的计算机程序来产生客户端和服务器的关系。A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-=server relationship to each other.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.
上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。The specific implementation manners described above do not limit the protection scope of the present disclosure. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110316646.0ACN112987932B (en) | 2021-03-24 | 2021-03-24 | Human-computer interaction and control method and device based on virtual image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110316646.0ACN112987932B (en) | 2021-03-24 | 2021-03-24 | Human-computer interaction and control method and device based on virtual image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112987932A CN112987932A (en) | 2021-06-18 |
| CN112987932Btrue CN112987932B (en) | 2023-04-18 |
Family
ID=76333573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110316646.0AActiveCN112987932B (en) | 2021-03-24 | 2021-03-24 | Human-computer interaction and control method and device based on virtual image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112987932B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114444768B (en)* | 2021-12-21 | 2025-03-28 | 科大讯飞股份有限公司 | Gamified learning system, gamified learning method and computing device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109584146A (en)* | 2018-10-15 | 2019-04-05 | 深圳市商汤科技有限公司 | U.S. face treating method and apparatus, electronic equipment and computer storage medium |
| CN112184921A (en)* | 2020-10-30 | 2021-01-05 | 北京百度网讯科技有限公司 | Avatar driving method, apparatus, device, and medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101359385A (en)* | 2008-09-23 | 2009-02-04 | 中国移动通信集团广东有限公司 | A non-contact precision marketing system and marketing method based on mobile network |
| US20130138499A1 (en)* | 2011-11-30 | 2013-05-30 | General Electric Company | Usage measurent techniques and systems for interactive advertising |
| CN108510917A (en)* | 2017-02-27 | 2018-09-07 | 北京康得新创科技股份有限公司 | Event-handling method based on explaining device and explaining device |
| EP3721325A1 (en)* | 2018-02-26 | 2020-10-14 | Seddi, Inc. | Avatar matching in on-line shopping |
| CN110767220B (en)* | 2019-10-16 | 2024-05-28 | 腾讯科技（深圳）有限公司 | Interaction method, device and equipment of intelligent voice assistant and storage medium |
| CN111429907B (en)* | 2020-03-25 | 2023-10-20 | 北京百度网讯科技有限公司 | Voice service mode switching method, device, equipment and storage medium |
- 2021
- 2021-03-24CNCN202110316646.0Apatent/CN112987932B/enactiveActive
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109584146A (en)* | 2018-10-15 | 2019-04-05 | 深圳市商汤科技有限公司 | U.S. face treating method and apparatus, electronic equipment and computer storage medium |
| CN112184921A (en)* | 2020-10-30 | 2021-01-05 | 北京百度网讯科技有限公司 | Avatar driving method, apparatus, device, and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112987932A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220157036A1 (en) | Method for generating virtual character, electronic device, and storage medium | |
| CN114187394B (en) | Avatar generation method, apparatus, electronic device, and storage medium | |
| CN113362263B (en) | Method, apparatus, medium and program product for transforming an image of a virtual idol | |
| US20240297957A1 (en) | Aspect ratio conversion for automated image generation | |
| EP4396669A1 (en) | Conversation guided augmented reality experience | |
| US11842457B2 (en) | Method for processing slider for virtual character, electronic device, and storage medium | |
| US20240404225A1 (en) | Avatar generation from digital media content items | |
| CN116740238A (en) | Personalized configuration method, device, electronic equipment and storage medium | |
| CN116977486A (en) | Image generation method, device, equipment and storage medium | |
| CN112987932B (en) | Human-computer interaction and control method and device based on virtual image | |
| CN112083863A (en) | Image processing method, apparatus, electronic device and readable storage medium | |
| CN113537043B (en) | Image processing method, device, electronic device and storage medium | |
| CN118350883A (en) | Information processing method and wearable device | |
| CN114239241B (en) | Card generation method and device and electronic equipment | |
| CN112084983B (en) | Hair style recommendation method based on ResNet and application thereof | |
| KR20230044583A (en) | Recording medium on which hair style simulation program is recorded | |
| KR20230045635A (en) | An apparatus for providing a hair style recommendation service in consideration of fashion elements worn by a user and an operating method thereof | |
| CN114648601A (en) | Virtual image generation method, electronic device, program product and user terminal | |
| CN114333018A (en) | Shaping information recommendation method and device and electronic equipment | |
| KR20230044586A (en) | Apparatus and method for customized virtual hair experience according to effects of hair care products | |
| KR20230045636A (en) | Method for providing realistic hair style simulation service based on artificial intelligence model | |
| KR20230044582A (en) | An apparatus for realistic hairstyle simulations by using transformation of bare hair based on ai model and a method for operating it | |
| KR20230044587A (en) | Apparatus for providing virtual hair style experience service according to hair style search terms and method of operation thereof | |
| KR20230045634A (en) | A method for customized virtual hair experience | |
| KR20230045631A (en) | An apparatus for realistic hair style simulation based on analysis of bare hair and user information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |