CN110969682A

Movatterモバイル変換

Info

Publication number: CN110969682A
Application number: CN201911182020.4A
Authority: CN
Inventors: 杨国基
Original assignee: Shenzhen Zhuiyi Technology Co Ltd
Current assignee: Shenzhen Zhuiyi Technology Co Ltd
Priority date: 2019-11-27
Filing date: 2019-11-27
Publication date: 2020-04-07
Anticipated expiration: 2039-11-27
Also published as: CN110969682B

Abstract

The application discloses a virtual image switching method, a virtual image switching device, electronic equipment and a storage medium, wherein the method comprises the following steps: when a video including a virtual image of a target person is played, acquiring a real image of the target person; extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person; comparing the first characteristic parameter with the second characteristic parameter to obtain a comparison result, and updating the virtual image in the video based on the comparison result; and when the updated virtual image and the real image meet preset conditions, switching the updated virtual image in the video into the real image. When the picture is switched between the real person picture and the virtual image, the transition of the picture is smoother and more natural, so that the experience of a user is improved.

Description

Virtual image switching method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of electronic devices, and in particular, to a method and an apparatus for switching an avatar, an electronic device, and a storage medium.

Background

At present, the popularity of mobile terminal devices such as mobile phones and the like is higher and higher, and smart phones become essential personal belongings for people going out. With the rapid development of the mobile internet, various applications appear on the mobile terminal, and many of the applications can provide customer service functions for users, so that the users can perform services such as product consultation and the like through the customer service.

Generally, the customer service function provided for the user in the mobile terminal application usually includes two parts of visual robot customer service and manual customer service. In the face of some simple or common questions, the robot service can answer the questions of the user, and in the case of some complicated or special questions, the robot service can transfer the manual service to deal with the problems.

However, in the current customer service video, the pictures of the robot customer service and the manual customer service are directly switched when being switched, so that the two pictures cannot be well connected, the picture switching is unnatural, and the user experience is reduced.

Disclosure of Invention

In view of the above problems, the present application provides a method, an apparatus, an electronic device and a storage medium for switching an avatar, which can make a transition of a picture smoother and more natural when a picture of a real person and a picture of the avatar are switched with each other, thereby improving a user experience.

In a first aspect, an embodiment of the present application provides an avatar switching method, including: when a video including a virtual image of a target person is played, acquiring a real image of the target person; extracting a first characteristic parameter of a target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person; comparing the first characteristic parameter with the second characteristic parameter to obtain a comparison result, and updating the virtual image in the video based on the comparison result; and when the updated virtual image and the real image meet the preset condition, switching the updated virtual image in the video into the real image.

Further, updating the virtual image in the video based on the third characteristic parameter includes: generating a target virtual image based on the third characteristic parameter; and updating the virtual image in the video into a target virtual image.

Further, obtaining a third feature parameter based on the first feature parameter and the second feature parameter includes: and inputting the first characteristic parameter and the second characteristic parameter into a pre-trained prediction model, and acquiring a third characteristic parameter output by the pre-trained prediction model.

Further, the method further comprises: and when the similarity of the first characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, switching the virtual image in the video into a real image.

Further, before switching the updated virtual image in the video to the real image when the updated virtual image and the real image satisfy the preset condition, the method further includes: judging whether the similarity between the third characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value or not; and when the similarity of the third characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, determining that the updated virtual image and the updated real image meet the preset condition.

Further, the determining whether the similarity between the third feature parameter and the second feature parameter is smaller than the similarity threshold includes: judging whether the distance between the first key point and the second key point is smaller than a distance threshold value; and when the distance between the first key point and the second key point is smaller than the distance threshold, determining that the similarity between the third characteristic parameter and the second characteristic parameter is not smaller than the similarity threshold.

Further, generating a target virtual image based on the third feature parameter includes: and inputting the third characteristic parameter into a virtual image model trained in advance to obtain a target virtual image corresponding to the third characteristic parameter.

Further, before acquiring a real image of the target person when playing a video including a virtual image of the target person, the method further includes: obtaining a sample image of a target person; extracting sample characteristic parameters and sample virtual images of the target person from the sample images; and inputting the sample characteristic parameters and the sample virtual images into a machine learning model for training to obtain a pre-trained virtual image model.

Further, the update module includes:

the judging unit is used for judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value;

and the updating unit is used for updating the virtual image in the video based on the third characteristic parameter.

Further, the updating unit is specifically configured to generate a target virtual image based on the third characteristic parameter; and updating the virtual image in the video into a target virtual image.

Further, the third feature parameter generating unit is specifically configured to input the first feature parameter and the second feature parameter into a pre-trained prediction model, and obtain a third feature parameter output by the pre-trained prediction model.

In a third aspect, an embodiment of the present application provides an electronic device, which includes: a memory; one or more processors coupled with the memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of the first aspect as described above.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, in which program code is stored, and the program code can be called by a processor to execute the method according to the first aspect.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 shows a flowchart of an avatar switching method according to a first embodiment of the present application.

Fig. 2 is a flowchart illustrating an avatar switching method according to a second embodiment of the present application.

Fig. 3 is a flowchart illustrating an avatar switching method according to a third embodiment of the present application.

Fig. 4 is a flowchart illustrating an avatar switching method according to a fourth embodiment of the present application.

Fig. 5 is a flowchart illustrating an avatar switching method according to a fifth embodiment of the present application.

Fig. 6 is a flowchart illustrating an avatar switching method according to a sixth embodiment of the present application.

Fig. 7 is a flowchart illustrating an avatar switching method according to a seventh embodiment of the present application.

Fig. 8 is a block diagram illustrating an avatar switching apparatus according to an eighth embodiment of the present application.

Fig. 9 is a block diagram of an electronic device for performing an avatar switching method according to an embodiment of the present application in a ninth embodiment of the present application.

Fig. 10 is a storage unit for storing or carrying a program code implementing an avatar switching method according to a tenth embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Along with the development of science and technology, the requirement of people for humanized experience in the use process of various intelligent products is gradually increased, and in the process of communicating with customer service, a user also hopes that the user can not only obtain the reply of characters or voice, but also can communicate in a more natural interaction mode similar to interpersonal communication in actual life. Therefore, the current intelligent products can communicate with the user by playing the video containing the virtual image of the robot customer service so as to meet the visual demands of the user.

However, when the customer service robot encounters a question that cannot be answered, it needs to switch to manual customer service to answer the question of the user, and at the same time, the virtual image corresponding to the customer service robot displayed in the video is also converted into a real image corresponding to the manual customer service. However, the current switching mode is often to directly stop playing the virtual image and then start playing the real image, so that there is no linking process when the virtual image is switched to the real image, and the image switching causes the user to feel more obtrusive and unnatural, thereby reducing the user experience.

The inventor finds in research that if the characteristics of the action, expression, posture and the like of the artificial customer service in the real image and the virtual image of the customer service robot in the virtual image are kept consistent as much as possible and then switching is performed, the switching process of the two images can be more smooth, and therefore user experience can be improved.

However, in the actual research process, the inventor also finds that, because the virtual customer service robot frame and the real artificial customer service frame are always in a dynamic state, it is difficult to find a time when the frame of the customer service robot and the artificial customer service frame are synchronized, that is, a time when the virtual image of the customer service robot in the virtual image and the characteristics of the artificial customer service in the real image are consistent.

In order to improve the above problem, the inventor proposes an avatar switching method, an avatar switching apparatus, an electronic device, and a storage medium in the embodiments of the present application. The method and the system have the advantages that a section of transition video can be played before the customer service robot is switched to the real person customer service, so that the characteristic parameters of the customer service robot are gradually close to the characteristic parameters of the real person customer service, the characteristic parameters of the customer service robot can be kept approximately consistent with the characteristic parameters of the real person customer service when the customer service robot is switched, the fact that the process that the customer service robot is switched to the real person customer service in the video looks smoother is guaranteed, and user experience is improved.

The following describes in detail an avatar switching method, an avatar switching apparatus, an electronic device, and a storage medium according to embodiments of the present application.

First embodiment

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an avatar switching method according to an embodiment of the present application. The method can be applied to electronic devices. The electronic device may be various electronic devices having a display screen, a camera, an audio output function, and data input, including but not limited to a smart phone, a tablet computer, a laptop portable computer, a desktop computer, a wearable electronic device, and the like. Specifically, the data input may be the input of a voice based on a voice module electronically provided, the input of a character by a character input module, or the like.

The method may comprise the steps of:

s101, when playing a video including a virtual image of a target person, acquiring a real image of the target person.

In some embodiments, the electronic device may play a video including a virtual image of the target person, and in addition, the electronic device may capture a current real image of the target person in real time through the camera. It should be noted that the target person is a person actually existing in reality, and the virtual image of the target person may be an image generated according to the character features of the target person, so that the features of the target person (hereinafter, referred to as an avatar) displayed in the virtual image and the features of the target person in reality, such as appearance, body type, and expression, may be very similar. The character features may include facial features, body type features, posture features, and the like.

Alternatively, the real image may be an image of a real target person captured by the electronic device through a camera, such as a photograph, a video, and the like of the target person. The virtual image and the real image at least include the face of the target person, and optionally, the virtual image and the real image may also include the body type, the gesture, the action, and the like of the target person.

In some embodiments, the real image of the target person may be acquired by the electronic device through a camera of the electronic device, or may be remotely transmitted to the electronic device after the real image is acquired by another device, for example, through a video phone, an online video, and the like.

S102, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

The first characteristic parameter and the second characteristic parameter may include one or more combinations of characteristic parameters of key points, actions, postures, expressions, sizes, angles and the like of the target character. It will be appreciated that the angular characteristic parameter may be indicative of the angle of display of the target person in the real image or the virtual image, such as the side angle, the front angle, etc. of the target person. The size characteristic parameter may characterize the display size of the target person in the real image or the virtual image. The motion and posture characteristic parameters can represent the positions of all parts of the target character in the image of the current frame in the video.

As one mode, when the first feature parameter is extracted from the virtual image, when the first feature parameter is a key point, a coordinate of the key point in the virtual image may be extracted as the feature parameter. When the first characteristic parameter is large, the ratio of the outline of the target person to the size of the virtual image can be extracted as the characteristic parameter. When the first characteristic parameter is an expression, the expression parameter of the target character in the virtual image can be extracted and identified, the expression parameter is compared with the expression parameter which is marked in advance, for example, the expression parameters of four expressions of happiness, anger, sadness and happiness of the target character are marked in advance, after comparison, if the extracted expression parameter is matched with the expression parameter of happiness, the expression corresponding to the extracted expression parameter is determined to be happiness, and if the label corresponding to the expression parameter of happiness is 1, the characteristic parameter can be 1 when the first characteristic parameter is an expression. Alternatively, the feature parameter extraction of the motion and the gesture may refer to feature parameter extraction of the expression. Similarly, the extraction of the second feature parameter from the real image may refer to the way of extracting the first feature parameter from the virtual image.

For example, when the first characteristic parameter is an angle characteristic parameter of the target person, the second characteristic parameter is also the angle characteristic parameter of the target person. For another example, when the first characteristic parameter is a key point corresponding to an eye portion of the target person, the second characteristic parameter is also a key point corresponding to an eye portion of the target person.

S103, comparing the first characteristic parameter with the second characteristic parameter to obtain a comparison result, and updating the virtual image in the video based on the comparison result.

And S104, when the updated virtual image and the real image meet the preset conditions, switching the updated virtual image in the video into the real image.

In this embodiment, when the electronic device plays a video including a virtual image of a target person, a real image of the target person is obtained, a first characteristic parameter of the target person is extracted from the virtual image, a second characteristic parameter is extracted from the real image, the first characteristic parameter and the second characteristic parameter are compared to obtain a comparison result, and the virtual image in the video is updated based on the comparison result, so that a transition video before image switching can be generated in real time according to the characteristic parameters of the real image and the characteristic parameters of the virtual image, and in the transition video, the target person of the virtual image is continuously close to the target person of the real image. When the updated virtual image and the real image meet the preset conditions, the updated virtual image in the video is switched into the real image, so that the characteristic parameters of the real image and the virtual image can be very similar when the two images are switched, namely the characteristics of a target character in the real image and the characteristics of the target character in the virtual image, such as action, expression, posture and the like, can be approximately consistent, the switching between the virtual image and the real image is smoother and more natural, a user cannot feel the switching process, and the user experience is improved.

Second embodiment

Referring to fig. 2, fig. 2 is a flowchart illustrating an avatar switching method according to another embodiment of the present application. The method may comprise the steps of:

s201, when playing a video including a virtual image of a target person, acquiring a real image of the target person.

S202, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

The specific implementation of S201 and S202 may refer to S101 and S102, and therefore will not be described herein.

S203, judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

In some embodiments, a plurality of feature parameters in the first feature parameters may be used as a first vector, and a plurality of feature parameters in the second feature parameters may be used as a second vector, wherein the number and type of the feature parameters in the first vector are the same as the number and type of the feature parameters in the second vector. And then, the distance between the first characteristic parameter and the second characteristic parameter is obtained according to the first vector and the second vector, the distance can represent the similarity between the first characteristic parameter and the second characteristic parameter, and the similarity is larger when the distance is smaller.

As an example, it is assumed that the first feature parameter comprises n feature parameters, which may be referred to as frame parameters of a current frame of the virtual image, which are represented as a first vector x [1], x [2], x [3] … x [ n ]. Each feature parameter in the first vector may be used to represent a feature value of one dimension, for example, x [1] may represent coordinates of a key point of a target character in the virtual image, x [2] may represent a feature parameter of an expression of the target character in the virtual image, x [3] may represent a feature parameter of an action of the target character in the virtual image, and so on, the feature parameters of n dimensions may be obtained. Similarly, the second feature parameter may be represented as a vector y [1], y [2], y [3] … y [ n ]. For example, x [1] may be represented as the key point coordinates of the target person in the virtual image, and y [1] may be represented as the key point coordinates of the target person in the real image. The first and second feature parameters may then be substituted into a distance function f for calculating the feature parameter distance, the distance between the first and second feature parameters being calculated by f (x 1, x 2, x 3 … x n, y 1, y 2, y 3 … y n), wherein the distance output by the distance function f may be a float type value. And finally, the comparison result can be analyzed according to the distance.

And the type and the number of the characteristic parameters in the third characteristic parameter are the same as those of the first characteristic parameter.

In some embodiments, a preset number of feature parameters may be obtained from a local database of the electronic device, and then the second feature parameters are sequentially compared with the preset number of feature parameters in terms of similarity, so as to select one target feature parameter, where the similarity between the target feature parameter and the second feature parameter is the largest among the preset number of feature parameters. Then, whether the similarity between the target characteristic parameter and the second characteristic parameter is greater than the similarity between the first characteristic parameter and the second characteristic parameter is compared, and if so, the target characteristic parameter can be used as a third characteristic parameter.

And S205, updating the virtual image in the video based on the third characteristic parameter.

In some embodiments, a new virtual image of the target person may be generated according to the third characteristic parameter, and then an image of a next frame in the video may be replaced with the new virtual image, so as to update the virtual image in the video.

And S206, when the updated virtual image and the real image meet the preset conditions, switching the updated virtual image in the video into the real image.

In some embodiments, the avatar switching method in this embodiment may further include S207.

And S207, when the similarity of the first characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, switching the virtual image in the video into a real image.

In this embodiment, the comparison result is obtained by determining whether the similarity between the first characteristic parameter and the second characteristic parameter is smaller than the similarity threshold, so that whether the characteristic of the target person in the virtual image is close to the characteristic of the target person in the real image can be accurately determined, that is, the switching condition is satisfied, and it is ensured that the virtual image and the real image are switched more naturally and smoothly.

Third embodiment

Referring to fig. 3, fig. 3 is a flowchart illustrating an avatar switching method according to another embodiment of the present application. The method may comprise the steps of:

s301, when a video including a virtual image of a target person is played, a real image of the target person is acquired.

S302, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

S303, judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

S304, when the similarity of the first characteristic parameter and the second characteristic parameter is smaller than the similarity threshold, a third characteristic parameter is obtained based on the first characteristic parameter and the second characteristic parameter, wherein the similarity of the third characteristic parameter and the second characteristic parameter is larger than the similarity of the first characteristic parameter and the second characteristic parameter.

The specific implementation of S301 to S304 can refer to S201 to S204, and therefore will not be described herein.

And S305, generating a target virtual image based on the third characteristic parameter.

The third characteristic parameter includes characteristics such as an expression, a key point, a motion, and a posture of the target person, and thus can be used to generate a character representation of the target person, that is, a target virtual image.

S306, the virtual image in the video is updated to be the target virtual image.

In some embodiments, when the electronic device plays a video of a target person, the electronic device replaces a next frame image of the video with a target virtual image to update the virtual image, and determines whether the updated virtual image meets a preset condition, and if not, repeatedly executes S305 and S306 until whether the updated virtual image meets the preset condition.

And S307, when the updated virtual image and the real image meet the preset condition, switching the updated virtual image in the video into the real image.

And S308, when the similarity of the first characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, switching the virtual image in the video into a real image.

In this embodiment, a transition video can be generated between switching of the virtual image to the real image by obtaining the third characteristic parameter based on the first characteristic parameter and the second characteristic parameter, and updating the virtual image in the video to the target virtual image, and the virtual image in the transition video is continuously close to the target person in the real image, so that the problem that the electronic device is difficult to find the time when the picture of the customer service robot is synchronized with the picture of the artificial customer service robot for switching is avoided.

Fourth embodiment

Referring to fig. 4, fig. 4 is a flowchart illustrating an avatar switching method according to another embodiment of the present application. The method may comprise the steps of:

s401, when the video including the virtual image of the target person is played, a real image of the target person is acquired.

S402, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

And S403, judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

The specific implementation of S401 to S403 may refer to S301 to S303, and therefore, will not be described herein.

And S405, updating the virtual image in the video based on the third characteristic parameter.

S406, when the updated virtual image and the real image meet the preset conditions, the updated virtual image in the video is switched into the real image.

S407, when the similarity of the first characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, the virtual image in the video is switched into a real image.

In this embodiment, a third feature parameter that is more similar to the second feature parameter than the first feature parameter can be predicted quickly by the pre-trained prediction model, so as to update the virtual image in the video.

Fifth embodiment

Referring to fig. 5, fig. 5 is a flowchart illustrating an avatar switching method according to another embodiment of the present application. The method may comprise the steps of:

s501, when playing a video including a virtual image of a target person, acquiring a real image of the target person.

S502, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

S503, judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

S504, when the similarity of the first characteristic parameter and the second characteristic parameter is smaller than the similarity threshold, a third characteristic parameter is obtained based on the first characteristic parameter and the second characteristic parameter, wherein the similarity of the third characteristic parameter and the second characteristic parameter is larger than the similarity of the first characteristic parameter and the second characteristic parameter.

And S505, updating the virtual image in the video based on the third characteristic parameter.

The specific implementation of S501-S305 can refer to S301-S305, and therefore is not described herein.

S506, judging whether the similarity between the third characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

The specific implementation of determining whether the similarity between the third characteristic parameter and the second characteristic parameter is smaller than the similarity threshold may refer to the implementation of determining whether the similarity between the first characteristic parameter and the second characteristic parameter is smaller than the similarity threshold in S203.

And S507, when the similarity between the third characteristic parameter and the second characteristic parameter is not less than the similarity threshold, determining that the updated virtual image and the updated real image meet the preset condition.

As an example, assuming that the preset similarity threshold is 80, when the similarity of the third characteristic parameter and the second characteristic parameter is 95, it may be determined that the updated virtual image and the real image satisfy the preset condition.

And S508, when the updated virtual image and the real image meet the preset conditions, switching the updated virtual image in the video into the real image.

S509, when the similarity of the first characteristic parameter and the second characteristic parameter is not smaller than the similarity threshold, the virtual image in the video is switched to a real image.

In this embodiment, whether the updated virtual image and the updated real image satisfy the preset condition is determined by determining whether the similarity between the third characteristic parameter and the second characteristic parameter is smaller than the similarity threshold, so that it can be ensured that the characteristics of the target person in the virtual image and the characteristics of the target person in the real image are very close to each other when the preset condition is satisfied, and further, smooth transition can be ensured when the virtual image and the real image are switched.

Sixth embodiment

Referring to fig. 6, fig. 6 is a flowchart illustrating an avatar switching method according to another embodiment of the present application. The method may comprise the steps of:

s601, when playing a video including a virtual image of a target person, acquiring a real image of the target person.

S602, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

S603, judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

S604, when the similarity between the first characteristic parameter and the second characteristic parameter is smaller than the similarity threshold, obtaining a third characteristic parameter based on the first characteristic parameter and the second characteristic parameter, wherein the similarity between the third characteristic parameter and the second characteristic parameter is larger than the similarity between the first characteristic parameter and the second characteristic parameter.

And S605, updating the virtual image in the video based on the third characteristic parameter.

The second feature parameter may include a first key point, and the third feature parameter may include a second key point, where the first key point and the second key point belong to the same feature point of the target person. For example, the first keypoint and the second keypoint belong to keypoints of an eye portion of the target person.

S606, judging whether the distance between the first key point and the second key point is smaller than a distance threshold value.

S607, when the distance between the first key point and the second key point is smaller than the distance threshold, it is determined that the similarity between the third feature parameter and the second feature parameter is not smaller than the similarity threshold.

As an example, assume that the distance threshold is 3cm and the similarity threshold is 80. When the distance between the first key point and the second key point is smaller than the distance threshold, the similarity between the corresponding third characteristic parameter and the second characteristic parameter is 85; when the distance between the first key point and the second key point is not less than the distance threshold, the similarity between the corresponding third feature parameter and the second feature parameter is 70. Therefore, when the distance between the first keypoint and the second keypoint is 2cm, it may be determined that the similarity between the third feature parameter and the second feature parameter is not less than the similarity threshold corresponding to the similarity between the third feature parameter and the second feature parameter being 85. When the distance between the first keypoint and the second keypoint is 4cm, it may be determined that the similarity between the third feature parameter and the second feature parameter is less than the similarity threshold, corresponding to the similarity between the third feature parameter and the second feature parameter being 70.

Considering that the key points of the person can better reflect the posture of the person than other features of the person, the similarity condition of the second feature parameter and the third feature parameter can be effectively judged according to the distance between the first key point of the second feature parameter and the second key point of the third feature parameter.

S607, when the similarity between the third characteristic parameter and the second characteristic parameter is not less than the similarity threshold, determining that the updated virtual image and the updated real image meet the preset condition.

And S608, when the updated virtual image and the real image meet the preset condition, switching the updated virtual image in the video into the real image.

And S609, when the similarity of the first characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, switching the virtual image in the video into a real image.

In the embodiment, whether the similarity between the third characteristic parameter and the second characteristic parameter is not less than the similarity threshold value is determined according to the key points of the target person in the virtual image and the real image, so that whether the similarity between the third characteristic parameter and the second characteristic parameter meets the requirement can be quickly and effectively judged, and the judgment efficiency is improved.

Seventh embodiment

Referring to fig. 7, fig. 7 is a flowchart illustrating an avatar switching method according to another embodiment of the present application. The method may comprise the steps of:

s701, acquiring a sample image of the target person.

In some embodiments, a sample image of the target person may be captured by the camera, wherein the sample image may include a picture, a video, and the like. When the sample image of the target person is stored in the local or cloud of the electronic device, the sample image can be directly extracted from the local or cloud of the electronic device by the electronic device.

S702, extracting sample characteristic parameters and sample virtual images of the target person from the sample images.

And S703, inputting the sample characteristic parameters and the sample virtual images into a machine learning model for training to obtain a pre-trained virtual image model.

S704, when the video including the virtual image of the target person is played, a real image of the target person is acquired.

S705, extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person.

S706, judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

And S708, inputting the third characteristic parameter into a pre-trained virtual image model to obtain a target virtual image corresponding to the third characteristic parameter.

And S709, updating the virtual image in the video into a target virtual image.

And S710, when the updated virtual image and the real image meet the preset conditions, switching the updated virtual image in the video into the real image.

And S711, when the similarity of the first characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, switching the virtual image in the video into a real image.

In the embodiment, the virtual image is generated by extracting the characteristics of the target person, so that the virtual image and the appearance of the target person can be highly similar, the user can not easily perceive the switching between the virtual image and the real target person, the switching is more natural, and the user experience is improved. And the third characteristic parameter is input into the virtual image model trained in advance to obtain the target virtual image corresponding to the third characteristic parameter, so that the generation efficiency of the target virtual image can be improved, and the transition video can be more smoothly played before the virtual image and the real image are switched.

Eighth embodiment

Referring to fig. 8, fig. 8 is a block diagram illustrating an avatar switching apparatus 800 according to an embodiment of the present application. The device 800 may be applied to an electronic device having a display screen or other image output device 800, and the electronic device may be an electronic device such as a smart phone, a tablet computer, a projector, a wearable smart terminal, and the like.

As will be explained below with respect to the block diagram of fig. 8, the apparatus 800 includes: a real image acquisition module 810, a feature parameter extraction module 820, an update module 830 and a switching module 840.

The real image obtaining module 810 is configured to obtain a real image of the target person when playing a video including a virtual image of the target person.

The feature parameter extraction module 820 is configured to extract a first feature parameter of a target person from the virtual image and a second feature parameter of the real image, where the first feature parameter and the second feature parameter are the same feature parameter of the target person.

The updating module 830 is configured to compare the first characteristic parameter and the second characteristic parameter to obtain a comparison result, and update the virtual image in the video based on the comparison result.

The switching module 840 is configured to switch the updated virtual image in the video to the real image when the updated virtual image and the real image satisfy a preset condition.

Further, the update module 830 includes:

and the first judging unit is used for judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value.

Further, the switching module 840 is further configured to switch the virtual image in the video to the real image when the similarity between the first feature parameter and the second feature parameter is not less than the similarity threshold.

Further, the update module 830 further includes: the second judging unit is used for judging whether the similarity between the third characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value or not; and when the similarity of the third characteristic parameter and the second characteristic parameter is not less than the similarity threshold value, determining that the updated virtual image and the updated real image meet the preset condition.

Further, the second feature parameter includes a first key point, the third feature parameter includes a second key point, and the first key point and the second key point belong to the same feature point of the target person. The second judging unit is specifically configured to judge whether a distance between the first key point and the second key point is smaller than a distance threshold; and when the distance between the first key point and the second key point is smaller than the distance threshold, determining that the similarity between the third characteristic parameter and the second characteristic parameter is not smaller than the similarity threshold.

Further, the updating unit is specifically configured to input the third feature parameter into a pre-trained avatar model to obtain a target virtual image corresponding to the third feature parameter.

Further, the apparatus 800 further comprises: the model training module is used for acquiring a sample image of a target person; extracting sample characteristic parameters and sample virtual images of the target person from the sample images; and inputting the sample characteristic parameters and the sample virtual images into a machine learning model for training to obtain a pre-trained virtual image model.

The avatar switching apparatus provided in the embodiment of the present application is used to implement the corresponding avatar switching method in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again.

As will be clearly understood by those skilled in the art, the avatar switching apparatus provided in the embodiment of the present application can implement each process in the foregoing method embodiment, and for convenience and brevity of description, the specific working processes of the above-described apparatus and modules may refer to the corresponding processes in the foregoing method embodiment, and are not described herein again.

In the embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or modules may be in an electrical, mechanical or other form.

In addition, each functional module in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.

Ninth embodiment

Referring to fig. 9, a block diagram of anelectronic device 900 according to an embodiment of the present disclosure is shown. Theelectronic device 900 may be a smart phone, a tablet computer, or other electronic device capable of running an application. Theelectronic device 900 in the present application may include one or more of the following components: a processor 910, amemory 920, and one or more applications, wherein the one or more applications may be stored in thememory 920 and configured to be executed by the one or more processors 910, the one or more programs configured to perform a method as described in the aforementioned method embodiments.

Processor 910 may include one or more processing cores. The processor 910 interfaces with various components throughout theelectronic device 900 using various interfaces and circuitry to perform various functions of theelectronic device 900 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in thememory 920 and invoking data stored in thememory 920. Alternatively, the processor 910 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 910 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 910, but may be implemented by a communication chip.

TheMemory 920 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Thememory 920 may be used to store instructions, programs, code sets, or instruction sets. Thememory 920 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The data storage area may also store data created during use by the electronic device 900 (e.g., phone books, audio-visual data, chat log data), and so forth.

Tenth embodiment

Referring to fig. 10, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable storage medium 1000 has stored therein program code that can be invoked by a processor to perform the methods described in the above-described method embodiments.

The computer-readable storage medium 1000 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 1000 includes a non-volatile computer-readable storage medium. The computer readable storage medium 1000 has storage space forprogram code 1010 for performing any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. Theprogram code 1010 may be compressed, for example, in a suitable form.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. An avatar switching method, the method comprising:

when a video including a virtual image of a target person is played, acquiring a real image of the target person;

extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person;

comparing the first characteristic parameter with the second characteristic parameter to obtain a comparison result, and updating the virtual image in the video based on the comparison result;

and when the updated virtual image and the real image meet preset conditions, switching the updated virtual image in the video into the real image.

2. The method according to claim 1, wherein comparing the first characteristic parameter and the second characteristic parameter to obtain a comparison result, and updating the virtual image in the video based on the comparison result comprises:

judging whether the similarity of the first characteristic parameter and the second characteristic parameter is smaller than a similarity threshold value;

updating the virtual image in the video based on the third characteristic parameter.

3. The method of claim 2, wherein the updating the virtual image in the video based on the third feature parameter comprises:

generating a target virtual image based on the third characteristic parameter;

and updating the virtual image in the video into the target virtual image.

4. The method of claim 2, wherein the deriving a third feature parameter based on the first feature parameter and the second feature parameter comprises:

and inputting the first characteristic parameter and the second characteristic parameter into a pre-trained prediction model, and acquiring the third characteristic parameter output by the pre-trained prediction model.

5. The method according to any one of claims 2-4, further comprising:

6. The method according to any one of claims 2 to 5, wherein before the switching the updated virtual image in the video to the real image when the updated virtual image and the real image satisfy a preset condition, the method further comprises:

judging whether the similarity between the third characteristic parameter and the second characteristic parameter is smaller than the similarity threshold value;

7. The method of claim 6, wherein the second feature parameter comprises a first key point, the third feature parameter comprises a second key point, the first key point and the second key point belong to the same feature point of the target person, and the determining whether the similarity between the third feature parameter and the second feature parameter is smaller than the similarity threshold value comprises:

judging whether the distance between the first key point and the second key point is smaller than a distance threshold value;

when the distance between the first key point and the second key point is smaller than the distance threshold, determining that the similarity of the third feature parameter and the second feature parameter is not smaller than the similarity threshold.

8. The method according to any one of claims 3-7, wherein the generating a target virtual image based on the third feature parameter comprises:

and inputting the third characteristic parameter into a pre-trained virtual image model to obtain the target virtual image corresponding to the third characteristic parameter.

9. The method according to claim 8, wherein before the acquiring the real image of the target person when the video including the virtual image of the target person is played, further comprising:

obtaining a sample image of a target person;

extracting sample characteristic parameters and sample virtual images of the target person from the sample images;

and inputting the sample characteristic parameters and the sample virtual images into a machine learning model for training to obtain a pre-trained virtual image model.

10. A virtual character switching device, characterized in that the device comprises:

a real image acquisition module for acquiring a real image of a target person when a video including a virtual image of the target person is played;

the characteristic parameter extraction module is used for extracting a first characteristic parameter of the target person from the virtual image and extracting a second characteristic parameter from the real image, wherein the first characteristic parameter and the second characteristic parameter are the same characteristic parameter of the target person;

the updating module is used for comparing the first characteristic parameter with the second characteristic parameter to obtain a comparison result and updating the virtual image in the video based on the comparison result;

and the switching module is used for switching the updated virtual image in the video into the real image when the updated virtual image and the real image meet preset conditions.

11. The virtual character switching apparatus of claim 10, wherein the updating module comprises:

a third feature parameter generating unit, configured to obtain a third feature parameter based on the first feature parameter and the second feature parameter when a similarity between the first feature parameter and the second feature parameter is smaller than a similarity threshold, where a similarity between the third feature parameter and the second feature parameter is greater than a similarity between the first feature parameter and the second feature parameter;

12. The virtual character switching apparatus according to claim 11, wherein the updating unit is specifically configured to generate a target virtual image based on the third feature parameter; and updating the virtual image in the video into the target virtual image.

13. The virtual character switching apparatus according to claim 11, wherein the third feature parameter generating unit is specifically configured to input the first feature parameter and the second feature parameter into a pre-trained prediction model, and acquire the third feature parameter output by the pre-trained prediction model.

14. An electronic device, comprising:

a memory;

one or more processors coupled with the memory;

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-9.

15. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 9.