CN116266408A - Body type estimating method, body type estimating device, storage medium and electronic equipment - Google Patents

Abstract

A body type estimating method, device, storage medium and electronic apparatus. The method comprises the steps of obtaining an object image of an object human body, and carrying out posture estimation on the object image through a posture estimation model to obtain human body posture estimation information and body type estimation information of the object human body and equipment posture estimation information of image acquisition equipment of the object image; performing skeleton point detection on the object image to obtain skeleton point information of the object human body; detecting the human body contour of the object image to obtain human body contour information of the object human body; and optimizing the body type estimation information according to the human body posture estimation information and the equipment posture estimation information by taking the skeleton point information and the human body contour information as constraints to obtain target body type information of the human body of the object. The method and the device can improve the efficiency of obtaining the human body type information.

Description

Translated fromChinese

体型估计方法、装置、存储介质及电子设备Body size estimation method, device, storage medium and electronic equipment

技术领域technical field

本申请涉及图像处理技术领域，具体涉及一种体型估计方法、装置、存储介质及电子设备。The present application relates to the technical field of image processing, and in particular to a body shape estimation method, device, storage medium and electronic equipment.

背景技术Background technique

体型信息作为人体的一个重要指标，在生活中的方方面面都会使用到。比如，用户需要参考自身人体的体型信息来购买合适的衣物，根据自身人体的体型信息来监测自身的健康状况等。相关技术中，通常采用人工的方式测量对象人体的体型信息，效率较低。As an important indicator of the human body, body shape information is used in all aspects of life. For example, users need to refer to their own body shape information to purchase suitable clothes, and to monitor their own health status according to their own body shape information. In related technologies, the body shape information of the subject's human body is usually measured manually, and the efficiency is low.

发明内容Contents of the invention

本申请提供了一种体型估计方法、装置、存储介质及电子设备，能够提高获得人体体型信息的效率。The present application provides a body shape estimation method, device, storage medium and electronic equipment, which can improve the efficiency of obtaining human body shape information.

本申请提供的体型估计方法，包括：The body size estimation methods provided by this application include:

获取对象人体的对象图像，并通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息；acquiring a subject image of the subject human body, and performing pose estimation on the subject image through a pose estimation model, obtaining human body pose estimation information, body shape estimation information of the subject human body, and device pose estimation information of an image acquisition device of the subject image;

对对象图像进行骨骼点检测，得到对象人体的骨骼点信息；Carry out bone point detection on the object image to obtain the bone point information of the object human body;

对对象图像进行人体轮廓检测，得到对象人体的人体轮廓信息；Perform human body contour detection on the object image to obtain human body contour information of the object human body;

以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。With the skeleton point information and human body contour information as constraints, the body shape estimation information is optimized according to the human body pose estimation information and device pose estimation information, and the target body shape information of the target human body is obtained.

本申请提供的体型估计装置，包括：The body size estimating device provided by this application includes:

估计模块，用于获取对象人体的对象图像，并通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息；The estimation module is used to obtain the object image of the target human body, and perform pose estimation on the object image through the pose estimation model, so as to obtain human body pose estimation information, body shape estimation information of the object human body, and device pose estimation information of the image acquisition device of the object image;

第一检测模块，用于对对象图像进行骨骼点检测，得到对象人体的骨骼点信息；The first detection module is used for performing skeleton point detection on the object image to obtain skeleton point information of the object human body;

第二检测模块，用于对对象图像进行人体轮廓检测，得到对象人体的人体轮廓信息；The second detection module is used to perform human body contour detection on the object image to obtain human body contour information of the object human body;

优化模块，用于以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。The optimization module is used to optimize the body shape estimation information according to the human body pose estimation information and the equipment pose estimation information with the constraints of the skeleton point information and the human body contour information, so as to obtain the target body shape information of the target human body.

本申请提供的存储介质，其上存储有计算机程序，当计算机程序被处理器加载时执行如本申请提供的体型估计方法中的步骤。The storage medium provided by the present application has a computer program stored thereon, and when the computer program is loaded by the processor, the steps in the body size estimation method provided by the present application are executed.

本申请提供的电子设备，包括处理器和存储器，存储器存有计算机程序，处理器通过加载计算机程序，用于执行本申请提供的体型估计方法中的步骤。The electronic device provided in the present application includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the steps in the body size estimation method provided in the present application by loading the computer program.

本申请中，利用姿态估计模型对对象人体的对象图像进行姿态估计，得到对象人体在三维空间内的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备在三维空间内的设备姿态估计信息，此外，还从对象图像中提取出对象人体在二维空间内的骨骼点信息和人体轮廓信息，并以该骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。相较于相关技术，本申请通过采用基于人工智能的姿态估计方式来代替传统的人工测量，能够减少人力操作，提高获得体型信息的效率，此外，以对象人体的骨骼点信息和人体轮廓信息为约束对体型估计信息进行优化处理，从而得到与对象人体的骨骼点信息和人体轮廓信息均匹配的目标体型信息，能够更准确的反映对象人体的体型。In this application, the pose estimation model is used to estimate the pose of the object image of the subject human body to obtain the human pose estimation information and body shape estimation information of the subject human body in the three-dimensional space, and the equipment pose estimation of the image acquisition device of the object image in the three-dimensional space In addition, the skeleton point information and human body contour information of the target human body in two-dimensional space are extracted from the object image, and the skeleton point information and human body contour information are used as constraints, and the human body pose estimation information and device pose estimation information The body shape estimation information is optimized to obtain the target body shape information of the target human body. Compared with related technologies, the present application replaces traditional manual measurement with an artificial intelligence-based attitude estimation method, which can reduce manpower operations and improve the efficiency of obtaining body shape information. The constraints optimize the body shape estimation information, so as to obtain target body shape information that matches both the skeleton point information and the human body contour information of the target human body, which can more accurately reflect the body shape of the target human body.

附图说明Description of drawings

为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

图1是本申请实施例提供的体型估计系统的场景示意图。FIG. 1 is a schematic diagram of a scene of a body size estimation system provided by an embodiment of the present application.

图2是本申请实施例提供的体型估计方法的流程示意图。Fig. 2 is a schematic flowchart of a body size estimation method provided by an embodiment of the present application.

图3是本申请实施例中掩膜图像形式的人体轮廓信息。FIG. 3 is human body contour information in the form of a mask image in the embodiment of the present application.

图4是本申请实施例从对象图像中截取人体子图像的示例图。Fig. 4 is an example diagram of intercepting a sub-image of a human body from an object image according to an embodiment of the present application.

图5是本申请实施例中提供的衣物选择接口的示例图。Fig. 5 is an example diagram of the clothes selection interface provided in the embodiment of the present application.

图6是本申请实施例中获得每一衣物三角面片的形变矢量的一示意图。FIG. 6 is a schematic diagram of obtaining the deformation vector of each triangular patch of clothing in the embodiment of the present application.

图7是本申请实施例中获得每一衣物三角面片的形变矢量的另一示意图。Fig. 7 is another schematic diagram of obtaining the deformation vector of each triangular patch of clothing in the embodiment of the present application.

图8是本申请实施例提供的体型估计装置的结构框图。Fig. 8 is a structural block diagram of a body shape estimation device provided by an embodiment of the present application.

图9是本申请实施例提供的电子设备的结构框图。FIG. 9 is a structural block diagram of an electronic device provided by an embodiment of the present application.

具体实施方式Detailed ways

需要说明的是，本申请的原理是以实施在一适当的运算环境中来举例说明。以下的说明是基于所例示的本申请具体实施例，其不应被视为限制本申请未在此详述的其他具体实施例。基于本申请中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。It should be noted that the principle of the present application is illustrated by being implemented in an appropriate computing environment. The following description is based on illustrated specific embodiments of the present application, which should not be construed as limiting other specific embodiments of the present application that are not described in detail here. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of this application.

本申请以下实施例中所涉及的诸如第一和第二等关系术语仅用于将一个对象或者操作与另一个对象或者操作区分开来，并不用于限定这些对象或操作之间存在着实际的顺序关系。在本申请实施例的描述中，“多个”的含义是两个或两个以上，除非另有明确具体的限定。The relational terms such as first and second involved in the following embodiments of the present application are only used to distinguish one object or operation from another object or operation, and are not used to limit the actual relationship between these objects or operations. sequence relationship. In the description of the embodiments of the present application, "plurality" means two or more, unless otherwise specifically defined.

人工智能(Artificial Intelligence,AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能、感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个综合技术，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the nature of intelligence and produce a new kind of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

人工智能技术是一门综合学科，涉及领域广泛，既有硬件层面的技术也有软件层面的技术。人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互系统、机电一体化等技术。人工智能软件技术主要包括机器学习(Machine Learning,ML)技术，其中，深度学习(Deep Learning，DL)是机器学习中一个新的研究方向，它被引入机器学习以使其更接近于最初的目标，即人工智能。目前，深度学习主要应用在计算机视觉、自然语言处理等领域。Artificial intelligence technology is a comprehensive subject that involves a wide range of fields, including both hardware-level technology and software-level technology. Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes machine learning (Machine Learning, ML) technology, among which, deep learning (Deep Learning, DL) is a new research direction in machine learning, which is introduced into machine learning to make it closer to the original goal , that is, artificial intelligence. At present, deep learning is mainly used in computer vision, natural language processing and other fields.

深度学习是学习样本数据的内在规律和表示层次，这些学习过程中获得的信息对诸如文字、图像和声音数据的解释有很大的帮助。利用深度学习技术，以及对应的训练数据集，能够训练得到实现不同功能的网络模型，比如，基于一训练数据集能够训练得到用于性别分类的深度学习网络，基于另一训练数据集能够训练得到图像优化的深度学习网络等。Deep learning is to learn the internal laws and representation levels of sample data. The information obtained during these learning processes is of great help to the interpretation of text, image and sound data. Using deep learning technology and corresponding training data sets, network models that implement different functions can be trained. For example, a deep learning network for gender classification can be trained based on a training data set, and a deep learning network for gender classification can be trained based on another training data set. Deep Learning Networks for Image Optimization, etc.

为了能够提高获得体型信息的效率，本申请将深度学习引入到发音检测中，相应提供一种体型估计方法、体型估计装置、存储介质以及电子设备。其中，体型估计方法可由电子设备执行。In order to improve the efficiency of obtaining body shape information, the present application introduces deep learning into pronunciation detection, and accordingly provides a body shape estimation method, a body shape estimation device, a storage medium, and an electronic device. Wherein, the body shape estimation method can be executed by electronic equipment.

请参照图1，本申请还提供一种体型估计系统，如图1所示，该体型估计系统包括电子设备100，比如，当电子设备100配置有摄像头等图像采集设备时，可以通过摄像头拍摄对象人体(若配置有多个摄像头，则可以通过其中一个摄像头进行拍摄)，获取到对象人体的对象图像，并将该对象图像输入到已训练的姿态估计模型，通过姿态估计模型对对象图像进行姿态估计，得到包括对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息在内的估计结果；此外，还分别对对象图像进行骨骼点检测和人体轮廓检测，得到对象人体的骨骼点信息和人体轮廓信息；之后，以该骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。Please refer to FIG. 1, the present application also provides a body shape estimation system, as shown in FIG. 1, the body shape estimation system includes anelectronic device 100, for example, when theelectronic device 100 is equipped with an image acquisition device such as a camera, the object can be photographed through the camera Human body (if configured with multiple cameras, one of the cameras can be used to shoot), obtain the object image of the object human body, and input the object image into the trained pose estimation model, and pose the object image through the pose estimation model Estimating, obtaining estimation results including human body posture estimation information, body shape estimation information of the subject human body, and equipment posture estimation information of the image acquisition device of the subject image; in addition, bone point detection and human body contour detection are also performed on the subject image respectively, Obtain the skeletal point information and human body contour information of the target human body; then, with the skeletal point information and human body contour information as constraints, optimize the body shape estimation information according to the human body pose estimation information and equipment pose estimation information, and obtain the target body shape of the target human body information.

其中，电子设备100可以是任何配置有处理器而具备处理能力的设备，比如智能手机、平板电脑、掌上电脑、笔记本电脑等具备处理器的移动式电子设备，或者台式电脑、电视、服务器等具备处理器的固定式电子设备。Wherein, theelectronic device 100 may be any device equipped with a processor and capable of processing, such as a mobile electronic device equipped with a processor such as a smart phone, a tablet computer, a handheld computer, or a notebook computer, or a desktop computer, a TV, a server, etc. Processor stationary electronics.

另外，如图1所示，该体型估计系统还可以包括存储设备200，用于存储数据，包括但不限于体型估计过程中得到的原始数据、中间数据以及结果数据等，比如，电子设备100可以将获取到的对象图像，由对象图像估计得到的人体姿态估计信息、体型估计信息、设备姿态估计信息，由对象图像提取得到的骨骼点信息和人体轮廓信息，以及最终优化得到的体型估计信息存入存储设备200中。In addition, as shown in FIG. 1 , the body shape estimation system may also include astorage device 200 for storing data, including but not limited to raw data, intermediate data, and result data obtained during the body shape estimation process. For example, theelectronic device 100 may Store the acquired object image, human body pose estimation information, body shape estimation information, equipment pose estimation information obtained from the object image estimation, bone point information and human body contour information extracted from the object image, and finally optimized body shape estimation information. into thestorage device 200.

需要说明的是，图1所示的体型估计系统的场景示意图仅仅是一个示例，本申请实施例描述的体型估计系统以及场景是为了更加清楚的说明本申请实施例的技术方案，并不构成对于本申请实施例提供的技术方案的限定，本领域普通技术人员可知，随着体型估计系统的演变和新业务场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。It should be noted that the scene diagram of the body size estimation system shown in FIG. 1 is only an example. The body size estimation system and the scene described in the embodiment of the application are for more clearly illustrating the technical solution of the embodiment of the application, and do not constitute a The limitations of the technical solutions provided in the embodiments of the present application, those skilled in the art know that, with the evolution of the body size estimation system and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

请参照图2，图2为本申请实施例提供的体型估计方法的流程示意图。如图2所示，本申请实施例提供的体型估计方法的流程可以如下：Please refer to FIG. 2 . FIG. 2 is a schematic flow chart of a body size estimation method provided in an embodiment of the present application. As shown in Figure 2, the flow of the body size estimation method provided by the embodiment of the present application may be as follows:

在S310中，获取对象人体的对象图像，并通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息。In S310, the object image of the target human body is obtained, and the pose estimation is performed on the object image through the pose estimation model to obtain human body pose estimation information, body shape estimation information of the object human body, and equipment pose estimation information of the image acquisition device of the object image.

应当说明的是，对象人体可以是任意需要进行体型估计的人体，比如，若需要对A用户进行体型估计，则A用户的人体即为对象人体。体型估计可以的理解为以非实际测量的方式估计得到对象人体的体型信息，包括但不限于身高、胸围、臀围、腰围、臂宽以及腿长等信息。It should be noted that the target human body may be any human body whose body size needs to be estimated. For example, if the body size of user A needs to be estimated, then the human body of user A is the target human body. Body shape estimation can be understood as estimating and obtaining the body shape information of the subject's human body in a non-actual measurement manner, including but not limited to information such as height, chest circumference, hip circumference, waist circumference, arm width, and leg length.

相应的，本实施例中，电子设备首先获取到需要进行体型估计的对象人体的对象图像。此处对于电子设备如何获取对象人体的对象图像不作具体限制，可由本领域技术人员根据实际需要进行配置。Correspondingly, in this embodiment, the electronic device first acquires an object image of an object human body for which body size estimation needs to be performed. Here, there is no specific limitation on how the electronic device acquires the object image of the object human body, and it can be configured by those skilled in the art according to actual needs.

比如，当电子设备配置有摄像头等图像采集设备时，可以通过配置的摄像头拍摄对象人体，从而获取得到对象人体的对象图像；For example, when the electronic device is equipped with an image acquisition device such as a camera, the human body of the subject can be captured by the configured camera, so as to obtain an object image of the human body of the subject;

又比如，电子设备还可以从其它电子设备处获取到该其它电子设备所拍摄得到的对象人体的对象图像；For another example, the electronic device may also acquire the subject image of the subject's human body captured by the other electronic device from other electronic devices;

又比如，电子设备还可以从网络中下载对象人体的对象图像。For another example, the electronic device may also download the subject image of the subject's human body from the network.

应当说明的是，本申请预先训练有姿态估计模型，该姿态估计模型被配置为以包括人体的对象图像为输入，依据对象图像对人体进行姿态估计，以人体姿态信息、体型信息以及对象图像的图像采集设备的设备姿态信息为输出。此处对于姿态估计模型的结构以及训练方式均不作具体限制，可由本领域技术人员根据实际需要进行选取。It should be noted that the present application pre-trains a pose estimation model, and the pose estimation model is configured to take an object image including a human body as input, perform pose estimation on the human body according to the object image, and use human body pose information, body shape information, and object image The device pose information of the image acquisition device is output. Here, there are no specific limitations on the structure and training methods of the pose estimation model, which can be selected by those skilled in the art according to actual needs.

比如，姿态估计模型可以由三大部分组成，分别为特征编码网络、特征解码网络以及估计网络，其中估计网络包括第一估计子网络、第二估计子网络以及第三估计子网络。For example, the pose estimation model can be composed of three parts, which are feature encoding network, feature decoding network and estimation network, wherein the estimation network includes a first estimation subnetwork, a second estimation subnetwork and a third estimation subnetwork.

其中，特征编码网络被配置为对输入的对象图像进行特征编码，得到对象图像的编码特征；Wherein, the feature encoding network is configured to perform feature encoding on the input object image to obtain the encoding feature of the object image;

特征解码网络被配置为对特征编码网络输出的编码特征进行特征解码，得到解码特征；The feature decoding network is configured to perform feature decoding on the encoded features output by the feature encoding network to obtain decoded features;

第一估计子网络被配置为根据解码特征估计得到人体姿态信息；The first estimation sub-network is configured to obtain human body posture information according to decoding feature estimation;

第二估计子网络被配置为根据解码特征估计得到体型信息；The second estimating sub-network is configured to obtain body shape information according to decoding feature estimation;

第三估计子网络被配置为根据解码特征估计得到设备姿态信息。The third estimating sub-network is configured to estimate device pose information according to the decoded features.

相应的，基于以上姿态估计模型，电子设备在获取到对象人体的对象图像之后，将获取到的对象图像输入姿态估计模型，通过姿态估计模型对输入的对象图像进行姿态估计，将姿态估计模型输出的人体姿态信息记为人体姿态估计信息，将姿态估计模型输出的体型信息记为体型估计信息，将姿态估计模型输出的设备姿态信息记为设备姿态估计信息。Correspondingly, based on the above pose estimation model, after the electronic device obtains the object image of the subject human body, it inputs the obtained object image into the pose estimation model, performs pose estimation on the input object image through the pose estimation model, and outputs the pose estimation model The human body pose information is recorded as human body pose estimation information, the body shape information output by the pose estimation model is recorded as body shape estimation information, and the device pose information output by the pose estimation model is recorded as device pose estimation information.

其中，人体姿态估计信息至少描述了对象人体的骨骼点在三维空间内的平移信息和旋转信息，设备姿态估计信息至少描述了对象图像的图像采集设备在三维空间的平移信息和旋转信息。Wherein, the human body pose estimation information at least describes the translation information and rotation information of the skeletal points of the target human body in the three-dimensional space, and the device pose estimation information at least describes the translation information and rotation information of the image acquisition device of the object image in the three-dimensional space.

在S320中，对对象图像进行骨骼点检测，得到对象人体的骨骼点信息。In S320, bone point detection is performed on the target image to obtain bone point information of the target human body.

本实施例中，除了对对象图像进行三维空间内的姿态估计之外，电子设备还按照配置的骨骼点检测算法，在二维空间内对对象图像进行骨骼点检测，得到对象人体的骨骼点信息。其中，骨骼点信息用于描述对象人体的骨骼点(包括但不限于头、颈、肩、手、臀、膝以及脚等骨骼点)在对象图像的二维空间内位置。此处对于采用何种骨骼点检测算法不作具体限制，可由本领域技术人员根据实际需要进行配置。In this embodiment, in addition to performing pose estimation on the object image in three-dimensional space, the electronic device also performs skeleton point detection on the object image in two-dimensional space according to the configured skeleton point detection algorithm to obtain skeleton point information of the object human body . Wherein, the bone point information is used to describe the position of the bone points of the subject's human body (including but not limited to head, neck, shoulder, hand, hip, knee, and foot) in the two-dimensional space of the target image. There is no specific limitation on which bone point detection algorithm is used here, and it can be configured by those skilled in the art according to actual needs.

在S330中，对对象图像进行人体轮廓检测，得到对象人体的人体轮廓信息。In S330, human body contour detection is performed on the target image to obtain human body contour information of the target human body.

本实施例中，除了对对象图像进行三维空间内的姿态估计之外，电子设备还按照配置的人体轮廓检测算法，在二维空间内对对象图像进行人体轮廓检测，得到对象人体的人体轮廓信息。其中，人体轮廓用于描述对象人体在对象图像的轮廓。此处对于采用何种人体轮廓检测算法不作具体限制，可由本领域技术人员根据实际需要进行配置。In this embodiment, in addition to performing pose estimation on the object image in three-dimensional space, the electronic device also performs human body contour detection on the object image in two-dimensional space according to the configured human body contour detection algorithm to obtain human body contour information of the target human body . Wherein, the human body contour is used to describe the contour of the subject's human body in the object image. Here, there is no specific limitation on which human body contour detection algorithm is used, and it can be configured by those skilled in the art according to actual needs.

示例性的，电子设备按照配置的人体轮廓检测算法对对象图像进行人体轮廓检测，输出掩膜图像形式的人体轮廓信息，其中，可以采用像素值为0的像素点表征人体、像素值为255的像素值表征非人体，也可以采用像素值为255的像素值表征人体，像素值为0的像素点表征非人体。比如，请参照图3，示出了一掩膜图像，该掩膜图像黑白分明，仅由0和255两个像素值构成，其中，像素值为0的像素点构成黑色区域表征人体，像素值为255的像素点构成白色区域表征非人体区域，从而整体描述出人体轮廓。Exemplarily, the electronic device performs human body contour detection on the object image according to the configured human body contour detection algorithm, and outputs human body contour information in the form of a mask image, wherein the human body can be represented by a pixel with a pixel value of 0, and a human body with a pixel value of 255. A pixel value represents a non-human body, and a pixel value with a pixel value of 255 may be used to represent a human body, and a pixel with a pixel value of 0 may represent a non-human body. For example, please refer to Figure 3, which shows a mask image, which is black and white, and consists of only two pixel values of 0 and 255, wherein the pixel points with a pixel value of 0 form a black area to represent the human body, and the pixel value The 255 pixels form a white area to represent the non-human body area, thereby describing the outline of the human body as a whole.

应当说明的是，以上对对象图像的姿态估计、骨骼点检测和人体轮廓检测不受序号大小的限制，可以按照序号大小依序执行，也可以按照其它顺序依序执行，还可以并行执行等。It should be noted that the pose estimation, skeletal point detection, and human body contour detection of the object image above are not limited by the size of the sequence number, and can be executed in sequence according to the sequence number, or in other sequences, or in parallel.

比如，电子设备可以运行三个线程，通过三个线程并行的对对象图像进行姿态估计、进行骨骼点检测以及进行人体轮廓检测。For example, the electronic device can run three threads, and perform pose estimation, bone point detection, and human body contour detection on the object image in parallel through the three threads.

在S340中，以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。In S340 , with the skeleton point information and human body contour information as constraints, the body shape estimation information is optimized according to the human body pose estimation information and the device pose estimation information, to obtain target body shape information of the target human body.

如上，在利用姿态估计模型估计得到对象人体在三维空间的人体姿态估计信息、体型估计信息，对象图像的图像采集设备在三维空间的设备姿态估计信息，以及从对象图像中检测得到对象人体在二维空间的骨骼点信息和人体轮廓信息之后，以骨骼点信息和人体轮廓信息为约束，按照配置的非线性优化策略，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行迭代优化，得到与骨骼点信息和人体轮廓信息均匹配的优化后体型信息，记为目标体型信息。此处对于采用何种非线性优化策略不作具体限制，可由本领域技术人员根据实际需要进行配置。As above, after using the pose estimation model to estimate the human body pose estimation information and body shape estimation information of the target human body in the three-dimensional space, the device pose estimation information of the image acquisition device of the object image in the three-dimensional space, and obtain the target human body in two dimensions from the object image. After skeletal point information and human body contour information in three-dimensional space, with the skeleton point information and human body contour information as constraints, according to the configured nonlinear optimization strategy, iteratively optimizes the body shape estimation information according to the human body pose estimation information and equipment pose estimation information, and obtains The optimized body shape information that matches both the bone point information and the human body contour information is recorded as the target body shape information. There is no specific limitation on which nonlinear optimization strategy is used here, and it can be configured by those skilled in the art according to actual needs.

由上可知，本申请利用姿态估计模型对对象人体的对象图像进行姿态估计，得到对象人体在三维空间内的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备在三维空间内的设备姿态估计信息，此外，还从对象图像中提取出对象人体在二维空间内的骨骼点信息和人体轮廓信息，并以该骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。相较于相关技术，本申请通过采用基于人工智能的姿态估计方式来代替传统的人工测量，能够减少人力操作，提高获得体型信息的效率，此外，以对象人体的骨骼点信息和人体轮廓信息为约束对体型估计信息进行优化处理，从而得到与对象人体的骨骼点信息和人体轮廓信息均匹配的目标体型信息，能够更准确的反映对象人体的体型。It can be seen from the above that this application uses the pose estimation model to perform pose estimation on the object image of the subject human body, and obtain the human body pose estimation information and body shape estimation information of the object human body in the three-dimensional space, as well as the image acquisition equipment of the object image in the three-dimensional space. Pose estimation information, in addition, the skeleton point information and human body contour information of the target human body in two-dimensional space are extracted from the object image, and with the skeleton point information and human body contour information as constraints, according to the human body pose estimation information and device pose The estimation information optimizes the body shape estimation information to obtain target body shape information of the subject human body. Compared with related technologies, the present application replaces traditional manual measurement with an artificial intelligence-based attitude estimation method, which can reduce manpower operations and improve the efficiency of obtaining body shape information. The constraints optimize the body shape estimation information, so as to obtain target body shape information that matches both the skeleton point information and the human body contour information of the target human body, which can more accurately reflect the body shape of the target human body.

在一可选的实施例中，为进一步提高体型估计的准确性，获取的对象图像包括对象人体的多个预设角度的对象图像，以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息，包括：In an optional embodiment, in order to further improve the accuracy of body shape estimation, the acquired object images include object images of multiple preset angles of the object human body, constrained by the skeleton point information and human body contour information, according to the human body pose estimation The information and equipment attitude estimation information optimize the body shape estimation information to obtain the target body shape information of the target human body, including:

融合多个预设角度的对象图像对应的体型估计信息，得到对象人体的初始体型信息；fusing the body shape estimation information corresponding to the object images at multiple preset angles to obtain the initial body shape information of the object;

以多个预设角度的对象图像对应的骨骼点信息和人体轮廓信息为约束，根据多个预设角度的对象图像对应的人体姿态估计信息和设备姿态估计信息，对初始体型信息进行优化处理，得到对象人体的目标体型信息。The initial body shape information is optimized according to the human body pose estimation information and device pose estimation information corresponding to the object images at multiple preset angles, constrained by the skeleton point information and human body contour information corresponding to the object images at multiple preset angles, The target body shape information of the subject human body is obtained.

本实施例中，对于预设角度的数量及取值不作具体限制，可由本领域技术人员根据实际需要进行配置。In this embodiment, there is no specific limitation on the number and value of the preset angles, which can be configured by those skilled in the art according to actual needs.

比如，在电子设备配置有摄像头等图像采集设备时，可以通过摄像头按照多个不同的预设角度拍摄对象人体，由此获得对象人体的多个预设角度的对象图像。For example, when the electronic device is equipped with an image acquisition device such as a camera, the camera can take pictures of the subject's human body at multiple different preset angles, thereby obtaining object images of the subject's human body at multiple preset angles.

其中，可以通过多种方式来拍摄得到对象人体的多个预设角度的对象图像，对象人体可以站定不动，通过移动电子设备的方式来获得不同的拍摄角度；也可以固定电子设备，由对象人体原地旋转的方式来获得不同的拍摄角度。以下以多个预设角度为正面角度、左面角度、右面角度以及背面角度为例进行说明。Among them, multiple methods can be used to capture the subject images of multiple preset angles of the subject's human body. The subject's human body can stand still and obtain different shooting angles by moving the electronic device; the electronic device can also be fixed, by The way the subject's body is rotated in situ to obtain different shooting angles. In the following, the multiple preset angles are the front angle, the left angle, the right angle and the back angle as an example for illustration.

比如，在对对象人体进行拍摄时，被拍摄的对象人体站定不动，由他人手持电子设备在对象人体正面、背面、左面以及右面分别对对象人体进行拍摄，得到以上四个不同预设角度的对象图像。For example, when photographing the subject's human body, the photographed subject's human body stands still, and another person holds an electronic device to photograph the subject's human body on the front, back, left and right sides of the subject's human body respectively, and obtains the above four different preset angles object image.

又比如，在对对象人体进行拍摄时，可由对象人体将电子设备固定，并设置定时拍摄时间，以及拍摄张数为4；在完成前述设置之后，对象人体可以先正面朝向电子设备，待电子设备完成第一拍摄时(此时电子设备将拍摄到对象人体正面的对象图像)，对象人体原地顺时针旋转90度(即身体左面朝向电子设备)，待电子设备完成第二次拍摄时(此时电子设备将拍摄到对象人体左面的对象图像)，对象人体再原地顺时针旋转90度(即身体背面朝向电子设备)，待电子设备完成第三拍摄时(此时电子设备将拍摄到对象人体背面的对象图像)，对象人体再原地瞬时针旋转90度(即身体右面朝向电子设备)，待电子设备完成第四次拍摄，由此，电子设备将同样拍摄到对象人体正面、背面、左面以及右面共四个不同预设角度的对象图像。For another example, when shooting the subject's human body, the electronic device can be fixed by the subject's human body, and the timing shooting time can be set, and the number of shots can be 4; When the first shooting is completed (at this time, the electronic device will capture the subject image of the front of the subject's human body), the subject's human body is rotated 90 degrees clockwise on the spot (that is, the left side of the body faces the electronic device), and when the electronic device completes the second shooting (this time At this time, the electronic device will capture the subject image on the left side of the subject's body), and the subject's body will rotate 90 degrees clockwise (that is, the back of the body faces the electronic device), and when the electronic device completes the third shooting (at this time, the electronic device will capture the The object image on the back of the human body), the subject’s human body is rotated 90 degrees instantaneously (that is, the right side of the body faces the electronic device), and the electronic device completes the fourth shooting, so that the electronic device will also capture the front, back, and back of the subject’s human body. Object images from four different preset angles on the left and right.

按照以上实施例中描述的姿态估计方式、骨骼点检测方式以及人体轮廓检测方式，电子设备对每一预设角度的对象图像均进行姿态估计、骨骼点检测和人体轮廓检测，相应得到每一预设角度的对象图像的人体姿态估计信息和设备姿态估计信息，以及每一预设角度的对象图像的骨骼点信息和人体轮廓信息。According to the pose estimation method, bone point detection method and human body contour detection method described in the above embodiments, the electronic device performs pose estimation, bone point detection and human body contour detection on the object image at each preset angle, and obtains each preset Human body pose estimation information and device pose estimation information of the object image at a preset angle, and skeleton point information and human body contour information of the object image at each preset angle.

本实施例中，在进行体型信息的优化，并不分别对每一预设角度的体型信息进行优化，而是先按照配置的融合策略融合多个预设角度的对象图像所对应的体型估计信息，并将融合得到的体型信息记为初始体型信息，然后，再以多个预设角度的对象图像对应的骨骼点信息和人体轮廓信息为约束，按照配置的非线性优化策略，根据多个预设角度的对象图像对应的人体姿态估计信息和设备姿态估计信息，对初始体型信息进行迭代优化，得到与对象人体在多个不同预设角度的骨骼点信息和人体轮廓信息均匹配的目标体型信息。In this embodiment, when optimizing the body shape information, the body shape information at each preset angle is not optimized separately, but the body shape estimation information corresponding to the object images at multiple preset angles is first fused according to the configured fusion strategy , and the fused body shape information is recorded as the initial body shape information, and then, with the skeleton point information and human body contour information corresponding to the object images at multiple preset angles as constraints, according to the configured nonlinear optimization strategy, according to multiple preset The human body pose estimation information and device pose estimation information corresponding to the object image at the set angle, iteratively optimize the initial body shape information, and obtain the target body shape information that matches the skeleton point information and human body contour information of the object human body at multiple preset angles .

应当说明的是，本实施例中对于融合策略的配置不作具体限制，可由本领域技术人员根据实际需要进行配置。It should be noted that there is no specific limitation on the configuration of the fusion policy in this embodiment, and it can be configured by those skilled in the art according to actual needs.

比如，融合策略可以配置为：取各预设角度的对象图像所对应的体型估计信息的平均值；For example, the fusion strategy can be configured as: take the average value of the body shape estimation information corresponding to the object image at each preset angle;

又比如，融合策略可以配置为：按照各预设角度的预分配权重，对多个预设角度的对象图像所对应的体型估计信息进行加权求和等。For another example, the fusion strategy may be configured as: performing weighted summation on body shape estimation information corresponding to object images at multiple preset angles according to the pre-assigned weights of each preset angle.

在一可选的实施例中，为进一步提高体型估计的准确性和效率，通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息之前，还包括：In an optional embodiment, in order to further improve the accuracy and efficiency of body shape estimation, pose estimation is performed on the object image through the pose estimation model to obtain human body pose estimation information, body shape estimation information of the object human body, and image acquisition of the object image Before the device pose estimation information of the device, it also includes:

对对象图像进行人体区域检测，得到对象图像的人体边界框；Perform human body area detection on the object image to obtain the human body bounding box of the object image;

根据人体边界框，截取对象图像中的人体子图像；According to the bounding box of the human body, the sub-image of the human body in the object image is intercepted;

通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息，包括：Perform pose estimation on the object image through the pose estimation model to obtain human body pose estimation information, body shape estimation information of the object human body, and equipment pose estimation information of the image acquisition device of the object image, including:

通过姿态估计模型对人体子图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息。Perform pose estimation on the sub-image of the human body through the pose estimation model to obtain human body pose estimation information, body shape estimation information of the target human body, and equipment pose estimation information of the image acquisition device of the target image.

本实施例中，并不将对象图像完整的输入姿态估计模型进行姿态估计，而是将其中与人体相关的部分图像内容输入姿态估计模型进行姿态估计。In this embodiment, the complete object image is not input into the pose estimation model for pose estimation, but part of the image content related to the human body is input into the pose estimation model for pose estimation.

其中，电子设备首先按照配置的人体区域检测策略对对象图像进行人体区域检测，得到对象图像的人体边界框，即对象图像中人体的最小外接矩形框。此处对于人体区域检测策略的配置不作具体限制，可由本领域技术人员根据实际需要进行配置。Wherein, the electronic device first detects the human body region of the object image according to the configured human body region detection strategy, and obtains the human body bounding box of the object image, that is, the minimum circumscribed rectangular frame of the human body in the object image. The configuration of the human body area detection strategy is not specifically limited here, and can be configured by those skilled in the art according to actual needs.

比如，本实施例中，还预先训练有人体检测模型，该人体检测模型被配置为以包括人体的图像为输入，以人体边界框为输出，该人体边界框之内的区域即为人体区域。此处对于人体检测模型的架构以及训练方式均不作具体限制，可由本领域技术人员根据实际需要进行选取。For example, in this embodiment, a human body detection model is also pre-trained, and the human body detection model is configured to take an image including a human body as input, and output a human body bounding box, and the area within the human body bounding box is the human body area. There are no specific limitations on the architecture and training methods of the human body detection model, which can be selected by those skilled in the art according to actual needs.

相应的，在对对象图像进行人体区域检测时，电子设备可以将对象图像输入人体检测模型，通过人体检测模型对对象图像进行人体区域检测，得到人体边界框。之后，电子设备进一步根据该人体边界框，从对象图像中截取出人体子图像，并将该人体子图像输入姿态估计模型进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息。Correspondingly, when performing human body area detection on an object image, the electronic device may input the object image into a human body detection model, and perform human body area detection on the object image through the human body detection model to obtain a human body bounding box. Afterwards, the electronic device further intercepts a sub-image of the human body from the object image according to the bounding box of the human body, and inputs the sub-image of the human body into the pose estimation model for pose estimation, and obtains human body pose estimation information, body shape estimation information, and object The device pose estimation information of the image capture device.

比如，请参照图4，对象图像中除了存在人体之外，还存在其它物体。电子设备将该对象图像输入到人体检测模型中进行人体区域检测，得到对应对象图像的人体边界框，然后，按照该人体边界框从对象图像中截取出人体子图像。For example, referring to FIG. 4 , in addition to the human body, there are other objects in the object image. The electronic device inputs the object image into the human body detection model to detect the human body area, obtains the human body bounding box corresponding to the object image, and then intercepts the human body sub-image from the object image according to the human body bounding box.

可以理解的是，人们在购买衣物时，通常存在不方便试衣的情况，使得人们难以挑选到合身的衣物。比如，在实体店选购衣物时，需要排队试衣，等候试衣需要耗费漫长的等待时间，而且，试衣时也通常需要对衣物进行反复穿脱，浪费大量时间和精力，也不一定能够挑选到合身的衣物。又比如，在通过网络选购衣物时，将面临无法试衣的境地，通常只能通过商家展示的模特图片来假想自己的穿衣效果，往往难以挑选到合身的衣物。因此，为了满足用户的试衣需求，本申请实施例还利用估计得到的目标体型信息实现虚拟试衣服务。本实施例中，以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息之后，还包括：It can be understood that when people buy clothes, it is usually inconvenient to try on clothes, which makes it difficult for people to choose suitable clothes. For example, when buying clothes in a physical store, you need to line up to try on clothes, and it takes a long time to wait for the fitting. Moreover, you usually need to put on and take off the clothes repeatedly when trying on clothes, wasting a lot of time and energy. Pick out clothes that fit. For another example, when purchasing clothes online, you will face the situation that you cannot try on the clothes. Usually, you can only imagine your dressing effect through the pictures of models displayed by the merchants, and it is often difficult to choose clothes that fit. Therefore, in order to meet the user's fitting needs, the embodiment of the present application also utilizes the estimated target body shape information to implement a virtual fitting service. In this embodiment, with the skeleton point information and human body contour information as constraints, the body shape estimation information is optimized according to the human body posture estimation information and device posture estimation information, and after obtaining the target body shape information of the target human body, it also includes:

根据目标体型信息，获取与对象人体的体型所匹配的目标人体模型；Obtaining a target human body model that matches the body shape of the subject human body according to the target body shape information;

响应于对衣物模型的选取操作，将选取操作指定的目标衣物模型融合至目标人体模型进行显示。In response to the selection operation on the clothing model, the target clothing model specified by the selection operation is fused to the target human body model for display.

本实施例中，电子设备在优化得到对象人体的目标体型信息之后，进一步获取到与对象人体的体型所匹配的三维的人体模型，记为目标人体模型。比如，电子设备可以从包括多个不同体型的人体模型的模型库中获取到与对象人体的体型所匹配的目标人体模型，也可以直接生成与对象人体的体型所匹配的目标人体模型等。In this embodiment, after obtaining the target body shape information of the target human body through optimization, the electronic device further acquires a three-dimensional human body model that matches the body shape of the target human body, which is recorded as the target human body model. For example, the electronic device may obtain a target human body model that matches the body shape of the target human body from a model library including multiple human body models of different body shapes, or may directly generate a target human body model that matches the body shape of the target human body.

此外，在本实施例中，电子设备还提供有试衣界面，同时提供有用于触发电子设备显示试衣界面的“启动接口”。此处对于该启动接口的设置位置以及展现形式等不作具体限制，可由本领域技术人员根据实际需要进行设置。比如，可以将启动接口设置在电子设备的桌面，并以“试衣”图标的形式将试衣控件设置在桌面上，可以点击试衣图标触发电子设备显示试衣界面。In addition, in this embodiment, the electronic device is also provided with a fitting interface, and at the same time, a "starting interface" for triggering the electronic device to display the fitting interface is provided. Here, there is no specific limitation on the setting position and presentation form of the startup interface, which can be set by those skilled in the art according to actual needs. For example, the startup interface can be set on the desktop of the electronic device, and the fitting control can be set on the desktop in the form of a "fitting" icon, and clicking the fitting icon can trigger the electronic device to display the fitting interface.

其中，试衣界面包括衣物选择接口，衣物选择接口被配置为接收输入的针对衣物(包括但不限于上装、下装、帽子以及成套的上装和下装等)的选取操作。Wherein, the fitting interface includes a clothing selection interface, and the clothing selection interface is configured to receive input selection operations for clothing (including but not limited to tops, bottoms, hats, and sets of tops and bottoms, etc.).

示例性的，请参照图5，该衣物选择接口可以滑动选择框的形式展现。比如，如图5所示，可以点击滑动选择框中衣物图标(该衣物图标用于表示不同衣物，分别与对应的三维的衣物模型关联)的方式来输入选取操作，并可以在选择框中左/右滑动以切换当前可选的衣物图标，由此来选择期望试穿的衣物。For example, please refer to FIG. 5 , the clothing selection interface may be displayed in the form of a sliding selection box. For example, as shown in Figure 5, you can click on the clothing icon in the sliding selection box (the clothing icon is used to represent different clothing, respectively associated with the corresponding three-dimensional clothing model) to input the selection operation, and you can click on the left in the selection box / Swipe right to switch the currently available clothing icons, so as to select the clothing you want to try on.

相应的，电子设备在接收到输入的针对衣物模型的选取操作时，响应于该选取操作，将选取操作指定的目标衣物模型融合至目标人体模型进行显示，实现给目标人体模型穿衣的效果，从而达到虚拟试衣的目的。Correspondingly, when the electronic device receives an input selection operation for the clothing model, in response to the selection operation, it fuses the target clothing model specified by the selection operation into the target human body model for display, and achieves the effect of dressing the target human body model. So as to achieve the purpose of virtual fitting.

在一可选的实施例中，为了提高目标人体模型的获取效率，根据目标体型信息，获取与对象人体的体型所匹配的目标人体模型，包括：In an optional embodiment, in order to improve the acquisition efficiency of the target human body model, the target human body model that matches the body shape of the target human body is acquired according to the target body shape information, including:

获取标准体型的预设人体模型；Get preset mannequins of standard body types;

根据目标体型信息对预设人体模型进行形变处理，得到与对象人体的体型所匹配的目标人体模型。Deformation processing is performed on the preset human body model according to the target body shape information to obtain a target human body model that matches the body shape of the target human body.

其中，电子设备首先获取一标准体型的预设人体模型，并以该标准体型的预设人体模型为基础，按照目标体型信息对其进行形变处理，从而得到与对象人体的体型所匹配的目标人体模型。Wherein, the electronic device first acquires a preset human body model of a standard body type, and based on the preset human body model of a standard body type, deforms it according to the target body type information, thereby obtaining a target human body that matches the body type of the target human body Model.

示例性的，本实施例中可以采用SMPL(A Skinned Multi-Person Linear，采用人皮肤的多线性)模型作为标准体型的预设人体模型。其中，SMPL模型是一种基于顶点的采用人皮肤的多线性人体三维模型，通过对SMPL模型的体型参数和姿态参数进行配置，能够精确地表示不同体型和姿态的人体。相应的，在本实施例中，按照目标体型信息对SMPL模型的体型参数进行配置，即可得到与对象人体的体型所匹配的体型调整后的SMPL模型，将该体型调整后的SMPL模型作为目标人体模型。Exemplarily, in this embodiment, an SMPL (A Skinned Multi-Person Linear) model may be used as a preset human body model of a standard body type. Among them, the SMPL model is a vertex-based multi-linear human body three-dimensional model using human skin. By configuring the body shape parameters and posture parameters of the SMPL model, it can accurately represent human bodies with different body shapes and postures. Correspondingly, in this embodiment, the body shape parameters of the SMPL model are configured according to the target body shape information to obtain a body shape-adjusted SMPL model that matches the body shape of the target human body, and the body shape-adjusted SMPL model is used as the target mannequin.

在一可选的实施例中，在目标衣物模型对应标准体型时，将选取操作指定的目标衣物模型融合至目标人体模型进行显示，包括：In an optional embodiment, when the target clothing model corresponds to a standard body type, the target clothing model specified by the selection operation is fused to the target human body model for display, including:

获取目标人体模型与预设人体模型之间的形变信息；Acquiring deformation information between the target human body model and the preset human body model;

根据形变信息对目标衣物模型进行形变处理，得到形变后的目标衣物模型；Deforming the target clothing model according to the deformation information to obtain the deformed target clothing model;

将形变后的目标衣物模型融合至目标人体模型进行显示。The deformed target clothing model is fused to the target human body model for display.

可以理解的是，预先构建不同体型的衣物模型需要花费大量的时间，以及计算资源等成本，因此，本实施例中，衣物模型均按照标准体型构建。相应的，本实施例中的目标衣物模型也将对应标准体型，由于目标人体模型与标准体型的预设人体模型之间存在形变，若直接将对应标准体型的目标衣物模型融合至目标人体模型进行显示，将无法获得最佳的显示效果，影响试衣体验。因此，本实施例中，电子设备先获取目标人体模型与预设人体模型之间的形变信息，并利用该形变信息对目标衣物模型进行形变处理，得到形变后的目标衣物模型，再将该形变后的目标衣物模型融合至目标人体模型进行显示。It can be understood that pre-constructing clothing models of different body shapes takes a lot of time and costs such as computing resources. Therefore, in this embodiment, the clothing models are constructed according to standard body shapes. Correspondingly, the target clothing model in this embodiment will also correspond to the standard body type. Since there is deformation between the target human body model and the preset human body model of the standard body type, if the target clothing model corresponding to the standard body type is directly fused to the target human body model for display, the best display effect will not be obtained, which will affect the fitting experience. Therefore, in this embodiment, the electronic device first obtains the deformation information between the target human body model and the preset human body model, and uses the deformation information to perform deformation processing on the target clothing model to obtain the deformed target clothing model, and then transforms the The final target clothing model is fused to the target human body model for display.

采用本实施例通过的虚拟试衣功能，只要电子设备具备单目摄像头即可低成本的快速完成虚拟试衣功能的部署，同时可以与AR\VR应用相结合，增强AR\VR应用的可玩性及虚拟试衣的使用场景，与线上线下服饰门店结合，让用户通过虚拟试衣功能更加方便快速地寻找到符合自身需求的衣物，同时也可以作为娱乐化的工具让用户体验到虚拟衣物角色扮演的乐趣等。Using the virtual fitting function adopted in this embodiment, as long as the electronic device has a monocular camera, the deployment of the virtual fitting function can be quickly completed at low cost, and at the same time, it can be combined with AR\VR applications to enhance the playability of AR\VR applications Combining with online and offline clothing stores, users can find clothes that meet their needs more conveniently and quickly through the virtual fitting function. At the same time, it can also be used as an entertainment tool to let users experience virtual clothes. Role-playing fun and more.

在一可选的实施例中，获取目标人体模型与预设人体模型之间的形变信息，包括：In an optional embodiment, obtaining deformation information between the target human body model and the preset human body model includes:

获取目标人体模型与预设人体模型之间每一组对应的人体模型单元的形变信息；Acquiring deformation information of each group of corresponding human body model units between the target human body model and the preset human body model;

根据形变信息对目标衣物模型进行形变处理，得到形变后的目标衣物模型，包括：According to the deformation information, the target clothing model is deformed to obtain the deformed target clothing model, including:

根据每一组对应的人体模型单元的形变信息，获取每一组对应的人体模型单元在目标衣物模型中对应的衣物模型单元的形变矢量；According to the deformation information of each group of corresponding human body model units, the deformation vectors of the clothing model units corresponding to each group of corresponding human body model units in the target clothing model are obtained;

根据每一衣物模型单元的形变矢量对每一衣物模型单元进行形变处理，得到每一形变后的衣物模型单元；performing deformation processing on each clothing model unit according to the deformation vector of each clothing model unit to obtain each deformed clothing model unit;

根据每一形变后的衣物模型单元，得到形变后的目标衣物模型。According to each deformed clothing model unit, a deformed target clothing model is obtained.

应当说明的是，本实施例中衣物模型和预设人体模型均采用相同形状的模型单元(比如三角面片)构建得到，相应的，本实施例中以每一模型单元为形变对象进行形变处理。It should be noted that both the clothing model and the preset human body model in this embodiment are constructed using model units (such as triangular faces) of the same shape. Correspondingly, in this embodiment, each model unit is used as the deformation object for deformation processing .

其中，电子设备可以采用交叉参数化的方式来获取目标人体模型与预设人体模型之间每一组对应的人体模型单元的形变信息，该形变信息用于描述预设人体模型中的一人体模型单元经过何种形变(包括形变的方向以及大小等信息)变化为目标人体模型中对应的人体模型单元。Wherein, the electronic device can obtain the deformation information of each group of corresponding human body model units between the target human body model and the preset human body model by means of cross parameterization, and the deformation information is used to describe a human body model in the preset human body model What kind of deformation (including information such as the direction and size of the deformation) the unit undergoes changes into the corresponding human body model unit in the target human body model.

本实施例中，根据每一组对应的人体模型单元的形变信息，电子设备获取每一组对应的人体模型单元在目标衣物模型中对应的衣物模型单元的形变矢量(包括形变大小和形变方向)，以此，可以获得目标衣物模型中每一衣物模型单元的形变矢量。之后，电子设备根据每一衣物模型单元的形变矢量对每一衣物模型单元进行形变处理，得到每一形变后的衣物模型单元。相应的，根据根据每一形变后的衣物模型单元，即可得到形变后的目标衣物模型。In this embodiment, according to the deformation information of each group of corresponding human body model units, the electronic device acquires the deformation vector (including the deformation magnitude and deformation direction) of the clothing model unit corresponding to each group of corresponding human body model units in the target clothing model , so that the deformation vector of each clothing model unit in the target clothing model can be obtained. Afterwards, the electronic device deforms each clothing model unit according to the deformation vector of each clothing model unit to obtain each deformed clothing model unit. Correspondingly, according to each deformed clothing model unit, a deformed target clothing model can be obtained.

请结合参照图6和图7，以下以人体模型单元和衣物模型单元均为三角面片进行说明：Please refer to Fig. 6 and Fig. 7 together, and the following is an illustration that both the human body model unit and the clothing model unit are triangular faces:

将一衣物三角面片的顶点记为pg，将与顶点pg最近的骨骼点连线的垂直交点记为pb(反映顶点pg到最近两个骨骼点连线的最短距离)，将线段pbpg与预设人体模型中某个人体三角面片的交点记为pm，根据衣物三角面片对应的一组人体三角面片的形变信息，计算出对交点pm形变后的形变交点pm’，将形变交点pm’与目标人体模型最近的骨骼点连线上的垂直交点记为pb’，根据原始pg与pm之间的距离结合pb’和pm’即可得到对应于目标人体模型的形变顶点pg’。至此，矢量

即为衣物三角面片的形变矢量。The vertex of a clothing triangular patch is recorded as pg, and the vertical intersection point of the closest bone point line with vertex pg is marked as pb (reflecting the shortest distance from the vertex pg to the nearest two bone point lines), and the line segment pbpg and the preset Assuming that the intersection point of a human body triangle in the human body model is recorded as pm, according to the deformation information of a group of human body triangles corresponding to the clothing triangle, the deformation intersection point pm' after the deformation of the intersection point pm is calculated, and the deformation intersection point pm 'Pb' is the vertical intersection point on the line closest to the bone point of the target human model, and the deformed vertex pg' corresponding to the target human model can be obtained by combining pb' and pm' according to the distance between the original pg and pm. So far, the vector

That is, the deformation vector of the triangle surface of the clothing.

在一可选的实施例中，将选取操作指定的目标衣物模型融合至目标人体模型进行显示之后，还包括：In an optional embodiment, after the target clothing model specified by the selection operation is fused to the target human body model for display, it also includes:

获取对象人体的实时对象图像；Acquiring a real-time object image of the subject's human body;

通过姿态估计模型对实时对象图像进行姿态估计，得到对象人体的实时人体姿态估计信息、实时对象图像的图像采集设备的实时设备姿态估计信息；Estimating the pose of the real-time object image through the pose estimation model to obtain real-time human pose estimation information of the subject human body and real-time device pose estimation information of the image acquisition device of the real-time object image;

若实时设备姿态估计信息基于弱透视投影，则对实时设备姿态估计信息进行投影转换，得到基于全透视投影的全透视实时设备姿态估计信息；If the real-time device attitude estimation information is based on weak perspective projection, the real-time device attitude estimation information is projected and transformed to obtain full-perspective real-time device attitude estimation information based on full perspective projection;

根据全透视实时设备姿态估计信息确定目标人体模型的目标显示位置，以及根据实时人体姿态估计信息确定目标人体模型的目标显示姿态；determining the target display position of the target human body model according to the full-perspective real-time equipment posture estimation information, and determining the target display posture of the target human body model according to the real-time human body posture estimation information;

按照目标显示位置以及目标显示姿态，融合显示目标人体模型和目标衣物模型。According to the target display position and the target display posture, the target human body model and the target clothing model are fused and displayed.

为了进一步丰富虚拟试衣功能，本实施例进一步提供动态的虚拟试衣功能。In order to further enrich the virtual fitting function, this embodiment further provides a dynamic virtual fitting function.

其中，电子设备可以获取对象人体的实时对象图像。比如，当电子设备配置有摄像头等图像采集设备时，通过配置的摄像头实时拍摄对象人体，从而得到对象人体的实时对象图像。此时，电子设备进一步将获取到的实时对象图像输入姿态估计模型，通过姿态估计模型对该实时对象图像进行姿态估计，得到姿态估计模型输出的人体姿态信息、体型信息以及实时对象图像的图像采集设备的设备姿态信息，由于已经获得了对象人体的目标体型信息，此时仅取姿态估计模型输出的人体姿态信息和设备姿态信息，分别记为实时人体姿态估计信息和实时设备姿态估计信息。Wherein, the electronic device can acquire real-time object images of the object's human body. For example, when the electronic device is configured with an image acquisition device such as a camera, the configured camera captures the human body of the subject in real time, so as to obtain a real-time object image of the human body of the subject. At this point, the electronic device further inputs the acquired real-time object image into the attitude estimation model, performs attitude estimation on the real-time object image through the attitude estimation model, and obtains human body attitude information, body shape information and image acquisition of the real-time object image output by the attitude estimation model For the equipment pose information of the device, since the target body shape information of the target human body has been obtained, only the human pose information and device pose information output by the pose estimation model are taken at this time, which are respectively recorded as real-time human pose estimation information and real-time device pose estimation information.

本实施例中，为提高模型训练效率，姿态估计模型基于弱透视投影训练得到。相应的，利用该姿态估计模型获取到的实时设备姿态估计信息也将是基于弱透视投影的，为了能够更正准确的反映对象人体与图像采集设备之间真实的位置关系，电子设备进一步对获取到的实时设备姿态估计信息进行投影转换，将实时设备姿态估计信息由弱透视投影转换为全透视投影，并将转换后的实时设备姿态估计信息记为全透视实时设备姿态估计信息。In this embodiment, in order to improve the efficiency of model training, the pose estimation model is trained based on weak perspective projection. Correspondingly, the real-time device pose estimation information obtained by using the pose estimation model will also be based on weak perspective projection. In order to correct and accurately reflect the real positional relationship between the subject human body and the image acquisition device, the electronic device further The real-time device pose estimation information is projected and transformed, and the real-time device pose estimation information is converted from weak perspective projection to full perspective projection, and the converted real-time device pose estimation information is recorded as full perspective real-time device pose estimation information.

其中，全透视投影是指物体投影时，投影大小与物体距离负相关，以同一物体为例，该物体不同位置处的距离越大，则投影越小，从而呈现近大远小的效果。弱透视投影是在全透视投影基础上的简化，物体投影时，同一物体各个位置的距离用平均距离代替，无法呈现出近大远小的效果。Among them, full perspective projection means that when an object is projected, the size of the projection is negatively correlated with the distance of the object. Taking the same object as an example, the larger the distance between different positions of the object, the smaller the projection, thus presenting the effect that the near distance is large and the distance is small. Weak perspective projection is a simplification on the basis of full perspective projection. When an object is projected, the distance of each position of the same object is replaced by the average distance, which cannot show the effect of being near large and far small.

进一步的，电子设备根据该全透视实时设备姿态估计信息，确定对象人体与图像采集设备的实时相对位置，并根据配置的相对位置和显示位置的对应关系，将对应该实时相对位置的显示位置确定为目标人体模型的目标显示位置，以及根据实时人体姿态估计信息确定目标人体模型的目标显示姿态，也即是将实时人体姿态估计信息所描述的人体姿态确定为目标人体模型的目标显示姿态。Further, the electronic device determines the real-time relative position of the subject's human body and the image acquisition device based on the full perspective real-time device attitude estimation information, and determines the display position corresponding to the real-time relative position according to the corresponding relationship between the configured relative position and the display position Determining the target display position of the target human body model, and determining the target display posture of the target human body model according to the real-time human body posture estimation information, that is, determining the human body posture described by the real-time human body posture estimation information as the target display posture of the target human body model.

如上，在确定目标人体模型的目标显示位置以及目标显示姿态之后，电子设备即按照该目标显示位置和目标显示姿态，融合显示目标人体模型和目标衣物模型。As above, after determining the target display position and the target display posture of the target human body model, the electronic device fuses and displays the target human body model and the target clothing model according to the target display position and target display posture.

通过采用本实施例提供的动态虚拟试衣功能，能够使得融合的目标人体模型和目标衣物模型跟随对象人体移动，并做出对应姿态，使得虚拟试衣功能更为逼真，就像衣物穿在真实的人体之上一样。By adopting the dynamic virtual fitting function provided by this embodiment, the fused target human body model and target clothing model can be moved with the target human body and make corresponding gestures, making the virtual fitting function more realistic, just like wearing clothes in real life. the same as on the human body.

在一可选的实施例中，按照目标显示位置以及目标显示姿态，融合显示目标人体模型和目标衣物模型之前，还包括：In an optional embodiment, before fusion displaying the target human body model and the target clothing model according to the target display position and the target display posture, it also includes:

对目标显示位置和目标显示姿态进行平滑处理，得到平滑后的目标显示位置和平滑后目标显示姿态；Smoothing the target display position and target display attitude to obtain the smoothed target display position and smoothed target display attitude;

按照目标显示位置以及目标显示姿态，融合显示目标人体模型和目标衣物模型，包括：According to the target display position and target display posture, the target human body model and target clothing model are fused and displayed, including:

按照平滑后的目标显示位置以及平滑后的目标显示姿态，融合显示目标人体模型和目标衣物模型。According to the smoothed target display position and the smoothed target display posture, the target human body model and the target clothing model are fused and displayed.

为了进一步提高虚拟试衣的效果，本实施例中并不直接利用获得的目标显示位置和目标显示姿态进行虚拟试衣，而是先按照配置的平滑滤波算法，对目标显示位置和目标显示姿态进行平滑滤波处理，得到平滑后的目标显示位置和平滑后的目标显示姿态，再按照平滑后的目标显示位置和平滑后的目标显示姿态，融合显示目标人体模型和目标衣物模型。In order to further improve the effect of virtual fitting, in this embodiment, the obtained target display position and target display posture are not directly used for virtual fitting, but the target display position and target display posture are firstly adjusted according to the configured smoothing filter algorithm. The smoothing filtering process obtains the smoothed target display position and the smoothed target display attitude, and then fuses and displays the target human body model and the target clothing model according to the smoothed target display position and the smoothed target display attitude.

应当说明的是，本实施例中对于采用何种平滑滤波算法不作具体限制，可由本领域技术人员根据实际需要进行配置，比如，本实施例中可以采用one-euro滤波算法对目标显示位置和目标显示姿态进行平滑滤波处理。It should be noted that there is no specific limitation on which smoothing filter algorithm is used in this embodiment, and it can be configured by those skilled in the art according to actual needs. The display attitude is smoothed and filtered.

请参照图8，为更好的执行本申请所提供的体型估计方法，本申请进一步提供一种体型估计装置400，如图8所示，该体型估计装置400包括：Please refer to FIG. 8. In order to better implement the body shape estimation method provided by the present application, the present application further provides a body shape estimation device 400. As shown in FIG. 8, the body shape estimation device 400 includes:

估计模块410，用于获取对象人体的对象图像，并通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息；Estimation module 410, configured to acquire the object image of the target human body, and perform pose estimation on the object image through the pose estimation model, to obtain human body pose estimation information, body shape estimation information of the target human body, and equipment pose estimation information of the image acquisition device of the object image ;

第一检测模块420，用于对对象图像进行骨骼点检测，得到对象人体的骨骼点信息；The first detection module 420 is configured to perform skeleton point detection on the subject image to obtain skeleton point information of the subject human body;

第二检测模块430，用于对对象图像进行人体轮廓检测，得到对象人体的人体轮廓信息；The second detection module 430 is configured to perform human body contour detection on the target image to obtain human body contour information of the target human body;

优化模块440，用于以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。The optimization module 440 is configured to optimize the body shape estimation information according to the human body pose estimation information and device pose estimation information, with the constraints of the skeleton point information and the human body contour information, to obtain the target body shape information of the target human body.

在一可选的实施例中，对象图像包括对象人体的多个预设角度的对象图像，优化模块440用于：In an optional embodiment, the object image includes object images of multiple preset angles of the object human body, and the optimization module 440 is used for:

融合多个预设角度的对象图像对应的体型估计信息，得到对象人体的初始体型信息；fusing the body shape estimation information corresponding to the object images at multiple preset angles to obtain the initial body shape information of the object;

以多个预设角度的对象图像对应的骨骼点信息和人体轮廓信息为约束，根据多个预设角度的对象图像对应的人体姿态估计信息和设备姿态估计信息，对初始体型信息进行优化处理，得到对象人体的目标体型信息。The initial body shape information is optimized according to the human body pose estimation information and device pose estimation information corresponding to the object images at multiple preset angles, constrained by the skeleton point information and human body contour information corresponding to the object images at multiple preset angles, The target body shape information of the subject human body is obtained.

在一可选的实施例中，本申请提供的体型估计装置400还包括第三检测模块，用于：In an optional embodiment, the body shape estimating device 400 provided in the present application further includes a third detection module, which is used for:

对对象图像进行人体区域检测，得到对象图像的人体边界框；Perform human body area detection on the object image to obtain the human body bounding box of the object image;

根据人体边界框，截取对象图像中的人体子图像；According to the bounding box of the human body, the sub-image of the human body in the object image is intercepted;

估计模块410用于通过姿态估计模型对人体子图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息。The estimation module 410 is used to perform pose estimation on the sub-image of the human body through the pose estimation model, to obtain human body pose estimation information, body shape estimation information of the target human body, and device pose estimation information of the image acquisition device of the target image.

在一可选的实施例中，本申请提供的体型估计装置400还包括显示模块，用于：In an optional embodiment, the body shape estimating device 400 provided by the present application further includes a display module, which is used for:

根据目标体型信息，获取与对象人体的体型所匹配的目标人体模型；Obtaining a target human body model that matches the body shape of the subject human body according to the target body shape information;

响应于对衣物模型的选取操作，将选取操作指定的目标衣物模型融合至目标人体模型进行显示。In response to the selection operation on the clothing model, the target clothing model specified by the selection operation is fused to the target human body model for display.

在一可选的实施例中，显示模块用于：In an optional embodiment, the display module is used for:

获取标准体型的预设人体模型；Get preset mannequins of standard body types;

根据目标体型信息对预设人体模型进行形变处理，得到与对象人体的体型所匹配的目标人体模型。Deformation processing is performed on the preset human body model according to the target body shape information to obtain a target human body model that matches the body shape of the target human body.

在一可选的实施例中，在目标衣物模型对应标准体型时，显示模块用于：In an optional embodiment, when the target clothing model corresponds to a standard body shape, the display module is used to:

获取目标人体模型与预设人体模型之间的形变信息；Acquiring deformation information between the target human body model and the preset human body model;

根据形变信息对目标衣物模型进行形变处理，得到形变后的目标衣物模型；Deforming the target clothing model according to the deformation information to obtain the deformed target clothing model;

将形变后的目标衣物模型融合至目标人体模型进行显示。The deformed target clothing model is fused to the target human body model for display.

在一可选地实施例中，显示模块用于：In an optional embodiment, the display module is used for:

获取目标人体模型与预设人体模型之间每一组对应的人体模型单元的形变信息；Acquiring deformation information of each group of corresponding human body model units between the target human body model and the preset human body model;

根据形变信息对目标衣物模型进行形变处理，得到形变后的目标衣物模型，包括：According to the deformation information, the target clothing model is deformed to obtain the deformed target clothing model, including:

根据每一组对应的人体模型单元的形变信息，获取每一组对应的人体模型单元在目标衣物模型中对应的衣物模型单元的形变矢量；According to the deformation information of each group of corresponding human body model units, the deformation vectors of the clothing model units corresponding to each group of corresponding human body model units in the target clothing model are obtained;

根据每一衣物模型单元的形变矢量对每一衣物模型单元进行形变处理，得到每一形变后的衣物模型单元；performing deformation processing on each clothing model unit according to the deformation vector of each clothing model unit to obtain each deformed clothing model unit;

根据每一形变后的衣物模型单元，得到形变后的目标衣物模型。According to each deformed clothing model unit, a deformed target clothing model is obtained.

在一可选的实施例中，显示模块还用于：In an optional embodiment, the display module is also used for:

获取对象人体的实时对象图像；Acquiring a real-time object image of the subject's human body;

通过姿态估计模型对实时对象图像进行姿态估计，得到对象人体的实时人体姿态估计信息、实时对象图像的图像采集设备的实时设备姿态估计信息；Estimating the pose of the real-time object image through the pose estimation model to obtain real-time human pose estimation information of the subject human body and real-time device pose estimation information of the image acquisition device of the real-time object image;

若实时设备姿态估计信息基于弱透视投影，则对实时设备姿态估计信息进行投影转换，得到基于全透视投影的全透视实时设备姿态估计信息；If the real-time device attitude estimation information is based on weak perspective projection, the real-time device attitude estimation information is projected and transformed to obtain full-perspective real-time device attitude estimation information based on full perspective projection;

根据全透视实时设备姿态估计信息确定目标人体模型的目标显示位置，以及根据实时人体姿态估计信息确定目标人体模型的目标显示姿态；determining the target display position of the target human body model according to the full-perspective real-time equipment posture estimation information, and determining the target display posture of the target human body model according to the real-time human body posture estimation information;

按照目标显示位置以及目标显示姿态，融合显示目标人体模型和目标衣物模型。According to the target display position and the target display posture, the target human body model and the target clothing model are fused and displayed.

在一可选的实施例中，本申请提供的体型估计装置400还包括平滑模块，用于对目标显示位置和目标显示姿态进行平滑处理，得到平滑后的目标显示位置和平滑后目标显示姿态；In an optional embodiment, the body shape estimating device 400 provided in the present application further includes a smoothing module, which is used to perform smoothing processing on the target display position and target display posture to obtain the smoothed target display position and smoothed target display posture;

显示模块用于按照平滑后的目标显示位置以及平滑后的目标显示姿态，融合显示目标人体模型和目标衣物模型。The display module is used to fuse and display the target human body model and the target clothing model according to the smoothed target display position and the smoothed target display posture.

应当说明的是，本申请实施例提供的体型估计装置400与上文实施例中的体型估计方法属于同一构思，其具体实现过程详见以上相关实施例，此处不再赘述。It should be noted that the body shape estimating apparatus 400 provided in the embodiment of the present application belongs to the same idea as the body shape estimating method in the above embodiment, and its specific implementation process is detailed in the above related embodiments, and will not be repeated here.

本申请实施例还提供一种电子设备，包括存储器和处理器，其中处理器通过调用存储器中存储的计算机程序，用于执行本实施例提供的体型估计方法中的步骤。The embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the processor is used to execute the steps in the body size estimation method provided in this embodiment by calling a computer program stored in the memory.

请参照图9，图9为本申请实施例提供的电子设备100的结构示意图。Please refer to FIG. 9 , which is a schematic structural diagram of anelectronic device 100 provided by an embodiment of the present application.

该电子设备100可以包括网络接口110、存储器120、处理器130以及屏幕组件等部件。本领域技术人员可以理解，图9中示出的电子设备100结构并不构成对电子设备100的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。Theelectronic device 100 may include components such as anetwork interface 110, amemory 120, aprocessor 130, and a screen component. Those skilled in the art can understand that the structure of theelectronic device 100 shown in FIG. 9 does not constitute a limitation on theelectronic device 100, and may include more or less components than those shown in the illustration, or combine certain components, or different components. layout.

网络接口110可以用于进行设备之间的网络连接。Thenetwork interface 110 can be used for network connection between devices.

存储器120可用于存储计算机程序和数据。存储器120存储的计算机程序中包含有可执行代码。计算机程序可以划分为各种功能模块。处理器130通过运行存储在存储器120的计算机程序，从而执行各种功能应用以及数据处理。Memory 120 may be used to store computer programs and data. The computer programs stored in thememory 120 include executable codes. A computer program can be divided into various functional modules. Theprocessor 130 executes various functional applications and data processing by executing the computer program stored in thememory 120 .

处理器130是电子设备100的控制中心，利用各种接口和线路连接整个电子设备100的各个部分，通过运行或执行存储在存储器120内的计算机程序，以及调用存储在存储器120内的数据，执行电子设备100的各种功能和处理数据，从而对电子设备100进行整体控制。Theprocessor 130 is the control center of theelectronic device 100. It uses various interfaces and lines to connect various parts of the entireelectronic device 100. By running or executing the computer program stored in thememory 120 and calling the data stored in thememory 120, theprocessor 130 executes Various functions and processing data of theelectronic device 100 , so as to control theelectronic device 100 as a whole.

在本申请实施例中，电子设备100中的处理器130会按照如下的指令，将一个或一个以上的计算机程序对应的可执行代码加载到存储器120中，并由处理器130来执行本申请提供的体型估计方法中的步骤，比如：In this embodiment of the application, theprocessor 130 in theelectronic device 100 will load the executable code corresponding to one or more computer programs into thememory 120 according to the following instructions, and theprocessor 130 will execute the program provided by this application. The steps in the body size estimation method, such as:

获取对象人体的对象图像，并通过姿态估计模型对对象图像进行姿态估计，得到对象人体的人体姿态估计信息、体型估计信息，以及对象图像的图像采集设备的设备姿态估计信息；acquiring a subject image of the subject human body, and performing pose estimation on the subject image through a pose estimation model, obtaining human body pose estimation information, body shape estimation information of the subject human body, and device pose estimation information of an image acquisition device of the subject image;

对对象图像进行骨骼点检测，得到对象人体的骨骼点信息；Carry out bone point detection on the object image to obtain the bone point information of the object human body;

对对象图像进行人体轮廓检测，得到对象人体的人体轮廓信息；Perform human body contour detection on the object image to obtain human body contour information of the object human body;

以骨骼点信息和人体轮廓信息为约束，根据人体姿态估计信息和设备姿态估计信息对体型估计信息进行优化处理，得到对象人体的目标体型信息。With the skeleton point information and human body contour information as constraints, the body shape estimation information is optimized according to the human body pose estimation information and device pose estimation information, and the target body shape information of the target human body is obtained.

应当说明的是，本申请实施例提供的电子设备100与上文实施例中的体型估计方法属于同一构思，其具体实现过程详见以上相关实施例，此处不再赘述。It should be noted that theelectronic device 100 provided in the embodiment of the present application is based on the same idea as the body size estimation method in the above embodiment, and its specific implementation process is detailed in the above related embodiments, and will not be repeated here.

本申请还提供一种计算机可读的存储介质，其上存储有计算机程序，当其存储的计算机程序在本申请实施例提供的电子设备的处理器上执行时，使得电子设备的处理器执行以上任一适于电子设备的体型估计方法中的步骤。其中，存储介质可以是磁碟、光盘、只读存储器(Read Only Memory，ROM)或者随机存取器(Random Access Memory，RAM)等。The present application also provides a computer-readable storage medium on which a computer program is stored. When the stored computer program is executed on the processor of the electronic device provided in the embodiment of the present application, the processor of the electronic device executes the above Steps in any body size estimation method suitable for an electronic device. Wherein, the storage medium may be a magnetic disk, an optical disk, a read only memory (Read Only Memory, ROM), or a random access device (Random Access Memory, RAM), and the like.

以上对本申请所提供的一种体型估计方法、装置、存储介质及电子设备进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。A body shape estimation method, device, storage medium and electronic equipment provided by this application have been introduced in detail above. In this paper, specific examples are used to illustrate the principle and implementation of this application. The description of the above embodiments is only for To help understand the method and its core idea of this application; at the same time, for those skilled in the art, according to the idea of this application, there will be changes in the specific implementation and application scope. In summary, the content of this specification does not It should be understood as a limitation on the present application.

Claims (12)

1. A body type estimation method, comprising:

acquiring an object image of an object human body, and carrying out posture estimation on the object image through a posture estimation model to obtain human body posture estimation information and body type estimation information of the object human body and equipment posture estimation information of image acquisition equipment of the object image;

performing skeleton point detection on the object image to obtain skeleton point information of the object human body;

detecting the human body contour of the object image to obtain human body contour information of the object human body;

and taking the skeleton point information and the human body contour information as constraints, and carrying out optimization processing on the body type estimation information according to the human body posture estimation information and the equipment posture estimation information to obtain target body type information of the target human body.

2. The body type estimation method according to claim 1, wherein the object image includes object images of a plurality of preset angles of the object human body, and the optimizing processing is performed on the body type estimation information according to the human body posture estimation information and the device posture estimation information with the skeletal point information and the human body contour information as constraints to obtain target body type information of the object human body, comprising:

Fusing body type estimation information corresponding to the object images with the preset angles to obtain initial body type information of the object human body;

and taking skeleton point information and human body contour information corresponding to the object images with the plurality of preset angles as constraints, and carrying out optimization processing on the initial body type information according to human body posture estimation information and equipment posture estimation information corresponding to the object images with the plurality of preset angles to obtain target body type information of the object human body.

3. The body type estimation method according to claim 1, wherein before performing pose estimation on the object image by the pose estimation model to obtain the human body pose estimation information, the body type estimation information, and the device pose estimation information of the image acquisition device of the object image, further comprising:

detecting the human body region of the object image to obtain a human body boundary box of the object image;

according to the human body boundary box, human body sub-images in the object image are intercepted;

the performing gesture estimation on the object image through a gesture estimation model to obtain human body gesture estimation information and body type estimation information of the human body of the object and equipment gesture estimation information of an image acquisition equipment of the object image, including:

And carrying out posture estimation on the human body sub-images through the posture estimation model to obtain human body posture estimation information and body type estimation information of the human body of the object and equipment posture estimation information of the image acquisition equipment of the image of the object.

4. A body type estimating method according to any one of claims 1 to 3, wherein said optimizing said body type estimating information based on said body posture estimating information and said device posture estimating information with said skeletal point information and said body contour information as constraints, after obtaining target body type information of said subject's body, further comprises:

acquiring a target human body model matched with the body type of the target human body according to the target body type information;

and in response to the selection operation of the clothes model, fusing the target clothes model appointed by the selection operation to the target human body model for display.

5. The body type estimation method according to claim 4, wherein the acquiring a target human body model matching the body type of the subject human body based on the target body type information includes:

acquiring a preset human body model of a standard body type;

And carrying out deformation processing on the preset human body model according to the target body type information to obtain a target human body model matched with the body type of the target human body.

6. The body type estimation method according to claim 5, wherein when the target clothing model corresponds to the standard body type, the fusing the target clothing model specified by the selecting operation to the target human body model for display includes:

obtaining deformation information between the target human body model and the preset human body model;

performing deformation treatment on the target clothing model according to the deformation information to obtain a deformed target clothing model;

and fusing the deformed target clothing model to the target human body model for display.

7. The body type estimation method according to claim 6, wherein the acquiring deformation information between the target human body model and the preset human body model includes:

obtaining deformation information of each group of corresponding human model units between the target human model and the preset human model;

performing deformation processing on the target clothes model according to the deformation information to obtain a deformed target clothes model, wherein the deformation processing comprises the following steps:

According to the deformation information of each group of corresponding human body model units, obtaining deformation vectors of corresponding clothes model units of each group of corresponding human body model units in the target clothes model;

performing deformation processing on each clothing model unit according to the deformation vector of each clothing model unit to obtain each deformed clothing model unit;

and obtaining the deformed target clothing model according to each deformed clothing model unit.

8. The body type estimation method according to claim 4, wherein after the fusing the target clothing model specified by the selecting operation to the target human model for display, further comprising:

acquiring a real-time object image of the object human body;

carrying out gesture estimation on the real-time object image through the gesture estimation model to obtain real-time human body gesture estimation information of the object human body and real-time equipment gesture estimation information of image acquisition equipment of the real-time object image;

if the real-time equipment posture estimation information is based on weak perspective projection, performing projection conversion on the real-time equipment posture estimation information to obtain full perspective real-time equipment posture estimation information based on full perspective projection;

Determining a target display position of the target human body model according to the full-perspective real-time equipment posture estimation information, and determining a target display posture of the target human body model according to the real-time human body posture estimation information;

and according to the target display position and the target display gesture, the target human body model and the target clothes model are displayed in a fusion mode.

9. The body type estimation method according to claim 8, wherein before the fusion display of the target mannequin and the target clothing model according to the target display position and the target display posture, further comprising:

performing smoothing processing on the target display position and the target display gesture to obtain a smoothed target display position and a smoothed target display gesture;

and according to the target display position and the target display gesture, the target human body model and the target clothes model are displayed in a fusion mode, and the method comprises the following steps:

and according to the smoothed target display position and the smoothed target display gesture, the target human body model and the target clothes model are displayed in a fusion mode.

10. A body type estimating apparatus, comprising:

The estimating module is used for acquiring an object image of an object human body, carrying out gesture estimation on the object image through a gesture estimating model, and obtaining human body gesture estimating information and body type estimating information of the object human body and equipment gesture estimating information of image acquisition equipment of the object image;

the first detection module is used for detecting skeleton points of the object image to obtain skeleton point information of the object human body;

the second detection module is used for detecting the human body contour of the object image to obtain the human body contour information of the object human body;

and the optimization module is used for optimizing the body type estimation information according to the human body posture estimation information and the equipment posture estimation information by taking the skeleton point information and the human body contour information as constraints to obtain target body type information of the human body of the object.

11. A storage medium having stored thereon a computer program, which when loaded by a processor performs the steps of the body shape estimation method according to any of claims 1-9.

12. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that the processor is adapted to perform the steps of the body shape estimation method according to any of claims 1 to 9 by loading the computer program.

Images