技术领域technical field
本发明属于图像处理技术领域，尤其是涉及一种数据生成方法、装置及电子设备。The invention belongs to the technical field of image processing, and in particular relates to a data generation method, device and electronic device.
背景技术Background technique
近年来，深度学习技术不断发展，在目标检测、行为识别等领域得到了广泛应用。当前的深度学习技术多基于卷积神经网络实现。在训练卷积神经网络时，对训练数据的数量和多样性有较高的需求，卷积神经网络的最终性能和训练数据的丰富程度呈正比。In recent years, with the continuous development of deep learning technology, it has been widely used in object detection, behavior recognition and other fields. Current deep learning techniques are mostly based on convolutional neural networks. When training a convolutional neural network, there is a high demand for the quantity and diversity of training data, and the final performance of the convolutional neural network is proportional to the richness of the training data.
目前多采用将图像进行随机尺度变化、随机旋转或随机水平旋转的方式来增加训练数据的数量。采用该方法得到的训练数据训练出的卷积神经网络，在对具有重叠的部件或杂乱的背景的图像进行检测时，很容易输出错误的检测结果。因此上述方法无法保证卷积神经网络处理复杂背景图像的稳定性。At present, random scale changes, random rotations or random horizontal rotations of images are often used to increase the amount of training data. The convolutional neural network trained with the training data obtained by this method is prone to output wrong detection results when detecting images with overlapping parts or cluttered backgrounds. Therefore, the above methods cannot guarantee the stability of the convolutional neural network in processing complex background images.
发明内容SUMMARY OF THE INVENTION
有鉴于此，本发明的目的在于提供一种数据生成方法、装置及电子设备，可以增加用于训练的数据量，提高数据的多样性，有利于提高训练得到的模型的稳定性。In view of this, the purpose of the present invention is to provide a data generation method, device and electronic device, which can increase the amount of data used for training, improve the diversity of data, and help improve the stability of the model obtained by training.
为了实现上述目的，本发明实施例采用的技术方案如下：In order to achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:
第一方面，本发明实施例提供了一种数据生成方法，包括：In a first aspect, an embodiment of the present invention provides a data generation method, including:
获取背景图像，并获取目标对象的多个实例图；Get the background image, and get multiple instance graphs of the target object;
将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置，使所述多个实例图中任意两个实例图之间的距离在预设范围内；adding the plurality of instance graphs to the background image, determining the positions of the plurality of instance graphs, so that the distance between any two instance graphs in the plurality of instance graphs is within a preset range;
按照确定的位置将所述多个实例图和所述背景图像合成为训练图像。The multiple instance images and the background image are synthesized into a training image according to the determined positions.
结合第一方面，本发明实施例提供了第一方面的第一种可能的实施方式，其中，所述获取背景图像的步骤，包括：In conjunction with the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, wherein the step of acquiring a background image includes:
从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像；Extract the first image that does not contain the target object from the pre-acquired training data set as the background image;
获取目标对象的多个实例图的步骤，包括：Steps to obtain multiple instance graphs of the target object, including:
从所述训练数据集中抽取包含目标对象的至少一个第二图像；extracting at least one second image containing the target object from the training data set;
从所述至少一个第二图像中分割出所述目标对象的多个实例图。A plurality of instance maps of the target object are segmented from the at least one second image.
结合第一方面，本发明实施例提供了第一方面的第二种可能的实施方式，其中，所述获取背景图像，并获取目标对象的多个实例图的步骤，包括：In conjunction with the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, wherein the step of acquiring a background image and acquiring multiple instance images of the target object includes:
从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像，生成候选背景集；Extract the first image that does not contain the target object from the pre-acquired training data set as a background image to generate a candidate background set;
从所述训练数据集中抽取包含目标对象的第二图像，从所述第二图像中分割出所述目标对象的实例图，生成候选实例集；Extract a second image containing the target object from the training data set, segment an instance graph of the target object from the second image, and generate a candidate instance set;
从所述候选背景集中选择一张背景图像，从所述候选实例集中选择多个实例图，将选择的背景图像和多个实例图对应组成实例背景对；所述实例背景对中的实例图和背景图像用于合成一张训练图像。A background image is selected from the candidate background set, multiple instance images are selected from the candidate instance set, and the selected background image and multiple instance images are correspondingly formed into an instance background pair; the instance images in the instance background pair and The background image is used to synthesize a training image.
结合第一方面的第二种可能的实施方式，本发明实施例提供了第一方面的第三种可能的实施方式，其中，所述将选择的背景图像和多个实例图对应组成实例背景对的步骤，包括：In conjunction with the second possible implementation manner of the first aspect, this embodiment of the present invention provides a third possible implementation manner of the first aspect, wherein the selected background image and multiple instance images correspond to form an instance background pair steps, including:
对从所述第二图像中分割出的实例图进行尺度变换；performing scale transformation on the instance graph segmented from the second image;
将选择的背景图像、从所述第二图像中分割出的实例图和经尺度变换后的实例图对应组成实例背景对。The selected background image, the instance image segmented from the second image, and the scale-transformed instance image correspond to form an instance-background pair.
结合第一方面，本发明实施例提供了第一方面的第四种可能的实施方式，其中，将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置的步骤，包括：In conjunction with the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, wherein the multiple instance images are added to the background image, and the location of the multiple instance images is determined. steps, including:
将所述多个实例图随机添加至所述背景图像的任意位置；randomly adding the plurality of instance images to any position of the background image;
根据所述多个实例图中的每个实例图在所述背景图像上的位置确定邻近损失值；determining a proximity loss value according to the position of each instance map of the plurality of instance maps on the background image;
根据所述邻近损失值调整所述实例图在所述背景图像上的位置。The position of the instance map on the background image is adjusted according to the proximity loss value.
结合第一方面的第四种可能的实施方式，本发明实施例提供了第一方面的第五种可能的实施方式，其中，根据所述多个实例图中的每个实例图在所述背景图像上的位置确定邻近损失值的步骤，包括：In conjunction with the fourth possible implementation manner of the first aspect, this embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein, according to each example diagram of the plurality of example diagrams, in the background The location on the image determines the steps of the proximity loss value, including:
根据每个实例图在所述背景图像上的位置，分别确定整体拉力损失值和整理推力损失值；According to the position of each instance image on the background image, the overall tension loss value and the finishing thrust loss value are respectively determined;
根据所述整体拉力损失值和所述整体推力损失值确定所述邻近损失值。The proximity loss value is determined based on the overall pull loss value and the overall thrust loss value.
结合第一方面的第五种可能的实施方式，本发明实施例提供了第一方面的第六种可能的实施方式，其中，根据每个实例图在所述背景图像上的位置，确定整体拉力损失值的步骤，包括：With reference to the fifth possible implementation manner of the first aspect, the embodiment of the present invention provides the sixth possible implementation manner of the first aspect, wherein the overall tensile force is determined according to the position of each instance map on the background image The steps to lose value, including:
逐一将所述背景图像上的每个实例图作为当前实例图，计算所述当前实例图与所述背景图像上的其他的每个实例图之间的距离；确定与所述当前实例图距离最近的相邻实例图，将所述当前实例图和所述相邻实例图组成一组相近实例图对；Taking each instance graph on the background image as the current instance graph one by one, calculating the distance between the current instance graph and each other instance graph on the background image; determining that the distance to the current instance graph is the closest The adjacent instance graph of , the current instance graph and the adjacent instance graph form a group of close instance graph pairs;
将得到的所有相近实例图对组成实例图对集合；All the obtained similar instance graph pairs are formed into instance graph pair sets;
确定所述实例图对集合中每一组相近实例图对的拉力损失值；determining the tensile force loss value of each group of similar instance graph pairs in the instance graph pair set;
将每一组相近实例图对的拉力损失值之和作为整体拉力损失值。The sum of the tension loss values of each group of similar instance graph pairs is taken as the overall tension loss value.
结合第一方面的第六种可能的实施方式，本发明实施例提供了第一方面的第七种可能的实施方式，其中，根据每个实例图在所述背景图像上的位置，确定整体推力损失值的步骤，包括：In conjunction with the sixth possible implementation manner of the first aspect, the embodiment of the present invention provides the seventh possible implementation manner of the first aspect, wherein the overall thrust is determined according to the position of each instance map on the background image The steps to lose value, including:
将所述背景图像上的任意两个实例图组成一组实例图对，得到多组实例图对；Forming any two instance graphs on the background image into a group of instance graph pairs to obtain multiple groups of instance graph pairs;
从所述多组实例图对中选出所述实例图对集合之外的非相近实例图对，计算每一组非相近实例图对的推力损失值；Select non-similar instance graph pairs other than the instance graph pair set from the multiple groups of instance graph pairs, and calculate the thrust loss value of each group of non-similar instance graph pairs;
将每一组非相近实例图对的推力损失值之和，作为整体推力损失值。The sum of the thrust loss values of each group of non-similar instance graph pairs is taken as the overall thrust loss value.
结合第一方面的第五种可能的实施方式，本发明实施例提供了第一方面的第八种可能的实施方式，其中，根据所述整体拉力损失值和所述整体推力损失值确定所述邻近损失值的步骤，包括：With reference to the fifth possible implementation manner of the first aspect, the embodiment of the present invention provides the eighth possible implementation manner of the first aspect, wherein the The steps for proximity loss values include:
将所述整体拉力损失值与预设系数相乘后加上所述整体推力损失值，得到所述邻近损失值。The adjacent loss value is obtained by multiplying the overall tensile force loss value by a preset coefficient and then adding the overall thrust loss value.
结合第一方面，本发明实施例提供了第一方面的第九种可能的实施方式，其中，将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置的步骤之后，所述方法还包括：In conjunction with the first aspect, an embodiment of the present invention provides a ninth possible implementation manner of the first aspect, wherein the multiple instance images are added to the background image, and the position of the multiple instance images is determined. After the step, the method further includes:
确定所述背景图像上的每个实例图的被遮挡面积比；所述被遮挡面积比为所述实例图中被遮挡部分的面积与所述实例图总面积的比值；Determine the occluded area ratio of each example image on the background image; the occluded area ratio is the ratio of the area of the occluded portion in the example image to the total area of the example image;
将所述被遮挡面积比大于设定阈值的实例图删除。The instance graphs whose occluded area ratio is greater than the set threshold are deleted.
结合第一方面，本发明实施例提供了第一方面的第十种可能的实施方式，其中，将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置的步骤之后，所述方法还包括：In conjunction with the first aspect, an embodiment of the present invention provides a tenth possible implementation manner of the first aspect, wherein the multiple instance images are added to the background image, and the position of the multiple instance images is determined. After the step, the method further includes:
将所述背景图像上的实例图的被遮挡部分的标签设为遮挡。Set the label of the occluded part of the instance graph on the background image as occluded.
第二方面，本发明实施例还提供一种数据生成装置，包括：In a second aspect, an embodiment of the present invention further provides a data generation device, including:
元素获取模块，用于获取背景图像，并获取目标对象的多个实例图；The element acquisition module is used to acquire the background image and acquire multiple instance images of the target object;
位置确定模块，用于将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置，使所述多个实例图中任意两个实例图之间的距离在预设范围内；The position determination module is used to add the multiple instance images to the background image, determine the positions of the multiple instance images, and make the distance between any two instance images in the multiple instance images within the predetermined range. within the set range;
数据生成模块，用于按照确定的位置将所述多个实例图和所述背景图像合成为训练图像。The data generation module is used for synthesizing the plurality of instance images and the background image into a training image according to the determined position.
结合第二方面，本发明实施例提供了第二方面的第一种可能的实施方式，其中，元素获取模块还用于：In conjunction with the second aspect, the embodiment of the present invention provides a first possible implementation manner of the second aspect, wherein the element acquisition module is further configured to:
从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像；Extract the first image that does not contain the target object from the pre-acquired training data set as the background image;
从所述训练数据集中抽取包含目标对象的至少一个第二图像；extracting at least one second image containing the target object from the training data set;
从所述至少一个第二图像中分割出所述目标对象的多个实例图。A plurality of instance maps of the target object are segmented from the at least one second image.
结合第二方面，本发明实施例提供了第二方面的第二种可能的实施方式，其中，元素获取模块还用于：In conjunction with the second aspect, the embodiment of the present invention provides a second possible implementation manner of the second aspect, wherein the element acquisition module is further configured to:
从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像，生成候选背景集；Extract the first image that does not contain the target object from the pre-acquired training data set as a background image to generate a candidate background set;
从所述训练数据集中抽取包含目标对象的第二图像，从所述第二图像中分割出所述目标对象的实例图，生成候选实例集；Extract a second image containing the target object from the training data set, segment an instance graph of the target object from the second image, and generate a candidate instance set;
从所述候选背景集中选择一张背景图像，从所述候选实例集中选择多个实例图，将选择的背景图像和多个实例图对应组成实例背景对；所述实例背景对中的实例图和背景图像用于合成一张训练图像。A background image is selected from the candidate background set, multiple instance images are selected from the candidate instance set, and the selected background image and multiple instance images are correspondingly formed into an instance background pair; the instance images in the instance background pair and The background image is used to synthesize a training image.
结合第二方面的第二种可能的实施方式，本发明实施例提供了第二方面的第三种可能的实施方式，其中，元素获取模块还用于：对从所述第二图像中分割出的实例图进行尺度变换；将选择的背景图像、从所述第二图像中分割出的实例图和经尺度变换后的实例图对应组成实例背景对。With reference to the second possible implementation manner of the second aspect, this embodiment of the present invention provides a third possible implementation manner of the second aspect, wherein the element acquisition module is further configured to: Scale transformation is performed on the instance image of the second image; the selected background image, the instance image segmented from the second image, and the instance image after scale transformation correspond to form an instance background pair.
结合第二方面，本发明实施例提供了第二方面的第四种可能的实施方式，其中，所述位置确定模块还用于：In conjunction with the second aspect, the embodiment of the present invention provides a fourth possible implementation manner of the second aspect, wherein the position determination module is further configured to:
将所述多个实例图随机添加至所述背景图像的任意位置；randomly adding the plurality of instance images to any position of the background image;
根据所述多个实例图中的每个实例图在所述背景图像上的位置确定邻近损失值；determining a proximity loss value according to the position of each instance map of the plurality of instance maps on the background image;
根据所述邻近损失值调整所述实例图在所述背景图像上的位置。The position of the instance map on the background image is adjusted according to the proximity loss value.
结合第二方面的第四种可能的实施方式，本发明实施例提供了第二方面的第五种可能的实施方式，其中，所述位置确定模块还用于：With reference to the fourth possible implementation manner of the second aspect, the embodiment of the present invention provides the fifth possible implementation manner of the second aspect, wherein the position determination module is further configured to:
根据每个实例图在所述背景图像上的位置，分别确定整体拉力损失值和整理推力损失值；According to the position of each instance image on the background image, the overall tension loss value and the finishing thrust loss value are respectively determined;
根据所述整体拉力损失值和所述整体推力损失值确定所述邻近损失值。The proximity loss value is determined based on the overall pull loss value and the overall thrust loss value.
结合第二方面的第五种可能的实施方式，本发明实施例提供了第二方面的第六种可能的实施方式，其中，所述位置确定模块还用于：With reference to the fifth possible implementation manner of the second aspect, the embodiment of the present invention provides the sixth possible implementation manner of the second aspect, wherein the position determination module is further configured to:
逐一将所述背景图像上的每个实例图作为当前实例图，计算所述当前实例图与所述背景图像上的其他的每个实例图之间的距离；Taking each instance graph on the background image as the current instance graph one by one, calculating the distance between the current instance graph and each other instance graph on the background image;
将与所述当前实例图距离最近的实例图作为当前实例图的相邻实例图，将所述当前实例图和所述相邻实例图组成一组相近实例图对；Taking the instance graph with the closest distance from the current instance graph as the adjacent instance graph of the current instance graph, and forming a group of close instance graph pairs with the current instance graph and the adjacent instance graph;
将得到的所有相近实例图对组成实例图对集合；All the obtained similar instance graph pairs are formed into instance graph pair sets;
确定所述实例图对集合中每一组相近实例图对的拉力损失值；determining the tensile force loss value of each group of similar instance graph pairs in the instance graph pair set;
将每一组相近实例图对的拉力损失值之和作为整体拉力损失值。The sum of the tension loss values of each group of similar instance graph pairs is taken as the overall tension loss value.
结合第二方面的第五种可能的实施方式，本发明实施例提供了第二方面的第七种可能的实施方式，其中，所述位置确定模块还用于：With reference to the fifth possible implementation manner of the second aspect, the embodiment of the present invention provides the seventh possible implementation manner of the second aspect, wherein the position determination module is further configured to:
将所述背景图像上的任意两个实例图组成一组实例图对，得到多组实例图对；Forming any two instance graphs on the background image into a group of instance graph pairs to obtain multiple groups of instance graph pairs;
从所述多组实例图对中选出所述实例图对集合之外的非相近实例图对，计算每一组非相近实例图对的推力损失值；Select non-similar instance graph pairs other than the instance graph pair set from the multiple groups of instance graph pairs, and calculate the thrust loss value of each group of non-similar instance graph pairs;
将每一组相近实例图对的推力损失值之和，作为整体推力损失值。The sum of the thrust loss values of each group of similar instance graph pairs is taken as the overall thrust loss value.
结合第二方面的第五种可能的实施方式，本发明实施例提供了第二方面的第八种可能的实施方式，其中，所述位置确定模块还用于：With reference to the fifth possible implementation manner of the second aspect, the embodiment of the present invention provides the eighth possible implementation manner of the second aspect, wherein the position determination module is further configured to:
将所述整体拉力损失值与预设系数相乘后加上所述整体推力损失值，得到所述邻近损失值。The adjacent loss value is obtained by multiplying the overall tensile force loss value by a preset coefficient and then adding the overall thrust loss value.
结合第二方面的第四种可能的实施方式，本发明实施例提供了第二方面的第十种可能的实施方式，其中，所述位置确定模块还用于：With reference to the fourth possible implementation manner of the second aspect, the embodiment of the present invention provides the tenth possible implementation manner of the second aspect, wherein the position determination module is further configured to:
确定所述背景图像上的每个实例图的被遮挡面积比；所述被遮挡面积比为所述实例图中被遮挡部分的面积与所述实例图总面积的比值；Determine the occluded area ratio of each example image on the background image; the occluded area ratio is the ratio of the area of the occluded portion in the example image to the total area of the example image;
将所述被遮挡面积比大于设定阈值的实例图删除。The instance graphs whose occluded area ratio is greater than the set threshold are deleted.
结合第二方面的第四种可能的实施方式，本发明实施例提供了第二方面的第十一种可能的实施方式，其中，所述位置确定模块还用于：With reference to the fourth possible implementation manner of the second aspect, the embodiment of the present invention provides an eleventh possible implementation manner of the second aspect, wherein the position determination module is further configured to:
将所述背景图像上的实例图的被遮挡部分的标签设为遮挡。Set the label of the occluded part of the instance graph on the background image as occluded.
第三方面，本发明实施例提供了一种电子设备，包括存储器、处理器；In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor;
所述存储器中存储有可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述第一方面中任一项所述的方法的步骤。A computer program executable on the processor is stored in the memory, and when the processor executes the computer program, the steps of the method according to any one of the above-mentioned first aspects are implemented.
第四方面，本发明实施例提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器运行时执行上述第一方面任一项所述的方法的步骤。In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, any one of the foregoing first aspects is executed. steps of the method.
本发明实施例提供了一种数据生成方法、装置及电子设备，通过将目标对象的实例图添加到背景图像中，合成训练图像的方式，增加训练数据集的数据量，可以提高训练数据的多样性，有利于提高模型的训练效果，增强训练得到的模型的稳定性。The embodiments of the present invention provide a data generation method, device, and electronic equipment. By adding an instance image of a target object to a background image, and synthesizing a training image, the data volume of the training data set can be increased, and the diversity of the training data can be improved. It is beneficial to improve the training effect of the model and enhance the stability of the model obtained by training.
本发明的其他特征和优点将在随后的说明书中阐述，或者，部分特征和优点可以从说明书推知或毫无疑义地确定，或者通过实施本发明的上述技术即可得知。Additional features and advantages of the present invention will be set forth in the description which follows, or some may be inferred or unambiguously determined from the description, or may be learned by practicing the above-described techniques of the present invention.
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below, and are described in detail as follows in conjunction with the accompanying drawings.
附图说明Description of drawings
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the specific embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the specific embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.
图1示出了本发明实施例所提供的一种电子设备的结构示意图；FIG. 1 shows a schematic structural diagram of an electronic device provided by an embodiment of the present invention;
图2示出了本发明实施例所提供的一种数据生成方法的流程图；2 shows a flowchart of a data generation method provided by an embodiment of the present invention;
图3示出了本发明实施例所提供的另一种数据生成方法的流程图；3 shows a flowchart of another data generation method provided by an embodiment of the present invention;
图4示出了本发明实施例所提供的一种数据生成装置的结构框图；4 shows a structural block diagram of a data generating apparatus provided by an embodiment of the present invention;
图5示出了采用本发明实施例所提供的数据生成方法得到的一种训练图像的示意图；5 shows a schematic diagram of a training image obtained by using the data generation method provided by an embodiment of the present invention;
图6示出了采用本发明实施例所提供的数据生成方法得到的另一种训练图像的示意图。FIG. 6 shows a schematic diagram of another training image obtained by using the data generation method provided by the embodiment of the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合附图对本发明的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. example. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
为了改善利用现有的训练数据集训练得到的模型在处理复杂背景图像是稳定性不足的问题，本发明实施例提供了一种数据生成方法、装置、电子设备和计算机存储介质。以下结合具体实施例和附图对本发明实施例提供的数据生成方法、装置、电子设备和计算机存储介质进行详细介绍。In order to improve the problem that the model obtained by using the existing training data set has insufficient stability in processing complex background images, the embodiments of the present invention provide a data generation method, apparatus, electronic device and computer storage medium. The data generation method, apparatus, electronic device, and computer storage medium provided by the embodiments of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.
实施例一：Example 1:
首先，参照图1来描述用于实现本发明实施例的数据生成方法的示例电子设备100。该示例电子设备100可以是计算机或服务器，也可以是其它的电子设备，本发明不作具体限定。First, an example electronic device 100 for implementing the data generation method of an embodiment of the present invention is described with reference to FIG. 1 . The exemplary electronic device 100 may be a computer or a server, or may be other electronic devices, which are not specifically limited in the present invention.
如图1所示，电子设备100包括一个或多个处理器102、一个或多个存储器104、输入装置106、输出装置108以及通讯装置110，这些组件通过总线系统112和/或其它形式的连接机构(未示出)互连。应当注意，图1所示的电子设备100的组件和结构只是示例性的，而非限制性的，根据需要，所述电子设备也可以具有其他组件和结构。As shown in FIG. 1, electronic device 100 includes one or more processors 102, one or more memories 104, input devices 106, output devices 108, and communication devices 110, these components are connected by a bus system 112 and/or other forms Mechanisms (not shown) are interconnected. It should be noted that the components and structures of the electronic device 100 shown in FIG. 1 are only exemplary and not restrictive, and the electronic device may also have other components and structures as required.
所述处理器102可以是中央处理器(CPU)、图形处理器(Graphics ProcessingUnit，GPU)或者具有数据处理能力、图像处理能力和/或指令执行能力的其它形式的处理单元，并且可以控制所述电子设备100中的其它组件以执行期望的功能。The processor 102 may be a central processing unit (CPU), a graphics processing unit (Graphics Processing Unit, GPU), or other forms of processing units with data processing capability, image processing capability, and/or instruction execution capability, and may control the other components in the electronic device 100 to perform the desired functions.
所述存储器104可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器102可以运行所述程序指令，以实现下文所述的本发明实施例中(由处理器实现)的图像分割功能以及/或者其它期望的功能。在所述计算机可读存储介质中还可以存储各种应用程序和各种数据，例如所述应用程序使用和/或产生的各种图像等。The memory 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like. The non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the image segmentation function (implemented by the processor) in the embodiments of the present invention described below and/or other desired functionality. Various application programs and various data, such as various images used and/or generated by the application program, may also be stored in the computer-readable storage medium.
所述输入装置106可以是用户用来输入指令的装置，并且可以包括键盘、鼠标、麦克风和触摸屏等中的一个或多个。The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.
所述输出装置108可以向外部(例如，用户)输出各种信息(例如，图像或声音)，并且可以包括显示器、扬声器等中的一个或多个。The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.
所述通讯装置110可以包括数据传输接口或网络接口，用于与其它电子设备及其它网络单元连接。例如，电子设备100可以通过通讯装置110连接远程服务器，从远程服务器上下载训练数据集。The communication device 110 may include a data transmission interface or a network interface for connecting with other electronic devices and other network units. For example, the electronic device 100 may connect to a remote server through the communication device 110, and download the training data set from the remote server.
可选地，电子设备100还可以包括图像采集装置。图像采集装置可以拍摄用户期望的图像(例如照片、视频等)，并且将所拍摄的图像存储在所述存储装置中以供其它组件使用。Optionally, the electronic device 100 may further include an image capturing device. The image capture device may capture images desired by the user (eg, photos, videos, etc.) and store the captured images in the storage device for use by other components.
实施例二：Embodiment 2:
本实施例提供了一种数据生成方法，可以增加训练数据的数据量和训练数据的多样性。图2示出了该数据生成方法的流程图。该需要说明的是，在图2的流程图中示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。以下对本实施例进行详细介绍。This embodiment provides a data generation method, which can increase the data volume of training data and the diversity of training data. FIG. 2 shows a flow chart of the data generation method. It should be noted that the steps shown in the flowchart of FIG. 2 may be performed in a computer system, such as a set of computer-executable instructions, and although a logical sequence is shown in the flowchart, in some cases In the following, steps shown or described may be performed in an order different from that herein. This embodiment will be described in detail below.
如图2所示，本实施例提供的数据生成方法，包括如下步骤：As shown in Figure 2, the data generation method provided by this embodiment includes the following steps:
步骤S202，获取背景图像，并获取目标对象的多个实例图。In step S202, a background image is acquired, and multiple instance images of the target object are acquired.
其中，背景图像为不包含目标对象的场景图像。背景图像可以是电子设备通过图像采集装置采集的，也可以是从网络上搜集的，或者从远程服务器下载的。The background image is a scene image that does not contain the target object. The background image may be collected by an electronic device through an image collection device, or may be collected from a network, or downloaded from a remote server.
目标对象根据待训练的模型的用途不同而不同，目标对象可以是但不限于行人、车辆、动物、植物或其它感兴趣的目标等，目标对象还可以是动物的一部分或植物的一部分。例如，如果待训练的模型用于进行人体姿态估计(Human pose estimation)，则目标对象可以是人体，如处于不同姿态的人体。如果待训练的模型用于进行车辆检测，则目标对象可以是车辆。The target object varies according to the purpose of the model to be trained. The target object can be, but is not limited to, pedestrians, vehicles, animals, plants, or other objects of interest. The target object can also be a part of an animal or a part of a plant. For example, if the model to be trained is used for human pose estimation, the target object may be a human body, such as a human body in different poses. If the model to be trained is used for vehicle detection, the target object can be a vehicle.
目标对象的实例图可以采用图像分割的方法从包含目标对象的图像中分割得到。目标对象的实例图可以理解为图像中的目标对象区域，采用如掩膜网络可以从图像中分割出目标对象的实例图。包含目标对象的图像可以通过图像采集装置采集，也可以从网络上搜集得到，或者从远程服务器下载。The instance graph of the target object can be obtained from the image containing the target object by using the method of image segmentation. The instance graph of the target object can be understood as the target object area in the image, and the instance graph of the target object can be segmented from the image by using, for example, a mask network. The image containing the target object can be collected by an image acquisition device, or can be collected from the network, or downloaded from a remote server.
在一种可选的实施例中，可以从远程服务器下载公开的训练数据集，从预先获取的训练数据集中分别抽取不包含目标对象的第一图像和包含目标对象的第二图像，将第一图像作为背景图像。从第二图像中分割出目标对象的多个实例图，例如，可以采用现有的图像分割方法(如掩膜网络)从第二图像中分割出目标对象的实例图。如果第二图像中包含多个目标对象，则可以从第二图像中分割出目标对象的多个实例图。如果第二图像中仅包含一个目标对象区域，则可以从第二图像中分割出目标对象的一个实例图，此时，可以从训练数据集中抽取多个第二图像，从每个第二图像中均可以分离出目标对象的实例图，从而可以得到目标对象的多个实例图。In an optional embodiment, a public training data set may be downloaded from a remote server, a first image not containing the target object and a second image containing the target object are respectively extracted from the pre-acquired training data set, and the first image image as background image. A plurality of instance graphs of the target object are segmented from the second image. For example, an existing image segmentation method (such as a mask network) can be used to segment instance graphs of the target object from the second image. If the second image contains multiple target objects, multiple instance graphs of the target objects can be segmented from the second image. If the second image contains only one target object area, an instance graph of the target object can be segmented from the second image. At this time, multiple second images can be extracted from the training data set, and each second image can be extracted from The instance graph of the target object can be separated, so that multiple instance graphs of the target object can be obtained.
还可以对得到的实例图进行尺度变换，得到不同尺度的实例图，添加到背景图像中，可以进一步增加训练数据的多样性。The obtained instance graph can also be scaled to obtain instance graphs of different scales, which can be added to the background image to further increase the diversity of training data.
在另一种可选的实施例中，可以从远程服务器下载公开的训练数据集，从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像，生成候选背景集；从训练数据集中抽取包含目标对象的第二图像，从第二图像中分割出目标对象的实例图，生成候选实例集；从候选背景集中选择一张背景图像，从候选实例集中选择多个实例图，将选择的背景图像和多个实例图对应组成实例背景对；所述实例背景对中的实例图和背景图像用于合成一张训练图像。In another optional embodiment, a public training data set may be downloaded from a remote server, a first image that does not contain the target object is extracted from the pre-acquired training data set as a background image, and a candidate background set is generated; Centrally extract the second image containing the target object, segment the instance graph of the target object from the second image, and generate a candidate instance set; select a background image from the candidate background set, select multiple instance graphs from the candidate instance set, and select The background image and multiple instance images corresponding to form an instance background pair; the instance image and the background image in the instance background pair are used to synthesize a training image.
步骤S204，将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置，使所述多个实例图中任意两个实例图之间的距离在预设范围内。Step S204, adding the multiple instance images to the background image, and determining the positions of the multiple instance images, so that the distance between any two instance images in the multiple instance images is within a preset range .
为了使实例图之间的距离更适当，使不同的实例图之间既不大面积重叠，也不分离的过远，可以采用如下的方法设置实例图在背景图像上的位置：先将所有实例图随机添加至背景图像的任意位置，其中，实例图位于背景图像的上方。然后根据每个实例图在背景图像上的位置确定邻近损失值，根据邻近损失值调整实例图在背景图像上的位置。其中，邻近损失值包括推力损失值和拉力损失值，推力损失值用于控制各个实例图之间的距离不要太远，拉力损失值用于控制各个相邻的实例图之间的重叠程度不要太大。通过推力损失值和拉力损失值，可以将多个实例图中任意两个实例图之间的距离控制在预设范围内，即两个实例图之间不会离得太远，也不会重叠程度太大。In order to make the distance between the instance graphs more appropriate, so that the different instance graphs do not overlap in a large area, nor are they separated too far, you can use the following method to set the position of the instance graphs on the background image: Graphs are randomly added anywhere on the background image, with the instance graph on top of the background image. Then, the proximity loss value is determined according to the position of each instance graph on the background image, and the position of the instance graph on the background image is adjusted according to the proximity loss value. Among them, the adjacent loss value includes the thrust loss value and the pull loss value. The thrust loss value is used to control the distance between each instance graph not too far, and the pull loss value is used to control the overlap between the adjacent instance graphs not too much. big. The distance between any two instance graphs in multiple instance graphs can be controlled within a preset range through the thrust loss value and the pull force loss value, that is, the two instance graphs will not be too far apart and will not overlap. Too much.
步骤S206，按照确定的位置将所述多个实例图和所述背景图像合成为训练图像。Step S206, synthesizing the multiple instance images and the background image into a training image according to the determined position.
确定实例图在背景图像上的位置之后，可以将实例图的像素直接覆盖在背景图像上，将多个实例图与背景图像合成为训练图像。After determining the position of the instance graph on the background image, the pixels of the instance graph can be directly overlaid on the background image, and multiple instance graphs and the background image can be synthesized into a training image.
在一种应用场景中，可以将训练图像添加至训练数据集，用于对模型进行训练。所述训练数据集用于对模型进行训练。通过上述方法可以生成大量包含目标对象的训练图像，将生成的训练图像添加至训练数据集中，可以改善原有的训练数据集包含大比例负样本的情况。负样本指不包含目标对象的训练图像。In one application scenario, training images can be added to the training dataset for training the model. The training dataset is used to train the model. Through the above method, a large number of training images containing target objects can be generated, and the generated training images can be added to the training data set, which can improve the situation that the original training data set contains a large proportion of negative samples. Negative samples refer to training images that do not contain the target object.
本发明实施例提供的数据生成方法，通过将目标对象的实例图添加到背景图像中，合成训练图像的方式，增加训练数据集的数据量，可以提高训练数据的多样性，有利于提高模型的训练效果，增强训练得到的模型的稳定性。本发明实施例并不对具体的应用场景作限制，利用本发明实施例得到的训练数据集训练的模型可以应用于各种不同的实际应用场景中。The data generation method provided by the embodiment of the present invention increases the data volume of the training data set by adding the instance image of the target object to the background image and synthesizing the training image, which can improve the diversity of the training data, which is conducive to improving the model's performance. The training effect enhances the stability of the model obtained by training. The embodiments of the present invention do not limit specific application scenarios, and the model trained by using the training data set obtained in the embodiments of the present invention can be applied to various practical application scenarios.
实施例三：Embodiment three:
在上述方法实施例的基础上，本发明实施例还提供了另一种数据生成方法，如图3所示，该方法包括如下步骤：On the basis of the above method embodiment, the embodiment of the present invention also provides another data generation method, as shown in FIG. 3 , the method includes the following steps:
步骤S302，从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像，生成候选背景集。Step S302, extracting a first image that does not contain the target object from a pre-acquired training data set as a background image to generate a candidate background set.
训练数据集可以是从网络或远程服务器上下载的，例如，可以从微软官方网站上下载MS COCO数据集作为训练数据集。从训练数据集中的不包含目标对象的图像中抽取一部分或全部，生成候选背景集。The training dataset can be downloaded from the network or a remote server, for example, the MS COCO dataset can be downloaded from Microsoft's official website as a training dataset. Extract some or all of the images in the training dataset that do not contain the target object to generate a candidate background set.
步骤S304，从训练数据集中抽取包含目标对象的第二图像，从第二图像中分割出目标对象的实例图，生成候选实例集。Step S304, extracting a second image containing the target object from the training data set, segmenting an instance graph of the target object from the second image, and generating a candidate instance set.
从训练数据集中的包含目标对象的图像中抽取一部分或全部，作为第二图像。采用图像分割方法从每张第二图像中的分割出目标对象的实例图。根据分割得到的所有实例图生成候选实例集。Part or all of the images containing the target object in the training dataset are extracted as the second image. An instance map of the target object is segmented from each second image using an image segmentation method. A candidate instance set is generated from all instance graphs obtained by segmentation.
步骤S306，从候选背景集中选择一张背景图像，从候选实例集中选择多个实例图，将选择的背景图像和多个实例图对应组成实例背景对。Step S306, select a background image from the candidate background set, select multiple instance images from the candidate instance set, and form an instance background pair corresponding to the selected background image and the multiple instance images.
所述实例背景对中的实例图和背景图像用于合成一张训练图像。为了得到多种尺度的实例图，可以对从第二图像中分割出的实例图进行尺度变换，将选择的背景图像、从第二图像中分割出的实例图和经尺度变换后的实例图对应组成实例背景对。例如，可以采用傅里叶变换或可变高斯函数对实例图进行尺度变换，使得实例图平均每个关节点拥有400个像素。The instance map and the background image in the instance-background pair are used to synthesize a training image. In order to obtain instance graphs of various scales, scale transformation can be performed on the instance graphs segmented from the second image, and the selected background image, instance graphs segmented from the second image and the instance graphs after scale transformation correspond to Consists of instance-background pairs. For example, a Fourier transform or a variable Gaussian function can be used to scale the instance graph so that the instance graph has an average of 400 pixels per joint point.
重复执行步骤S306，可以得到多组实例背景对。对于每一组实例背景对，均可以按照下述步骤S308至步骤S314得到一张训练图像。Step S306 is repeatedly executed to obtain multiple sets of instance-background pairs. For each group of instance-background pairs, a training image can be obtained according to the following steps S308 to S314.
步骤S308，将实例背景对中的实例图随机添加至该实例背景对的背景图像的任意位置。Step S308, randomly adding the instance image in the instance background pair to any position of the background image of the instance background pair.
例如，可以用(Insts,Bg)表示一组实例背景对，其中，Bg代表背景图像，Insts代表实例背景对中的多个实例图组成的实例图集合。例如，Insts中可以包括K个实例图，表示为Insts＝{p1,p2,……pK}，pK为第K个实例图。For example, a set of instance-background pairs can be represented by (Insts, Bg ), where Bg represents a background image, and Insts represents a set of instance graphs composed of multiple instance graphs in the instance-background pair. For example, Insts may include K instance graphs, which are represented as Insts={p1 , p2 , ...... pK }, where pK is the K th instance graph.
步骤S310，根据每个实例图在背景图像上的位置确定邻近损失值。Step S310: Determine the proximity loss value according to the position of each instance image on the background image.
邻近损失值(ProximityLoss)包括两部分，第一部分为整体推力损失值(pushLoss)，目的是让各个实例图之间不要距离太远。第二部分为整体拉力损失值(pullLoss)，目的是将最邻近的两个实例图的重叠程度控制在一个给定的范围[T1，T2]内。可以采用如下方法确定邻近损失值：根据每个实例图在背景图像上的位置，分别确定整体拉力损失值和整理推力损失值；根据整体拉力损失值和整体推力损失值确定邻近损失值。以下分别介绍确定整体拉力损失值和整理推力损失值的步骤。The proximity loss value (ProximityLoss) consists of two parts, the first part is the overall thrust loss value (pushLoss), the purpose is to make the distance between each instance graph not too far. The second part is the overall pull loss value (pullLoss), which aims to control the overlapping degree of the two nearest instance graphs within a given range [T1 , T2 ]. The following methods can be used to determine the adjacent loss value: according to the position of each instance image on the background image, determine the overall tension loss value and the finishing thrust loss value respectively; determine the adjacent loss value according to the overall tension loss value and the overall thrust loss value. The steps for determining the overall pull loss value and for sorting the thrust loss value are described below.
根据每个实例图在背景图像上的位置确定整体拉力损失值，可以包括如下步骤：Determine the overall tension loss value based on the position of each instance image on the background image, which can include the following steps:
(1)逐一将背景图像上的每个实例图作为当前实例图，计算当前实例图与背景图像上的每个实例图之间的距离；将与所述当前实例图距离最近的实例图作为当前实例图的相邻实例图，将当前实例图和相邻实例图组成一组相近实例图对。(1) Take each instance graph on the background image as the current instance graph one by one, and calculate the distance between the current instance graph and each instance graph on the background image; take the instance graph closest to the current instance graph as the current instance graph The adjacent instance graph of the instance graph, which combines the current instance graph and the adjacent instance graph into a set of similar instance graph pairs.
可以采用如下方法计算背景图像上的两个实例图之间的距离。例如，对于第i个实例图pi，其位置可以表示为：其中，为第i个实例图的左上角坐标，为第i个实例图的右下角坐标。为实例图pi的面积。The distance between two instance images on the background image can be calculated as follows. For example, for the i-th instance graph pi , its position can be expressed as: in, is the coordinate of the upper left corner of the i-th instance graph, is the coordinate of the lower right corner of the i-th instance graph. is the area of the instance graphpi .
定义矩阵RD∈RK*K*2，其中RDi,j表示实例图pi与实例图pj的右下角之间的最近距离，该距离包括横向距离和纵向距离。定义矩阵LU∈RK*K*2，其中LUi,j表示实例图pi与实例图pj的左上角之间的最远距离，该距离包括横向距离和纵向距离。定义矩阵DIFF＝RD-LU，DIFFi,j反映了实例图pi与实例图pj的位置关系。若DIFFi,j,1＞0，代表实例图pi与实例图pj在横轴是相交的，若DIFFi,j,2＞0，代表实例图pi与实例图pj在纵轴是相交的，反之，若DIFFi,j,1＜0，代表实例图pi与实例图pj在横轴不相交，DIFFi,j,1越小，说明实例图pi与实例图pj横向相距越远。若DIFFi,j,2＜0，代表实例图pi与实例图pj在纵轴不相交，DIFFi,j,2越小，说明实例图pi与实例图pj纵向相距越远。定义距离矩阵Distance∈RK*K。Define the matrix RD∈RK*K*2 , where RDi,j represents theshortest distance between the instance graph pi and the lower right corner of the instance graph p j, and the distance includes a horizontal distance and a vertical distance. Define the matrix LU∈RK*K*2 , where LUi,j represents the farthest distance between the instance graph pi and the upper left corner of the instance graph pj , and the distance includes a horizontal distance and a vertical distance. The definition matrix DIFF=RD-LU, DIFFi,j reflects the positional relationship between the instance graph pi and the instance graph pj . If DIFFi,j,1 > 0, it means that the instance graph pi and instance graph pj intersect on the horizontal axis; if DIFFi,j,2> 0, it means that the instance graph pi and instance graph pjare on the vertical axis On the contrary, if DIFFi,j,1 < 0, it means that the instance graph pi and the instance graph pj do not intersect on the horizontal axis, and the smaller the DIFFi,j,1is , the case graphpi and the instance graph pj is farther apart laterally. If DIFFi,j,2 < 0, it means that the instance graph pi and the instance graph pj do not intersect on the vertical axis. The smaller the DIFFi,j,2 , the farther the instance graphpi and the instance graph pj are vertically. Define the distance matrix Distance∈RK*K .
Distance＝-DIFFi,j,1-DIFFi,j,2-102×interi,j+103×EDistance=-DIFFi,j,1 -DIFFi,j,2 -102 ×interi,j +103 ×E
其中，interi,j表示实例图pi与实例图pj之间重叠部分的面积。E为预设的单位矩阵。-102×interi,j可以保证相交的实例图之间的距离小于不相交的实例图之间的距离。103×E可以保证实例图与自身的距离远大于与其他实例图之间的距离。Among them, interi,j represents the area of the overlapping part between the instance graph pi and the instance graph pj . E is the preset identity matrix. -102 ×interi,j can guarantee that the distance between intersecting instance graphs is smaller than the distance between disjoint instance graphs. 103 ×E can ensure that the distance between the instance graph and itself is much greater than the distance between other instance graphs.
对于任意一个实例图pi，将作为实例图pi当前实例图，找到一个距离实例图pi最近的实例图pǐ，实例图pǐ可以表示为pǐ＝argminj Distancei，Distancei∈RK，Distancei代表实例图pi与实例图集合Insts中的所有实例图(即背景图像上的每个实例图)之间的距离。将实例图pǐ作为实例图pi的相邻实例图，将实例图pi和实例图pǐ组成一组相近实例图对。For any instance graph pi , it will be used as the current instance graph of instance graphpi , and find an instance graph pǐ that is closest to the instance graph pi , and the instance graph pǐ can be expressed as pǐ =argminj Distancei , Distancei∈ RK , Distancei represents the distance between the instance graphpi and all instance graphs in the instance graph set Insts (ie, each instance graph on the background image). The instance graphpǐ is regarded as the adjacent instance graph of the instance graph pi, and the instance graph p iand the instance graphpǐ are formed into a set of similar instance graph pairs.
(2)将得到的所有相近实例图对组成实例图对集合。(2) All the obtained similar instance graph pairs are formed into instance graph pair sets.
按照步骤(1)的方法，最终可以得到K对相近实例图对，可以组成实例图对集合According to the method of step (1), K pairs of similar instance graph pairs can be finally obtained, which can form a set of instance graph pairs
(3)确定实例图对集合中每一组相近实例图对的拉力损失值。(3) Determine the tension loss value of each group of similar instance graph pairs in the instance graph pair set.
对于实例图对集合PInsts中的每一组相近实例图对，可以通过如下公式确定一组相近实例图对的拉力损失值pullLossi,ǐ。For each group of similar instance graph pairs in the instance graph pair set PInsts, the pullLossi,ǐ of a group of similar instance graph pairs can be determined by the following formula.
(4)将每一组相近实例图对的拉力损失值之和作为整体拉力损失值。(4) The sum of the tensile force loss values of each group of similar example pairs is taken as the overall tensile force loss value.
整体拉力损失值pullLoss可以表示为：The overall pull loss value pullLoss can be expressed as:
根据每个实例图在背景图像上的位置确定整体推力损失值，可以包括如下步骤：Determine the overall thrust loss value based on the position of each instance map on the background image, which can include the following steps:
(a)将背景图像上的任意两个实例图组成一组实例图对，得到多组实例图对。(a) Composing any two instance graphs on the background image into a set of instance graph pairs to obtain multiple sets of instance graph pairs.
也可以理解为，将实例图集合Insts中的任意两个实例图组成一组实例图对，共可以得到组实例图对。It can also be understood that by combining any two instance graphs in the instance graph set Insts into a set of instance graph pairs, a total of Group instance graph pairs.
(b)从多组实例图对中选出实例图对集合之外的非相近实例图对，计算每一组非相近实例图对的推力损失值。(b) Select non-similar instance graph pairs out of the instance graph pair set from multiple groups of instance graph pairs, and calculate the thrust loss value of each group of non-similar instance graph pairs.
从步骤(a)中得到的多组实例图对中选出所有不属于实例图对集合PInsts的实例图对，作为非相近实例图对，计算每一组非相近实例图对的推力损失值。其中，一组非相近实例图对的推力损失值pullLossi,j＝interi,j。All instance graph pairs that do not belong to the instance graph pair set PInsts are selected from the multiple groups of instance graph pairs obtained in step (a) as non-similar instance graph pairs, and the thrust loss value of each group of non-similar instance graph pairs is calculated. Among them, the thrust loss value pullLossi,j =interi,j of a set of non-similar instance graph pairs.
(c)将每一组非相近实例图对的推力损失值之和，作为整体推力损失值。(c) Take the sum of the thrust loss values of each pair of non-similar instance diagrams as the overall thrust loss value.
整体推力损失值pushLoss可以表示为：The overall thrust loss value pushLoss can be expressed as:
将整体拉力损失值pullLoss与预设系数相乘后加上整体推力损失值pushLoss，可以得到邻近损失值ProximintyLoss。邻近损失值ProximintyLoss可以表示为：ProximintyLoss＝pushLoss+λ×pullLoss，其中，λ为预设系数。例如，在一具体实施例中，λ＝10。Multiply the overall pull loss value pullLoss by the preset coefficient and add the overall thrust loss value pushLoss to obtain the adjacent loss value ProximintyLoss. The proximity loss value ProximintyLoss can be expressed as: ProximintyLoss=pushLoss+λ×pullLoss, where λ is a preset coefficient. For example, in a specific embodiment, λ=10.
步骤S312，根据邻近损失值调整实例图在背景图像上的位置。Step S312, adjust the position of the instance image on the background image according to the adjacent loss value.
根据邻近损失值ProximintyLoss，采用随机梯度下降法(Stochastic gradientdescent，SGD)，调整实例图在背景图像上的位置，使得邻近损失值ProximintyLoss最小化，将背景图像上距离最近的两个实例图的重叠程度控制在预设的重叠范围内，同时控制各个实例图之间的距离不超过预设的距离范围，从而可以得到较理想的训练图像。例如，采用随机梯度下降法，可以通过迭代的方式使邻近损失值不断减小。在一具体实施例中，使用Adam优化器，将迭代步长(或称学习率)设定为0.01，最大迭代次数设定为5000次，同时设定若连续5次迭代的邻近损失值不变，可以提前终止迭代过程。According to the proximity loss value ProximintyLoss, the stochastic gradient descent (SGD) method is used to adjust the position of the instance graph on the background image, so that the proximity loss value ProximintyLoss is minimized, and the overlap degree of the two closest instance graphs on the background image is reduced. It is controlled within a preset overlapping range, and at the same time, the distance between each instance image is controlled not to exceed the preset distance range, so that an ideal training image can be obtained. For example, using stochastic gradient descent, the adjacent loss value can be continuously reduced in an iterative manner. In a specific embodiment, the Adam optimizer is used, the iteration step size (or learning rate) is set to 0.01, the maximum number of iterations is set to 5000 times, and the adjacent loss value is set to remain unchanged for 5 consecutive iterations. , the iterative process can be terminated early.
经过上述步骤S310和步骤S312，可以使实例图在背景图像中分布地更均匀。After the above steps S310 and S312, the instance images can be distributed more evenly in the background image.
步骤S314，对实例图进行调优，将调优后的实例图和背景图像合成为训练图像。In step S314, the instance graph is optimized, and the optimized instance graph and the background image are synthesized into a training image.
在一种可选的实施例中，通过步骤S312确定实例图在背景图像上的位置之后，可以删除不符合要求的实例图。例如，删除被遮挡面积较大的实例图。具体地，确定背景图像上的每个实例图的被遮挡面积比，所述被遮挡面积比为所述实例图中被遮挡部分的面积与所述实例图总面积的比值；将被遮挡面积比大于设定阈值的实例图删除。示例性地，设定阈值根据需要可以设定为0.1或0.2。In an optional embodiment, after the position of the instance graph on the background image is determined in step S312, the instance graph that does not meet the requirements may be deleted. For example, delete instance maps with large occluded areas. Specifically, the occluded area ratio of each instance image on the background image is determined, and the occluded area ratio is the ratio of the area of the occluded portion in the instance graph to the total area of the instance graph; the occluded area ratio is Instance graphs larger than the set threshold are deleted. Exemplarily, the set threshold may be set to 0.1 or 0.2 as required.
在另一种可选的实施例中，通过步骤S312确定实例图在背景图像上的位置之后，可以将背景图像上的实例图的被遮挡部分的标签设为遮挡。例如，原本可见的关节点，在实例图位置调整后，可能被其它实例图遮挡。此时，可以将被遮挡的关节点的标签由可见改为遮挡。In another optional embodiment, after the position of the instance graph on the background image is determined in step S312, the label of the shaded part of the instance graph on the background image may be set as occluded. For example, the originally visible joint points may be occluded by other instance graphs after the instance graph position is adjusted. At this point, the labels of the occluded joint points can be changed from visible to occluded.
经过上述调优步骤之后，可以将实例图的未被遮挡部分(可见部分)的像素直接覆盖在背景图像上，将实例图与背景图像合成为训练图像。After the above-mentioned tuning steps, the pixels of the unoccluded part (visible part) of the instance graph can be directly covered on the background image, and the instance graph and the background image can be synthesized into a training image.
可以理解的是，在一部分实施例中，也可以不进行调优，在步骤S312之后，直接将调整位置后的实例图与背景图像合成为训练图像。在另一部分实施例中，还可以不执行步骤S310和步骤S312，将实例背景对中的实例图随机添加至该实例背景对的背景图像中，即可将实例图与背景图像合成为训练图像。It can be understood that, in some embodiments, tuning may not be performed, and after step S312 , the instance image and the background image after the position adjustment are directly synthesized into a training image. In another part of the embodiment, steps S310 and S312 may not be performed, and the instance image in the instance background pair is randomly added to the background image of the instance background pair, so that the instance image and the background image can be synthesized into a training image.
在一具体实施例中，根据上述步骤可以得到图5所示的训练图像。例如，在MS COCO数据集中，可以分别收集不包含人的图像和包含人的图像，从包含人的图像中分割出人的实例图。从不包含人的图像中选择一幅图像作为背景图像，从人的实例图中选择四个实例图。将四个实例图随机地添加至背景图像上的任意位置，即可得到图5中(a)所示的训练图像。根据上述步骤S310和步骤S312对实例图进行位置规划，进一步调整四个实例图在背景图像上的位置，使四个实例图的相对位置更合理，可以得到图5中(b)所示的训练图像。In a specific embodiment, the training image shown in FIG. 5 can be obtained according to the above steps. For example, in the MS COCO dataset, images that do not contain people and images that contain people can be collected separately, and the instance graphs of people can be segmented from the images that contain people. One image is selected as the background image from images that do not contain people, and four instance maps are selected from the instance maps of people. The training image shown in (a) in Figure 5 can be obtained by randomly adding four instance images to any position on the background image. According to the above steps S310 and S312, the location planning of the instance graphs is performed, and the positions of the four instance graphs on the background image are further adjusted to make the relative positions of the four instance graphs more reasonable, and the training shown in (b) in FIG. 5 can be obtained. image.
在另一具体实施例中，采用矩形框表示目标对象的实例图，通过上述方法可以得到图6所示的训练图像。图6中的(a)表示将实例图随机地添加至背景图像上的任意位置得到的训练图像，图6中的(b)表示对实例图进行进一步位置规划后得到的训练图像。In another specific embodiment, a rectangular frame is used to represent the instance graph of the target object, and the training image shown in FIG. 6 can be obtained by the above method. (a) in Figure 6 represents a training image obtained by randomly adding an instance map to any position on the background image, and (b) in Figure 6 represents a training image obtained after further location planning is performed on the instance map.
通过上述方法，可以根据需要得到多张训练图像，将训练图像添加至训练数据集，对模型进行训练，可以增加训练数据集的数据量，提高训练数据的多样性，有利于提高模型的训练效果，采用训练得到的模型处理复杂背景的图像时，可以增强模型的稳定性，使模型输出的结果更加可靠。Through the above method, multiple training images can be obtained as needed, the training images can be added to the training data set, and the model can be trained, which can increase the data volume of the training data set, improve the diversity of training data, and help improve the training effect of the model. , when the model obtained by training is used to process images with complex backgrounds, the stability of the model can be enhanced, and the results of the model output are more reliable.
实施例四：Embodiment 4:
对应于上述方法实施例，本实施例提供了一种数据生成装置，参见图4所示的一种数据生成装置的结构示意图，该装置包括：Corresponding to the above method embodiments, the present embodiment provides a data generation device. Referring to the schematic structural diagram of a data generation device shown in FIG. 4 , the device includes:
元素获取模块41，用于获取背景图像，并获取目标对象的多个实例图；The element acquisition module 41 is used to acquire the background image and acquire multiple instance diagrams of the target object;
位置确定模块42，用于将所述多个实例图添加至所述背景图像中，确定所述多个实例图的位置，使所述多个实例图中任意两个实例图之间的距离在预设范围内；The position determination module 42 is configured to add the multiple instance images to the background image, determine the positions of the multiple instance images, and make the distance between any two instance images in the multiple instance images within within the preset range;
数据生成模块43，用于按照确定的位置将所述多个实例图和所述背景图像合成为训练图像。The data generation module 43 is configured to synthesize the plurality of instance images and the background image into a training image according to the determined position.
在一种可选的实施例中，元素获取模块41还可以用于：从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像；从所述训练数据集中抽取包含目标对象的至少一个第二图像；从所述至少一个第二图像中分割出所述目标对象的多个实例图。In an optional embodiment, the element acquisition module 41 may also be used to: extract a first image that does not contain the target object from a pre-acquired training data set as a background image; extract a first image containing the target object from the training data set at least one second image; segmenting a plurality of instance maps of the target object from the at least one second image.
在另一种可选的实施例中，元素获取模块41还可以用于：从预先获取的训练数据集中抽取不包含目标对象的第一图像作为背景图像，生成候选背景集；从所述训练数据集中抽取包含目标对象的第二图像，从所述第二图像中分割出所述目标对象的实例图，生成候选实例集；从所述候选背景集中选择一张背景图像，从所述候选实例集中选择多个实例图，将选择的背景图像和多个实例图对应组成实例背景对；所述实例背景对中的实例图和背景图像用于合成一张训练图像。元素获取模块41还可以用于：对从所述第二图像中分割出的实例图进行尺度变换；将选择的背景图像、从所述第二图像中分割出的实例图和经尺度变换后的实例图对应组成实例背景对。In another optional embodiment, the element acquisition module 41 may also be used to: extract a first image that does not contain the target object from a pre-acquired training data set as a background image, and generate a candidate background set; Centrally extract a second image containing the target object, segment the instance graph of the target object from the second image, and generate a candidate instance set; select a background image from the candidate background set, and select a background image from the candidate instance set. Multiple instance images are selected, and the selected background image and the multiple instance images are correspondingly formed into an instance background pair; the instance image and the background image in the instance background pair are used to synthesize a training image. The element acquisition module 41 can also be used to: perform scale transformation on the instance image segmented from the second image; The instance graph corresponds to the composition instance background pair.
可选地，位置确定模块42还可以用于：将所述多个实例图随机添加至所述背景图像的任意位置；根据所述多个实例图中的每个实例图在所述背景图像上的位置确定邻近损失值；根据所述邻近损失值调整所述实例图在所述背景图像上的位置。Optionally, the position determination module 42 can also be used to: randomly add the multiple instance graphs to any position of the background image; according to each instance graph of the multiple instance graphs on the background image The position of , determines a proximity loss value; the position of the instance map on the background image is adjusted according to the proximity loss value.
位置确定模块42还可以用于：根据每个实例图在所述背景图像上的位置，分别确定整体拉力损失值和整理推力损失值；根据所述整体拉力损失值和所述整体推力损失值确定所述邻近损失值。The position determination module 42 can also be used to: determine the overall pull loss value and the finishing thrust loss value respectively according to the position of each instance map on the background image; determine the overall pull loss value and the overall thrust loss value according to the the proximity loss value.
位置确定模块42还可以用于：逐一将所述背景图像上的每个实例图作为当前实例图，计算所述当前实例图与所述背景图像上的每个实例图之间的距离；将与所述当前实例图距离最近的实例图作为当前实例图的相邻实例图，将所述当前实例图和所述相邻实例图组成一组相近实例图对；将得到的所有相近实例图对组成实例图对集合；确定所述实例图对集合中每一组相近实例图对的拉力损失值；将每一组相近实例图对的拉力损失值之和作为整体拉力损失值。以及用于：将所述背景图像上的任意两个实例图组成一组实例图对，得到多组实例图对；从所述多组实例图对中选出所述实例图对集合之外的非相近实例图对，计算每一组非相近实例图对的推力损失值；将每一组相近实例图对的推力损失值之和，作为整体推力损失值。以及用于：将所述整体拉力损失值与预设系数相乘后加上所述整体推力损失值，得到所述邻近损失值。The position determination module 42 can also be used to: take each instance graph on the background image as the current instance graph one by one, and calculate the distance between the current instance graph and each instance graph on the background image; The instance graph with the closest distance to the current instance graph is used as the adjacent instance graph of the current instance graph, and the current instance graph and the adjacent instance graph are formed into a group of similar instance graph pairs; all the obtained similar instance graph pairs are composed of A set of instance graph pairs; determining the tensile force loss value of each group of similar instance graph pairs in the instance graph pair set; taking the sum of the tensile force loss values of each group of similar instance graph pairs as the overall tensile force loss value. and is used to: combine any two instance graphs on the background image into a set of instance graph pairs, and obtain multiple sets of instance graph pairs; For non-similar instance graph pairs, calculate the thrust loss value of each group of non-similar instance graph pairs; take the sum of the thrust loss values of each group of similar instance graph pairs as the overall thrust loss value. and is used for: multiplying the overall pull loss value by a preset coefficient and then adding the overall thrust loss value to obtain the adjacent loss value.
位置确定模块42还可以用于：确定所述背景图像上的每个实例图的被遮挡面积比；所述被遮挡面积比为所述实例图中被遮挡部分的面积与所述实例图总面积的比值；将所述被遮挡面积比大于设定阈值的实例图删除。以及用于：将所述背景图像上的实例图的被遮挡部分的标签设为遮挡。The position determination module 42 can also be used to: determine the occluded area ratio of each instance image on the background image; the occluded area ratio is the area of the occluded part in the instance graph and the total area of the instance graph The ratio of the occluded area is greater than the set threshold, delete the instance map. and used for: setting the label of the occluded part of the instance image on the background image as occlusion.
本发明实施例提供了一种数据生成装置，通过将目标对象的实例图添加到背景图像中，合成训练图像的方式，增加训练数据集的数据量，可以提高训练数据的多样性，有利于提高模型的训练效果，增强训练得到的模型的稳定性。The embodiment of the present invention provides a data generation device. By adding an instance image of a target object to a background image and synthesizing a training image, the data volume of the training data set can be increased, the diversity of training data can be improved, and the The training effect of the model enhances the stability of the trained model.
本实施例所提供的装置，其实现原理及产生的技术效果和前述实施例相同，为简要描述，装置实施例部分未提及之处，可参考前述方法实施例中相应内容。The implementation principle and the technical effects of the device provided in this embodiment are the same as those in the foregoing embodiments. For brief description, for the parts not mentioned in the device embodiment, reference may be made to the corresponding content in the foregoing method embodiments.
本发明实施例还提供了一种电子设备，包括存储器和处理器。所述存储器中存储有可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现前述方法实施例所记载的方法。An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program executable on the processor is stored in the memory, and when the processor executes the computer program, the method described in the foregoing method embodiments is implemented.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的电子设备的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the electronic device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here.
进一步，本实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器运行时执行上述前述方法实施例所提供的方法的步骤，具体实现可参见方法实施例，在此不再赘述。Further, this embodiment also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the methods provided by the foregoing method embodiments are executed. , and the specific implementation can refer to the method embodiment, which is not repeated here.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .
最后应说明的是：以上所述实施例，仅为本发明的具体实施方式，用以说明本发明的技术方案，而非对其限制，本发明的保护范围并不局限于此，尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本发明实施例技术方案的精神和范围，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present invention, and are used to illustrate the technical solutions of the present invention, but not to limit them. The protection scope of the present invention is not limited thereto, although referring to the foregoing The embodiment has been described in detail the present invention, those of ordinary skill in the art should understand: any person skilled in the art who is familiar with the technical field within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be covered in the present invention. within the scope of protection. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811523178.9ACN109583509B (en) | 2018-12-12 | 2018-12-12 | Data generation method and device and electronic equipment |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811523178.9ACN109583509B (en) | 2018-12-12 | 2018-12-12 | Data generation method and device and electronic equipment |
| Publication Number | Publication Date |
|---|---|
| CN109583509Atrue CN109583509A (en) | 2019-04-05 |
| CN109583509B CN109583509B (en) | 2020-11-03 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811523178.9AActiveCN109583509B (en) | 2018-12-12 | 2018-12-12 | Data generation method and device and electronic equipment |
| Country | Link |
|---|---|
| CN (1) | CN109583509B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070540A (en)* | 2019-04-28 | 2019-07-30 | 腾讯科技（深圳）有限公司 | Image generating method, device, computer equipment and storage medium |
| CN111325767A (en)* | 2020-02-17 | 2020-06-23 | 杭州电子科技大学 | Method for synthesizing image set of citrus trees based on real scene |
| CN113298913A (en)* | 2021-06-07 | 2021-08-24 | Oppo广东移动通信有限公司 | Data enhancement method and device, electronic equipment and readable storage medium |
| CN113688887A (en)* | 2021-08-13 | 2021-11-23 | 百度在线网络技术（北京）有限公司 | Training and image recognition method and device of image recognition model |
| WO2022021287A1 (en)* | 2020-07-31 | 2022-02-03 | 华为技术有限公司 | Data enhancement method and training method for instance segmentation model, and related apparatus |
| CN115082795A (en)* | 2022-07-04 | 2022-09-20 | 梅卡曼德（北京）机器人科技有限公司 | Method, device, device, medium and product for generating virtual image |
| CN115205628A (en)* | 2022-06-30 | 2022-10-18 | 北京三快在线科技有限公司 | Visual perception model training method, device, equipment and storage medium |
| CN115359319A (en)* | 2022-08-23 | 2022-11-18 | 京东方科技集团股份有限公司 | Image set generation method, device, equipment and computer-readable storage medium |
| WO2023056833A1 (en)* | 2021-10-09 | 2023-04-13 | 北京字节跳动网络技术有限公司 | Background picture generation method and apparatus, image fusion method and apparatus, and electronic device and readable medium |
| WO2024008081A1 (en)* | 2022-07-04 | 2024-01-11 | 梅卡曼德（北京）机器人科技有限公司 | Image generation method and model training method |
| US12198302B2 (en) | 2021-07-15 | 2025-01-14 | Boe Technology Group Co., Ltd. | Image processing method and device, and training method of image processing model and training method thereof |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105593901A (en)* | 2013-06-28 | 2016-05-18 | 日本电气株式会社 | Training data generation device, method and program and crowd state recognition device, method and program |
| US20180068463A1 (en)* | 2016-09-02 | 2018-03-08 | Artomatix Ltd. | Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures |
| US20180189607A1 (en)* | 2016-12-29 | 2018-07-05 | Elektrobit Automotive Gmbh | Generating training images for machine learning-based objection recognition systems |
| CN108257119A (en)* | 2018-01-08 | 2018-07-06 | 浙江大学 | A kind of immediate offshore area floating harmful influence detection method for early warning based near ultraviolet image procossing |
| CN108305262A (en)* | 2017-11-22 | 2018-07-20 | 腾讯科技（深圳）有限公司 | File scanning method, device and equipment |
| CN108492343A (en)* | 2018-03-28 | 2018-09-04 | 东北大学 | A kind of image combining method for the training data expanding target identification |
| CN108537859A (en)* | 2017-03-02 | 2018-09-14 | 奥多比公司 | Use the image masks of deep learning |
| CN108875732A (en)* | 2018-01-11 | 2018-11-23 | 北京旷视科技有限公司 | Model training and example dividing method, device and system and storage medium |
| CN108876791A (en)* | 2017-10-23 | 2018-11-23 | 北京旷视科技有限公司 | Image processing method, device and system and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105593901A (en)* | 2013-06-28 | 2016-05-18 | 日本电气株式会社 | Training data generation device, method and program and crowd state recognition device, method and program |
| US20180068463A1 (en)* | 2016-09-02 | 2018-03-08 | Artomatix Ltd. | Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures |
| US20180189607A1 (en)* | 2016-12-29 | 2018-07-05 | Elektrobit Automotive Gmbh | Generating training images for machine learning-based objection recognition systems |
| CN108537859A (en)* | 2017-03-02 | 2018-09-14 | 奥多比公司 | Use the image masks of deep learning |
| CN108876791A (en)* | 2017-10-23 | 2018-11-23 | 北京旷视科技有限公司 | Image processing method, device and system and storage medium |
| CN108305262A (en)* | 2017-11-22 | 2018-07-20 | 腾讯科技（深圳）有限公司 | File scanning method, device and equipment |
| CN108257119A (en)* | 2018-01-08 | 2018-07-06 | 浙江大学 | A kind of immediate offshore area floating harmful influence detection method for early warning based near ultraviolet image procossing |
| CN108875732A (en)* | 2018-01-11 | 2018-11-23 | 北京旷视科技有限公司 | Model training and example dividing method, device and system and storage medium |
| CN108492343A (en)* | 2018-03-28 | 2018-09-04 | 东北大学 | A kind of image combining method for the training data expanding target identification |
| Title |
|---|
| DAVID BREEDEN 等: "Synthesizing Object-Background Data for Large 3-d Datasets", 《CITESEER》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070540B (en)* | 2019-04-28 | 2023-01-10 | 腾讯科技（深圳）有限公司 | Image generation method and device, computer equipment and storage medium |
| CN110070540A (en)* | 2019-04-28 | 2019-07-30 | 腾讯科技（深圳）有限公司 | Image generating method, device, computer equipment and storage medium |
| CN111325767A (en)* | 2020-02-17 | 2020-06-23 | 杭州电子科技大学 | Method for synthesizing image set of citrus trees based on real scene |
| WO2022021287A1 (en)* | 2020-07-31 | 2022-02-03 | 华为技术有限公司 | Data enhancement method and training method for instance segmentation model, and related apparatus |
| CN113298913A (en)* | 2021-06-07 | 2021-08-24 | Oppo广东移动通信有限公司 | Data enhancement method and device, electronic equipment and readable storage medium |
| US12198302B2 (en) | 2021-07-15 | 2025-01-14 | Boe Technology Group Co., Ltd. | Image processing method and device, and training method of image processing model and training method thereof |
| CN113688887A (en)* | 2021-08-13 | 2021-11-23 | 百度在线网络技术（北京）有限公司 | Training and image recognition method and device of image recognition model |
| WO2023056833A1 (en)* | 2021-10-09 | 2023-04-13 | 北京字节跳动网络技术有限公司 | Background picture generation method and apparatus, image fusion method and apparatus, and electronic device and readable medium |
| CN115205628A (en)* | 2022-06-30 | 2022-10-18 | 北京三快在线科技有限公司 | Visual perception model training method, device, equipment and storage medium |
| CN115205628B (en)* | 2022-06-30 | 2025-08-29 | 北京三快在线科技有限公司 | Visual perception model training method, device, equipment and storage medium |
| WO2024008081A1 (en)* | 2022-07-04 | 2024-01-11 | 梅卡曼德（北京）机器人科技有限公司 | Image generation method and model training method |
| CN115082795A (en)* | 2022-07-04 | 2022-09-20 | 梅卡曼德（北京）机器人科技有限公司 | Method, device, device, medium and product for generating virtual image |
| CN115359319A (en)* | 2022-08-23 | 2022-11-18 | 京东方科技集团股份有限公司 | Image set generation method, device, equipment and computer-readable storage medium |
| Publication number | Publication date |
|---|---|
| CN109583509B (en) | 2020-11-03 |
| Publication | Publication Date | Title |
|---|---|---|
| CN109583509B (en) | Data generation method and device and electronic equipment | |
| CN107704838B (en) | Target object attribute identification method and device | |
| WO2020119527A1 (en) | Human action recognition method and apparatus, and terminal device and storage medium | |
| CN108898145A (en) | A kind of image well-marked target detection method of combination deep learning | |
| CN110246181A (en) | Attitude estimation model training method, Attitude estimation method and system based on anchor point | |
| CN108229533A (en) | Image processing method, model pruning method, device and equipment | |
| CN110781962B (en) | Target detection method based on lightweight convolutional neural network | |
| CN110276264A (en) | A Crowd Density Estimation Method Based on Foreground Segmentation Map | |
| CN116051953A (en) | Small Object Detection Method Based on Selectable Convolution Kernel Network and Weighted Bidirectional Feature Pyramid | |
| CN112085835A (en) | Three-dimensional cartoon face generation method and device, electronic equipment and storage medium | |
| CN113763249A (en) | Text image super-resolution reconstruction method and related equipment | |
| WO2017206400A1 (en) | Image processing method, apparatus, and electronic device | |
| CN113569809B (en) | Image processing method, device and computer readable storage medium | |
| WO2018082308A1 (en) | Image processing method and terminal | |
| CN113570615A (en) | An image processing method, electronic device and storage medium based on deep learning | |
| CN107392085A (en) | The method for visualizing convolutional neural networks | |
| WO2019090901A1 (en) | Image display selection method and apparatus, intelligent terminal and storage medium | |
| CN119339302B (en) | A method, device and medium for inter-frame image segmentation based on recursive neural network | |
| CN111177811A (en) | Automatic fire point location layout method applied to cloud platform | |
| CN104700384B (en) | Display systems and methods of exhibiting based on augmented reality | |
| US20250166135A1 (en) | Fine-grained controllable video generation | |
| CN108009549A (en) | A kind of iteration cooperates with conspicuousness detection method | |
| CN115527233A (en) | Anti-blocking pedestrian re-recognition method for guiding attention by posture | |
| CN113705304B (en) | Image processing method, device, storage medium and computer equipment | |
| CN113408452A (en) | Expression redirection training method and device, electronic equipment and readable storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | Denomination of invention:Data generation methods, devices, and electronic devices Effective date of registration:20230404 Granted publication date:20201103 Pledgee:Shanghai Yunxin Venture Capital Co.,Ltd. Pledgor:BEIJING KUANGSHI TECHNOLOGY Co.,Ltd.|NANJING KUANGYUN TECHNOLOGY Co.,Ltd. Registration number:Y2023990000195 | |
| PC01 | Cancellation of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right | Granted publication date:20201103 Pledgee:Shanghai Yunxin Venture Capital Co.,Ltd. Pledgor:BEIJING KUANGSHI TECHNOLOGY Co.,Ltd.|NANJING KUANGYUN TECHNOLOGY Co.,Ltd. Registration number:Y2023990000195 |