技术领域Technical field
本申请属于缺陷检测技术领域，尤其涉及一种基于小样本的自适应缺陷检测方法及终端设备。The present application belongs to the field of defect detection technology, and in particular relates to a small sample-based adaptive defect detection method and terminal equipment.
背景技术Background technique
随着工业制造技术的不断提高以及人们对电子产品的需求的不断增大，电池的生产数量急剧增加。在电池生产过程中，难免会产生一些具有缺陷（例如表面存在凸起）的缺陷电池，不仅会影响电池的外观，而且带来安全隐患。With the continuous improvement of industrial manufacturing technology and the increasing demand for electronic products, the production quantity of batteries has increased dramatically. During the battery production process, it is inevitable that some defective batteries with defects (such as bulges on the surface) will be produced, which will not only affect the appearance of the battery, but also pose safety risks.
基于此，在电池生产出来之后，通常需要对电池进行缺陷检测。相关技术通常是采用基于深度学习算法的目标检测模型对电池图像进行缺陷检测，而目标检测模型的训练通常需要大量的缺陷电池样本图像，但实际生产过程中产生的缺陷电池相对较少，导致没有足够的缺陷电池样本图像对目标检测模型进行训练，这样会使目标检测模型的泛化能力不足，从而降低了目标检测模型的缺陷检测准确度，也就是说，在小样本情况下，基于深度学习算法的目标检测模型的缺陷检测准确度较低。Based on this, after the battery is produced, it is usually necessary to detect defects in the battery. Related technology usually uses a target detection model based on deep learning algorithms to detect defects in battery images. The training of the target detection model usually requires a large number of defective battery sample images, but the actual production process produces relatively few defective batteries, resulting in no Enough defective battery sample images to train the target detection model will make the generalization ability of the target detection model insufficient, thereby reducing the defect detection accuracy of the target detection model. That is to say, in the case of small samples, based on deep learning The target detection model of the algorithm has low defect detection accuracy.
发明内容Contents of the invention
有鉴于此，本申请实施例提供一种基于小样本的自适应缺陷检测方法及终端设备，用于解决小样本情况下，基于深度学习算法的目标检测模型的缺陷检测准确度较低的技术问题。In view of this, embodiments of the present application provide an adaptive defect detection method and terminal equipment based on small samples to solve the technical problem of low defect detection accuracy of target detection models based on deep learning algorithms in the case of small samples. .
第一方面，本申请实施例提供一种基于小样本的自适应缺陷检测方法，包括：In the first aspect, embodiments of the present application provide an adaptive defect detection method based on small samples, including:
为正常样本图像集中的每张正常电池图像分别添加一个掩码区域，将添加了掩码区域的所述正常电池图像确定为掩码图像，并记录每张所述掩码图像中的掩码区域的位置信息和尺寸信息；Add a mask area to each normal battery image in the normal sample image set, determine the normal battery image to which the mask area is added as a mask image, and record the mask area in each mask image location information and size information;
通过训练好的缺陷图像生成模型分别对各张所述掩码图像进行处理，得到各张所述掩码图像分别对应的预测缺陷图像；所述缺陷图像生成模型包括n个级联的缺陷预测网络；每一级所述缺陷预测网络均包括一个缺陷判别模块和一个缺陷生成模块；每一级缺陷预测网络中的缺陷判别模块用于对上一级缺陷预测网络输出的上一级预测缺陷图像进行缺陷判别，并提取所述上一级预测缺陷图像中的掩码区域的梯度信息，且向本级缺陷预测网络中的缺陷生成模块发送所述缺陷判别结果和所述梯度信息；每一级缺陷预测网络中的缺陷生成模块用于基于所述判别结果、所述上一级预测缺陷图像以及所述梯度信息，生成本级预测缺陷图像，并向下一级缺陷预测网络发送所述本级预测缺陷图像；Each of the mask images is processed separately through the trained defect image generation model to obtain a predicted defect image corresponding to each of the mask images; the defect image generation model includesn cascaded defect prediction networks ; Each level of the defect prediction network includes a defect identification module and a defect generation module; the defect identification module in each level of the defect prediction network is used to perform prediction on the upper level predicted defect image output by the upper level defect prediction network. Defect identification, extract the gradient information of the mask area in the upper-level predicted defect image, and send the defect identification result and the gradient information to the defect generation module in the current-level defect prediction network; each level of defect The defect generation module in the prediction network is used to generate the current level prediction defect image based on the discrimination result, the upper level prediction defect image and the gradient information, and send the current level prediction to the next level defect prediction network. defective images;
基于所有所述掩码图像对应的预测缺陷图像以及所有所述掩码图像中的掩码区域的位置信息和尺寸信息，生成第一训练集，并通过所述第一训练集对缺陷检测模型进行训练；Based on the predicted defect images corresponding to all the mask images and the position information and size information of the mask areas in all the mask images, a first training set is generated, and the defect detection model is tested using the first training set. train;
通过训练好的缺陷检测模型对待检测电池图像进行缺陷检测。Defects are detected on the battery image to be detected through the trained defect detection model.
在第一方面的一种可选的实现方式中，所述缺陷判别结果包括：所述上一级预测缺陷图像中的掩码区域不是缺陷区域，以及所述上一级预测缺陷图像中的掩码区域是缺陷区域；对应地，所述通过训练好的缺陷图像生成模型分别对各张所述掩码图像进行处理，包括：In an optional implementation of the first aspect, the defect identification result includes: the mask area in the upper-level predicted defect image is not a defect area, and the mask area in the upper-level predicted defect image is not a defect area. The code area is the defective area; correspondingly, the trained defect image generation model processes each of the mask images separately, including:
在第i级缺陷预测网络中，通过所述第i级缺陷预测网络中的缺陷判别模块对上一级预测缺陷图像进行缺陷判别，并提取所述上一级预测缺陷图像中的掩码区域的梯度信息，且向所述第i级缺陷预测网络中的缺陷生成模块发送所述缺陷判别结果和所述梯度信息；其中，第1级缺陷预测网络对应的上一级预测缺陷图像为所述掩码图像；In thei-th level defect prediction network, the defect identification module in thei-th level defect prediction network performs defect identification on the upper-level predicted defect image, and extracts the mask area in the upper-level predicted defect image. Gradient information, and send the defect identification result and the gradient information to the defect generation module in the i-th level defect prediction network; wherein the upper-level predicted defect image corresponding to the first-level defect prediction network is the mask code image;
在所述缺陷判别结果为所述上一级预测缺陷图像中的掩码区域不是缺陷区域的情况下，通过所述第i级缺陷预测网络中的缺陷生成模块基于所述判别结果、所述上一级预测缺陷图像以及所述梯度信息，生成本级预测缺陷图像，并向第i+1级缺陷预测网络输出所述本级预测缺陷图像；When the defect discrimination result is that the mask area in the upper-level predicted defect image is not a defect area, the defect generation module in the i-th level defect prediction network is based on the discrimination result and the upper-level defect prediction network. The first-level predicted defect image and the gradient information generate a current-level predicted defect image, and output the current-level predicted defect image to thei +1th level defect prediction network;
在所述缺陷判别结果为所述上一级预测缺陷图像中的掩码区域是缺陷区域的情况下，通过所述第i级缺陷预测网络中的缺陷生成模块直接将所述上一级预测缺陷图像输出至所述缺陷图像生成模型的输出端，作为所述掩码图像对应的预测缺陷图像。When the defect discrimination result is that the mask area in the upper-level predicted defect image is a defect area, the upper-level predicted defect is directly generated by the defect generation module in the i-th level defect prediction network. The image is output to the output end of the defect image generation model as the predicted defect image corresponding to the mask image.
在第一方面的一种可选的实现方式中，所述通过所述第i级缺陷预测网络中的缺陷生成模块基于所述判别结果、所述上一级预测缺陷图像以及所述梯度信息，生成本级预测缺陷图像，包括：In an optional implementation of the first aspect, the defect generation module in thei-th level defect prediction network is based on the discrimination result, the upper-level predicted defect image and the gradient information, Generates native-level predicted defect images, including:
所述第i级缺陷预测网络中的缺陷生成模块随机生成一个尺寸与所述掩码区域的尺寸相同的噪声图像；The defect generation module in thei-th level defect prediction network randomly generates a noise image with the same size as the mask area;
所述第i级缺陷预测网络中的缺陷生成模块基于所述判别结果、所述上一级预测缺陷图像、所述梯度信息以及所述噪声图像，采用以下公式生成本级预测缺陷图像：The defect generation module in the i-th level defect prediction network uses the following formula to generate this level's predicted defect image based on the discrimination result, the upper-level predicted defect image, the gradient information and the noise image:
； ;
其中，Xi为所述本级预测缺陷图像，zi为所述第i级缺陷预测网络对应的噪声系数，Xt-1为所述上一级预测缺陷图像，Gi为所述噪声图像，Ti-1为所述上一级预测缺陷图像中掩码区域的梯度信息；1>z1>z2>z3>……>zn-1>zn>0。Among them,Xi isthe predicteddefect image at thislevel ,zi is the noise coefficient corresponding to thei-th level defect prediction network, ,Ti-1 is the gradient information of the mask area in the upper-level predicted defect image; 1>z1 >z2 >z3 >...>zn-1 >zn >0.
在第一方面的一种可选的实现方式中，在所述通过训练好的缺陷图像生成模型分别对各张所述掩码图像进行处理之前，还包括：In an optional implementation of the first aspect, before processing each of the mask images through the trained defect image generation model, the method further includes:
基于缺陷样本图像集和所述正常样本图像集制作第二训练集；Create a second training set based on the defective sample image set and the normal sample image set;
通过所述第二训练集训练预设二分类器，并将训练完成的所述二分类器与梯度信息提取单元的组合确定为所述缺陷判别模块。A preset two classifier is trained through the second training set, and the combination of the trained two classifier and the gradient information extraction unit is determined as the defect discrimination module.
在第一方面的一种可选的实现方式中，所述基于缺陷样本图像集和所述正常样本图像集制作第二训练集，包括：In an optional implementation of the first aspect, preparing a second training set based on the defective sample image set and the normal sample image set includes:
从所述缺陷样本图像集的每张缺陷电池图像中裁剪出缺陷区域，将裁剪出的所述缺陷区域分别确定为相应缺陷电池图像对应的局部缺陷图像；Cut out defective areas from each defective battery image in the defective sample image set, and determine the cropped defective areas as local defect images corresponding to the corresponding defective battery images;
从所述正常样本图像集的多张正常电池图像中裁剪出多张局部正常图像，每张所述局部正常图像对应一张所述局部缺陷图像，每张所述局部正常图像的尺寸与对应的所述局部缺陷图像的尺寸相同；A plurality of partially normal images are cut out from multiple normal battery images in the normal sample image set. Each of the partially normal images corresponds to one of the local defect images. The size of each of the partially normal images is the same as the corresponding The local defect images have the same size;
将裁剪出的所述局部缺陷图像和所述局部正常图像分别作为第二训练集的正样本集和负样本集，得到所述第二训练集。The cropped local defective image and the local normal image are respectively used as the positive sample set and the negative sample set of the second training set to obtain the second training set.
在第一方面的一种可选的实现方式中，所述为正常样本图像集中的每张正常电池图像分别添加一个掩码区域，包括：In an optional implementation of the first aspect, adding a mask area to each normal battery image in the normal sample image set includes:
在预设存储区域存储有预设信息列表的情况下，基于所述预设信息列表中各个目标区域的位置信息和尺寸信息，分别为各张正常电池图像添加掩码区域；In the case where a preset information list is stored in the preset storage area, based on the position information and size information of each target area in the preset information list, a mask area is added to each normal battery image respectively;
在所述预设存储区域未存储所述预设信息列表的情况下，分别为各张所述正常电池图像随机添加掩码区域。When the preset information list is not stored in the preset storage area, a mask area is randomly added to each of the normal battery images.
在第一方面的一种可选的实现方式中，所述掩码区域用于遮盖所述正常电池图像中的目标区域；所述为正常样本图像集中的每张正常电池图像分别添加一个掩码区域，包括：In an optional implementation of the first aspect, the mask area is used to cover the target area in the normal battery image; and a mask is added to each normal battery image in the normal sample image set. areas, including:
将所述正常电池图像中目标区域的所有像素的像素值均修改为0。Modify the pixel values of all pixels in the target area in the normal battery image to 0.
第二方面，本申请实施例提供一种终端设备，包括：In a second aspect, embodiments of the present application provide a terminal device, including:
掩码生成单元，用于为正常样本图像集中的每张正常电池图像分别添加一个掩码区域，将添加了掩码区域的所述正常电池图像确定为掩码图像，并记录每张所述掩码图像中的掩码区域的位置信息和尺寸信息；A mask generation unit configured to add a mask area to each normal battery image in the normal sample image set, determine the normal battery image to which the mask area is added as a mask image, and record each of the mask images. Position information and size information of the mask area in the code image;
缺陷生成单元，用于通过训练好的缺陷图像生成模型分别对各张所述掩码图像进行处理，得到各张所述掩码图像分别对应的预测缺陷图像；所述缺陷图像生成模型包括n个级联的缺陷预测网络；每一级所述缺陷预测网络均包括一个缺陷判别模块和一个缺陷生成模块；每一级缺陷预测网络中的缺陷判别模块用于对上一级缺陷预测网络输出的上一级预测缺陷图像进行缺陷判别，并提取所述上一级预测缺陷图像中的掩码区域的梯度信息，且向本级缺陷预测网络中的缺陷生成模块发送所述缺陷判别结果和所述梯度信息；每一级缺陷预测网络中的缺陷生成模块用于基于所述判别结果、所述上一级预测缺陷图像以及所述梯度信息，生成本级预测缺陷图像，并向下一级缺陷预测网络发送所述本级预测缺陷图像；A defect generation unit is used to process each of the mask images through a trained defect image generation model to obtain a predicted defect image corresponding to each of the mask images; the defect image generation model includesn Cascading defect prediction network; each level of the defect prediction network includes a defect identification module and a defect generation module; the defect identification module in each level of defect prediction network is used to improve the output of the previous level defect prediction network The first-level predicted defect image performs defect discrimination, extracts the gradient information of the mask area in the upper-level predicted defect image, and sends the defect discrimination result and the gradient to the defect generation module in the current-level defect prediction network Information; the defect generation module in each level of defect prediction network is used to generate the predicted defect image of this level based on the discrimination result, the upper level predicted defect image and the gradient information, and pass it to the next level defect prediction network. Send the predicted defect image of this level;
第一训练单元，用于基于所有所述掩码图像对应的预测缺陷图像以及所有所述掩码图像中的掩码区域的位置信息和尺寸信息，生成第一训练集，并通过所述第一训练集对缺陷检测模型进行训练；The first training unit is configured to generate a first training set based on the predicted defect images corresponding to all the mask images and the position information and size information of the mask areas in all the mask images, and use the first The training set is used to train the defect detection model;
缺陷检测单元，用于通过训练好的缺陷检测模型对待检测电池图像进行缺陷检测。The defect detection unit is used to detect defects in the battery image to be detected through the trained defect detection model.
第三方面，本申请实施例提供另一种终端设备，包括存储器以及存储在所述存储器中并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如上述第一方面任一可选实现方式所述的基于小样本的自适应缺陷检测方法。In a third aspect, embodiments of the present application provide another terminal device, including a memory and a computer program stored in the memory and executable on a processor. When the processor executes the computer program, the above first step is implemented. The small sample-based adaptive defect detection method described in any optional implementation of the aspect.
第四方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如上述第一方面任一可选实现方式所述的基于小样本的自适应缺陷检测方法。In a fourth aspect, embodiments of the present application provide a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, any optional implementation method of the first aspect is implemented as described above. The adaptive defect detection method based on small samples.
实施本申请实施例提供的基于小样本的自适应缺陷检测方法、终端设备及计算机可读存储介质具有以下有益效果：Implementing the small sample-based adaptive defect detection method, terminal equipment and computer-readable storage medium provided by the embodiments of this application has the following beneficial effects:
本申请实施例提供的基于小样本的自适应缺陷检测方法，通过构建并训练缺陷图像生成模型，通过缺陷图像生成模型来生成大量用于训练缺陷检测模型的预测缺陷图像，使得在真实的缺陷电池样本图像不足的情况下，缺陷检测模型也能够具有充足的训练样本，从而提高了缺陷检测模型的缺陷检测准确度。The adaptive defect detection method based on small samples provided by the embodiments of this application builds and trains a defect image generation model, and uses the defect image generation model to generate a large number of predicted defect images for training the defect detection model, so that in real defective batteries Even when sample images are insufficient, the defect detection model can also have sufficient training samples, thereby improving the defect detection accuracy of the defect detection model.
此外，生成预测缺陷图像的过程相当于是通过缺陷图像生成模型对掩码图像的掩码区域进行去燥的过程，由于缺陷图像生成模型中设置有多个级联的缺陷预测网络，且每一级缺陷预测网络中均设置有用于引导缺陷生成模块生成预测缺陷图像的缺陷判别模块，且缺陷判别模块是基于真实的缺陷样本图像集训练得到的，因此，在逐级缺陷判别模块输出的缺陷判别结果以及梯度信息的引导下，缺陷图像生成模型可以快速且高效地生成高质量的预测缺陷图像，如此使得缺陷检测模型的训练样本的数量和质量均得到了提高，从而进一步提高了缺陷检测模型的缺陷检测准确度。In addition, the process of generating predicted defect images is equivalent to the process of de-drying the mask area of the mask image through the defect image generation model. Since the defect image generation model is equipped with multiple cascaded defect prediction networks, and each level The defect prediction network is equipped with a defect identification module that guides the defect generation module to generate predicted defect images, and the defect identification module is trained based on the real defect sample image set. Therefore, the defect identification results output by the level-by-level defect identification module And guided by gradient information, the defect image generation model can quickly and efficiently generate high-quality predicted defect images, which improves the number and quality of training samples for the defect detection model, thereby further improving the defect detection model. Detection accuracy.
附图说明Description of the drawings
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the purpose of the present application. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.
图1为本申请实施例提供的一种基于小样本的自适应缺陷检测方法的示意性流程图；Figure 1 is a schematic flow chart of an adaptive defect detection method based on small samples provided by an embodiment of the present application;
图2为本申请实施例提供的一种掩码图像的生成过程示意图；Figure 2 is a schematic diagram of a mask image generation process provided by an embodiment of the present application;
图3为本申请实施例提供的一种缺陷图像生成模型的示意性结构图；Figure 3 is a schematic structural diagram of a defect image generation model provided by an embodiment of the present application;
图4为本申请实施例提供的另一种基于小样本的自适应缺陷检测方法的示意性流程图；Figure 4 is a schematic flow chart of another small sample-based adaptive defect detection method provided by an embodiment of the present application;
图5为本申请实施例提供的一种基于小样本的自适应缺陷检测方法中S41的具体实现流程图；Figure 5 is a specific implementation flow chart of S41 in a small sample-based adaptive defect detection method provided by the embodiment of the present application;
图6为本申请实施例提供的一种第二训练集的制作过程示意图；Figure 6 is a schematic diagram of the production process of a second training set provided by an embodiment of the present application;
图7为本申请实施例提供的一种终端设备的结构示意图；Figure 7 is a schematic structural diagram of a terminal device provided by an embodiment of the present application;
图8为本申请另一实施例提供的一种终端设备的结构示意图。FIG. 8 is a schematic structural diagram of a terminal device provided by another embodiment of the present application.
具体实施方式Detailed ways
以下实施例仅用于更加清楚地说明本申请的技术方案，因此只作为示例，而不能以此来限制本申请的保护范围。The following examples are only used to illustrate the technical solution of the present application more clearly, and are therefore only used as examples and cannot be used to limit the protection scope of the present application.
需要说明的是，除非另有规定，本申请实施例使用的所有技术术语与属于本申请的技术领域的技术人员通常理解的含义相同。本申请实施例使用的技术术语仅用于对本申请的具体实施例进行解释，而非旨在限定本申请。It should be noted that, unless otherwise specified, all technical terms used in the embodiments of this application have the same meanings as commonly understood by those skilled in the technical field of this application. The technical terms used in the embodiments of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.
在本申请实施例的描述中，技术术语“包括”、“包含”、“具有”及它们的任意变形等都意味着“包括但不限于”，除非是以其他方式另外特别强调。在本申请实施例的描述中，除非另有说明，技术术语“多个”是指两个或多于两个，技术术语“至少一个”、“一个或多个”是指一个、两个或两个以上。技术术语“第一”“第二”等仅用于区别不同对象，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量、特定顺序或主次关系。技术术语“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。In the description of the embodiments of this application, the technical terms "include", "include", "have" and any variations thereof mean "including but not limited to" unless otherwise specifically emphasized. In the description of the embodiments of this application, unless otherwise stated, the technical term "plurality" refers to two or more than two, and the technical term "at least one", "one or more" refers to one, two or more Two or more. The technical terms "first", "second", etc. are only used to distinguish different objects and cannot be understood as indicating or implying the relative importance or implicitly indicating the quantity, specific order or primary and secondary relationship of the indicated technical features. The technical term "and/or" is just an association relationship that describes related objects, indicating that there can be three relationships, such as A and/or B, which can mean: A alone exists, A and B exist simultaneously, and B alone exists. situation. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.
本申请实施例的描述中提及的“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。Reference to "an embodiment" in the description of the embodiments of the present application means that a specific feature, structure or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art understand, both explicitly and implicitly, that the embodiments described herein may be combined with other embodiments.
随着工业制造技术的不断提高以及人们对电子产品的需求的不断增大，电池的生产数量急剧增加。在电池生产过程中，难免会产生一些具有缺陷（例如表面存在凸起）的缺陷电池，不仅会影响电池的外观，而且带来安全隐患。With the continuous improvement of industrial manufacturing technology and the increasing demand for electronic products, the production quantity of batteries has increased dramatically. During the battery production process, it is inevitable that some defective batteries with defects (such as bulges on the surface) will be produced, which will not only affect the appearance of the battery, but also pose safety risks.
基于此，在电池生产出来之后，通常需要对电池进行缺陷检测。相关技术通常是采用基于深度学习算法的目标检测模型对电池图像进行缺陷检测，而目标检测模型的训练通常需要大量的缺陷电池样本图像，但实际生产过程中产生的缺陷电池相对较少，导致没有足够的缺陷电池样本图像对目标检测模型进行训练，这样会使目标检测模型的泛化能力不足，从而降低目标检测模型的缺陷检测准确度，也就是说，在小样本情况下，基于深度学习算法的目标检测模型的缺陷检测准确度较低。Based on this, after the battery is produced, it is usually necessary to detect defects in the battery. Related technology usually uses a target detection model based on deep learning algorithms to detect defects in battery images. The training of the target detection model usually requires a large number of defective battery sample images, but the actual production process produces relatively few defective batteries, resulting in no Enough defective battery sample images to train the target detection model will make the generalization ability of the target detection model insufficient, thereby reducing the defect detection accuracy of the target detection model. That is to say, in the case of small samples, based on deep learning algorithms The defect detection accuracy of the target detection model is low.
此外，用于训练目标检测模型的缺陷电池样本图像通常是由人工进行标注的，而电池上一些细小的缺陷很难通过人眼识别出来，如此不仅耗费人力，而且标注出的缺陷部分不一定准确，从而会影响目标检测模型的缺陷检测准确度。In addition, defective battery sample images used to train target detection models are usually annotated manually, and some small defects on the battery are difficult to identify with the human eye. This is not only labor-intensive, but the marked defective parts may not be accurate. , which will affect the defect detection accuracy of the target detection model.
有鉴于此，本申请实施例提供一种基于小样本的自适应缺陷检测方法，通过构建并训练缺陷图像生成模型，通过缺陷图像生成模型来生成大量用于训练缺陷检测模型的预测缺陷图像，使得在真实的缺陷电池样本图像不足的情况下，缺陷检测模型也能够具有充足的训练样本，从而提高了缺陷检测模型的缺陷检测准确度。此外，生成预测缺陷图像的过程相当于是通过缺陷图像生成模型对掩码图像的掩码区域进行去燥的过程，由于缺陷图像生成模型中设置有多个级联的缺陷预测网络，且每一级缺陷预测网络中均设置有用于引导缺陷生成模块生成预测缺陷图像的缺陷判别模块，且缺陷判别模块是基于真实的缺陷样本图像集训练得到的，因此，在逐级缺陷判别模块输出的缺陷判别结果以及梯度信息的引导下，缺陷图像生成模型可以快速且高效地生成高质量的预测缺陷图像，如此使得缺陷检测模型的训练样本的数量和质量均得到了提高，从而进一步提高了缺陷检测模型的缺陷检测准确度。In view of this, embodiments of the present application provide an adaptive defect detection method based on small samples. By constructing and training a defect image generation model, a large number of predicted defect images for training the defect detection model are generated through the defect image generation model, so that Even when there are insufficient real defective battery sample images, the defect detection model can also have sufficient training samples, thereby improving the defect detection accuracy of the defect detection model. In addition, the process of generating predicted defect images is equivalent to the process of de-drying the mask area of the mask image through the defect image generation model. Since the defect image generation model is equipped with multiple cascaded defect prediction networks, and each level The defect prediction network is equipped with a defect identification module that guides the defect generation module to generate predicted defect images, and the defect identification module is trained based on the real defect sample image set. Therefore, the defect identification results output by the level-by-level defect identification module And guided by gradient information, the defect image generation model can quickly and efficiently generate high-quality predicted defect images, which improves the number and quality of training samples for the defect detection model, thereby further improving the defect detection model. Detection accuracy.
本申请实施例提供的基于小样本的自适应缺陷检测方法的执行主体为终端设备。终端设备可以包括但不限于计算机终端或者移动通信终端，例如个人电脑、手机或平板电脑等。在具体应用中，可以通过对终端设备配置目标脚本文件，由该目标脚本文件描述本申请实施例提供的基于小样本的自适应缺陷检测方法，令终端设备在需要进行缺陷检测时执行该目标脚本文件，进而执行本申请实施例提供的基于小样本的自适应缺陷检测方法中的各个步骤。The execution subject of the small sample-based adaptive defect detection method provided by the embodiments of this application is the terminal device. Terminal devices may include but are not limited to computer terminals or mobile communication terminals, such as personal computers, mobile phones, or tablet computers. In a specific application, a target script file can be configured on the terminal device, and the target script file describes the small sample-based adaptive defect detection method provided by the embodiments of the present application, so that the terminal device can execute the target script when defect detection is required. file, and then execute each step in the small sample-based adaptive defect detection method provided by the embodiment of the present application.
请参阅图1，为本申请实施例提供的一种基于小样本的自适应缺陷检测方法的示意性流程图。如图1所示，该基于小样本的自适应缺陷检测方法可以包括S11~S14，详述如下：Please refer to Figure 1, which is a schematic flow chart of an adaptive defect detection method based on small samples provided by an embodiment of the present application. As shown in Figure 1, the adaptive defect detection method based on small samples can include S11~S14, which are detailed as follows:
S11，为正常样本图像集中的每张正常电池图像分别添加一个掩码区域，将添加了掩码区域的正常电池图像确定为掩码图像，并记录每张掩码图像中的掩码区域的位置信息和尺寸信息。S11, add a mask area to each normal battery image in the normal sample image set, determine the normal battery image with the mask area added as the mask image, and record the position of the mask area in each mask image. information and size information.
其中，正常样本图像集可以包括多张正常电池图像。正常电池图像是对表面不存在缺陷的正常电池进行拍摄得到的可见光图像。The normal sample image set may include multiple normal battery images. A normal battery image is a visible light image taken of a normal battery with no defects on its surface.
掩码区域用于对正常电池图像的目标区域进行遮盖。正常电池图像中被掩码区域遮盖的目标区域即为要生成电池缺陷的区域。电池缺陷例如可以包括表面凸起、表面凹坑和/或表面裂纹等，此处对电池缺陷的类型不做限定。The mask area is used to cover the target area of the normal battery image. The target area covered by the mask area in the normal battery image is the area where battery defects are to be generated. Battery defects may include, for example, surface protrusions, surface pits, and/or surface cracks, etc. The type of battery defects is not limited here.
在一个可选的实现方式中，目标区域的位置和尺寸可以是电子设备随机确定的。该实现方式中，由于随机确定目标区域具有较大的不确定性，因此可以增加最终生成的缺陷电池图像的多样性，从而能够使缺陷检测模型的训练样本更加丰富，进而可以提高训练完成的缺陷检测模型的缺陷识别准确度。In an alternative implementation, the location and size of the target area may be randomly determined by the electronic device. In this implementation, since the randomly determined target area has great uncertainty, it can increase the diversity of the final generated defective battery images, thereby enriching the training samples of the defect detection model, and thus improving the number of defects after training. Defect recognition accuracy of the inspection model.
在另一种可选的实现方式中，目标区域的位置和尺寸可以是电子设备根据用户需求确定的。示例性的，当用户想让电子设备生成一些在指定区域内有电池缺陷的缺陷电池图像时，用户可以向终端设备输入目标区域的位置信息和尺寸信息。其中，位置信息可以通过目标区域的中心点在预设坐标系中的坐标表示。预设坐标系例如可以是以正常电池图像的中心点为原点，以相交于该原点的任意两条垂直的射线分别为x轴和y轴建立的平面直角坐标系。尺寸信息可以通过目标区域的长和宽表示。该实现方式中，由于目标区域的位置和尺寸可以由用户自行设置，因此能够提高最终生成的缺陷电池图像中缺陷区域的位置和尺寸的可控性，如此能够根据实际的生产场景，生成适用于不同生产场景的缺陷电池图像，进而扩大了自适应缺陷检测方法的适用范围。In another optional implementation, the location and size of the target area may be determined by the electronic device according to user requirements. For example, when the user wants the electronic device to generate some defective battery images with battery defects in a specified area, the user can input the location information and size information of the target area to the terminal device. The position information can be represented by the coordinates of the center point of the target area in the preset coordinate system. The preset coordinate system may be, for example, a plane rectangular coordinate system established with the center point of the normal battery image as the origin and any two vertical rays intersecting the origin as the x-axis and y-axis respectively. Dimensional information can be represented by the length and width of the target area. In this implementation, since the position and size of the target area can be set by the user, the controllability of the position and size of the defective area in the final generated defective battery image can be improved, so that it can generate a suitable Images of defective batteries in different production scenarios, thereby expanding the scope of application of the adaptive defect detection method.
基于此，电子设备在为各个正常电池图像添加掩码区域时，可以先确定预设存储区域是否存储有预设信息列表。其中，预设信息列表用于记录用户预先输入的各个目标区域的位置信息和尺寸信息。预设存储区域可以根据实际需求设置，此处对其不做特别限定。Based on this, when adding a mask area to each normal battery image, the electronic device can first determine whether the preset storage area stores a preset information list. The preset information list is used to record the position information and size information of each target area input in advance by the user. The preset storage area can be set according to actual needs, and is not specifically limited here.
可选的，在预设存储区域存储有预设信息列表的情况下，电子设备可以从预设存储区域中获取该预设信息列表，并基于预设信息列表中各个目标区域的位置信息和尺寸信息，为各个正常电池图像添加掩码区域。Optionally, in the case where the preset information list is stored in the preset storage area, the electronic device can obtain the preset information list from the preset storage area and based on the position information and size of each target area in the preset information list information, adding a masked area to each normal battery image.
可选的，在预设存储区域未存储预设信息列表的情况下，电子设备可以为各个正常电池图像随机添加掩码区域。Optionally, when the preset storage area does not store the preset information list, the electronic device can randomly add a mask area to each normal battery image.
在一个具体的实现方式中，为正常电池图像添加掩码区域具体可以包括：将正常电池图像中目标区域的所有像素的像素值均修改为0。In a specific implementation, adding a mask area to the normal battery image may include: modifying the pixel values of all pixels in the target area in the normal battery image to 0.
由于某个像素的像素值为0时，该像素会呈现出黑色，因此，将目标区域的所有像素的像素值均修改为0，相当于通过黑色掩码对该目标区域进行了遮盖，即，修改了像素值之后的目标区域可以看作是为正常电池图像添加的掩码区域。为了便于描述，以下将添加了掩码区域的正常电池图像称为掩码图像。Since the pixel value of a certain pixel is 0, the pixel will appear black. Therefore, modifying the pixel values of all pixels in the target area to 0 is equivalent to covering the target area with a black mask, that is, The target area after modifying the pixel values can be viewed as a mask area added to the normal battery image. For the convenience of description, the normal battery image with the mask area added is referred to as the mask image below.
示例性的，请参阅图2，为本申请实施例提供的一种掩码图像的生成过程示意图。如图2所示，通过为正常样本图像集中的每张正常电池图像21分别添加一个掩码区域，可以得到多张分别与正常电池图像21对应的掩码图像22。For example, please refer to FIG. 2 , which is a schematic diagram of a mask image generation process provided by an embodiment of the present application. As shown in FIG. 2 , by adding a mask area to each normal battery image 21 in the normal sample image set, multiple mask images 22 corresponding to the normal battery images 21 can be obtained.
可以理解的是，在电子设备为各张正常电池图像随机添加掩码区域的情况下，电子设备可以在得到各张正常电池图像分别对应的掩码图像后，记录掩码图像中的掩码区域的位置信息和尺寸信息。其中，掩码图像中的掩码区域的位置信息即为正常电池图像中的目标区域的位置信息；掩码图像中的掩码区域的尺寸信息即为正常电池图像中的目标区域的尺寸信息。It can be understood that when the electronic device randomly adds a mask area to each normal battery image, the electronic device can record the mask area in the mask image after obtaining the mask image corresponding to each normal battery image. location information and size information. Wherein, the position information of the mask area in the mask image is the position information of the target area in the normal battery image; the size information of the mask area in the mask image is the size information of the target area in the normal battery image.
在一个具体的实现方式中，终端设备记录掩码图像中的掩码区域的位置信息和尺寸信息，可以包括：将掩码图像中的掩码区域的位置信息和尺寸信息与该掩码图像进行关联存储。In a specific implementation, the terminal device records the position information and size information of the mask area in the mask image, which may include: comparing the position information and size information of the mask area in the mask image with the mask image. Associative storage.
本申请实施例通过预先对掩码区域（即待生成电池缺陷的目标区域）的位置信息和尺寸信息进行记录，可以节省后续对预测出的缺陷电池图像中的缺陷区域进行人工标注的步骤，从而节省人工成本。By recording the position information and size information of the mask area (i.e., the target area where battery defects are to be generated) in advance, the embodiment of the present application can save the subsequent steps of manual annotation of the defect area in the predicted defective battery image, thus Save labor costs.
S12，通过训练好的缺陷图像生成模型分别对各张掩码图像进行处理，得到各张掩码图像分别对应的预测缺陷图像。S12: Process each mask image separately through the trained defect image generation model to obtain predicted defect images corresponding to each mask image.
本申请实施例中，通过训练好的缺陷图像生成模型对掩码图像进行处理的目的是为了对掩码图像中的掩码区域进行缺陷预测，从而将掩码图像中的掩码区域处理为包含电池缺陷的缺陷区域，且保持掩码图像中的非掩码区域不变。即，每张预测缺陷图像相对于对应的掩码图像而言，图像中的非掩码区域保持不变，图像中的掩码区域变为了包含电池缺陷的缺陷区域。也就是说，预测缺陷图像是通过对掩码图像中的掩码区域进行缺陷预测而得到的缺陷电池图像。In the embodiment of the present application, the purpose of processing the mask image through the trained defect image generation model is to predict defects in the mask area in the mask image, thereby processing the mask area in the mask image to include The defective area of the battery defect, while keeping the non-mask area in the mask image unchanged. That is, for each predicted defect image relative to the corresponding mask image, the non-mask area in the image remains unchanged, and the mask area in the image becomes a defect area containing battery defects. That is, the predicted defect image is a defective battery image obtained by performing defect prediction on the mask area in the mask image.
可以理解的是，为了区分真实的缺陷电池图像与通过缺陷图像生成模型生成的缺陷电池图像，本申请实施例将通过缺陷图像生成模型生成的缺陷电池图像定义为预测缺陷图像。其中，真实的缺陷电池图像指对实际生产过程中产生的表面存在缺陷的缺陷电池进行拍摄得到的可见光图像。It can be understood that in order to distinguish real defective battery images from defective battery images generated by the defect image generation model, embodiments of the present application define the defective battery images generated by the defect image generation model as predicted defect images. Among them, the real defective battery image refers to the visible light image obtained by photographing defective batteries with surface defects produced during the actual production process.
请参阅图3，为本申请实施例提供的一种缺陷图像生成模型的示意性结构图。如图3所示，该缺陷图像生成模型可以包括n个级联的缺陷预测网络31。其中，n为大于1的整数。每个缺陷预测网络31均可以包括缺陷判别模块310和缺陷生成模块320。其中，每一级缺陷预测网络31中的缺陷判别模块310的输入端与本级缺陷预测网络31中的缺陷生成模块320的第一输入端共接，并作为本级缺陷预测网络31的输入端；每一级缺陷预测网络31中的缺陷判别模块310的输出端与本级缺陷预测网络31中的缺陷生成模块320的第二输入端连接；每一级缺陷预测网络31中的缺陷生成模块320的输出端作为本级缺陷预测网络31中的输出端，并与下一级缺陷预测网络31的输入端连接；第一级缺陷预测网络31的输入端作为缺陷图像生成模型的输入端；任意一级缺陷预测网络31的输出端均可以作为陷图像生成模型的输出端。Please refer to Figure 3, which is a schematic structural diagram of a defect image generation model provided by an embodiment of the present application. As shown in FIG. 3 , the defect image generation model may includen cascaded defect prediction networks 31 . Among them,n is an integer greater than 1. Each defect prediction network 31 may include a defect identification module 310 and a defect generation module 320. Among them, the input end of the defect identification module 310 in each level of defect prediction network 31 is commonly connected with the first input end of the defect generation module 320 in the current level defect prediction network 31, and serves as the input end of the current level defect prediction network 31. ; The output end of the defect identification module 310 in each level of defect prediction network 31 is connected to the second input end of the defect generation module 320 in the current level defect prediction network 31 ; the defect generation module 320 in each level of defect prediction network 31 The output end of the defect prediction network 31 of this level is used as the output end of the defect prediction network 31 of this level, and is connected with the input end of the next level defect prediction network 31; the input end of the first level defect prediction network 31 is used as the input end of the defect image generation model; any one The output terminals of the level defect prediction network 31 can be used as the output terminals of the trap image generation model.
每一级缺陷预测网络31均可以预先配置有一个噪声系数。例如，第i级缺陷预测网络31的噪声系数可以为zi，1≤i≤n，且1>z1>z2>z3>……>zn-1>zn>0。其中，z1为最靠近缺陷图像生成模型的输入端的缺陷预测网络31对应的噪声系数，zn为最靠近缺陷图像生成模型的输入端的缺陷预测网络31对应的噪声系数。Each level of defect prediction network 31 may be pre-configured with a noise coefficient. For example, the noise coefficient of thei-th level defect prediction network 31 may bezi , 1≤i ≤n , and 1>z1 >z2 >z3 >...>zn-1 >zn >0. Among them,z1 is the noise coefficient corresponding to the defect prediction network 31 closest to the input end of the defect image generation model, andzn is the noise coefficient corresponding to the defect prediction network 31 closest to the input end of the defect image generation model.
在具体应用中，每一级缺陷预测网络31对应的噪声系数zi的初始值可以是由用户预先设置的，每一级缺陷预测网络31对应的噪声系数zi的最终值可以是缺陷图像生成模型在训练过程中学习得到的。其中，噪声系数zi的最终值指缺陷图像生成模型训练完成后各级缺陷预测网络31对应的噪声系数zi的值。In specific applications, the initial value of the noise coefficientzi corresponding to each level of the defect prediction network 31 can be preset by the user, and the final value of the noise coefficientzi corresponding to each level of the defect prediction network 31 can be generated by the defect image. The model learns during the training process. Among them, the final value of the noise coefficientzi refers to the value of the noise coefficientzi corresponding to the defect prediction network 31 at each level after the training of the defect image generation model is completed.
每一级缺陷预测网络31中的缺陷判别模块310可以用于引导本级缺陷预测网络31中的缺陷生成模块320生成本级缺陷预测网络31对应的预测缺陷图像。The defect identification module 310 in each level of defect prediction network 31 can be used to guide the defect generation module 320 in the current level of defect prediction network 31 to generate predicted defect images corresponding to the current level of defect prediction network 31 .
示例性的，缺陷判别模块310可以是基于真实的缺陷样本图像集训练得到的。缺陷样本图像集多张已被标注缺陷区域的缺陷电池图像。缺陷电池图像是对表面存在缺陷的缺陷电池进行拍摄得到的可见光图像。关于缺陷判别模块310的具体训练过程将在下一个实施例中进行详细说明，此处暂不详述。For example, the defect identification module 310 may be trained based on a real defect sample image set. The defect sample image set contains multiple defective battery images with defective areas marked. The defective battery image is a visible light image taken of a defective battery with defects on its surface. The specific training process of the defect identification module 310 will be described in detail in the next embodiment, and will not be described in detail here.
可以理解的是，为了便于描述，以下将上一级级缺陷预测网络31输出的预测缺陷图像称为上一级预测缺陷图像，将本级缺陷预测网络31生成的预测缺陷图像称为本级预测缺陷图像，将缺陷图像生成模型最终输出的预测缺陷图像称为输入端的掩码图像对应的预测缺陷图像。对于第1级缺陷预测网络31而言，其对应的上一级预测缺陷图像即为缺陷图像生成模型的输入端的掩码图像。It can be understood that, for the convenience of description, in the following, the predicted defect image output by the upper-level defect prediction network 31 will be called the upper-level predicted defect image, and the predicted defect image generated by the current-level defect prediction network 31 will be called the current-level prediction. Defect image, the predicted defect image finally output by the defect image generation model is called the predicted defect image corresponding to the mask image at the input end. For the first-level defect prediction network 31, its corresponding upper-level predicted defect image is the mask image at the input end of the defect image generation model.
具体地，每一级缺陷预测网络31中的缺陷判别模块310可以用于对接收到的上一级预测缺陷图像中的掩码区域进行进行缺陷判别，并提取上一级预测缺陷图像中的掩码区域的梯度信息，且向本级缺陷预测网络31中的缺陷生成模块320发送缺陷判别结果以及上述梯度信息。Specifically, the defect identification module 310 in each level of defect prediction network 31 can be used to perform defect identification on the mask area in the received upper-level predicted defect image, and extract the mask area in the upper-level predicted defect image. code the gradient information of the region, and send the defect identification result and the above-mentioned gradient information to the defect generation module 320 in the defect prediction network 31 of the current level.
其中，缺陷判别结果可以包括：上一级预测缺陷图像中的掩码区域不是缺陷区域，以及上一级预测缺陷图像中的掩码区域是缺陷区域。The defect discrimination results may include: the mask area in the upper-level predicted defect image is not a defect area, and the mask area in the upper-level predicted defect image is a defect area.
掩码区域的梯度信息可以通过尺寸与掩码区域的尺寸相同的梯度图像表示。梯度图像可以是基于现有的梯度特征提取网络对掩码区域进行处理得到的。梯度特征提取网络例如可以为基于索贝尔（sobel）算子的梯度特征提取网络。The gradient information of the mask area can be represented by a gradient image with the same size as the mask area. The gradient image can be obtained by processing the mask area based on the existing gradient feature extraction network. The gradient feature extraction network may be, for example, a gradient feature extraction network based on the Sobel operator.
在一种可选的实现方式中，在上一级预测缺陷图像中的掩码区域不是缺陷区域的情况下，本级缺陷预测网络31中的缺陷生成模块320可以基于上一级预测缺陷图像、上一级预测缺陷图像中掩码区域的梯度信息以及本级缺陷预测网络31对应的噪声系数，生成本级预测缺陷图像，且将本级预测缺陷图像输出至下一级缺陷生成模块320中。In an optional implementation, when the mask area in the upper-level predicted defect image is not a defect area, the defect generation module 320 in the current-level defect prediction network 31 can be based on the upper-level predicted defect image, The gradient information of the mask area in the upper level predicted defect image and the noise coefficient corresponding to the current level defect prediction network 31 generate the current level predicted defect image, and the current level predicted defect image is output to the next level defect generation module 320 .
在另一种可选的实现方式中，在上一级预测缺陷图像中的掩码区域是缺陷区域的情况下，本级缺陷预测网络31中的缺陷生成模块320可以用于直接将上一级预测缺陷图像输出至缺陷图像生成模型的输出端，作为掩码图像对应的预测缺陷图像端。该实现方式中，在任意一级缺陷预测网络中的缺陷判别模块310判别出上一级预测缺陷图像中的掩码区域是缺陷区域时，证明缺陷图像生成模型此时已经生成较为真实的预测缺陷图像，因此，通过将上一级预测缺陷图像直接输出至缺陷图像生成模型的输出端，既保证了生成的预测缺陷图像的真实性，可以提高缺陷图像生成模型的预测缺陷图像生成效率。In another optional implementation, when the mask area in the upper-level predicted defect image is a defect area, the defect generation module 320 in the current-level defect prediction network 31 can be used to directly convert the upper-level mask area into the defective image. The predicted defect image is output to the output end of the defect image generation model as the predicted defect image end corresponding to the mask image. In this implementation, when the defect identification module 310 in any one-level defect prediction network determines that the mask area in the upper-level predicted defect image is a defect area, it is proved that the defect image generation model has generated a more realistic predicted defect at this time. Therefore, by directly outputting the upper-level predicted defect image to the output end of the defect image generation model, it not only ensures the authenticity of the generated predicted defect image, but also improves the efficiency of the predicted defect image generation of the defect image generation model.
在一个具体的实现方式中，在上一级预测缺陷图像中的掩码区域不是缺陷区域的情况下，本级缺陷预测网络31中的缺陷生成模块320可以先随机生成一个尺寸与掩码区域的尺寸相同的噪声图像；再基于上一级预测缺陷图像、上一级预测缺陷图像中掩码区域的梯度信息、本级缺陷预测网络31对应的噪声系数以及本级缺陷预测网络31中的缺陷生成模块320生成的上述噪声图像，采用如下公式生成本级预测缺陷图像：In a specific implementation, when the mask area in the upper-level predicted defect image is not a defect area, the defect generation module 320 in the current-level defect prediction network 31 can first randomly generate a mask area with a size similar to that of the mask area. Noise images of the same size; then based on the upper level predicted defect image, the gradient information of the mask area in the upper level predicted defect image, the noise coefficient corresponding to the current level defect prediction network 31 and the defect generation in the current level defect prediction network 31 The above noise image generated by module 320 uses the following formula to generate this level predicted defect image:
； ;
其中，Xi为本级预测缺陷图像，zi为本级缺陷预测网络31对应的噪声系数，Xt-1为上一级预测缺陷图像，Gi为随机生成的噪声图像，Ti-1为上一级预测缺陷图像中掩码区域的梯度信息。Among them,Xiis the predicteddefect imageatthislevel ,zi is the noise coefficient corresponding to the defectprediction network31 at this level, Predict the gradient information of the mask area in the defect image for the upper level.
通过上述公式可以看出，每一级缺陷缺陷预测网络31中的缺陷生成模块320在基于上一级预测缺陷图像生成本级缺陷预测图像时，会在上一级预测缺陷图像的基础上，减去一个随机生成的噪声图像，并叠加一次上一级预测缺陷图像中掩码区域的梯度信息，如此能够进一步增强所预测的缺陷的边缘信息，加快缺陷图像生成模型生成预测缺陷图像的速度。It can be seen from the above formula that when the defect generation module 320 in each level of defect prediction network 31 generates a defect prediction image of this level based on the defect prediction image of the previous level, it will reduce the defect prediction image of the previous level based on the prediction defect image of the previous level. Remove a randomly generated noise image and superimpose the gradient information of the mask area in the upper-level predicted defect image. This can further enhance the edge information of the predicted defect and speed up the defect image generation model to generate predicted defect images.
在一个具体的实现方式中，Gi可以是电子设备从多元标准正态分布中采样得到的高斯噪声图像。In a specific implementation,Gi may be a Gaussian noise image sampled by the electronic device from a multivariate standard normal distribution.
本实施例中，生成预测缺陷图像的过程相当于通过缺陷图像生成模型对掩码图像的掩码区域进行去燥的过程，由于缺陷图像生成模型中设置有多个级联的缺陷预测网络，且每一级缺陷预测网络中均设置有用于引导缺陷生成模块生成预测缺陷图像的缺陷判别模块，且缺陷判别模块是基于真实的缺陷样本图像集训练得到的，因此，在每一级缺陷判别模块输出的缺陷判别结果以及梯度信息的引导下，缺陷图像生成模型能够快速且高效地生成高质量的预测缺陷图像。In this embodiment, the process of generating a predicted defect image is equivalent to the process of de-drying the mask area of the mask image through the defect image generation model. Since the defect image generation model is provided with multiple cascaded defect prediction networks, and Each level of the defect prediction network is equipped with a defect identification module that guides the defect generation module to generate predicted defect images, and the defect identification module is trained based on the real defect sample image set. Therefore, at each level, the defect identification module outputs Guided by the defect identification results and gradient information, the defect image generation model can quickly and efficiently generate high-quality predicted defect images.
S13，基于所有掩码图像对应的预测缺陷图像以及所有掩码图像中的掩码区域的位置信息和尺寸信息，生成第一训练集，并通过第一训练集对缺陷检测模型进行训练。S13. Generate a first training set based on the predicted defect images corresponding to all mask images and the position information and size information of the mask areas in all mask images, and train the defect detection model through the first training set.
在一个具体的实现方式中，基于所有掩码图像对应的预测缺陷图像以及所有掩码图像中的掩码区域的位置信息和尺寸信息，生成第一训练集，可以包括：In a specific implementation, the first training set is generated based on the predicted defect images corresponding to all mask images and the position information and size information of the mask areas in all mask images, which may include:
将各张掩码图像对应的预测缺陷图像分别与对应掩码图像中的掩码区域的位置信息和尺寸信息建立关联关系，将每一组具有关联关系的预测缺陷图像与掩码区域的位置信息和尺寸信息分别作为第一训练集的一条训练数据，得到第一训练集。Establish an association relationship between the predicted defect images corresponding to each mask image and the position information and size information of the mask area in the corresponding mask image, and associate each group of predicted defect images with an association relationship with the position information of the mask area. and size information are respectively used as a piece of training data in the first training set to obtain the first training set.
本申请实施例中，缺陷检测模型可以包括目标检测网络。In this embodiment of the present application, the defect detection model may include a target detection network.
示例性的，目标检测网络可以为YOLOv5（you only look once version 5）网络。需要说明的是，关于YOLOv5网络的具体结构以及工作原理等可以参考现有技术中的相关描述，此处不对其进行详述。For example, the target detection network can be a YOLOv5 (you only look once version 5) network. It should be noted that for the specific structure and working principle of the YOLOv5 network, please refer to the relevant descriptions in the prior art, which will not be described in detail here.
缺陷检测模型具体可以用于检测待检测电池图像中是否包含电池缺陷，以及在待检测电池图像中包含电池缺陷的情况下，对电池缺陷所在的缺陷区域进行定位，并输出电池缺陷所在的缺陷区域的位置信息和尺寸信息。Specifically, the defect detection model can be used to detect whether the battery image to be detected contains battery defects, and if the battery image to be detected contains battery defects, locate the defect area where the battery defect is located, and output the defect area where the battery defect is located. location information and size information.
在通过第一训练集对缺陷检测模型进行训练时，可以将第一训练集的每条训练数据中的预测缺陷图像作为缺陷检测模型的输入，将与该条训练数据中的预测缺陷图像具有关联关系的掩码区域的位置信息和尺寸信息作为缺陷检测模型的输出，对缺陷检测模型进行训练，从而使缺陷检测模型能够在训练过程中学习到其内部各个参数（例如每一级缺陷预测网络31对应的噪声系数zi）的值。When training the defect detection model through the first training set, the predicted defect image in each piece of training data of the first training set can be used as the input of the defect detection model, and will be associated with the predicted defect image in the piece of training data. The position information and size information of the related mask area are used as the output of the defect detection model to train the defect detection model, so that the defect detection model can learn its internal parameters (such as each level of defect prediction network 31 The corresponding value of the noise coefficientzi ).
S14，通过训练好的缺陷检测模型对待检测电池图像进行缺陷检测。S14, perform defect detection on the battery image to be detected through the trained defect detection model.
本申请实施例中，在通过第一训练集对缺陷检测模型进行训练之后，可以将训练好的缺陷检测模型应用于实际电池缺陷检测场景中。In the embodiment of the present application, after the defect detection model is trained through the first training set, the trained defect detection model can be applied to actual battery defect detection scenarios.
例如，当需要对实际生产过程中的待检测电池图像进行缺陷检测时，可以将待检测电池图像输入至训练好的缺陷检测模型中，训练好的缺陷检测模型便可以输出待检测电池图像中是否包含电池缺陷的缺陷检测结果；且在待检测电池图像中包含电池缺陷的情况下，训练好的缺陷检测模型还可以输出电池缺陷所在的缺陷区域的位置信息和尺寸信息，从而能够实现对电池缺陷的检测。For example, when it is necessary to perform defect detection on the battery image to be inspected in the actual production process, the battery image to be inspected can be input into the trained defect detection model, and the trained defect detection model can output whether the battery image to be inspected is Contains defect detection results of battery defects; and when the battery image to be detected contains battery defects, the trained defect detection model can also output the location information and size information of the defect area where the battery defect is located, so that battery defects can be detected detection.
请参阅图4，为本申请另一实施例提供的一种基于小样本的自适应缺陷检测方法的示意性流程图。与图1对应的实施例相比，本实施例提供的基于小样本的自适应缺陷检测方法，在S12之前，还可以包括S41，详述如下：Please refer to FIG. 4 , which is a schematic flow chart of an adaptive defect detection method based on small samples provided by another embodiment of the present application. Compared with the embodiment corresponding to Figure 1, the adaptive defect detection method based on small samples provided by this embodiment may also include S41 before S12, as detailed below:
S41，基于缺陷样本图像集和正常样本图像集制作第二训练集。S41. Create a second training set based on the defective sample image set and the normal sample image set.
需要说明的是，关于缺陷样本图像集和正常样本图像集的具体说明可以参考前述实施例中的相关描述，此处不对其进行赘述。It should be noted that for specific descriptions of the defective sample image set and the normal sample image set, reference can be made to the relevant descriptions in the foregoing embodiments, which will not be described again here.
本申请实施例中，第二训练集可以包括多张局部缺陷图像和多张局部正常图像。其中，局部缺陷图像可以是从缺陷样本图像集中得到的，局部正常图像可以是从正常样本图像集中得到的。基于此，在一个具体的实现方式中，S41可以包括如图5所示的S411~S413，详述如下：In this embodiment of the present application, the second training set may include multiple local defect images and multiple local normal images. Among them, the local defect image can be obtained from the defect sample image set, and the local normal image can be obtained from the normal sample image set. Based on this, in a specific implementation, S41 may include S411~S413 as shown in Figure 5, as detailed below:
S411，从缺陷样本图像集的每张缺陷电池图像中裁剪出缺陷区域，将裁剪出的缺陷区域分别确定为相应缺陷电池图像对应的局部缺陷图像。S411: Cut out defective areas from each defective battery image in the defective sample image set, and determine the cutout defective areas as local defect images corresponding to the corresponding defective battery images.
可以理解的是，由于每张缺陷电池图像的缺陷区域已被预先标注，因此，终端设备可以直接从每张缺陷电池图像中将被标注的缺陷区域裁剪出来，并将裁剪出的缺陷区域确定为相应缺陷电池图像对应的局部缺陷图像。即，每张局部缺陷图像属于对应的缺陷电池图像的一部分。It can be understood that since the defective area of each defective battery image has been marked in advance, the terminal device can directly crop out the marked defective area from each defective battery image and determine the cropped defective area as The local defect image corresponding to the corresponding defective battery image. That is, each local defect image belongs to a part of the corresponding defective battery image.
可以理解的是，由于不同缺陷电池图像的缺陷区域的大小可能相同，也可能不同，因此，裁剪出的不同局部缺陷图像的尺寸可以相同，也可以不同，本申请实施例对此不做特别限定。It can be understood that since the size of the defect areas of different defective battery images may be the same or different, the sizes of different cropped local defect images may be the same or different, which is not particularly limited in the embodiments of the present application. .
S412，从正常样本图像集的多张正常电池图像中裁剪出多张局部正常图像，每张局部正常图像对应一张局部缺陷图像，每张局部正常图像的尺寸与对应的局部缺陷图像的尺寸相同。S412, crop out multiple partial normal images from multiple normal battery images in the normal sample image set. Each partial normal image corresponds to a local defect image. The size of each partial normal image is the same as the size of the corresponding local defect image. .
在具体应用中，终端设备可以从正常样本图像集的任意多张正常图像中裁剪出多张尺寸分别与对应的局部缺陷图像的尺寸相同的局部正常图像。In a specific application, the terminal device can crop out multiple partial normal images with the same size as the corresponding local defect image from any multiple normal images in the normal sample image set.
S413，将裁剪出的局部缺陷图像和局部正常图像分别作为第二训练集的正样本集和负样本集，得到第二训练集。S413. Use the cropped local defective image and the local normal image as the positive sample set and the negative sample set of the second training set respectively to obtain the second training set.
本申请实施例中，第二训练集包括由多张局部缺陷图像组成的负样本集和由多张局部正常图像组成的正样本集。In this embodiment of the present application, the second training set includes a negative sample set composed of multiple local defective images and a positive sample set composed of multiple local normal images.
示例性的，请参阅图6，为本申请实施例提供的一种第二训练集的制作过程示意图。如图6所示，终端设备可以从缺陷样本图像集中的每张缺陷电池图像61中直接裁剪出被标注的缺陷区域，并将裁剪出的缺陷区域分别确定为相应缺陷电池图像61对应的局部缺陷图像610，且将由所有局部缺陷图像610组成的图像集确定为第二训练集的负样本集。For example, please refer to FIG. 6 , which is a schematic diagram of the production process of a second training set provided by an embodiment of the present application. As shown in Figure 6, the terminal device can directly crop out the marked defect areas from each defective battery image 61 in the defect sample image set, and determine the cropped defect areas as local defects corresponding to the corresponding defective battery images 61. Image 610, and the image set consisting of all local defect images 610 is determined as the negative sample set of the second training set.
终端设备可以从正常样本图像集中的各张正常电池图像62中随机裁剪出多张尺寸分别与对应的局部缺陷图像610的尺寸相同的局部正常图像620，并将由所有局部正常图像620组成的图像集确定为第二训练集的正样本集。The terminal device can randomly crop out multiple partial normal images 620 with the same size as the corresponding local defect image 610 from each normal battery image 62 in the normal sample image set, and create an image set composed of all partial normal images 620 Determine the positive sample set as the second training set.
S42，通过第二训练集训练预设二分类器，并将训练完成的预设二分类器确定为缺陷判别模块，将缺陷判别模块与梯度信息提取单元的组合确定为缺陷判别模块。S42, train the preset two classifiers through the second training set, determine the trained preset two classifiers as the defect identification module, and determine the combination of the defect identification module and the gradient information extraction unit as the defect identification module.
其中，预设二分类器和梯度信息提取单元均可以根据实际需求进行选择。示例性的，预设二分类器可以为U-net网络。梯度信息提取单元可以为基于sobel算子的梯度特征提取网络。Among them, the preset binary classifier and gradient information extraction unit can be selected according to actual needs. For example, the preset binary classifier may be a U-net network. The gradient information extraction unit can be a gradient feature extraction network based on the sobel operator.
由于用于训练缺陷判别模块的第二训练集来自于真实的缺陷电池图像和正常电池图像，因此，缺陷判别模块具有较高的缺陷判别准确率。Since the second training set used to train the defect identification module comes from real defective battery images and normal battery images, the defect identification module has a high defect identification accuracy.
需要说明的是，在对缺陷图像生成模型进行训练时，需要先对缺陷图像生成模型中的缺陷判别模块进行训练，在缺陷判别模块训练完成后，再基于训练好的缺陷判别模块，对缺陷图像生成模型中的缺陷生成模块进行训练。It should be noted that when training the defect image generation model, the defect discrimination module in the defect image generation model needs to be trained first. After the training of the defect discrimination module is completed, the defect image is then trained based on the trained defect discrimination module. The defect generation module in the generative model is trained.
在对缺陷图像生成模型中的缺陷生成模块进行训练时，可以预先生成一张像素为0且尺寸与局部缺陷图像的尺寸相同的第一图像，并将该第一图像作为缺陷图像生成模型的输入，将上述负样本集中的各张局部缺陷图像作为缺陷图像生成模型的输出，对缺陷图像生成模型中的缺陷生成模块进行训练，从而使缺陷图像生成模型在训练过程中学习到其内部各个参数的值。When training the defect generation module in the defect image generation model, a first image with a pixel of 0 and the same size as the local defect image can be pre-generated, and this first image can be used as the input of the defect image generation model. , use each local defect image in the above negative sample set as the output of the defect image generation model, and train the defect generation module in the defect image generation model, so that the defect image generation model can learn the parameters of its internal parameters during the training process. value.
可以理解的是，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。It can be understood that the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any influence on the implementation process of the embodiment of the present application. limited.
基于上述实施例所提供的基于小样本的自适应缺陷检测方法，本申请实施例进一步给出实现上述方法实施例的终端设备的实施例。请参阅图7，为本申请实施例提供的一种终端设备的结构示意图。为了便于说明，仅示出了与本实施例相关的部分。如图7所示，终端设备70可以包括：掩码生成单元71、缺陷生成单元72、第一训练单元73及缺陷检测单元74。其中：Based on the small sample-based adaptive defect detection method provided in the above embodiments, embodiments of the present application further provide embodiments of terminal equipment that implement the above method embodiments. Please refer to FIG. 7 , which is a schematic structural diagram of a terminal device provided by an embodiment of the present application. For convenience of explanation, only parts related to this embodiment are shown. As shown in FIG. 7 , the terminal device 70 may include: a mask generation unit 71 , a defect generation unit 72 , a first training unit 73 and a defect detection unit 74 . in:
掩码生成单元71用于为正常样本图像集中的每张正常电池图像分别添加一个掩码区域，将添加了掩码区域的所述正常电池图像确定为掩码图像，并记录每张所述掩码图像中的掩码区域的位置信息和尺寸信息。The mask generation unit 71 is configured to add a mask area to each normal battery image in the normal sample image set, determine the normal battery image to which the mask area is added as a mask image, and record each of the mask images. The position information and size information of the mask area in the code image.
缺陷生成单元72用于通过训练好的缺陷图像生成模型分别对各张所述掩码图像进行处理，得到各张所述掩码图像分别对应的预测缺陷图像；所述缺陷图像生成模型包括n个级联的缺陷预测网络；每一级所述缺陷预测网络均包括一个缺陷判别模块和一个缺陷生成模块；每一级缺陷预测网络中的缺陷判别模块用于对上一级预测缺陷图像进行缺陷判别，并提取所述上一级预测缺陷图像中的掩码区域的梯度信息，且向本级缺陷预测网络中的缺陷生成模块发送所述缺陷判别结果和所述梯度信息；每一级缺陷预测网络中的缺陷生成模块用于基于所述判别结果、所述上一级预测缺陷图像以及所述梯度信息，生成本级预测缺陷图像，并向下一级缺陷预测网络发送所述本级预测缺陷图像。The defect generation unit 72 is used to process each of the mask images through a trained defect image generation model to obtain a predicted defect image corresponding to each of the mask images; the defect image generation model includesn Cascading defect prediction network; each level of the defect prediction network includes a defect identification module and a defect generation module; the defect identification module in each level of defect prediction network is used to perform defect identification on the predicted defect images of the previous level , and extract the gradient information of the mask area in the upper-level predicted defect image, and send the defect discrimination result and the gradient information to the defect generation module in the current-level defect prediction network; each level of defect prediction network The defect generation module in is used to generate the current-level predicted defect image based on the discrimination result, the upper-level predicted defect image and the gradient information, and send the current-level predicted defect image to the next-level defect prediction network. .
第一训练单元73用于基于所有所述掩码图像对应的预测缺陷图像以及所有所述掩码图像中的掩码区域的位置信息和尺寸信息，生成第一训练集，并通过所述第一训练集对缺陷检测模型进行训练。The first training unit 73 is configured to generate a first training set based on the predicted defect images corresponding to all the mask images and the position information and size information of the mask areas in all the mask images, and use the first The training set is used to train the defect detection model.
缺陷检测单元74用于通过训练好的缺陷检测模型对待检测电池图像进行缺陷检测。The defect detection unit 74 is used to perform defect detection on the battery image to be detected through the trained defect detection model.
可选的，所述缺陷判别结果包括：所述上一级预测缺陷图像中的掩码区域不是缺陷区域，以及所述上一级预测缺陷图像中的掩码区域是缺陷区域；缺陷生成单元72具体用于：Optionally, the defect discrimination result includes: the mask area in the upper-level predicted defect image is not a defect area, and the mask area in the upper-level predicted defect image is a defect area; defect generation unit 72 Specifically used for:
在第i级缺陷预测网络中，通过所述第i级缺陷预测网络中的缺陷判别模块对上一级预测缺陷图像进行缺陷判别，并提取所述上一级预测缺陷图像中的掩码区域的梯度信息，且向所述第i级缺陷预测网络中的缺陷生成模块发送所述缺陷判别结果和所述梯度信息；其中，第1级缺陷预测网络对应的上一级预测缺陷图像为所述掩码图像；In thei-th level defect prediction network, the defect identification module in thei-th level defect prediction network performs defect identification on the upper-level predicted defect image, and extracts the mask area in the upper-level predicted defect image. Gradient information, and send the defect identification result and the gradient information to the defect generation module in the i-th level defect prediction network; wherein the upper-level predicted defect image corresponding to the first-level defect prediction network is the mask code image;
在所述缺陷判别结果为所述上一级预测缺陷图像中的掩码区域不是缺陷区域的情况下，通过所述第i级缺陷预测网络中的缺陷生成模块基于所述判别结果、所述上一级预测缺陷图像以及所述梯度信息，生成本级预测缺陷图像，并向第i+1级缺陷预测网络输出所述本级预测缺陷图像；When the defect discrimination result is that the mask area in the upper-level predicted defect image is not a defect area, the defect generation module in the i-th level defect prediction network is based on the discrimination result and the upper-level defect prediction network. The first-level predicted defect image and the gradient information generate a current-level predicted defect image, and output the current-level predicted defect image to thei +1th level defect prediction network;
在所述缺陷判别结果为所述上一级预测缺陷图像中的掩码区域是缺陷区域的情况下，通过所述第i级缺陷预测网络中的缺陷生成模块直接将所述上一级预测缺陷图像输出至所述缺陷图像生成模型的输出端，作为所述掩码图像对应的预测缺陷图像。When the defect discrimination result is that the mask area in the upper-level predicted defect image is a defect area, the upper-level predicted defect is directly generated by the defect generation module in the i-th level defect prediction network. The image is output to the output end of the defect image generation model as the predicted defect image corresponding to the mask image.
可选的，缺陷生成单元72具体用于：Optionally, the defect generation unit 72 is specifically used for:
所述第i级缺陷预测网络中的缺陷生成模块随机生成一个尺寸与所述掩码区域的尺寸相同的噪声图像；The defect generation module in thei-th level defect prediction network randomly generates a noise image with the same size as the mask area;
所述第i级缺陷预测网络中的缺陷生成模块基于所述判别结果、所述上一级预测缺陷图像、所述梯度信息以及所述噪声图像，采用以下公式生成本级预测缺陷图像：The defect generation module in the i-th level defect prediction network uses the following formula to generate this level's predicted defect image based on the discrimination result, the upper-level predicted defect image, the gradient information and the noise image:
； ;
其中，Xi为所述本级预测缺陷图像，zi为所述第i级缺陷预测网络对应的噪声系数，Xt-1为所述上一级预测缺陷图像，Gi为所述噪声图像，Ti-1为所述上一级预测缺陷图像中掩码区域的梯度信息；1>z1>z2>z3>……>zn-1>zn>0。Among them,Xi isthe predicteddefect image at thislevel ,zi is the noise coefficient corresponding to thei-th level defect prediction network, ,Ti-1 is the gradient information of the mask area in the upper-level predicted defect image; 1>z1 >z2 >z3 >...>zn-1 >zn >0.
可选的，终端设备70还可以包括：训练集制作单元和第二训练单元。其中：Optionally, the terminal device 70 may also include: a training set production unit and a second training unit. in:
训练集制作单元用于基于缺陷样本图像集和所述正常样本图像集制作第二训练集。The training set making unit is configured to make a second training set based on the defective sample image set and the normal sample image set.
第二训练单元用于通过所述第二训练集训练预设二分类器，并将训练完成的所述二分类器与梯度信息提取单元的组合确定为所述缺陷判别模块。The second training unit is configured to train a preset two-classifier through the second training set, and determine the combination of the trained two-classifier and the gradient information extraction unit as the defect discrimination module.
可选的，训练集制作单元具体用于：Optional, the training set production unit is specifically used for:
从所述缺陷样本图像集的每张缺陷电池图像中裁剪出缺陷区域，将裁剪出的所述缺陷区域分别确定为相应缺陷电池图像对应的局部缺陷图像；Cut out defective areas from each defective battery image in the defective sample image set, and determine the cropped defective areas as local defect images corresponding to the corresponding defective battery images;
从所述正常样本图像集的多张正常电池图像中裁剪出多张局部正常图像，每张所述局部正常图像对应一张所述局部缺陷图像，每张所述局部正常图像的尺寸与对应的所述局部缺陷图像的尺寸相同；A plurality of partially normal images are cut out from multiple normal battery images in the normal sample image set. Each of the partially normal images corresponds to one of the local defect images. The size of each of the partially normal images is the same as the corresponding The local defect images have the same size;
将裁剪出的所述局部缺陷图像和所述局部正常图像分别作为第二训练集的正样本集和负样本集，得到所述第二训练集。The cropped local defective image and the local normal image are respectively used as the positive sample set and the negative sample set of the second training set to obtain the second training set.
可选的，掩码生成单元71具体用于：Optionally, the mask generation unit 71 is specifically used for:
在预设存储区域存储有预设信息列表的情况下，基于所述预设信息列表中各个目标区域的位置信息和尺寸信息，分别为各张正常电池图像添加掩码区域；In the case where a preset information list is stored in the preset storage area, based on the position information and size information of each target area in the preset information list, a mask area is added to each normal battery image respectively;
在所述预设存储区域未存储所述预设信息列表的情况下，分别为各张所述正常电池图像随机添加掩码区域。When the preset information list is not stored in the preset storage area, a mask area is randomly added to each of the normal battery images.
可选的，所述掩码区域用于遮盖所述正常电池图像中的目标区域。掩码生成单元71具体用于：Optionally, the mask area is used to cover the target area in the normal battery image. The mask generation unit 71 is specifically used for:
将所述正常电池图像中目标区域的所有像素的像素值均修改为0。Modify the pixel values of all pixels in the target area in the normal battery image to 0.
需要说明的是，上述单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其具体功能及带来的技术效果，具体可参照方法实施例部分，此处不再赘述。It should be noted that the information interaction, execution process, etc. between the above units are based on the same concept as the method embodiments of the present application. For details of their specific functions and technical effects, please refer to the method embodiments section, which will not be discussed here. Again.
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元完成，即将终端设备的内部结构划分成不同的功能单元，以完成以上描述的全部或者部分功能。实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述终端设备中各单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional units is used as an example. In practical applications, the above functions can be allocated to different functional units according to needs, that is, The internal structure of the terminal device is divided into different functional units to complete all or part of the functions described above. Each functional unit in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware. , can also be implemented in the form of software functional units. In addition, the specific names of each functional unit are only for the convenience of distinguishing each other and are not used to limit the protection scope of the present application. For the specific working process of each unit in the above terminal device, reference can be made to the corresponding process in the foregoing method embodiment, and will not be described again here.
请参阅图8，图8为本申请另一实施例提供的一种终端设备的结构示意图。如图8所示，本实施例提供的终端设备8可以包括：处理器80、存储器81以及存储在存储器81中并可在处理器80上运行的计算机程序82，例如基于小样本的自适应缺陷检测方法对应的程序。处理器80执行计算机程序82时实现上述基于小样本的自适应缺陷检测方法实施例中的步骤，例如图1所示的S11~S14。或者，处理器80执行计算机程序82时实现上述终端设备实施例中各模块/单元的功能，例如图7所示的单71~74的功能。Please refer to FIG. 8 , which is a schematic structural diagram of a terminal device provided by another embodiment of the present application. As shown in Figure 8, the terminal device 8 provided by this embodiment may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and runable on the processor 80, such as adaptive defect detection based on small samples. Procedure corresponding to the detection method. When the processor 80 executes the computer program 82, it implements the steps in the above embodiment of the adaptive defect detection method based on small samples, such as S11 to S14 shown in Figure 1. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module/unit in the above terminal device embodiment, such as the functions of the units 71 to 74 shown in FIG. 7 .
示例性的，计算机程序82可以被分割成一个或多个模块/单元，一个或者多个模块/单元被存储在存储器81中，并由处理器80执行，以完成本申请。一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述计算机程序82在终端设备8中的执行过程。例如，计算机程序82可以被分割成掩码生成单元、缺陷生成单元、第一训练单元及缺陷检测单元，各单元的具体功能请参阅图7对应的实施例中的相关描述，此处不赘述。For example, the computer program 82 may be divided into one or more modules/units, and one or more modules/units are stored in the memory 81 and executed by the processor 80 to complete the present application. One or more modules/units may be a series of computer program instruction segments capable of completing specific functions. The instruction segments are used to describe the execution process of the computer program 82 in the terminal device 8 . For example, the computer program 82 can be divided into a mask generation unit, a defect generation unit, a first training unit and a defect detection unit. For the specific functions of each unit, please refer to the relevant descriptions in the corresponding embodiment of FIG. 7 and will not be described again here.
本领域技术人员可以理解，图8仅仅是终端设备8的示例，并不构成对终端设备8的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件。Those skilled in the art can understand that FIG. 8 is only an example of the terminal device 8 and does not constitute a limitation on the terminal device 8. It may include more or less components than shown in the figure, or some components may be combined, or different components may be used. .
处理器80可以是中央处理单元（central processing unit，CPU），还可以是其他通用处理器、数字信号处理器（digital signal processor，DSP）、专用集成电路（application specific integrated circuit，ASIC）、现成可编程门阵列（field-programmable gate array，FPGA）或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 80 may be a central processing unit (CPU), or other general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or an off-the-shelf processor. Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
存储器81可以是终端设备8的内部存储单元，例如终端设备8的硬盘或内存。存储器81也可以是终端设备8的外部存储设备，例如终端设备8上配备的插接式硬盘、智能存储卡（smart media card，SMC）、安全数字（secure digital，SD）卡或闪存卡（flash card）等。进一步地，存储器81还可以既包括终端设备8的内部存储单元也包括外部存储设备。存储器81用于存储计算机程序以及终端设备所需的其他程序和数据。存储器81还可以用于暂时地存储已经输出或者将要输出的数据。The memory 81 may be an internal storage unit of the terminal device 8 , such as a hard disk or memory of the terminal device 8 . The memory 81 may also be an external storage device of the terminal device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card or a flash memory card equipped on the terminal device 8. card) etc. Further, the memory 81 may also include both an internal storage unit of the terminal device 8 and an external storage device. The memory 81 is used to store computer programs and other programs and data required by the terminal device. The memory 81 may also be used to temporarily store data that has been output or is to be output.
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序被处理器执行时实现上述方法实施例所述的基于小样本的自适应缺陷检测方法中的各个步骤。Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the small sample-based adaptive defect detection method described in the above method embodiments is implemented. various steps in.
本申请实施例提供了一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备实现上述各个方法实施例中的步骤。Embodiments of the present application provide a computer program product. When the computer program product is run on a terminal device, the terminal device implements the steps in each of the above method embodiments.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参照其它实施例的相关描述。In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or recorded in a certain embodiment, you can refer to the relevant descriptions of other embodiments.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。The above-described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still implement the above-mentioned implementations. The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of this application, and should be included in within the protection scope of this application.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311324323.1ACN117094986B (en) | 2023-10-13 | 2023-10-13 | Adaptive defect detection method and terminal equipment based on small samples |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311324323.1ACN117094986B (en) | 2023-10-13 | 2023-10-13 | Adaptive defect detection method and terminal equipment based on small samples |
| Publication Number | Publication Date |
|---|---|
| CN117094986Atrue CN117094986A (en) | 2023-11-21 |
| CN117094986B CN117094986B (en) | 2024-04-05 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311324323.1AActiveCN117094986B (en) | 2023-10-13 | 2023-10-13 | Adaptive defect detection method and terminal equipment based on small samples |
| Country | Link |
|---|---|
| CN (1) | CN117094986B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118297923A (en)* | 2024-04-19 | 2024-07-05 | 北京瓦特曼智能科技有限公司 | Steel coil defect identification method and device |
| CN119904625A (en)* | 2025-03-27 | 2025-04-29 | 杭州萤石软件有限公司 | Target detection method, system and device based on image generation model |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190251401A1 (en)* | 2018-02-15 | 2019-08-15 | Adobe Inc. | Image composites using a generative adversarial neural network |
| US20190370955A1 (en)* | 2018-06-05 | 2019-12-05 | Kla-Tencor Corporation | Active learning for defect classifier training |
| CN111223093A (en)* | 2020-03-04 | 2020-06-02 | 武汉精立电子技术有限公司 | AOI defect detection method |
| WO2021017372A1 (en)* | 2019-08-01 | 2021-02-04 | 中国科学院深圳先进技术研究院 | Medical image segmentation method and system based on generative adversarial network, and electronic equipment |
| CN112396110A (en)* | 2020-11-20 | 2021-02-23 | 南京大学 | Method for generating anti-cascade network augmented image |
| CN114155244A (en)* | 2022-02-10 | 2022-03-08 | 北京阿丘科技有限公司 | Defect detection method, device, equipment and storage medium |
| CN114898177A (en)* | 2022-05-09 | 2022-08-12 | 梅卡曼德（北京）机器人科技有限公司 | Defect image generation method, model training method, equipment, medium and product |
| CN115115905A (en)* | 2022-06-13 | 2022-09-27 | 苏州大学 | High-mobility image countermeasure sample generation method based on generation model |
| CN115760823A (en)* | 2022-11-28 | 2023-03-07 | 杭州海康机器人股份有限公司 | Defect detection method and device and electronic equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190251401A1 (en)* | 2018-02-15 | 2019-08-15 | Adobe Inc. | Image composites using a generative adversarial neural network |
| US20190370955A1 (en)* | 2018-06-05 | 2019-12-05 | Kla-Tencor Corporation | Active learning for defect classifier training |
| WO2021017372A1 (en)* | 2019-08-01 | 2021-02-04 | 中国科学院深圳先进技术研究院 | Medical image segmentation method and system based on generative adversarial network, and electronic equipment |
| CN111223093A (en)* | 2020-03-04 | 2020-06-02 | 武汉精立电子技术有限公司 | AOI defect detection method |
| CN112396110A (en)* | 2020-11-20 | 2021-02-23 | 南京大学 | Method for generating anti-cascade network augmented image |
| CN114155244A (en)* | 2022-02-10 | 2022-03-08 | 北京阿丘科技有限公司 | Defect detection method, device, equipment and storage medium |
| CN114898177A (en)* | 2022-05-09 | 2022-08-12 | 梅卡曼德（北京）机器人科技有限公司 | Defect image generation method, model training method, equipment, medium and product |
| CN115115905A (en)* | 2022-06-13 | 2022-09-27 | 苏州大学 | High-mobility image countermeasure sample generation method based on generation model |
| CN115760823A (en)* | 2022-11-28 | 2023-03-07 | 杭州海康机器人股份有限公司 | Defect detection method and device and electronic equipment |
| Title |
|---|
| 王健 等: "基于生成对抗网络和RetinaNet的销钉缺陷识别", 华南理工大学学报(自然科学版), no. 02, pages 5 - 12* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118297923A (en)* | 2024-04-19 | 2024-07-05 | 北京瓦特曼智能科技有限公司 | Steel coil defect identification method and device |
| CN119904625A (en)* | 2025-03-27 | 2025-04-29 | 杭州萤石软件有限公司 | Target detection method, system and device based on image generation model |
| CN119904625B (en)* | 2025-03-27 | 2025-07-25 | 杭州萤石软件有限公司 | Target detection method, system and device based on image generation model |
| Publication number | Publication date |
|---|---|
| CN117094986B (en) | 2024-04-05 |
| Publication | Publication Date | Title |
|---|---|---|
| CN111353956B (en) | Image restoration method, device, computer equipment and storage medium | |
| CN117094986A (en) | Self-adaptive defect detection method based on small sample and terminal equipment | |
| WO2020024484A1 (en) | Method and device for outputting data | |
| CN110619656B (en) | Face detection tracking method and device based on binocular camera and electronic equipment | |
| CN111881803B (en) | An animal face recognition method based on improved YOLOv3 | |
| CN114596242A (en) | Defect detection method, apparatus, electronic device, and computer-readable storage medium | |
| CN118314345A (en) | A small sample segmentation method, system, medium and device for eliminating defects within metal surface defect classes | |
| CN110533020A (en) | A kind of recognition methods of text information, device and storage medium | |
| CN116071586A (en) | Image screening method, device, electronic equipment and computer readable storage medium | |
| CN115410281A (en) | Electronic signature identification method, device, equipment and storage medium | |
| CN114511702A (en) | Remote sensing image segmentation method and system based on multi-scale weighted attention | |
| CN112750065B (en) | Carrier object processing and watermark embedding method, device and electronic equipment | |
| CN118429880A (en) | Image recognition method, device, electronic device and computer-readable storage medium | |
| CN104851114B (en) | A kind of method and terminal for realizing image local discoloration | |
| CN114648751A (en) | Method, device, terminal and storage medium for processing video subtitles | |
| CN117292120A (en) | Light-weight visible light insulator target detection method and system | |
| CN113870210B (en) | Image quality assessment method, device, equipment and storage medium | |
| CN116486153A (en) | Image classification method, device, equipment and storage medium | |
| CN114463306A (en) | Pattern recognition method, device, electronic device and storage medium | |
| CN114201637A (en) | Similar picture searching method, device, equipment and storage medium | |
| CN114818958B (en) | Generation of adversarial samples, model training, image recognition method and device | |
| CN111899154A (en) | Cartoon video generation method, cartoon generation device, cartoon generation equipment and cartoon generation medium | |
| US12315102B2 (en) | Method for recognizing images, electronic device, and non-transitory storage medium | |
| CN111832491B (en) | Text detection method, device and processing equipment | |
| CN112733574B (en) | Face recognition method and device and computer readable storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |