CN114722228A

Movatterモバイル変換

Info

Publication number: CN114722228A
Application number: CN202210289027.1A
Authority: CN
Inventors: 张枫
Original assignee: Guangzhou Huya Technology Co Ltd
Current assignee: Guangzhou Huya Technology Co Ltd
Priority date: 2022-03-22
Filing date: 2022-03-22
Publication date: 2022-07-08

Abstract

The application discloses an image classification method, a related device and equipment, wherein the image classification method comprises the following steps: acquiring an image to be classified; performing feature extraction on the image to be classified to obtain a feature vector of the image to be classified; performing first classification processing, second classification processing, foreground detection and feature classification processing on the feature vectors to obtain a first classification result, a second classification result, a foreground detection result and a feature classification result of the feature vectors; obtaining a classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result; the classification type of the second classification processing comprises the classification type of the first classification processing and a subtype corresponding to the classification type of the first classification processing, and the classification type of the second classification processing is the same as the classification type of the feature classification. According to the scheme, the image classification precision can be improved.

Description

Translated fromChinese

技术领域technical field

本申请涉及图像分类的技术领域，特别是涉及图像分类方法以及相关装置、设备。The present application relates to the technical field of image classification, and in particular, to an image classification method and related devices and equipment.

背景技术Background technique

随着科学技术的不断发展，图像分类技术被广泛用来解决各种问题，例如，通过深度学习模型实现图像识别、行人检测以及医学诊断等问题。With the continuous development of science and technology, image classification technology is widely used to solve various problems, such as image recognition, pedestrian detection, and medical diagnosis through deep learning models.

图像分类方案通过监督学习把不同类别的图像区分开来。相比于检测，分割，跟踪技术，分类技术方案具有处理速度快，标注简单的优势。目前分类任务采用的监督信息是将单张图像分为一个大类，特征层主要提取的是全局类信息。Image classification schemes distinguish between different classes of images through supervised learning. Compared with detection, segmentation and tracking technology, the classification technology scheme has the advantages of fast processing speed and simple labeling. At present, the supervision information used in the classification task is to divide a single image into a large class, and the feature layer mainly extracts the global class information.

上述分类方案在提取图像特征时，模型始终关注全局特征。在处理复杂背景的图像时，分类精度往往较低。When the above classification scheme extracts image features, the model always pays attention to global features. When dealing with images with complex backgrounds, the classification accuracy tends to be low.

发明内容SUMMARY OF THE INVENTION

本申请提供了图像分类方法以及相关装置、设备，解决现有技术中存在的图像分类精度较低的问题。The present application provides an image classification method and related devices and equipment to solve the problem of low image classification accuracy existing in the prior art.

本申请提供了一种图像分类方法，包括：获取到待分类图像；对待分类图像进行特征提取，得到待分类图像的特征向量；对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、第二分类结果、前景检测结果以及特征分类结果；基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果；其中，第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，且第二分类处理的分类类型与特征分类的分类类型相同。The present application provides an image classification method, including: obtaining an image to be classified; performing feature extraction on the image to be classified to obtain a feature vector of the image to be classified; performing first classification processing, second classification processing, foreground detection on the feature vector, and Feature classification processing to obtain the first classification result, second classification result, foreground detection result, and feature classification result of the feature vector; based on the first classification result, the second classification result, the foreground detection result, and the feature classification result, the classification of the image to be classified is obtained The result; wherein, the classification type of the second classification process includes the classification type of the first classification process and its corresponding subtype, and the classification type of the second classification process is the same as the classification type of the feature classification.

其中，对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、第二分类结果、前景检测结果以及特征分类结果的步骤包括：对特征向量进行前景检测，得到特征向量的前景检测结果；分别对特征向量的前景检测结果进行第一分类处理、第二分类处理以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果以及特征向量的特征分类结果。Wherein, the first classification process, the second classification process, the foreground detection and the feature classification process are performed on the feature vector, and the steps of obtaining the first classification result, the second classification result, the foreground detection result and the feature classification result of the feature vector include: Perform foreground detection on the vector to obtain the foreground detection result of the feature vector; respectively perform the first classification process, the second classification process and the feature classification process on the foreground detection result of the feature vector to obtain the first classification result of the feature vector and the second classification result of the feature vector. The classification result and the feature classification result of the feature vector.

其中，基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果的步骤包括：响应于第二分类结果的分类类型与特征分类结果的分类类型相同，且第二分类结果为第一分类结果或第一分类结果对应的子类型，将第二分类结果以及前景检测结果确定为待分类图像的分类结果。Wherein, the step of obtaining the classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result includes: responding that the classification type of the second classification result is the same as the classification type of the feature classification result, and The second classification result is the first classification result or a subtype corresponding to the first classification result, and the second classification result and the foreground detection result are determined as the classification result of the image to be classified.

其中，第一分类结果包括第一分类处理的各分类类型的置信度，第二分类结果包括第二分类处理的各分类类型的置信度，特征分类结果包括特征分类处理的各分类类型的置信度；基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果的步骤包括：将第二分类处理的各分类类型的置信度分别与对应的第一分类处理的各分类类型的置信度以及对应的特征分类处理的各分类类型的置信度进行乘积处理，得到多个乘积数值；将数值最大的乘积数值对应的第二分类处理的分类类型以及前景检测结果进行逻辑组合，得到待分类图像的分类结果。The first classification result includes the confidence level of each classification type processed by the first classification process, the second classification result includes the confidence level of each classification type processed by the second classification process, and the feature classification result includes the confidence level of each classification type processed by the feature classification process. ; The step of obtaining the classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result comprises: comparing the confidence of each classification type of the second classification processing with the corresponding first classification processing The confidence of each classification type and the confidence of each classification type of the corresponding feature classification process are multiplied to obtain multiple product values; the classification type of the second classification process and the foreground detection result corresponding to the product value with the largest value Logical combination to obtain the classification result of the image to be classified.

其中，对待分类图像进行特征提取，得到待分类图像的特征向量的步骤包括：通过图像分类模型的特征提取网络对待分类图像进行特征提取，得到待分类图像的特征向量；分别对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果、特征向量的前景检测结果以及特征向量的特征分类结果的步骤包括：通过图像分类模型的前景检测网络对特征向量进行前景检测处理，得到特征向量的前景检测结果；以及通过图像分类模型的第一分类网络对特征向量进行第一分类处理，得到特征向量的第一分类结果；以及通过图像分类模型的第二分类网络对特征向量进行第二分类处理，得到特征向量的第二分类结果；以及利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果；其中，图像分类模型至少包括特征提取网络、第一分类网络、第二分类网络以及前景检测网络，特征提取网络分别与第一分类网络、第二分类网络以及前景检测网络相互级联。The step of performing feature extraction on the image to be classified to obtain the feature vector of the image to be classified includes: performing feature extraction on the image to be classified through a feature extraction network of the image classification model to obtain the feature vector of the image to be classified; The classification process, the second classification process, the foreground detection and the feature classification process, the steps of obtaining the first classification result of the feature vector, the second classification result of the feature vector, the foreground detection result of the feature vector, and the feature classification result of the feature vector include: The foreground detection network of the image classification model performs foreground detection processing on the feature vector to obtain a foreground detection result of the feature vector; and performs first classification processing on the feature vector through the first classification network of the image classification model to obtain the first classification result of the feature vector And carry out the second classification processing to the feature vector by the second classification network of the image classification model, obtain the second classification result of the feature vector; And utilize the feature standard library to carry out the feature classification process to the feature vector, obtain the feature classification result of the feature vector; The image classification model includes at least a feature extraction network, a first classification network, a second classification network and a foreground detection network, and the feature extraction network is cascaded with the first classification network, the second classification network and the foreground detection network respectively.

其中，利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果的步骤包括：将特征向量分别与特征标准库中的各标准特征向量进行余弦相似度处理，得到特征向量与各标准特征向量之间的相似度；将相似度最大的标准特征向量对应的分类类型确定为特征向量的特征分类结果。The step of using the feature standard library to perform feature classification processing on the feature vector to obtain the feature classification result of the feature vector includes: performing cosine similarity processing on the feature vector and each standard feature vector in the feature standard library respectively, and obtaining the feature vector and each standard feature vector. The similarity between standard feature vectors; the classification type corresponding to the standard feature vector with the greatest similarity is determined as the feature classification result of the feature vector.

其中，获取到待分类图像的步骤之前包括：获取到多张训练图像；通过训练中的图像分类模型的特征提取网络分别对各训练图像进行特征提取，得到各训练图像的训练特征向量；通过训练中的图像分类模型的分类网络分别对训练特征向量进行第一分类处理、第二分类处理、前景检测，得到训练特征向量的第一分类结果、训练特征向量的第二分类结果、训练特征向量的前景检测结果；利用各训练图像的第一分类结果、第二分类结果、前景检测结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型。Wherein, the step of obtaining the images to be classified includes: obtaining a plurality of training images; performing feature extraction on each training image through a feature extraction network of the image classification model under training, respectively, to obtain a training feature vector of each training image; The classification network of the image classification model in the image classification model performs the first classification process, the second classification process, and the foreground detection on the training feature vector respectively, and obtains the first classification result of the training feature vector, the second classification result of the training feature vector, and the result of the training feature vector. Foreground detection result; use the first classification result, second classification result, foreground detection result of each training image and the standard classification result corresponding to each training image to train the image classification model under training until the image classification model is obtained.

其中，获取到多张训练图像的步骤还包括：获取到多张训练图像；其中，各训练图像上标注有标准第一分类结果、标准第二分类结果以及标准前景结果；对训练图像进行特征提取，得到训练图像的标准特征向量，基于训练图像的多个标准第二分类结果确定标准特征向量对应的分类类型；利用各训练图像的标准第一分类结果、多个标准第二分类结果、标准前景结果以及标准特征向量，得到各训练图像的标准分类结果。The step of acquiring multiple training images further includes: acquiring multiple training images; wherein each training image is marked with a standard first classification result, a standard second classification result and a standard foreground result; and extracting features from the training images , obtain the standard feature vector of the training image, and determine the classification type corresponding to the standard feature vector based on multiple standard second classification results of the training image; use the standard first classification results of each training image, multiple standard second classification results, and standard foregrounds The result and the standard feature vector, the standard classification result of each training image is obtained.

其中，利用各训练图像的第一分类结果、第二分类结果、前景检测结果以及特征分类结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型的步骤包括：基于整体损失函数利用各训练图像的第一分类结果、第二分类结果、前景检测结果以及特征分类结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型。The image classification model under training is trained by using the first classification result, second classification result, foreground detection result and feature classification result of each training image and the standard classification result corresponding to each training image until the image classification model is obtained. Including: using the first classification result, second classification result, foreground detection result and feature classification result of each training image and the standard classification result corresponding to each training image based on the overall loss function to train the image classification model under training, until the image is obtained classification model.

其中，基于整体损失函数利用各训练图像的第一分类结果、第二分类结果、前景检测结果以及特征分类结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型的步骤包括：基于第一损失函数利用各训练特征向量的第一分类结果以及标准分类结果中对应的标准第一分类结果对训练中的图像分类模型进行训练；以及基于第二损失函数利用各训练特征向量的第二分类结果以及标准分类结果中对应的标准第二分类结果对训练中的图像分类模型进行训练；以及基于第三损失函数利用各训练特征向量的前景检测结果以及标准分类结果中对应的标准前景检测结果对训练中的图像分类模型进行训练；以及基于第四损失函数利用各训练特征向量以及对应的标准特征向量对训练中的图像分类模型进行训练；其中，通过加权处理后的第一损失函数、第二损失函数、第三损失函数以及第四损失函数之和确定整体损失函数。Wherein, based on the overall loss function, the image classification model under training is trained by using the first classification result, second classification result, foreground detection result and feature classification result of each training image and the standard classification result corresponding to each training image, until the image is obtained. The steps of classifying the model include: using the first classification result of each training feature vector and the corresponding standard first classification result in the standard classification result based on the first loss function to train the image classification model under training; and based on the second loss function using The second classification result of each training feature vector and the corresponding standard second classification result in the standard classification result are used to train the image classification model in training; and the foreground detection result and the standard classification result of each training feature vector are utilized based on the third loss function. The image classification model in training is trained with the corresponding standard foreground detection results in The sum of the first loss function, the second loss function, the third loss function, and the fourth loss function determines the overall loss function.

其中，图像分类方法应用于图像色情分类；第一分类结果的分类类型包括正常类型，性感类型，色情类型；第二分类结果和特征分类结果的分类类型包括正常类型、性感类型的子类型以及色情类型的子类型；前景检测结果包括各待分类图像上人体所在的区域的位置信息。The image classification method is applied to image pornography classification; the classification types of the first classification result include normal types, sexy types, and pornographic types; the classification types of the second classification results and feature classification results include normal types, subtypes of sexy types, and pornographic types The subtype of the type; the foreground detection result includes the position information of the area where the human body is located on each image to be classified.

其中，响应于前景检测结果的前景检测尺寸不大于预设尺寸，将待分类图像的分类结果确定为正常类型。Wherein, in response to the foreground detection size of the foreground detection result being not larger than the preset size, the classification result of the image to be classified is determined to be a normal type.

本申请还提供了一种图像分类装置，包括：获取模块，用于获取到待分类图像；特征提取模块，用于对待分类图像进行特征提取，得到待分类图像的特征向量；分类模块，用于分别对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果、特征向量的前景检测结果以及特征向量的特征分类结果；确定模块，用于基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果；其中，第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，且第二分类处理的分类类型与特征分类的分类类型相同。The present application also provides an image classification device, comprising: an acquisition module for acquiring an image to be classified; a feature extraction module for performing feature extraction on the image to be classified to obtain a feature vector of the image to be classified; a classification module for Perform first classification processing, second classification processing, foreground detection and feature classification processing on the feature vector respectively, and obtain the first classification result of the feature vector, the second classification result of the feature vector, the foreground detection result of the feature vector, and the feature of the feature vector. Classification result; a determination module for obtaining a classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result; wherein, the classification type of the second classification process includes the classification of the first classification process type and its corresponding subtype, and the classification type of the second classification process is the same as the classification type of the feature classification.

本申请还提供了一种电子设备，包括相互耦接的存储器和处理器，处理器用于执行存储器中存储的程序指令，以实现上述任意一图像分类方法。The present application also provides an electronic device, including a memory and a processor coupled to each other, and the processor is configured to execute program instructions stored in the memory, so as to implement any one of the above image classification methods.

本申请还提供了一种计算机可读存储介质，其上存储有程序指令，程序指令被处理器执行时实现上述任意一图像分类方法。The present application also provides a computer-readable storage medium on which program instructions are stored, and when the program instructions are executed by a processor, any one of the above image classification methods is implemented.

上述方案，本申请通过对待分类图像进行特征提取，得到待分类图像的特征向量，再对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、第二分类结果、前景检测结果以及特征分类结果，最后基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果，能够通过分别对特征向量进行多次互相独立地处理，能够避免各处理之间互相干扰，互相影响的情况发生，从而提高对特征向量进行整体分类的准确性和可靠性。且本申请将独立处理后的分类结果合并后得到最终的待分类图像的分类图像，能够在一定程度上弱化图像分类在复杂背景的待分类图像中对无效背景的过多关注，从而加强对待分类图像中特定前景的关注度，提高图像分类准确率和可靠性。In the above solution, the present application obtains the feature vector of the image to be classified by performing feature extraction on the image to be classified, and then performs the first classification process, the second classification process, the foreground detection and the feature classification process on the feature vector to obtain the first classification of the feature vector. result, the second classification result, the foreground detection result and the feature classification result, and finally the classification result of the image to be classified is obtained based on the first classification result, the second classification result, the foreground detection result and the feature classification result. The times are processed independently of each other, which can avoid the occurrence of mutual interference and mutual influence among the various processes, thereby improving the accuracy and reliability of the overall classification of the feature vector. In addition, the present application combines the independently processed classification results to obtain the final classified image of the image to be classified, which can weaken to a certain extent the excessive attention paid to the invalid background by the image classification in the image to be classified with complex background, thereby strengthening the classification image The attention of specific foreground in the image improves the accuracy and reliability of image classification.

附图说明Description of drawings

图1是本申请图像分类方法一实施例的流程示意图；1 is a schematic flowchart of an embodiment of an image classification method of the present application;

图2是本申请图像分类方法另一实施例的流程示意图；2 is a schematic flowchart of another embodiment of the image classification method of the present application;

图3是图2实施例中图像分类模型一实施方式的结构示意图；3 is a schematic structural diagram of an embodiment of an image classification model in the embodiment of FIG. 2;

图4是图2实施例中图像分类模型另一实施方式的结构示意图；4 is a schematic structural diagram of another embodiment of the image classification model in the embodiment of FIG. 2;

图5是图2实施例中图像分类模型的训练方法一实施例的流程示意图；5 is a schematic flowchart of an embodiment of a training method for an image classification model in the embodiment of FIG. 2;

图6是本申请图像分类装置一实施例的框架示意图；FIG. 6 is a schematic diagram of a framework of an embodiment of an image classification apparatus of the present application;

图7是本申请电子设备一实施例的框架示意图；7 is a schematic diagram of a framework of an embodiment of an electronic device of the present application;

图8为本申请计算机可读存储介质一实施例的框架示意图。FIG. 8 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application.

具体实施方式Detailed ways

下面结合说明书附图，对本申请实施例的方案进行详细说明。The solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、接口、技术之类的具体细节，以便透彻理解本申请。In the following description, for purposes of illustration and not limitation, specific details such as specific system structures, interfaces, techniques, etc. are set forth in order to provide a thorough understanding of the present application.

本文中术语“系统”和“网络”在本文中常被可互换使用。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，可以存在三种关系，例如，A和/或B，可以：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般前后关联对象是一种“或”的关系。此外，本文中的“多”两个或者多于两个。The terms "system" and "network" are often used interchangeably herein. The term "and/or" in this article is only an association relationship to describe the associated objects, there can be three kinds of relationships, for example, A and/or B, can be: A alone exists, A and B exist simultaneously, B alone exists this three conditions. In addition, the character "/" in this article, the general contextual object is an "or" relationship. Also, "multiple" as used herein is two or more than two.

请参阅图1，图1是本申请图像分类方法一实施例的流程示意图。Please refer to FIG. 1 , which is a schematic flowchart of an embodiment of an image classification method of the present application.

步骤S11：获取到待分类图像。Step S11: the image to be classified is acquired.

先获取到待分类图像。其中，待分类图像的分类标准可以基于图像上的任何对象进行构建，例如可以对待分类图像进行色情分类、人体分类、性别分类、颜色分类等，也可以基于待分类图像所针对的目标对象的不同，构建不同的分类标准，例如：植物分类、动物分类、形状分类等目标对象的分类，具体可以基于实际需求进行设置，在此不做限定。First obtain the images to be classified. Wherein, the classification standard of the image to be classified can be constructed based on any object on the image, for example, the image to be classified can be classified into pornography, human body classification, gender classification, color classification, etc. , to construct different classification standards, such as the classification of target objects such as plant classification, animal classification, shape classification, etc., which can be set based on actual needs, which is not limited here.

本步骤中获取的待分类图像可以为一张或多张。The images to be classified obtained in this step may be one or more images.

步骤S12：对待分类图像进行特征提取，得到待分类图像的特征向量。Step S12: Perform feature extraction on the image to be classified to obtain a feature vector of the image to be classified.

获取到待分类图像后，对待分类图像进行特征提取，得到待分类图像的特征向量。当待分类图像为多张时，分别对待分类图像依次进行特征提取，以得到各待分类图像的特征向量。After the image to be classified is acquired, feature extraction is performed on the image to be classified to obtain a feature vector of the image to be classified. When there are multiple images to be classified, feature extraction is performed on the images to be classified in sequence to obtain the feature vector of each image to be classified.

在一个具体的实施方式中，可以通过特征提取算法对待分类图像进行特征提取，得到待分类图像的特征向量。其中，特征提取算法至少包括LBP(Local Binary Patterns，局部二值模式)特征提取算法、HOG(Histogram of Oriented Gradient，HOG)特征提取算法、Haar特征提取算子、LoG(一阶边缘提取)特征提取算法中的一种或多种，具体在此不做限定。In a specific implementation manner, feature extraction can be performed on the image to be classified by a feature extraction algorithm to obtain a feature vector of the image to be classified. Among them, the feature extraction algorithm at least includes LBP (Local Binary Patterns, local binary pattern) feature extraction algorithm, HOG (Histogram of Oriented Gradient, HOG) feature extraction algorithm, Haar feature extraction operator, LoG (first-order edge extraction) feature extraction One or more of the algorithms, which are not specifically limited here.

在另一个具体的实施方式中，也可以通过训练完成的特征提取模型对待分类图像进行特征提取，得到待分类图像的特征向量。训练完成的特征提取模型可以基于BP(BackPropagation)神经网络、径向基函数(RBF-Radial Basis Function)神经网络、线性神经网络、卷积神经网络、循环神经网络等深度神经网络中的一种或多种进行构建，具体在此不做限定。In another specific embodiment, the feature extraction model of the image to be classified can also be used for feature extraction to obtain the feature vector of the image to be classified. The trained feature extraction model can be based on one of deep neural networks such as BP (BackPropagation) neural network, radial basis function (RBF-Radial Basis Function) neural network, linear neural network, convolutional neural network, and recurrent neural network. There are many kinds of constructions, which are not specifically limited here.

步骤S13：对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、第二分类结果、前景检测结果以及特征分类结果。Step S13: Perform the first classification process, the second classification process, the foreground detection and the feature classification process on the feature vector to obtain the first classification result, the second classification result, the foreground detection result and the feature classification result of the feature vector.

获取到特征向量后，对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、第二分类结果、前景检测结果以及特征分类结果。After the feature vector is obtained, the first classification process, the second classification process, the foreground detection and the feature classification process are performed on the feature vector to obtain the first classification result, the second classification result, the foreground detection result and the feature classification result of the feature vector.

在一个具体的应用场景中，可以分别对特征向量进行上述四种处理，具体地：对特征向量进行第一分类处理，得到特征向量的第一分类结果；以及对特征向量进行第二分类处理，得到特征向量的第二分类结果；以及对特征向量进行前景检测，得到特征向量的前景检测结果；以及对特征向量进行特征分类处理，得到特征向量的特征分类结果。In a specific application scenario, the above four kinds of processing can be respectively performed on the feature vector, specifically: performing a first classification process on the feature vector to obtain a first classification result of the feature vector; and performing a second classification process on the feature vector, obtaining the second classification result of the feature vector; and performing foreground detection on the feature vector to obtain the foreground detection result of the feature vector; and performing feature classification processing on the feature vector to obtain the feature classification result of the feature vector.

本应用场景中四种处理方式互相独立，并列进行，虽然四种处理方式的分类对象都为特征向量，但四种处理方式的分类手段以及分类结果不同。通过分别对特征向量进行多次互相独立地处理，能够避免各处理之间互相干扰，互相影响的情况发生，从而提高对特征向量进行整体分类的准确性和可靠性。In this application scenario, the four processing methods are independent of each other and are performed in parallel. Although the classification objects of the four processing methods are all feature vectors, the classification methods and classification results of the four processing methods are different. By separately processing the eigenvectors multiple times independently of each other, it is possible to avoid mutual interference and mutual influence among the processings, thereby improving the accuracy and reliability of the overall classification of the eigenvectors.

在另一个具体的应用场景中，也可以先对特征向量进行前景检测，得到特征向量的前景检测结果，再基于前景检测结果分别进行第一分类处理、第二分类处理以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果以及特征向量的特征分类结果。In another specific application scenario, foreground detection can also be performed on the feature vector to obtain the foreground detection result of the feature vector, and then the first classification process, the second classification process and the feature classification process are respectively performed based on the foreground detection result to obtain the feature. The first classification result of the vector, the second classification result of the feature vector, and the feature classification result of the feature vector.

本应用场景先对特征向量进行前景检测，再基于前景检测结果分别且互相独立地进行三种分类处理，从而预先通过前景检测，使得提高后续三种分类处理对待分类图像中特定前景的关注度，从而提高分类准确率和可靠性，减少在复杂背景的待分类图像中无法聚焦特定前景，导致多过关注到背景信息的情况发生。This application scenario first performs foreground detection on the feature vector, and then performs three classification processes based on the foreground detection results respectively and independently of each other, so as to pass the foreground detection in advance, so that the attention of the specific foreground in the image to be classified by the subsequent three classification processes can be improved. Thereby, the classification accuracy and reliability are improved, and the inability to focus on a specific foreground in an image to be classified with a complex background reduces the occurrence of the situation that more attention is paid to the background information.

其中，本实施例的第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，且第二分类处理的分类类型与特征分类的分类类型相同。The classification type of the second classification process in this embodiment includes the classification type of the first classification process and its corresponding subtype, and the classification type of the second classification process is the same as the classification type of the feature classification.

即，第一分类处理可以为对待分类图像的粗分类，第二分类处理可以为对待分类图像的细分类，而第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型。在一个具体的应用场景中，当图像分类应用于图像颜色分类时，第一分类处理可以为对待分类图像的颜色粗分类，其分类类型可以包括：白色、红色、蓝色。第二分类处理可以为对待分类图像的颜色细分类，其分类类型可以包括白色、粉红、玫红、血红、天蓝、深蓝。其中，第二分类处理的分类类型的粉红、玫红、血红为第一分类处理的分类类型的红色的子类型，第二分类处理的分类类型的天蓝、深蓝为第一分类处理的分类类型的蓝色的子类型，而第二分类处理的分类类型的白色与第一分类处理的分类类型的白色相同。其中，第一分类处理、第二分类处理以及特征分类处理的分类类型及其数量可以基于实际需求进行设置，在此不做限定。That is, the first classification process may be the rough classification of the images to be classified, the second classification process may be the sub-classification of the images to be classified, and the classification types of the second classification process include the classification types of the first classification process and their corresponding subtypes . In a specific application scenario, when image classification is applied to image color classification, the first classification process may be a rough classification of the color of the image to be classified, and the classification types may include: white, red, and blue. The second classification process may sub-classify the colors of the images to be classified, and the classification types may include white, pink, rose red, blood red, sky blue, and dark blue. Wherein, pink, rose red, and blood red of the classification type of the second classification process are the red subtypes of the classification type of the first classification process, and sky blue and dark blue of the classification type of the second classification process are the classification types of the first classification process. The sub-type of blue, and the white of the classification type of the second classification process is the same as the white of the classification type of the first classification process. The classification types and numbers of the first classification process, the second classification process, and the feature classification process can be set based on actual requirements, which are not limited here.

其中，虽然第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，但本实施例的第二分类不基于第一分类的第一分类结果进行，而是基于待分类图像的特征向量进行第二分类。本实施例的三种分类处理互相独立，以避免各处理之间互相干扰，互相影响的情况发生，从而提高对特征向量进行整体分类的准确性和可靠性。Wherein, although the classification type of the second classification process includes the classification type of the first classification process and its corresponding subtype, the second classification in this embodiment is not performed based on the first classification result of the first classification, but is based on the to-be-classified The feature vector of the image is used for the second classification. The three classification processes in this embodiment are independent of each other, so as to avoid mutual interference and mutual influence among the processes, thereby improving the accuracy and reliability of the overall classification of the feature vector.

前景检测指的是基于特征向量对待分类图像的目标对象进行检测，例如：当对图像进行色情分类时，待分类图像的目标对象可以为人体，则前景检测为基于特征向量对待分类图像上的人体进行检测。具体可以通过在待分类图像上的前景区域标注检测框或前景区域的位置信息来输出前景检测结果。Foreground detection refers to the detection of the target object of the image to be classified based on the feature vector. For example: when the image is classified as pornographic, the target object of the image to be classified can be a human body, and the foreground detection is based on the feature vector. The human body on the image to be classified test. Specifically, the foreground detection result can be output by labeling the detection frame or the position information of the foreground region in the foreground region of the image to be classified.

特征分类指的是基于特征向量直接进行特征相似度的判断，从而确定特征向量的特征分类结果。其中，特征向量的特征相似度判断的对象可以为图像分类过程中已进行分类的图像中的特征向量，也可以为预先基于图像的各分类类型构建的标准特征库，具体在此不做限定。Feature classification refers to directly judging feature similarity based on feature vectors, so as to determine the feature classification results of feature vectors. The object of the feature similarity judgment of the feature vector may be the feature vector in the image that has been classified in the image classification process, or may be a standard feature library constructed in advance based on each classification type of the image, which is not specifically limited here.

本实施例通过对同一特征向量分别做第一分类处理、第二分类处理以及特征分类处理，而第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，且第二分类处理的分类类型与特征分类的分类类型相同，即三种分类处理的处理过程互相独立，从而避免各处理之间互相干扰，互相影响的情况发生，进而提高第一分类结果、第二分类结果以及特征分类结果各自的准确率，但三种分类处理的分类标准互相关联，从而使得第一分类结果、第二分类结果以及特征分类结果能够互相参考、辅助判断，最终得到待分类图像的分类结果，从而进一步提高待分类图像的分类结果的准确率。In this embodiment, the first classification process, the second classification process, and the feature classification process are respectively performed on the same feature vector, and the classification type of the second classification process includes the classification type of the first classification process and its corresponding subtype, and the second classification process The classification type of the classification processing is the same as the classification type of the feature classification, that is, the processing processes of the three classification processing are independent of each other, so as to avoid mutual interference and mutual influence between the processing, thereby improving the first classification result and the second classification result. and the respective accuracy rates of the feature classification results, but the classification criteria of the three classification processes are related to each other, so that the first classification result, the second classification result and the feature classification result can refer to each other and assist in the judgment, and finally obtain the classification result of the image to be classified. , thereby further improving the accuracy of the classification results of the images to be classified.

步骤S14：基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果。Step S14: Obtain a classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result.

获取到第一分类结果、第二分类结果、特征分类结果以及前景检测结果后，基于第一分类结果、第二分类结果、特征分类结果以及前景检测结果得到待分类图像的分类结果。通过对第一分类结果、第二分类结果、特征分类结果以及前景检测结果进行合并，从而得到待分类图像的分类结果。After the first classification result, the second classification result, the feature classification result and the foreground detection result are obtained, the classification result of the image to be classified is obtained based on the first classification result, the second classification result, the feature classification result and the foreground detection result. By combining the first classification result, the second classification result, the feature classification result and the foreground detection result, the classification result of the image to be classified is obtained.

在一个具体的应用场景中，可以将第一分类结果、第二分类结果、特征分类结果以及前景检测结果进行逻辑组合后，将逻辑组合后的第一分类结果、第二分类结果、特征分类结果以及前景检测结果确定为待分类图像的分类结果。In a specific application scenario, after logically combining the first classification result, the second classification result, the feature classification result, and the foreground detection result, the logically combined first classification result, the second classification result, and the feature classification result And the foreground detection result is determined as the classification result of the image to be classified.

在一个具体的应用场景中，也可以基于第一分类结果、第二分类结果、特征分类结果进行对比判断，响应于上述三种分类结果的实质意义相同，则将逻辑组合后的第二分类结果以及前景检测结果确定为待分类图像的分类结果，或将逻辑组合后的第一分类结果以及前景检测结果确定为待分类图像的分类结果，或将逻辑组合后的特征分类结果以及前景检测结果确定为待分类图像的分类结果，具体基于实际情况进行选择。In a specific application scenario, the comparison and judgment can also be made based on the first classification result, the second classification result, and the feature classification result. And the foreground detection result is determined as the classification result of the image to be classified, or the first classification result after the logical combination and the foreground detection result are determined as the classification result of the image to be classified, or the feature classification result after the logical combination and the foreground detection result are determined. It is the classification result of the image to be classified, and is selected based on the actual situation.

其中，逻辑组合的形式可以包括通过文字、表格、图像等方式进行综合输出。Wherein, the form of logical combination may include comprehensive output through text, table, image, etc.

且本实施例分别对同一特征向量分别做不同的分类处理，再基于不同的分类结果进行合并，得到最终的待分类图像的分类图像，能够在一定程度上弱化图像分类在复杂背景的待分类图像中对无效背景的过多关注，从而加强对待分类图像中特定前景的关注度，提高图像分类准确率和可靠性。In addition, in this embodiment, different classification processing is performed on the same feature vector respectively, and then combined based on different classification results to obtain the final classified image of the image to be classified, which can weaken the image classification of the image to be classified in a complex background to a certain extent. In this way, too much attention is paid to the invalid background in the image, so as to strengthen the attention to the specific foreground in the image to be classified, and improve the accuracy and reliability of image classification.

通过上述步骤，本实施例的图像分类方法通过对待分类图像进行特征提取，得到待分类图像的特征向量，再对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、第二分类结果、前景检测结果以及特征分类结果，最后基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果，能够通过分别对特征向量进行多次互相独立地处理，能够避免各处理之间互相干扰，互相影响的情况发生，从而提高对特征向量进行整体分类的准确性和可靠性。且本实施例将独立处理后的分类结果合并后得到最终的待分类图像的分类图像，能够在一定程度上弱化图像分类在复杂背景的待分类图像中对无效背景的过多关注，从而加强对待分类图像中特定前景的关注度，提高图像分类准确率和可靠性。Through the above steps, the image classification method of this embodiment obtains the feature vector of the image to be classified by performing feature extraction on the image to be classified, and then performs the first classification process, the second classification process, the foreground detection and the feature classification process on the feature vector to obtain The first classification result, the second classification result, the foreground detection result and the feature classification result of the feature vector, and finally the classification result of the image to be classified is obtained based on the first classification result, the second classification result, the foreground detection result and the feature classification result, which can be obtained by The eigenvectors are processed independently of each other for many times, which can avoid the occurrence of mutual interference and mutual influence among the various processes, thereby improving the accuracy and reliability of the overall classification of the eigenvectors. And this embodiment combines the independently processed classification results to obtain the final classification image of the image to be classified, which can weaken the image classification to a certain extent in the image to be classified with complex background. Classify the attention of specific foregrounds in images to improve the accuracy and reliability of image classification.

请参阅图2，图2是本申请图像分类方法另一实施例的流程示意图。Please refer to FIG. 2 , which is a schematic flowchart of another embodiment of the image classification method of the present application.

步骤S21：获取到待分类图像。Step S21: the image to be classified is acquired.

本实施例与前述实施例的步骤S11相同，请参阅前文，在此不再赘述。This embodiment is the same as step S11 in the foregoing embodiment, please refer to the foregoing, and details are not repeated here.

步骤S22：通过图像分类模型的特征提取网络对待分类图像进行特征提取，得到待分类图像的特征向量。Step S22 : extracting features of the image to be classified through the feature extraction network of the image classification model to obtain a feature vector of the image to be classified.

通过图像分类模型的特征提取网络对待分类图像进行特征提取，得到待分类图像的特征向量。其中，图像分类模型至少包括相互级联的特征提取网络以及分类网络。The feature extraction network of the image classification model is used to extract the feature of the image to be classified, and the feature vector of the image to be classified is obtained. The image classification model includes at least a feature extraction network and a classification network that are cascaded to each other.

特征提取网络为预先基于待分类图像的分类标准进行训练后得到的特征提取网络，其可以基于BP(Back Propagation)神经网络、径向基函数(RBF-Radial BasisFunction)神经网络、线性神经网络、卷积神经网络、循环神经网络等深度神经网络中的一种或多种进行构建，具体在此不做限定。The feature extraction network is a feature extraction network obtained by pre-training based on the classification criteria of the images to be classified. It can be based on BP (Back Propagation) neural network, radial basis function (RBF-Radial BasisFunction) neural network, linear neural network, volume One or more of deep neural networks such as cumulative neural network and recurrent neural network are used to construct, which is not specifically limited here.

本步骤通过图像分类模型的特征提取网络对待分类图像进行特征提取，其中，特征提取网络进行特征提取过程中，得到中间特征图后，可以结合通道注意力与空间注意力依次推断出注意力权重，然后与待分类图像的原特征图相乘来对特征进行自适应调整，从而提高特征向量的准确率和可靠性。In this step, the feature extraction network of the image classification model is used to extract the feature of the image to be classified. During the feature extraction process of the feature extraction network, after obtaining the intermediate feature map, the attention weight can be inferred in turn by combining the channel attention and the spatial attention. Then, it is multiplied with the original feature map of the image to be classified to adjust the feature adaptively, thereby improving the accuracy and reliability of the feature vector.

在一个具体的应用场景中，在对待分类图像进行特征提取之前，还可以预先对待分类图像进行预处理，以尽量消除图像中无关的信息，预处理至少包括裁剪、去噪、色彩平衡、灰度处理、拉伸等中的一种或多种。In a specific application scenario, before the feature extraction of the image to be classified, the image to be classified can also be preprocessed to eliminate irrelevant information in the image as much as possible. The preprocessing at least includes cropping, denoising, color balance, grayscale One or more of processing, stretching, etc.

步骤S23：通过图像分类模型的前景检测网络对特征向量进行前景检测处理，得到特征向量的前景检测结果；以及通过图像分类模型的第一分类网络对特征向量进行第一分类处理，得到特征向量的第一分类结果；以及通过图像分类模型的第二分类网络对特征向量进行第二分类处理，得到特征向量的第二分类结果；以及利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果。Step S23: performing foreground detection processing on the feature vector through the foreground detection network of the image classification model to obtain the foreground detection result of the feature vector; and performing the first classification processing on the feature vector through the first classification network of the image classification model to obtain the The first classification result; and the second classification process is performed on the feature vector by the second classification network of the image classification model, and the second classification result of the feature vector is obtained; Feature classification results.

获取到待分类图像的特征向量后，通过图像分类模型的前景检测网络对特征向量进行前景检测处理，得到特征向量的前景检测结果；以及通过图像分类模型的第一分类网络对特征向量进行第一分类处理，得到特征向量的第一分类结果；以及通过图像分类模型的第二分类网络对特征向量进行第二分类处理，得到特征向量的第二分类结果；以及利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果。After obtaining the feature vector of the image to be classified, perform foreground detection processing on the feature vector through the foreground detection network of the image classification model to obtain the foreground detection result of the feature vector; and perform a first classification process on the feature vector through the first classification network of the image classification model. classification processing to obtain a first classification result of the feature vector; and performing a second classification process on the feature vector through the second classification network of the image classification model to obtain a second classification result of the feature vector; and using the feature standard library to characterize the feature vector The classification process is performed to obtain the feature classification result of the feature vector.

其中，利用特征分类处理的过程单独进行，其不属于图像分类模型的处理范围。从而在利用图像分类模型进行图像分类的基础上，引入图像分类模型外部的特征分类处理，并最后利用图像分类模型外部的特征分类处理的处理结果辅助判断图像分类模型自身的分类结果，提高待分类图像的分类结果的鲁棒性，进而提高待分类图像的分类结果的准确性和可靠性。Among them, the process of using feature classification processing is carried out separately, which does not belong to the processing scope of the image classification model. Therefore, on the basis of using the image classification model for image classification, the feature classification processing outside the image classification model is introduced, and finally the processing results of the feature classification processing outside the image classification model are used to assist in judging the classification results of the image classification model itself, so as to improve the classification results. The robustness of the classification result of the image, thereby improving the accuracy and reliability of the classification result of the image to be classified.

其中，图像分类模型至少包括特征提取网络、第一分类网络、第二分类网络以及前景检测网络，特征提取网络分别与第一分类网络、第二分类网络以及前景检测网络相互级联。The image classification model includes at least a feature extraction network, a first classification network, a second classification network and a foreground detection network, and the feature extraction network is cascaded with the first classification network, the second classification network and the foreground detection network respectively.

在一个具体的实施方式中，可以先对特征向量进行前景检测，得到特征向量的前景检测结果，再分别对特征向量的前景检测结果进行第一分类处理、第二分类处理以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果以及特征向量的特征分类结果。In a specific embodiment, foreground detection may be performed on the feature vector to obtain the foreground detection result of the feature vector, and then the foreground detection results of the feature vector are respectively subjected to the first classification process, the second classification process, and the feature classification process to obtain The first classification result of the feature vector, the second classification result of the feature vector, and the feature classification result of the feature vector.

请参阅图3，图3是图2实施例中图像分类模型一实施方式的结构示意图。Please refer to FIG. 3 , which is a schematic structural diagram of an implementation manner of the image classification model in the embodiment of FIG. 2 .

本实施例的图像分类模型10包括特征提取网络11、前景检测网络12、第一分类网络13以及第二分类网络14。其中，特征提取网络11与前景检测网络12相互级联，前景检测网络12分别与第一分类网络13以及第二分类网络14相互级联。Theimage classification model 10 of this embodiment includes afeature extraction network 11 , aforeground detection network 12 , afirst classification network 13 and asecond classification network 14 . Thefeature extraction network 11 and theforeground detection network 12 are cascaded with each other, and theforeground detection network 12 is respectively cascaded with thefirst classification network 13 and thesecond classification network 14 .

具体地，先通过特征提取网络11对待分类图像进行特征提取，得到待分类图像的特征向量，通过前景检测网络12对特征向量进行前景检测，得到特征向量的前景检测结果，再分别通过第一分类网络13以及第二分类网络14对同一前景检测结果进行不同分类处理，得到第二分类结果以及第二分类结果。Specifically, thefeature extraction network 11 is used to extract the feature of the image to be classified to obtain the feature vector of the image to be classified, and theforeground detection network 12 is used to perform foreground detection on the feature vector to obtain the foreground detection result of the feature vector. Thenetwork 13 and thesecond classification network 14 perform different classification processing on the same foreground detection result to obtain the second classification result and the second classification result.

且同时利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果。其中，利用特征标准库对特征向量进行特征分类处理的过程单独进行，不属于图像分类模型10的处理范围。At the same time, the feature standard library is used to perform feature classification processing on the feature vector, and the feature classification result of the feature vector is obtained. The process of performing feature classification processing on the feature vector by using the feature standard library is performed separately, and does not belong to the processing scope of theimage classification model 10 .

本实施方式预先通过前景检测，使得提高后续三种分类处理对待分类图像中特定前景的关注度，从而提高分类准确率和可靠性，减少在复杂背景的待分类图像中无法聚焦特定前景，导致多过关注到背景信息的情况发生。In this embodiment, the foreground detection is performed in advance, so that the attention of the specific foreground in the image to be classified in the subsequent three classification processes is improved, thereby improving the classification accuracy and reliability, and reducing the inability to focus on the specific foreground in the image to be classified with complex background, resulting in more This happens when you pay attention to background information.

在另一个具体的实施方式中，可以同时对特征向量进行前景检测、第一分类处理、第二分类处理以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果、特征向量的前景检测结果以及特征向量的特征分类结果。In another specific embodiment, foreground detection, first classification processing, second classification processing, and feature classification processing may be performed on the feature vector at the same time to obtain the first classification result of the feature vector, the second classification result of the feature vector, the feature vector The foreground detection result of the vector and the feature classification result of the feature vector.

请参阅图4，图4是图2实施例中图像分类模型另一实施方式的结构示意图。Please refer to FIG. 4 , which is a schematic structural diagram of another implementation manner of the image classification model in the embodiment of FIG. 2 .

本实施例的图像分类模型20包括特征提取网络21、前景检测网络22、第一分类网络23以及第二分类网络24。其中，特征提取网络21分别与前景检测网络22、第一分类网络23以及第二分类网络24相互级联。Theimage classification model 20 of this embodiment includes afeature extraction network 21 , aforeground detection network 22 , afirst classification network 23 and asecond classification network 24 . Thefeature extraction network 21 is cascaded with theforeground detection network 22 , thefirst classification network 23 and thesecond classification network 24 respectively.

先通过特征提取网络11对待分类图像进行特征提取，得到待分类图像的特征向量，再分别通过前景检测网络22、第一分类网络23以及第二分类网络24对特征向量进行处理，得到第一分类结果、第二分类结果、前景检测结果。且同时利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果。其中，利用特征标准库对特征向量进行特征分类处理的过程单独进行，不属于图像分类模型10的处理范围。Firstly, thefeature extraction network 11 is used to extract the feature of the image to be classified to obtain the feature vector of the image to be classified, and then the feature vector is processed through theforeground detection network 22, thefirst classification network 23 and thesecond classification network 24 respectively to obtain the first classification. Result, second classification result, foreground detection result. At the same time, the feature standard library is used to perform feature classification processing on the feature vector, and the feature classification result of the feature vector is obtained. The process of performing feature classification processing on the feature vector by using the feature standard library is performed separately, and does not belong to the processing scope of theimage classification model 10 .

本实施方式中四种处理方式互相独立，并列进行，能够避免各处理之间互相干扰，互相影响的情况发生，从而提高对特征向量进行整体分类的准确性和可靠性。In this embodiment, the four processing methods are independent of each other and are performed in parallel, which can avoid the occurrence of mutual interference and mutual influence among the processing, thereby improving the accuracy and reliability of the overall classification of the feature vector.

其中，本步骤中利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果的步骤具体包括：将特征向量分别与特征标准库中的各标准特征向量进行余弦相似度处理，得到特征向量与各标准特征向量之间的相似度，将相似度最大的标准特征向量对应的分类类型确定为特征向量的特征分类结果。Wherein, in this step, the feature standard library is used to perform feature classification processing on the feature vector, and the step of obtaining the feature classification result of the feature vector specifically includes: performing cosine similarity processing on the feature vector and each standard feature vector in the feature standard library, respectively, to obtain The similarity between the feature vector and each standard feature vector, and the classification type corresponding to the standard feature vector with the largest similarity is determined as the feature classification result of the feature vector.

而特征分类处理与第二分类处理的区别在于特征分类处理时利用特征标准库中的特征向量进行余弦相似度对比进行分类确定，而第二分类处理时利用图像分类模型训练学习的标签参数进行分类计算确定。The difference between the feature classification process and the second classification process is that in the feature classification process, the feature vector in the feature standard library is used to compare the cosine similarity for classification and determination, while the second classification process uses the image classification model training and learned label parameters for classification. Calculation OK.

本实施例的第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，即，第一分类处理可以为对待分类图像的粗分类，第二分类处理可以为对待分类图像的细分类。且第二分类处理的分类类型与特征分类的分类类型相同，通过对同一特征向量分别做第一分类处理、第二分类处理以及特征分类处理，而第二分类处理的分类类型包括第一分类处理的分类类型及其对应的子类型，且第二分类处理的分类类型与特征分类的分类类型相同，即三种分类处理的处理过程互相独立，从而避免各处理之间互相干扰，互相影响的情况发生，进而提高第一分类结果、第二分类结果以及特征分类结果各自的准确率，但三种分类处理的分类标准互相关联，从而使得第一分类结果、第二分类结果以及特征分类结果能够互相参考、辅助判断，最终得到待分类图像的分类结果，从而进一步提高待分类图像的分类结果的准确率。The classification types of the second classification process in this embodiment include the classification types of the first classification process and their corresponding subtypes, that is, the first classification process may be a rough classification of images to be classified, and the second classification process may be images to be classified sub-categories. And the classification type of the second classification process is the same as the classification type of the feature classification, by performing the first classification process, the second classification process and the feature classification process respectively on the same feature vector, and the classification type of the second classification process includes the first classification process. The classification type and its corresponding sub-type, and the classification type of the second classification process is the same as the classification type of the feature classification, that is, the processing processes of the three classification processes are independent of each other, so as to avoid the situation that the processes interfere with each other and affect each other. occurs, thereby improving the respective accuracy rates of the first classification result, the second classification result and the feature classification result, but the classification criteria of the three classification processes are related to each other, so that the first classification result, the second classification result and the feature classification result can be mutually With reference and auxiliary judgment, the classification result of the image to be classified is finally obtained, thereby further improving the accuracy of the classification result of the image to be classified.

在一个具体的应用场景中，当图像分类为图像色情分类时，第一分类结果的分类类型包括正常类型，性感类型，色情类型；第二分类结果和特征分类结果的分类类型包括正常类型、性感类型的子类型以及色情类型的子类型；前景检测结果包括各待分类图像上人体所在的区域的位置信息。In a specific application scenario, when the image is classified as image pornography, the classification types of the first classification result include normal type, sexy type, and pornographic type; the classification types of the second classification result and feature classification result include normal type, sexy type The sub-type of the type and the sub-type of the pornographic type; the foreground detection result includes the position information of the area where the human body is located on each image to be classified.

步骤S24：基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果。Step S24: Obtain a classification result of the image to be classified based on the first classification result, the second classification result, the foreground detection result and the feature classification result.

本步骤综合第一分类结果、第二分类结果、前景检测结果以及特征分类结果，以得到待分类图像的分类结果。In this step, the first classification result, the second classification result, the foreground detection result and the feature classification result are integrated to obtain the classification result of the image to be classified.

在一个具体的应用场景中，响应于第二分类结果的分类类型与特征分类结果的分类类型相同，且第二分类结果为第一分类结果或第一分类结果对应的子类型，将第二分类结果以及前景检测结果确定为待分类图像的分类结果。其中，由于第二分类结果的分类类型更为细致，则在三种分类结果互相匹配的情况下，将第二分类结果进行输出，从而提高待分类图像的分类结果的信息含量。而在其他应用场景中，也可以基于实际需求选择其他分类结果进行输出，在此不做限定。In a specific application scenario, in response to the classification type of the second classification result being the same as the classification type of the feature classification result, and the second classification result is the first classification result or a subtype corresponding to the first classification result, the second classification result is The result and the foreground detection result are determined as the classification result of the image to be classified. Wherein, since the classification type of the second classification result is more detailed, when the three classification results match each other, the second classification result is output, thereby improving the information content of the classification result of the image to be classified. In other application scenarios, other classification results can also be selected for output based on actual needs, which is not limited here.

在一个具体的应用场景中，第一分类结果包括第一分类处理的各分类类型的置信度，第二分类结果包括第二分类处理的各分类类型的置信度，特征分类结果包括特征分类处理的各分类类型的置信度。将第二分类处理的各分类类型的置信度分别与对应的第一分类处理的各分类类型的置信度以及对应的特征分类处理的各分类类型的置信度进行乘积处理，得到多个乘积数值，将数值最大的乘积数值对应的第二分类处理的分类类型以及前景检测结果进行逻辑组合，得到待分类图像的分类结果。In a specific application scenario, the first classification result includes the confidence level of each classification type processed by the first classification process, the second classification result includes the confidence level of each classification type processed by the second classification process, and the feature classification result includes the Confidence for each classification type. Multiply the confidence of each classification type of the second classification process with the corresponding confidence of each classification type of the first classification process and the confidence of each classification type of the corresponding feature classification process to obtain a plurality of product values, The classification type of the second classification process and the foreground detection result corresponding to the product value with the largest value are logically combined to obtain the classification result of the image to be classified.

例如：当第一分类结果的分类类型为正常类型、色情类型、性感类型时，第二分类结果和特征分类结果的分类类型可以分别为正常、第一色情子类型、第二色情子类型、第三色情子类型、第一性感子类型、第二性感子类型以及第三性感子类型。其中，色情子类型以及性感子类型的具体针对对象可以基于实际情况进行确定。For example: when the classification types of the first classification result are normal type, pornographic type, and sexy type, the classification types of the second classification result and feature classification result can be normal, the first pornographic subtype, the second pornographic subtype, the Three sexual subtypes, a first sexual subtype, a second sexual subtype, and a third sexual subtype. Wherein, the specific target object of the pornographic subtype and the sexy subtype can be determined based on the actual situation.

当获取到第一分类结果的分类类型及其置信度为：正常：0.2；性感：0.5；色情：0.3。When the classification type of the first classification result and its confidence level are obtained: normal: 0.2; sexy: 0.5; pornographic: 0.3.

第二分类结果包括正常：0.1；第一色情子类型：0.1；第二色情子类型：0.2；第三色情子类型：0.1；第一性感子类型：0.3；第二性感子类型：0.1以及第三性感子类型：0.1。The second classification results include normal: 0.1; first sexual subtype: 0.1; second sexual subtype: 0.2; third sexual subtype: 0.1; first sexual subtype: 0.3; Three sex subtypes: 0.1.

特征分类结果包括正常：0.2；第一色情子类型：0.1；第二色情子类型：0.2；第三色情子类型：0.1；第一性感子类型：0.3；第二性感子类型：0.1以及第三性感子类型：0.1。Feature classification results include normal: 0.2; first erotic subtype: 0.1; second erotic subtype: 0.2; third erotic subtype: 0.1; first sexual subtype: 0.3; second sexual subtype: 0.1 and third Sexy Subtype: 0.1.

将第二分类处理的各分类类型的置信度分别与对应的第一分类处理的各分类类型的置信度以及对应的特征分类处理的各分类类型的置信度进行乘积处理，得到多个乘积数值。例如：第二分类处理的正常类型的乘积处理为第二分类结果的正常类型的置信度*第一分类结果的正常类型的置信度*特征分类结果的正常类型的置信度，为“0.1*0.2*0.2＝0.004”，第二分类处理的第一性感子类型的乘积处理为：第二分类处理的第一性感子类型的乘积处理为第二分类结果的正常类型的置信度*第一分类结果的性感类型的置信度*特征分类结果的第一性感子类型的置信度，为“0.3*0.5*0.3＝0.045”，其他置信度的乘积数值的计算方式与上述类似，在此不做限定。Multiply the confidence of each classification type of the second classification process with the corresponding confidence of each classification type of the first classification process and the corresponding confidence of each classification type of the feature classification process to obtain multiple product values. For example: the product of the normal type of the second classification process is processed as the confidence level of the normal type of the second classification result * the confidence level of the normal type of the first classification result * the confidence level of the normal type of the feature classification result, which is "0.1*0.2 *0.2=0.004", the product processing of the first sexual subtype of the second classification process is: the product processing of the first sexual subtype of the second classification process is the confidence level of the normal type of the second classification result * the first classification result The confidence of the sexy type * the confidence of the first sexual sub-type of the feature classification result is "0.3*0.5*0.3=0.045", and the calculation method of the product value of other confidences is similar to the above, which is not limited here.

将数值最大的乘积数值对应的第二分类处理的分类类型以及前景检测结果进行逻辑组合，得到待分类图像的分类结果。The classification type of the second classification process and the foreground detection result corresponding to the product value with the largest value are logically combined to obtain the classification result of the image to be classified.

在一个具体的应用场景中，当图像分类为图像色情分类时，响应于前景检测结果的前景检测尺寸不大于预设尺寸，将待分类图像的分类结果确定为正常类型。其中，当图像分类检测到前景检测的尺寸不足以产生不良影响时，可以直接将待分类图像的分类结果确定为正常类型。In a specific application scenario, when an image is classified as an image pornography classification, in response to the foreground detection size of the foreground detection result being no larger than a preset size, the classification result of the image to be classified is determined to be a normal type. Wherein, when the image classification detects that the size of the foreground detection is insufficient to produce adverse effects, the classification result of the image to be classified can be directly determined as a normal type.

在一个具体的应用场景中，为便于乘积数值的相互对比，可以在对乘积数值进行归一化处理后，再进行相互对比，以选取数值最大的乘积数值，得到待分类图像的分类结果。In a specific application scenario, in order to facilitate the mutual comparison of the product values, the product values can be normalized and then compared with each other to select the product value with the largest value to obtain the classification result of the image to be classified.

通过上述综合方式确定待分类图像的分类结果，能够在一定程度上弱化图像分类在复杂背景的待分类图像中对无效背景的过多关注，从而加强对待分类图像中特定前景的关注度，提高图像分类准确率和可靠性。Determining the classification result of the image to be classified by the above comprehensive method can weaken the excessive attention paid to the invalid background in the image to be classified with complex background to a certain extent. Classification accuracy and reliability.

其中，在本实施例的图像分类之前，可以对图像分类模型进行训练。请参阅图5，图5是图2实施例中图像分类模型的训练方法一实施例的流程示意图。Wherein, before the image classification in this embodiment, the image classification model may be trained. Please refer to FIG. 5 . FIG. 5 is a schematic flowchart of an embodiment of the training method of the image classification model in the embodiment of FIG. 2 .

步骤S31：获取到多张训练图像。Step S31: Acquire multiple training images.

获取到多张训练图像，其中，多张训练图像可以包括全部的第一分类的分类类型、全部的第二分类的分类类型(全部的特征分类的类型)，从而保证图像分类模型对每张分类类型的分类都能够得到训练。Acquire a plurality of training images, wherein the plurality of training images may include all the classification types of the first classification and all the classification types of the second classification (all the types of feature classifications), so as to ensure that the image classification model can classify each image Types of classification can be trained.

且，各训练图像上标注有标准第一分类结果、标准第二分类结果以及标准前景结果。在一个具体的应用场景中，可以接收人工对各训练图像进行标注的标准第一分类结果、标准第二分类结果以及标准前景结果。在另一个具体的应用场景中，也可以通过训练完成的第一分类模型、第二分类模型以及前景检测模型对各训练图像进行标注，得到各训练图像的标准第一分类结果，标准第二分类结果以及标准前景结果。Moreover, each training image is marked with the standard first classification result, the standard second classification result and the standard foreground result. In a specific application scenario, the standard first classification result, the standard second classification result, and the standard foreground result of manually labeling each training image may be received. In another specific application scenario, each training image can also be marked by the first classification model, the second classification model and the foreground detection model completed after training, and the standard first classification result and the standard second classification result of each training image can be obtained. Results as well as standard outlook results.

在一个具体的应用场景中，当图像分类为色情分类时，可以采用人体检测对多张训练图像进行检测，获取人体前景区域，从而得到标准前景结果。In a specific application scenario, when an image is classified as pornographic, human body detection can be used to detect multiple training images to obtain a human foreground area, thereby obtaining a standard foreground result.

在一个具体的应用场景中，本步骤还可以获取到各训练图像的标准特征向量。其中，标准特征向量可以基于各训练图像通过特征提取算法或训练完成的特征提取网络进行获取。得到训练图像的标准特征向量后，基于训练图像的多个标准第二分类结果确定标准特征向量对应的分类类型，从而便于利用各训练图像的标准特征向量构建前述实施例中的标准特征库。In a specific application scenario, the standard feature vector of each training image can also be obtained in this step. The standard feature vector can be obtained based on each training image through a feature extraction algorithm or a trained feature extraction network. After the standard feature vector of the training image is obtained, the classification type corresponding to the standard feature vector is determined based on the multiple standard second classification results of the training image, so that the standard feature library in the foregoing embodiment can be constructed by using the standard feature vector of each training image.

步骤S32：通过训练中的图像分类模型的特征提取网络分别对各训练图像进行特征提取，得到各训练图像的训练特征向量。Step S32: Perform feature extraction on each training image through the feature extraction network of the image classification model under training to obtain a training feature vector of each training image.

通过训练中的图像分类模型的特征提取网络分别对各训练图像进行特征提取，得到各训练图像的训练特征向量。Through the feature extraction network of the image classification model in training, the features of each training image are extracted respectively, and the training feature vector of each training image is obtained.

其中，具体的特征提取过程与前述实施例步骤S22中的特征提取过程相同，请参阅前文，在此不再赘述。Wherein, the specific feature extraction process is the same as the feature extraction process in step S22 in the foregoing embodiment, please refer to the foregoing, and details are not repeated here.

步骤S33：通过训练中的图像分类模型的分类网络分别对训练特征向量进行第一分类处理、第二分类处理、前景检测，得到训练特征向量的第一分类结果、训练特征向量的第二分类结果、训练特征向量的前景检测结果。Step S33: Perform the first classification process, the second classification process, and the foreground detection on the training feature vector through the classification network of the image classification model in training, to obtain the first classification result of the training feature vector and the second classification result of the training feature vector. , the foreground detection result of the training feature vector.

其中，具体的第一分类、第二分类、特征分类以及前景检测的具体过程与前述实施例步骤S23相同，请参阅前文，在此不再赘述。The specific processes of the first classification, the second classification, the feature classification, and the foreground detection are the same as those of step S23 in the foregoing embodiment, please refer to the above, and are not repeated here.

步骤S34：利用各训练图像的第一分类结果、第二分类结果、前景检测结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型。Step S34: Use the first classification result, the second classification result, the foreground detection result of each training image and the standard classification result corresponding to each training image to train the image classification model under training until an image classification model is obtained.

获取到各训练特征向量的第一分类结果、第二分类结果以及前景检测结果后，利用各训练特征向量的第一分类结果、第二分类结果以及前景检测结果与对应的训练图像的标准分类结果对训练中的图像分类模型进行训练，直至训练得到图像分类模型。After obtaining the first classification result, second classification result and foreground detection result of each training feature vector, use the first classification result, second classification result and foreground detection result of each training feature vector and the standard classification result of the corresponding training image The image classification model under training is trained until an image classification model is obtained from the training.

在一个具体的应用场景中，可以基于整体损失函数利用各训练图像的第一分类结果、第二分类结果、前景检测结果以及训练特征向量与各训练图像对应的标准分类结果以及标准特征向量对训练中的图像分类模型进行训练，直至得到图像分类模型。In a specific application scenario, the first classification result, the second classification result, the foreground detection result of each training image, the standard classification result corresponding to each training image and the standard feature vector of each training image can be used based on the overall loss function. The image classification model in is trained until the image classification model is obtained.

在一个具体的应用场景中，可以基于第一损失函数利用各训练特征向量的第一分类结果以及标准分类结果中对应的标准第一分类结果对训练中的图像分类模型进行训练；以及基于第二损失函数利用各训练特征向量的第二分类结果以及标准分类结果中对应的多个标准第二分类结果对训练中的图像分类模型进行训练；以及基于第三损失函数利用各训练特征向量的前景检测结果以及标准分类结果中对应的标准前景检测结果对训练中的图像分类模型进行训练；以及基于第四损失函数利用各训练特征向量以及对应的标准特征向量对训练中的图像分类模型进行训练；其中，通过加权处理后的第一损失函数、第二损失函数、第三损失函数以及第四损失函数之和确定整体损失函数。In a specific application scenario, the image classification model under training can be trained by using the first classification result of each training feature vector and the corresponding standard first classification result in the standard classification result based on the first loss function; and based on the second The loss function uses the second classification result of each training feature vector and a plurality of standard second classification results corresponding to the standard classification results to train the image classification model in training; and utilizes the foreground detection of each training feature vector based on the third loss function The result and the corresponding standard foreground detection result in the standard classification result are used to train the image classification model in training; and the image classification model in training is trained by using each training feature vector and the corresponding standard feature vector based on the fourth loss function; wherein , the overall loss function is determined by the sum of the weighted first loss function, the second loss function, the third loss function and the fourth loss function.

通过加权处理能够使得各个损失函数处于同一个量级，进而使得分类网络的多个分支网络以及特征提取网络的优化精度相似，实现图像分类模型的训练平衡。Through the weighting process, each loss function can be in the same order of magnitude, so that the optimization accuracy of the multiple branch networks and the feature extraction network of the classification network is similar, and the training balance of the image classification model can be achieved.

在一个具体的应用场景中，第一损失函数和第二损失函数可以采用softmax函数，第三损失函数可以采用ciou损失函数，而第四损失函数可以采用am函数。在其他应用场景中，各损失函数也可以基于训练对象的特征采取其他类型的损失函数，在此不做限定。In a specific application scenario, the first loss function and the second loss function may use the softmax function, the third loss function may use the ciou loss function, and the fourth loss function may use the am function. In other application scenarios, each loss function may also adopt other types of loss functions based on the characteristics of the training object, which is not limited here.

本实施例分别通过四个损失函数对第一分类、第二分类、前景检测以及特征提取处理进行优化训练，使得训练中的图像分类模型能够基于上述四种对图像的处理进行独立训练收敛，从而能够提高图像分类模型的各处理的处理精度，进而提高整个图像分类模型的分类精度。In this embodiment, the first classification, the second classification, the foreground detection, and the feature extraction processing are optimized and trained through four loss functions respectively, so that the image classification model in training can be independently trained and converged based on the above four types of image processing, thereby The processing accuracy of each processing of the image classification model can be improved, thereby improving the classification accuracy of the entire image classification model.

且上述训练能够使图像分类模型在训练过程中学习到如何进行第一分类、第二分类、前景检测以及特征提取，进而使得训练完成的图像分类模型能够提高第一分类、第二分类、前景检测以及特征提取的精度，并利用对训练图像进行上述四种不同的处理训练，从而使得训练完成的图像分类模型能够更多地聚焦到待分类图像的前景区域，减少对无效背景区域的过度关注，提高模型进行第一分类、第二分类的针对性，进而提高图像分类模型的分类准确率和可靠性。进一步地，前景训练可以让图像分类模型在学习过程中尽可能忽略背景信息，聚焦前景，从而能够使得图像分类模型在不同场景的分类更加专注前景，适应力更强。且，本实施例在图像分类模型的图像分类过程采用多种分类处理方式进行分类，能够进一步提高图像分类模型的泛化能力，使其能够应用于多种分类处理方式所能够适应的应用场景，提高本实施例的图像分类方法的应用范围和鲁棒性。And the above training can enable the image classification model to learn how to perform the first classification, the second classification, the foreground detection and the feature extraction during the training process, so that the trained image classification model can improve the first classification, the second classification, and the foreground detection. and the accuracy of feature extraction, and using the above four different processing training on the training image, so that the trained image classification model can focus more on the foreground area of the image to be classified, and reduce excessive attention to the invalid background area. The pertinence of the model for the first classification and the second classification is improved, thereby improving the classification accuracy and reliability of the image classification model. Further, foreground training can make the image classification model ignore the background information as much as possible and focus on the foreground during the learning process, so that the image classification model can focus more on the foreground in the classification of different scenes, and has stronger adaptability. Moreover, in the image classification process of the image classification model in this embodiment, a variety of classification processing methods are used for classification, which can further improve the generalization ability of the image classification model, so that it can be applied to application scenarios that can be adapted by a variety of classification processing methods. The application range and robustness of the image classification method of this embodiment are improved.

通过上述步骤，本实施例的图像分类模型通过图像分类模型的特征提取网络对待分类图像进行特征提取，得到待分类图像的特征向量，再通过图像分类模型的分类网络分别对特征向量进行第一分类处理、多次第二分类处理以及前景检测，得到特征向量的第一分类结果、第二分类结果以及前景检测结果，最后基于第一分类结果、第二分类结果以及前景检测结果得到待分类图像的分类结果，使得整个图像分类能够避免在复杂背景的待分类图像中无法聚焦特定前景，导致多过关注到背景信息的情况发生，通过多种分类处理提高对待分类图像中特定前景的关注度，从而提高图像分类准确率和可靠性。本实施例能够通过分别对特征向量进行多次互相独立地处理，避免各处理之间互相干扰，互相影响的情况发生，从而提高对特征向量进行整体分类的准确性和可靠性。且本实施例将独立处理后的分类结果合并后得到最终的待分类图像的分类图像，能够在一定程度上弱化图像分类在复杂背景的待分类图像中对无效背景的过多关注，从而加强对待分类图像中特定前景的关注度，提高图像分类准确率和可靠性。且本实施例的图像分类模型的训练过程还能使得训练中的图像分类模型能够基于不同的图像处理进行独立训练收敛，从而能够提高图像分类模型的各处理的处理精度，进而提高整个图像分类模型的分类精度。且，本实施例在图像分类模型的图像分类过程采用多种分类处理方式进行分类，能够进一步提高图像分类模型的泛化能力，使其能够应用于多种分类处理方式所能够适应的应用场景，提高本实施例的图像分类方法的应用范围和鲁棒性。Through the above steps, the image classification model of this embodiment performs feature extraction on the image to be classified through the feature extraction network of the image classification model to obtain the feature vector of the image to be classified, and then performs the first classification on the feature vector through the classification network of the image classification model. processing, multiple second classification processing and foreground detection to obtain the first classification result, second classification result and foreground detection result of the feature vector, and finally obtain the image to be classified based on the first classification result, the second classification result and the foreground detection result. The classification result enables the entire image classification to avoid the failure to focus on a specific foreground in the image to be classified with a complex background, resulting in more attention to the background information. Improve image classification accuracy and reliability. In this embodiment, the feature vectors are processed independently of each other for multiple times, so as to avoid mutual interference and mutual influence between the processes, thereby improving the accuracy and reliability of the overall classification of the feature vectors. And this embodiment combines the independently processed classification results to obtain the final classification image of the image to be classified, which can weaken the image classification to a certain extent in the image to be classified with complex background. Classify the attention of specific foregrounds in images to improve the accuracy and reliability of image classification. In addition, the training process of the image classification model in this embodiment can also enable the image classification model under training to perform independent training and convergence based on different image processing, thereby improving the processing accuracy of each processing of the image classification model, thereby improving the entire image classification model. classification accuracy. Moreover, in the image classification process of the image classification model in this embodiment, a variety of classification processing methods are used for classification, which can further improve the generalization ability of the image classification model, so that it can be applied to application scenarios that can be adapted by a variety of classification processing methods. The application range and robustness of the image classification method of this embodiment are improved.

请参阅图6，图6是本申请图像分类装置一实施例的框架示意图。图像分类装置60包括获取模块61、特征提取模块62、分类模块63以及确定模块64。获取模块61，用于获取到待分类图像；特征提取模块62用于对待分类图像进行特征提取，得到待分类图像的特征向量；分类模块63，用于分别对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果、特征向量的前景检测结果以及特征向量的特征分类结果；确定模块64，用于基于第一分类结果、第二分类结果、前景检测结果以及特征分类结果得到待分类图像的分类结果。Please refer to FIG. 6 , which is a schematic diagram of a framework of an embodiment of an image classification apparatus of the present application. Theimage classification device 60 includes anacquisition module 61 , afeature extraction module 62 , aclassification module 63 and adetermination module 64 . Theacquisition module 61 is used to acquire the image to be classified; thefeature extraction module 62 is used to perform feature extraction on the image to be classified to obtain the feature vector of the image to be classified; Two-classification processing, foreground detection, and feature classification processing, to obtain the first classification result of the feature vector, the second classification result of the feature vector, the foreground detection result of the feature vector, and the feature classification result of the feature vector; the determiningmodule 64 is used for based on the first classification result of the feature vector The first classification result, the second classification result, the foreground detection result and the feature classification result obtain the classification result of the image to be classified.

分类模块63还用于对特征向量进行前景检测，得到特征向量的前景检测结果；分别对特征向量的前景检测结果进行第一分类处理、第二分类处理以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果以及特征向量的特征分类结果。Theclassification module 63 is also used to perform foreground detection on the feature vector to obtain the foreground detection result of the feature vector; respectively perform the first classification process, the second classification process and the feature classification process on the foreground detection result of the feature vector to obtain the first classification process of the feature vector. The classification result, the second classification result of the feature vector, and the feature classification result of the feature vector.

确定模块64还用于响应于第二分类结果的分类类型与特征分类结果的分类类型相同，且第二分类结果为第一分类结果或第一分类结果对应的子类型，将第二分类结果以及前景检测结果确定为待分类图像的分类结果。Thedetermination module 64 is further configured to, in response to the classification type of the second classification result being the same as the classification type of the feature classification result, and the second classification result being the first classification result or a subtype corresponding to the first classification result, the second classification result and The foreground detection result is determined as the classification result of the image to be classified.

确定模块64还用于将第二分类处理的各分类类型的置信度分别与对应的第一分类处理的各分类类型的置信度以及对应的特征分类处理的各分类类型的置信度进行乘积处理，得到多个乘积数值；将数值最大的乘积数值对应的第二分类处理的分类类型以及前景检测结果进行逻辑组合，得到待分类图像的分类结果。Thedetermination module 64 is further configured to multiply the confidence levels of each classification type of the second classification process with the corresponding confidence levels of each classification type of the first classification process and the confidence levels of each classification type of the corresponding feature classification process, respectively, and perform product processing, Obtaining multiple product values; logically combining the classification type of the second classification process and the foreground detection result corresponding to the product value with the largest value to obtain the classification result of the image to be classified.

特征提取模块62还用于通过图像分类模型的特征提取网络对待分类图像进行特征提取，得到待分类图像的特征向量；分别对特征向量进行第一分类处理、第二分类处理、前景检测以及特征分类处理，得到特征向量的第一分类结果、特征向量的第二分类结果、特征向量的前景检测结果以及特征向量的特征分类结果的步骤包括：通过图像分类模型的前景检测网络对特征向量进行前景检测处理，得到特征向量的前景检测结果；以及通过图像分类模型的第一分类网络对特征向量进行第一分类处理，得到特征向量的第一分类结果；以及通过图像分类模型的第二分类网络对特征向量进行第二分类处理，得到特征向量的第二分类结果；以及利用特征标准库对特征向量进行特征分类处理，得到特征向量的特征分类结果；其中，图像分类模型至少包括特征提取网络、第一分类网络、第二分类网络以及前景检测网络，特征提取网络分别与第一分类网络、第二分类网络以及前景检测网络相互级联。Thefeature extraction module 62 is further configured to perform feature extraction on the image to be classified through the feature extraction network of the image classification model to obtain the feature vector of the image to be classified; respectively perform the first classification process, the second classification process, the foreground detection and the feature classification on the feature vector. The steps of obtaining the first classification result of the feature vector, the second classification result of the feature vector, the foreground detection result of the feature vector, and the feature classification result of the feature vector include: performing foreground detection on the feature vector through the foreground detection network of the image classification model. process, obtain the foreground detection result of the feature vector; And carry out the first classification process to the feature vector by the first classification network of the image classification model, obtain the first classification result of the feature vector; And pass the second classification network of the image classification model to the feature vector The vector is subjected to the second classification process to obtain the second classification result of the feature vector; and the feature standard library is used to perform the feature classification process on the feature vector to obtain the feature classification result of the feature vector; wherein, the image classification model at least includes a feature extraction network, a first The classification network, the second classification network and the foreground detection network, and the feature extraction network are respectively cascaded with the first classification network, the second classification network and the foreground detection network.

分类模块63还用于将特征向量分别与特征标准库中的各标准特征向量进行余弦相似度处理，得到特征向量与各标准特征向量之间的相似度；将相似度最大的标准特征向量对应的分类类型确定为特征向量的特征分类结果。Theclassification module 63 is also used to perform cosine similarity processing on the feature vector with each standard feature vector in the feature standard library respectively to obtain the similarity between the feature vector and each standard feature vector; The classification type is determined as the feature classification result of the feature vector.

获取模块61还用于获取到多张训练图像；Theacquisition module 61 is also used to acquire multiple training images;

通过训练中的图像分类模型的特征提取网络分别对各训练图像进行特征提取，得到各训练图像的训练特征向量；通过训练中的图像分类模型的分类网络分别对训练特征向量进行第一分类处理、第二分类处理、前景检测，得到训练特征向量的第一分类结果、训练特征向量的第二分类结果、训练特征向量的前景检测结果；利用各训练图像的第一分类结果、第二分类结果、前景检测结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型。Perform feature extraction on each training image through the feature extraction network of the image classification model in training, to obtain the training feature vector of each training image; through the classification network of the image classification model in training, respectively perform the first classification processing, The second classification process and foreground detection are used to obtain the first classification result of the training feature vector, the second classification result of the training feature vector, and the foreground detection result of the training feature vector; using the first classification result, second classification result, The foreground detection result and the standard classification result corresponding to each training image are used to train the image classification model under training until the image classification model is obtained.

获取模块61还用于获取到多张训练图像；其中，各训练图像上标注有标准第一分类结果、标准第二分类结果以及标准前景结果；对训练图像进行特征提取，得到训练图像的标准特征向量，基于训练图像的多个标准第二分类结果确定标准特征向量对应的分类类型；利用各训练图像的标准第一分类结果、多个标准第二分类结果、标准前景结果以及标准特征向量，得到各训练图像的标准分类结果。Theacquisition module 61 is also used to acquire a plurality of training images; wherein, each training image is marked with a standard first classification result, a standard second classification result and a standard foreground result; feature extraction is performed on the training image to obtain the standard features of the training image vector, determine the classification type corresponding to the standard feature vector based on the multiple standard second classification results of the training image; use the standard first classification results, multiple standard second classification results, standard foreground results and standard feature vectors of each training image to obtain Standard classification results for each training image.

获取模块61还用于基于整体损失函数利用各训练图像的第一分类结果、第二分类结果、前景检测结果以及特征分类结果与各训练图像对应的标准分类结果对训练中的图像分类模型进行训练，直至得到图像分类模型。Theacquisition module 61 is further configured to use the first classification result, the second classification result, the foreground detection result and the feature classification result of each training image and the standard classification result corresponding to each training image based on the overall loss function to train the image classification model under training. , until the image classification model is obtained.

获取模块61还用于基于第一损失函数利用各训练特征向量的第一分类结果以及标准分类结果中对应的标准第一分类结果对训练中的图像分类模型进行训练；以及基于第二损失函数利用各训练特征向量的第二分类结果以及标准分类结果中对应的标准第二分类结果对训练中的图像分类模型进行训练；以及基于第三损失函数利用各训练特征向量的前景检测结果以及标准分类结果中对应的标准前景检测结果对训练中的图像分类模型进行训练；以及基于第四损失函数利用各训练特征向量以及对应的标准特征向量对训练中的图像分类模型进行训练；其中，通过加权处理后的第一损失函数、第二损失函数、第三损失函数以及第四损失函数之和确定整体损失函数。Theacquisition module 61 is further configured to use the first classification result of each training feature vector and the corresponding standard first classification result in the standard classification result based on the first loss function to train the image classification model in training; and based on the second loss function using The second classification result of each training feature vector and the corresponding standard second classification result in the standard classification result are used to train the image classification model in training; and the foreground detection result and the standard classification result of each training feature vector are utilized based on the third loss function. The image classification model in training is trained with the corresponding standard foreground detection results in The sum of the first loss function, the second loss function, the third loss function, and the fourth loss function determines the overall loss function.

其中，第一分类结果的分类类型包括正常类型，性感类型，色情类型；第二分类结果和特征分类结果的分类类型包括正常类型、性感类型的子类型以及色情类型的子类型；前景检测结果包括各待分类图像上人体所在的区域的位置信息。The classification types of the first classification result include normal types, sexy types, and pornographic types; the classification types of the second classification results and feature classification results include normal types, sub-types of sexy types, and sub-types of pornographic types; the foreground detection results include The position information of the area where the human body is located on each image to be classified.

确定模块64还用于响应于前景检测结果的前景检测尺寸不大于预设尺寸，将待分类图像的分类结果确定为正常类型。The determiningmodule 64 is further configured to determine the classification result of the image to be classified as a normal type in response to the foreground detection size of the foreground detection result being not greater than the preset size.

上述方案，能够提高图像分类精度。The above solution can improve the image classification accuracy.

请参阅图7，图7是本申请电子设备一实施例的框架示意图。电子设备70包括相互耦接的存储器71和处理器72，处理器72用于执行存储器71中存储的程序指令，以实现上述任一实施例的图像分类方法的步骤。在一个具体的实施场景中，电子设备70可以包括但不限于：微型计算机、服务器，此外，电子设备70还可以包括笔记本电脑、平板电脑等移动设备，在此不做限定。Please refer to FIG. 7 , which is a schematic diagram of a framework of an embodiment of an electronic device of the present application. Theelectronic device 70 includes amemory 71 and aprocessor 72 coupled to each other, and theprocessor 72 is configured to execute program instructions stored in thememory 71 to implement the steps of the image classification method in any of the foregoing embodiments. In a specific implementation scenario, theelectronic device 70 may include, but is not limited to, a microcomputer and a server. In addition, theelectronic device 70 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.

具体而言，处理器72用于控制其自身以及存储器71以实现上述任一图像分类方法实施例的步骤。处理器72还可以称为CPU(Central Processing Unit，中央处理单元)。处理器72可能是一种集成电路芯片，具有信号的处理能力。处理器72还可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application SpecificIntegrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。另外，处理器72可以由集成电路芯片共同实现。Specifically, theprocessor 72 is configured to control itself and thememory 71 to implement the steps of any of the image classification method embodiments described above. Theprocessor 72 may also be referred to as a CPU (Central Processing Unit, central processing unit). Theprocessor 72 may be an integrated circuit chip with signal processing capability. Theprocessor 72 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programming logic devices, discrete gate or transistor logic devices, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, theprocessor 72 may be jointly implemented by an integrated circuit chip.

请参阅图8，图8为本申请计算机可读存储介质一实施例的框架示意图。计算机可读存储介质80存储有能够被处理器运行的程序指令801，程序指令801用于实现上述任一实施例的图像分类方法的步骤。Please refer to FIG. 8 , which is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 80stores program instructions 801 that can be executed by the processor, and theprogram instructions 801 are used to implement the steps of the image classification method in any of the foregoing embodiments.

在本申请所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性、机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other divisions. For example, units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed over network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.

另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented as a software functional unit and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .