技术领域Technical field
本发明涉及机器学习和计算机视觉技术领域，尤其涉及的是一种基于特征调整的域泛化图像分类方法、系统、终端及介质。The present invention relates to the technical fields of machine learning and computer vision, and in particular to a domain generalization image classification method, system, terminal and medium based on feature adjustment.
背景技术Background technique
为解决在不同数据分布下模型泛化能力差的问题，域泛化旨在通过仅使用源数据进行模型学习来实现面向对象的泛化。域泛化广泛用于图像分类、目标检测、人脸识别、语音识别等实际应用中。To solve the problem of poor model generalization ability under different data distributions, domain generalization aims to achieve object-oriented generalization by using only source data for model learning. Domain generalization is widely used in practical applications such as image classification, target detection, face recognition, and speech recognition.
目前，大量域泛化方法致力于解决域偏移问题，包括学习域不变特征表示方法、数据增强方法以及模型学习策略等。其中，基于模型学习策略对模型的训练主要局限于在源域数据上如何训练模型，而忽略了在测试阶段利用目标域样本相关的个体信息进行针对性地自适应推理，使得在目标域上模型对部分样本性能不佳，导致域泛化能力及模型精度较低。Currently, a large number of domain generalization methods are dedicated to solving the domain shift problem, including learning domain-invariant feature representation methods, data enhancement methods, and model learning strategies. Among them, the training of the model based on the model learning strategy is mainly limited to how to train the model on the source domain data, but ignores the use of individual information related to the target domain samples in the test phase to perform targeted adaptive inference, so that the model in the target domain Poor performance on some samples results in low domain generalization capabilities and low model accuracy.
发明内容Contents of the invention
鉴于上述现有技术的不足，本发明的目的在于提供一种基于特征调整的域泛化图像分类方法、系统、终端及介质，旨在解决现有技术中存在的学习模型的域泛化能力较低的问题。In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a domain generalization image classification method, system, terminal and medium based on feature adjustment, aiming to solve the problem of poor domain generalization capabilities of learning models in the prior art. low question.
为了实现上述目的，本发明第一方面提供一种基于特征调整的域泛化图像分类方法，包括以下步骤：In order to achieve the above objectives, the first aspect of the present invention provides a domain generalization image classification method based on feature adjustment, which includes the following steps:
获取目标域的图像及图像类别标签，并获取训练好的域泛化图像分类模型，所述训练好的域泛化图像分类模型包括训练好的基础特征提取网络、训练好的域偏移感知网络、训练好的执行器网络和训练好的基础分类网络；Obtain images and image category labels of the target domain, and obtain a trained domain generalization image classification model. The trained domain generalization image classification model includes a trained basic feature extraction network and a trained domain shift perception network. , the trained actuator network and the trained basic classification network;
利用所述目标域的图像构建目标集，并将所述目标集输入至所述训练好的域泛化图像分类模型，对所述目标域的图像进行分类，获得最终的分类结果。A target set is constructed using images in the target domain, and the target set is input to the trained domain generalization image classification model to classify the images in the target domain to obtain a final classification result.
可选的，对所述训练好的域泛化图像分类模型进行训练的步骤，包括：Optionally, the step of training the trained domain generalization image classification model includes:
获取源域的图像，构建域泛化图像分类模型，所述域泛化图像分类模型包括基础特征提取网络、域偏移感知网络、执行器网络和基础分类网络；Obtain images from the source domain and construct a domain generalized image classification model. The domain generalized image classification model includes a basic feature extraction network, a domain shift perception network, an actuator network and a basic classification network;
基于所述源域的图像构建训练集，将所述训练集输入至所述基础特征提取网络，得到原始特征；Construct a training set based on the images in the source domain, input the training set to the basic feature extraction network, and obtain original features;
基于预设的约束条件，将所述原始特征输入至所述域偏移感知网络，获得约束偏差；Based on the preset constraint conditions, input the original features into the domain shift sensing network to obtain the constrained deviation;
基于所述约束偏差，将所述原始特征输入至所述执行器网络，获得调整后的特征；Based on the constrained deviation, input the original features to the actuator network to obtain adjusted features;
将所述调整后的特征输入到所述基础分类网络，获得分类结果；Input the adjusted features into the basic classification network to obtain classification results;
基于所述分类结果和所述图像类别标签计算联合损失，重复执行对所述域泛化图像分类模型进行训练的步骤，直至所述联合损失达到预设的联合损失阈值，获得训练好的域泛化图像分类模型。Calculate a joint loss based on the classification result and the image category label, and repeatedly perform the steps of training the domain generalization image classification model until the joint loss reaches a preset joint loss threshold, and obtain the trained domain generalization model. image classification model.
可选的，还包括利用扰动特征更新所述原始特征，具体包括：Optionally, it also includes using perturbation features to update the original features, specifically including:
基于所述原始特征的统计分布，获取所述原始特征的均值和方差；Based on the statistical distribution of the original features, obtain the mean and variance of the original features;
基于所述均值和所述方差，对所述原始特征的高斯分布进行缩放，获得缩放后的均值和缩放后的方差；Based on the mean and the variance, scale the Gaussian distribution of the original feature to obtain a scaled mean and a scaled variance;
利用所述缩放后的均值和所述缩放后的方差产生扰动样本，获得扰动特征；Using the scaled mean and the scaled variance to generate disturbance samples to obtain disturbance features;
利用所述扰动特征更新所述原始特征。The original features are updated using the perturbation features.
可选的，所述将所述原始特征输入至所述域偏移感知网络，获得约束偏差，包括：Optionally, inputting the original features into the domain shift sensing network to obtain constrained deviations includes:
获取所述图像的分类类别，及每个所述分类类别对应的类中心；Obtain the classification category of the image and the class center corresponding to each classification category;
基于所述原始特征和预设的结构约束条件，获得结构约束偏差；Based on the original features and preset structural constraints, obtain the structural constraint deviation;
基于所述原始特征、所述类中心和预设的分布约束条件，获得分布约束偏差；Based on the original features, the class center and the preset distribution constraint conditions, obtain the distribution constraint deviation;
基于所述结构约束偏差和所述分布约束偏差，获得所述原始特征的约束偏差。Based on the structural constraint deviation and the distribution constraint deviation, a constraint deviation of the original feature is obtained.
可选的，所述基于所述原始特征和预设的结构约束条件，获得结构约束偏差，包括：Optionally, the structural constraint deviation is obtained based on the original features and preset structural constraint conditions, including:
将所述原始特征投影到与所述原始特征具有相同维度的空间，获得投影后的特征；Project the original features to a space with the same dimension as the original features to obtain projected features;
基于所述图像的分类类别，对所述投影后的特征进行归一化处理，获得结构约束特征；Based on the classification category of the image, normalize the projected features to obtain structural constraint features;
计算所述投影后的特征与所述结构约束特征的距离，获得结构约束偏差。Calculate the distance between the projected feature and the structural constraint feature to obtain the structural constraint deviation.
可选的，所述基于所述原始特征、所述类中心和预设的分布约束条件，获得分布约束偏差，包括：Optionally, obtaining the distribution constraint deviation based on the original features, the class center and preset distribution constraint conditions includes:
基于所述预设的分布约束条件，获得所述原始特征与各个所述类中心之间的相关性；Based on the preset distribution constraints, obtain the correlation between the original features and each of the class centers;
基于所述相关性，获得所述原始特征与各个所述类中心的权重；Based on the correlation, obtain the weight of the original feature and each of the class centers;
基于所述原始特征与各个所述类中心之间的距离和所述权重，获得分布约束偏差。Based on the distance between the original feature and each of the class centers and the weight, a distribution constraint deviation is obtained.
可选的，所述基于所述分类结果和所述图像类别标签计算联合损失，包括：Optionally, calculating a joint loss based on the classification result and the image category label includes:
基于所述分类结果、所述图像类别标签及所述图像类别标签的数量，计算交叉熵损失；Calculate cross-entropy loss based on the classification result, the image category label, and the number of image category labels;
基于所述图像的分类类别，对投影后的特征进行归一化处理，获得结构约束条件；并基于所述结构约束条件和所述投影后的特征构建结构约束损失函数；Based on the classification category of the image, normalize the projected features to obtain structural constraints; and construct a structural constraint loss function based on the structural constraints and the projected features;
基于所述交叉熵损失和所述结构约束损失函数，计算联合损失。Based on the cross-entropy loss and the structural constraint loss function, a joint loss is calculated.
本发明第二方面提供一种基于特征调整的域泛化分类系统，所述系统包括：A second aspect of the present invention provides a domain generalization classification system based on feature adjustment. The system includes:
数据获取模块，用于获取目标域的图像及图像类别标签，并获取训练好的域泛化图像分类模型，所述训练好的域泛化图像分类模型包括训练好的基础特征提取网络、训练好的域偏移感知网络、训练好的执行器网络和训练好的基础分类网络；The data acquisition module is used to obtain images and image category labels of the target domain, and obtain a trained domain generalized image classification model. The trained domain generalized image classification model includes a trained basic feature extraction network, a trained domain generalized image classification model, and a trained domain generalized image classification model. The domain shift aware network, the trained actuator network and the trained basic classification network;
域泛化分类模块，用于利用所述目标域的图像构建目标集，并将所述目标集输入至所述域泛化图像分类模型，对所述目标域的图像进行分类，获得最终的分类结果。A domain generalization classification module, used to construct a target set using images in the target domain, input the target set into the domain generalization image classification model, classify the images in the target domain, and obtain the final classification result.
本发明第三方面提供一种智能终端，所述智能终端包括存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的基于特征调整的域泛化分类程序，所述基于特征调整的域泛化分类程序被所述处理器执行时实任意一项上述基于特征调整的域泛化图像分类方法的步骤。A third aspect of the present invention provides an intelligent terminal. The intelligent terminal includes a memory, a processor, and a domain generalization classification program based on feature adjustment that is stored in the memory and can be run on the processor. When the feature-adjusted domain generalization classification program is executed by the processor, any one of the steps of the above-mentioned feature-adjusted domain generalization image classification method is implemented.
本发明第四方面提供一种计算机可读存储介质，所述计算机可读存储介质上存储有基于特征调整的域泛化分类程序，所述基于特征调整的域泛化分类程序被处理器执行时实现任意一项上述基于特征调整的域泛化图像分类方法的步骤。A fourth aspect of the present invention provides a computer-readable storage medium. A domain generalization classification program based on feature adjustment is stored on the computer-readable storage medium. When the domain generalization classification program based on feature adjustment is executed by a processor, Steps to implement any of the above domain generalization image classification methods based on feature adjustment.
与现有技术相比，本发明的有益效果如下：Compared with the prior art, the beneficial effects of the present invention are as follows:
本发明通过获取目标域的图像及图像类别标签，并获取训练好的域泛化图像分类模型，且训练好的域泛化图像分类模型包括训练好的基础特征提取网络、训练好的域偏移感知网络、训练好的执行器网络和训练好的基础分类网络；利用目标域的图像构建目标集，并将目标集输入至训练好的域泛化图像分类模型，对目标域的图像进行分类，获得最终的分类结果。The present invention obtains the image and image category label of the target domain and obtains the trained domain generalization image classification model, and the trained domain generalization image classification model includes the trained basic feature extraction network and the trained domain offset. Perception network, trained actuator network and trained basic classification network; use images in the target domain to construct a target set, and input the target set into the trained domain generalization image classification model to classify images in the target domain, Obtain the final classification result.
可见，本发明设计的训练好的域泛化图像分类模型是一种新型的传感器-执行器网络模型，该模型中的各个子模块能够通过引入域偏移传感器来感知由域偏移引起的约束偏差，并根据源域图像特征的统计分布特点对源域图像特征的统计参数进行缩放得到扰动后的源域图像特征，并根据约束偏差对缩放扰动后的源域图像特征进行调整优化，获得调整后的特征，各个子模块的功能相辅相成，能够提高分类效率，并提高图片分类模型的精度与泛化能力，而且由于子模块具备良好的自适应调节能力，无需依赖复杂的数据生成和增强，使得模型在不同的环境下都能够保持良好的性能，具有良好的鲁棒性和稳健性。It can be seen that the trained domain generalization image classification model designed by the present invention is a new type of sensor-actuator network model. Each sub-module in the model can sense the constraints caused by domain offset by introducing domain offset sensors. deviation, and scale the statistical parameters of the source domain image features according to the statistical distribution characteristics of the source domain image features to obtain the perturbed source domain image features, and adjust and optimize the scaled and perturbed source domain image features according to the constraint deviation to obtain the adjustment After the features are obtained, the functions of each sub-module complement each other, which can improve the classification efficiency and improve the accuracy and generalization ability of the image classification model. Moreover, because the sub-modules have good adaptive adjustment capabilities, there is no need to rely on complex data generation and enhancement, making The model can maintain good performance in different environments and has good robustness and robustness.
附图说明Description of drawings
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其它的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings in the following description are only illustrative of the present invention. For some embodiments, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.
图1为本发明的域泛化图像分类策略流程图；Figure 1 is a flow chart of the domain generalization image classification strategy of the present invention;
图2为本发明的域泛化图像分类系统结构示意图；Figure 2 is a schematic structural diagram of the domain generalization image classification system of the present invention;
图3为本发明的基于特征调整的域泛化图像分类方法流程图；Figure 3 is a flow chart of the domain generalization image classification method based on feature adjustment of the present invention;
图4为本发明的域泛化图像分类模型训练流程图；Figure 4 is a flow chart of domain generalization image classification model training flow chart of the present invention;
图5为本发明的投影后的特征仿真图及相应的结构约束偏差仿真图；Figure 5 is a projected characteristic simulation diagram and corresponding structural constraint deviation simulation diagram of the present invention;
图6为本发明的执行器网络调整前、后特征分布对比示意图；Figure 6 is a schematic diagram comparing the characteristic distribution before and after adjustment of the actuator network of the present invention;
图7为本发明的利用所构建的域泛化图像分类模型进行分类的流程示意图；Figure 7 is a schematic flow chart of classification using the constructed domain generalization image classification model of the present invention;
图8为本发明的基于特征调整的域泛化图像分类系统示意图；Figure 8 is a schematic diagram of the domain generalization image classification system based on feature adjustment of the present invention;
图9为本发明的智能终端原理框图。Figure 9 is a functional block diagram of the intelligent terminal of the present invention.
具体实施方式Detailed ways
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本发明实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本发明。在其它情况下，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本发明的描述。In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention in unnecessary detail.
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。It will be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components but does not exclude one or more other features , the presence or addition of a whole, a step, an operation, an element, a component, and/or a collection thereof.
还应当理解，在本发明说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本发明。如在本发明说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。It should also be understood that the terminology used in the description of the present invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly dictates otherwise.
还应当进一步理解，在本发明说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。It will be further understood that the term "and/or" as used in the specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items. .
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当…时”或“一旦”或“响应于确定”或“响应于检测到”。类似的，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述的条件或事件]”或“响应于检测到[所描述条件或事件]”。As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, to mean "once determined" or "in response to a determination" or "once the [described condition or event] is detected" event]” or “in response to detection of [the described condition or event]”.
下面结合本发明实施例的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明的一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.
在下面的描述中阐述了很多具体细节以便于充分理解本发明，但是本发明还可以采用其它不同于在此描述的其它方式来实施，本领域技术人员可以在不违背本发明内涵的情况下做类似推广，因此本发明不受下面公开的具体实施例的限制。Many specific details are set forth in the following description to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described here. Those skilled in the art can do so without departing from the connotation of the present invention. Similar generalizations are made, and therefore the present invention is not limited to the specific embodiments disclosed below.
现有方法忽略了在测试阶段利用目标域样本相关的个体信息进行针对性地自适应推理优化，针对这个问题，本方法受自动控制理论启发，设计了一个传感器-执行器网络模型，提供域偏移的检测机制和显示的特征调整机制。我们的传感器-执行器网络模型由一个用于检测域偏移的传感器网络和一个能够根据传感器信号自适应地调整特征的执行器网络组成。其中域偏移传感器包含两个部分：约束网络和数据转换网络。我们对基础特征提取网络提取到的原始特征引入了两个约束条件，并在源域中学习了一个约束网络以验证输出特征是否满足这些约束条件，从而在推理阶段，约束网络会分析目标域样本的特征是否满足这些约束条件，若不满足，则和约束条件之间的偏差被称为约束偏差，这种约束偏差正是由域偏移引起的。我们随后学习一个数据转换网络将这些约束偏差映射到有效的指导信号中。在接收到指导信号后，执行器网络用于将原始特征调整为更具有区分度的新特征，新特征将被输入基础分类网络进行分类决策。该方法能够在推理阶段，利用目标域样本自身信息，捕获目标域样本的域偏移并产生相应指导信号，引导具有偏移的原始特征进行调整优化，提高了图片分类模型的精度与泛化能力。我们引入的传感器-执行器网络模型参数量小，计算资源和时间消耗少；在测试时利用目标域样本自身信息和特定约束进行域偏移检测，无需依赖复杂的数据生成和增强。Existing methods ignore the use of individual information related to target domain samples in the testing phase for targeted adaptive inference optimization. To address this problem, this method is inspired by automatic control theory and designs a sensor-actuator network model to provide domain bias. Shift detection mechanism and display feature adjustment mechanism. Our sensor-actuator network model consists of a sensor network for detecting domain shifts and an actuator network capable of adaptively adjusting features based on sensor signals. The domain offset sensor consists of two parts: the constraint network and the data conversion network. We introduced two constraints on the original features extracted by the basic feature extraction network, and learned a constraint network in the source domain to verify whether the output features meet these constraints, so that in the inference stage, the constraint network will analyze the target domain samples Whether the features satisfy these constraints, if not, the deviation from the constraints is called constraint deviation, and this constraint deviation is caused by domain shift. We then learn a data transformation network to map these constrained biases into effective guidance signals. After receiving the guidance signal, the actuator network is used to adjust the original features into new features that are more discriminative, and the new features will be input into the basic classification network for classification decisions. This method can use the target domain sample's own information to capture the domain offset of the target domain sample and generate corresponding guidance signals during the inference phase, guiding the original features with offsets to be adjusted and optimized, thereby improving the accuracy and generalization ability of the image classification model. . The sensor-actuator network model we introduced has a small number of parameters and consumes less computing resources and time; during testing, it uses the target domain sample's own information and specific constraints to detect domain offsets, without relying on complex data generation and enhancement.
示例性方法Example methods
本发明实施例提供一种基于特征调整的域泛化图像分类方法，部署于电脑、服务器等电子设备上，应用领域为医疗诊断、图像分类、目标检测、人脸识别、语音识别等等，应用场景为从源域数据训练的模型泛化到领域不同但相关的具有任意数据分布的目标域，主要目的是基于域泛化图像分类策略来构建域泛化图像分类系统，域泛化图像分类策略如图1所示，即通过生成基础特征提取网络获取源域数据的原始特征，并基于传感器网络感知原始特征在目标域上的域偏移，然后利用执行器网络对感知到的域偏移进行特征调整，得到调整后的特征，最后利用基础分类网络对调整后的特征进行分类决策。Embodiments of the present invention provide a domain generalization image classification method based on feature adjustment, which is deployed on electronic equipment such as computers and servers. The application fields include medical diagnosis, image classification, target detection, face recognition, speech recognition, etc. Applications The scenario is to generalize the model trained from source domain data to a target domain with arbitrary data distribution in a different domain but related. The main purpose is to build a domain generalization image classification system based on the domain generalization image classification strategy. The domain generalization image classification strategy As shown in Figure 1, the original features of the source domain data are obtained by generating a basic feature extraction network, and the domain offset of the original features in the target domain is perceived based on the sensor network, and then the actuator network is used to perform the perceived domain offset. The features are adjusted to obtain the adjusted features, and finally the basic classification network is used to make classification decisions on the adjusted features.
具体地，域泛化图像分类系统如图2所示，包括基础特征提取网络、域偏移感知网络、数据转换网络/>、执行器网络/>和基础分类网络/>。其中，基础特征提取网络/>用于基于源域的图像构建训练集，并基于训练集提取原始特征，并将原始特征输出至域偏移感知网络和执行器网络。域偏移感知网络用于基于接收到的原始特征和预设的结构约束条件及分布约束条件，获得结构约束偏差，并将约束偏差输出至执行器网络，且域偏移感知网络包括结构约束网络/>和分布约束网络/>。其中，结构约束网络/>用于将原始特征输入训练好的结构约束网络，获得结构约束偏差；分布约束网络/>用于将原始特征和源域数据的类中心输入训练好的分布约束网络，获得分布约束偏差。数据转换网络/>用于将结构约束偏差和分布约束偏差映射成为指导信号，并将指导信号输出至执行器网络。执行器网络/>用于基于接收到的约束偏差和原始特征，获得调整后的特征，并将调整后的特征输出至基础分类网络。基础分类网络/>用于基于接收到的调整后的特征，获得分类结果。基于分类结果和图像类别标签计算联合损失，并根据预设的联合损失阈值训练各个子网络，直至联合损失达到预设的联合损失阈值，获得训练好的域泛化图像分类模型。然后利用训练好的域泛化图像分类模型对目标域的图像进行分类，获得最终的分类结果。Specifically, the domain generalization image classification system is shown in Figure 2, including a basic feature extraction network , domain shift aware network, data conversion network/> , actuator network/> and basic classification network/> . Among them, the basic feature extraction network/> Used to construct a training set based on images in the source domain, extract original features based on the training set, and output the original features to the domain shift perception network and the actuator network. The domain shift-aware network is used to obtain the structural constraint deviation based on the received original features and preset structural constraints and distribution constraints, and output the constraint deviation to the actuator network, and the domain shift-aware network includes a structural constraint network /> and distribution constraint network/> . Among them, structural constraint network/> Used to input original features into the trained structural constraint network to obtain the structural constraint deviation; distribution constraint network/> It is used to input the original features and the class center of the source domain data into the trained distribution constraint network to obtain the distribution constraint deviation. Data conversion network/> It is used to map the structural constraint deviation and distribution constraint deviation into guidance signals, and output the guidance signals to the actuator network. Actuator Network/> Used to obtain adjusted features based on the received constrained deviations and original features, and output the adjusted features to the basic classification network. Basic classification network/> Used to obtain classification results based on the received adjusted features. The joint loss is calculated based on the classification results and image category labels, and each sub-network is trained according to the preset joint loss threshold until the joint loss reaches the preset joint loss threshold, and a trained domain generalization image classification model is obtained. Then use the trained domain generalization image classification model to classify the images in the target domain to obtain the final classification result.
基于该系统，设计的基于特征调整的域泛化图像分类方法流程，如图3所示，主要包括以下步骤：Based on this system, the designed domain generalization image classification method process based on feature adjustment is shown in Figure 3, which mainly includes the following steps:
步骤S100：获取目标域的图像及图像类别标签，并获取训练好的域泛化图像分类模型，其中，训练好的域泛化图像分类模型包括训练好的基础特征提取网络、训练好的域偏移感知网络、训练好的执行器网络和训练好的基础分类网络；Step S100: Obtain the image and image category label of the target domain, and obtain the trained domain generalization image classification model, where the trained domain generalization image classification model includes the trained basic feature extraction network, the trained domain bias Shift sensing network, trained actuator network and trained basic classification network;
步骤S200：利用目标域的图像构建目标集，并将目标集输入至训练好的域泛化图像分类模型，对目标域的图像进行分类，获得最终的分类结果。Step S200: Construct a target set using images in the target domain, input the target set into the trained domain generalization image classification model, classify the images in the target domain, and obtain the final classification result.
其中，对于步骤S100中获取的训练好的域泛化图像分类模型进行训练的步骤，如图4所示，包括：Among them, the steps of training the trained domain generalization image classification model obtained in step S100 are as shown in Figure 4, including:
步骤S110：获取源域的图像，构建域泛化图像分类模型，域泛化图像分类模型包括基础特征提取网络、域偏移感知网络、执行器网络和基础分类网络；Step S110: Obtain the image of the source domain and construct a domain generalized image classification model. The domain generalized image classification model includes a basic feature extraction network, a domain shift perception network, an actuator network and a basic classification network;
具体地，基于源域数据样本，获取源域的图像数据，及各个图像的类别标签。需要注意的是，不同的图像可能有相同的类别标签，但是每个图像最多只有一个类别标签。其中，图像的类别标签是图片所属的应用领域专家根据经验进行标定得到的。Specifically, based on the source domain data sample, the image data of the source domain and the category label of each image are obtained. It should be noted that different images may have the same category label, but each image has at most one category label. Among them, the category label of the image is calibrated based on experience by experts in the application field to which the image belongs.
构建域泛化图像分类模型，域泛化图像分类模型包括基础特征提取网络、域偏移感知网络、执行器网络和基础分类网络。Construct a domain generalization image classification model. The domain generalization image classification model includes a basic feature extraction network, a domain shift perception network, an actuator network and a basic classification network.
具体地，构建域泛化图像分类模型，初始化该模型中的各个子网络，即基础特征提取网络、域偏移感知网络、执行器网络和基础分类网络，并通过设定的条件对各个子网络进行迭代训练，直至满足所设定的条件。下面详细介绍域泛化图像分类模型的训练过程。其中，域偏移感知网络包括结构约束网络和分布约束网络。Specifically, a domain generalization image classification model is constructed, and each sub-network in the model is initialized, namely the basic feature extraction network, the domain shift perception network, the actuator network and the basic classification network, and each sub-network is evaluated according to the set conditions. Carry out iterative training until the set conditions are met. The following describes the training process of the domain generalization image classification model in detail. Among them, domain shift-aware networks include structural constraint networks and distribution constraint networks.
步骤S120：利用源域的图像构建训练集，并将训练集输入至基础特征提取网络，得到原始特征。Step S120: Construct a training set using images in the source domain, and input the training set into the basic feature extraction network to obtain original features.
具体地，利用现有的经验风险最小化方法，将所有的源域图像数据合并为一个训练集。利用所构建的训练集，以交叉熵损失为目标函数，训练以Resnet-50卷积神经网络为骨干的基础特征提取网络和由一层全连接层构造的基础分类网络/>。Specifically, existing empirical risk minimization methods are used to merge all source domain image data into a training set. Use the constructed training set and use cross-entropy loss as the objective function to train a basic feature extraction network with Resnet-50 convolutional neural network as the backbone. and a basic classification network constructed by a layer of fully connected layers/> .
步骤S130：基于预设的约束条件，将原始特征输入至域偏移感知网络，获得约束偏差；Step S130: Based on the preset constraint conditions, input the original features into the domain shift sensing network to obtain the constrained deviation;
具体地，由于是在源域上训练基础特征提取网络，源域样本特征可以在特征中间很好地根据类别聚类。这里定义源域特征的类中心为/>，其中表示类中心的数量。但在目标域上，由于域偏移的存在，这些聚类的特征分布遭受一定程度的破坏而变得离散，部分特征变得远离正确的类中心，原始特征/>与各个类中心之间会往往会存在一定的偏差，本实施例通过捕获这些偏差生成分布约束偏差，以便于对域偏移进行统计分析。Specifically, since the basic feature extraction network is trained on the source domain , the source domain sample features can be well clustered according to categories among the features. Here the class center of the source domain features is defined as/> ,in Indicates the number of class centers. However, in the target domain, due to the existence of domain offset, the feature distribution of these clusters suffers a certain degree of damage and becomes discrete, and some features become far away from the correct cluster center. The original features/> There will often be certain deviations from each class center. This embodiment generates distribution constraint deviations by capturing these deviations to facilitate statistical analysis of domain offsets.
本实施例通过对域偏移进行感知，也就是通过在源域中学习约束条件，并基于交叉熵损失检验学习到的约束条件是否得到满足来检测域偏移。因此，本实施例在源域中定义一组约束，其中，/>表示定义的每一项约束，i表示该组约束的数量，N表示源域中数据的一个维度。对于目标域样本而言，基于基础特征提取网络生成的原始特征/>可能会由于域偏移的存在而不再满足约束条件，即原始特征/>相对于约束存在约束偏差/>。本实施例引入结构约束网络和分布约束网络生成对应的约束条件来感知域偏移量，以从不同角度进行偏移感知得到由域偏移引起的综合偏差，能够提高对域偏移感知的准确性。This embodiment detects domain shifts by sensing domain shifts, that is, by learning constraints in the source domain, and checking whether the learned constraints are satisfied based on cross-entropy loss. Therefore, this embodiment defines a set of constraints in the source domain , where,/> represents each constraint defined, i represents the number of constraints in the group, and N represents a dimension of the data in the source domain. For target domain samples, the original features generated based on the basic feature extraction network/> The constraints may no longer be satisfied due to the presence of domain offsets, i.e. the original features/> Relative to constraints There is a constraint deviation/> . This embodiment introduces the structural constraint network and the distribution constraint network to generate corresponding constraints to sense the domain offset, so as to perform offset sensing from different angles to obtain the comprehensive deviation caused by the domain offset, which can improve the accuracy of domain offset perception. sex.
本实施例直接基于原始特征，利用结构约束网络和分布约束网络生成对应的约束条件来感知域偏移量，作为其他优选实施例，还可以对原始特征施加扰动之后得到扰动特征，并利用扰动特征更新原始特征之后，利用结构约束网络和分布约束网络生成对应的约束条件来感知域偏移量。This embodiment is directly based on the original features, and uses the structural constraint network and the distribution constraint network to generate corresponding constraints to perceive the domain offset. As other preferred embodiments, the perturbation features can also be obtained after perturbation is applied to the original features, and the perturbation features are used After updating the original features, the structural constraint network and distribution constraint network are used to generate corresponding constraints to sense the domain offset.
由于目标域是不可访问的，执行器网络需要在源域上进行训练，从而获得自适应调整特征的能力。为此，在训练阶段对于源域数据样本，获取原始特征的高斯分布的均值和方差；基于原始特征的高斯分布的均值和方差，对原始特征的高斯分布进行缩放，获得缩放后的均值和缩放后的方差；利用缩放后的均值和缩放后的方差产生扰动样本，获得扰动特征，即：Since the target domain is inaccessible, the actuator network needs to be trained on the source domain to gain the ability to adaptively adjust features. To this end, in the training phase, for the source domain data samples, the mean and variance of the Gaussian distribution of the original features are obtained; based on the mean and variance of the Gaussian distribution of the original features, the Gaussian distribution of the original features is scaled to obtain the scaled mean and scaled variance; use the scaled mean and the scaled variance to generate perturbation samples to obtain the perturbation characteristics, that is:
， ,
其中，表示扰动特征，/>表示原始特征，/>表示扰动量。in, Represents disturbance characteristics,/> Represents original features,/> Represents the amount of disturbance.
然后，基于扰动特征感知域偏移量，也就是感知结构约束偏差和分布约束偏差。Then, based on the perturbation feature perceptual domain offset, that is, perceptual structure constraint deviation and distribution constraint deviation.
下面详细介绍结构约束网络和分布约束网络的生成和相应约束偏差的获取过程：The following describes in detail the generation of structural constraint networks and distribution constraint networks and the acquisition process of corresponding constraint deviations:
步骤S131：基于原始特征和预设的结构约束条件，获得结构约束偏差；Step S131: Obtain the structural constraint deviation based on the original features and preset structural constraint conditions;
具体地，基于原始特征初始化结构约束网络，基于预设的结构约束条件，构建结构约束损失函数，并利用结构约束损失函数训练结构约束网络，获得训练好的结构约束网络。Specifically, the structural constraint network is initialized based on the original features, and the structural constraint loss function is constructed based on the preset structural constraint conditions. , and use the structural constraint loss function to train the structural constraint network to obtain the trained structural constraint network.
在一个实施例中，利用扰动特征更新原始特征，利用结构约束损失函数训练结构约束网络，获得训练好的结构约束网络。In one embodiment, the perturbation features are used to update the original features, and the structural constraint loss function is used to train the structural constraint network to obtain a trained structural constraint network.
具体地，将更新后的原始特征投影到与原始特征具有相同维度的空间，获得投影后的特征；基于预设的结构约束条件，构建结构约束损失函数；基于投影后的特征和结构约束损失函数，获得结构约束偏差。其中，结构约束损失函数是基于图像的分类类别，对投影后的特征进行归一化处理，获得结构约束条件；并基于结构约束条件和投影后的特征构建而成。Specifically, the updated original features are projected into a space with the same dimension as the original features to obtain the projected features; based on the preset structural constraints, a structural constraint loss function is constructed; based on the projected features and the structural constraint loss function , obtain the structural constraint deviation. Among them, the structural constraint loss function is based on the classification category of the image, normalizing the projected features to obtain the structural constraints; and is constructed based on the structural constraints and projected features.
本实施例通过结构约束网络将从源域图像数据中提取的D*D维，（其中D表示正整数）（例如256*256维的数据）的原始特征/>投影到与源域图像数据具有相同维度的空间得到投影后的特征/>，/>，/>，…，/>分别表示各个维度的投影后的特征，并对/>施加结构约束。This embodiment constrains the network through structure Original features of D*D dimensions extracted from the source domain image data, (where D represents a positive integer) (for example, 256*256-dimensional data)/> Project to a space with the same dimensions as the source domain image data to obtain the projected features/> ,/> ,/> ,…,/> Represent the projected features of each dimension respectively, and compare/> Impose structural constraints.
本实施例对于源域图像数据的每一个分类类别c，随机选取一个长度为d的下标子集，其中每一项分别表示在该分类类别中选取的每一个源域图像的下标，/>，/>；并将剩余的下标组成另一个下标子集，记作，其中每一项分别表示该分类类别中剩余的每一个源域图像的下标。然后利用所生成的两个下标子集构建结构约束特征/>，即：In this embodiment, for each classification category c of the source domain image data, a subscript subset of length d is randomly selected. , each item represents the subscript of each source domain image selected in the classification category,/> ,/> ; and the remaining subscripts form another subscript subset, denoted as , where each item represents the subscript of each remaining source domain image in the classification category. The two generated subscript subsets are then used to construct structural constraint features/> ,Right now:
， ,
也就是通过将源域图像随机分为两大类，并进行归一化处理，以消除奇异样本数据的影响。That is, the source domain images are randomly divided into two categories and normalized to eliminate the influence of singular sample data.
然后，基于预设的结构约束条件，构建结构约束损失函数，并利用结构约束损失函数训练结构约束网络，获得训练好的结构约束网络，即：Then, based on the preset structural constraints, a structural constraint loss function is constructed , and use the structural constraint loss function to train the structural constraint network, and obtain the trained structural constraint network, that is:
， ,
基于结构约束损失函数，将原始特征投影到与训练好的结构约束网络具有相同维度的空间，获得投影后的特征。那么，结构约束偏差表示投影特征和结构约束特征/>之间的偏差，即结构约束偏差，记作/>。Based on the structural constraint loss function, the original features are projected into a space with the same dimension as the trained structural constraint network, and the projected features are obtained. Then, the structural constraint deviation represents the projected feature and structural constraint features/> The deviation between them, that is, the structural constraint deviation, is denoted as/> .
基于维度D=256的源域图像数据进行仿真实验，得到图5所示的投影后的特征仿真图及相应的结构约束偏差仿真图，其中，投影后的特征仿真图如图5中的（a）所示，基于投影后的特征得到的结构约束偏差仿真图如图5中的（b）所示，其中，横坐标表示维度D的数值，纵坐标表示投影后的特征的归一化值。A simulation experiment was conducted based on the source domain image data with dimension D=256, and the projected feature simulation diagram and the corresponding structural constraint deviation simulation diagram shown in Figure 5 were obtained. Among them, the projected feature simulation diagram in Figure 5 (a ), the structural constraint deviation simulation diagram based on the projected features is shown in (b) in Figure 5, where the abscissa represents the value of dimension D, and the ordinate represents the normalized value of the projected features.
在一个实施例中，通过结构约束网络将从源域图像数据中提取的原始特征/>施加扰动后得到扰动特征/>，然后将扰动特征/>投影到与源域图像数据具有相同维度的空间得到投影后的特征，并基于同样的原理，获得结构约束偏差。In one embodiment, the network is constrained by structure Original features extracted from source domain image data/> Obtain the perturbation characteristics after applying perturbation/> , and then perturb the features/> Projecting to a space with the same dimension as the source domain image data obtains the projected features, and based on the same principle, obtains the structural constraint deviation.
步骤S132：基于原始特征、类中心和预设的分布约束条件，获得分布约束偏差；Step S132: Obtain the distribution constraint deviation based on the original features, class center and preset distribution constraint conditions;
具体地，基于预设的分布约束条件，获得原始特征与各个类中心之间的相关性；基于相关性，获得原始特征与各个类中心的权重；基于原始特征与各个类中心之间的距离及权重，获得分布约束偏差。Specifically, based on the preset distribution constraints, the correlation between the original features and each class center is obtained; based on the correlation, the weight of the original features and each class center is obtained; based on the distance between the original features and each class center and weights to obtain distribution constraint deviations.
由于基础特征提取网络在源域上训练，源域样本特征可以在特征中间很好地根据类别聚类。但在目标域上，由于域偏移的存在，这些聚类的特征分布遭受一定程度的破坏而变得离散，部分特征变得远离正确的类中心。在目标域上，根据上述定义的源域特征的类中心，以及原始特征/>到各个类中心的距离，将分布约束偏差定义为原始特征/>与各个类中心的距离的加权和，即：Since the basic feature extraction network is trained on the source domain, the source domain sample features can be well clustered according to categories among the features. However, in the target domain, due to the existence of domain offset, the feature distribution of these clusters is damaged to a certain extent and becomes discrete, and some features become far away from the correct class center. On the target domain, the class center of the source domain features defined above , and original features/> The distance to each class center defines the distribution constraint deviation as the original feature/> The weighted sum of distances from each class center, that is:
， ,
其中，，分布约束网络/>用于计算原始特征和各个类中心的相关性，并将相关性转换为原始特征与对应的类中心之间的偏差的权重。分布约束网络是通过扰动特征和源域特征的类中心训练学习生成的。in, , distribution constraint network/> It is used to calculate the correlation between the original feature and each class center, and convert the correlation into a weight of the deviation between the original feature and the corresponding class center. distributed constraint network It is generated by class center training learning of perturbation features and source domain features.
在一个实施例中，将分布约束偏差定义为原始特征与各个类中心偏差的加权和，以获得分布约束偏差。In one embodiment, the distribution constrained deviation is defined as the original feature Weighted sum of deviations from respective class centers to obtain distribution constrained deviations.
步骤S140：基于约束偏差，将原始特征输入至执行器网络，获得调整后的特征。Step S140: Based on the constrained deviation, input the original features to the actuator network to obtain adjusted features.
具体地，利用数据转换网络将结构约束偏差和分布约束偏差转换为指导信号。具体地，基于本实施例的学习模型生成一个数据转换网络，上述结构约束偏差和分布约束偏差映射成为指导信号/>，即：Specifically, the data conversion network is used to convert the structural constraint deviation and the distribution constraint deviation into guidance signals. Specifically, a data conversion network is generated based on the learning model of this embodiment , the above structural constraint deviation and distribution constraint deviation mapping become a guidance signal/> ,Right now:
， ,
将指导信号输入给执行器网络，引导执行器网络进行原始特征/>的调整。本实施例通过指导信号/>综合表征源域数据的特点，有利于后续学习对偏移进行精确调整优化。will guide the signal Input to the actuator network to guide the actuator network to perform original features/> adjustment. This embodiment uses the guidance signal/> Comprehensive representation of the characteristics of the source domain data is conducive to subsequent learning to accurately adjust and optimize the offset.
为了能够实现更好的域泛化，本实施例旨在学习到一种自适应调整特征的能力，从而最大化特征被准确分类的能力，以达到更好的分类性能。在传感器网络的指导信号的引导下，本实施例基于原始特征和指导信号/>生成一个执行器网络/>，通过执行器网络/>获取特征调整量/>，以对原始特征进行调整得到调整后的特征/>，即：In order to achieve better domain generalization, this embodiment aims to learn an ability to adaptively adjust features, thereby maximizing the ability of features to be accurately classified to achieve better classification performance. Guided by the guidance signal of the sensor network, this embodiment is based on the original features and guidance signals/> Generate an executor network/> , through the actuator network/> Get the feature adjustment amount/> , to adjust the original features to obtain the adjusted features/> ,Right now:
。 .
在一个实施例中，由于目标域是不可访问的，执行器网络需要在源域上进行训练，从而获得自适应调整特征的能力。为此，在训练阶段对于源域样本，获取原始特征的高斯分布的均值和方差；基于原始特征的高斯分布的均值和方差，对原始特征的高斯分布进行缩放，获得缩放后的均值和缩放后的方差；利用缩放后的均值和缩放后的方差产生扰动样本，获得扰动特征，即：In one embodiment, since the target domain is inaccessible, the executor network needs to be trained on the source domain to obtain the ability to adaptively adjust features. To this end, for the source domain samples in the training phase, the mean and variance of the Gaussian distribution of the original features are obtained; based on the mean and variance of the Gaussian distribution of the original features, the Gaussian distribution of the original features is scaled to obtain the scaled mean and scaled The variance; use the scaled mean and the scaled variance to generate perturbation samples to obtain the perturbation characteristics, that is:
， ,
并对扰动特征进行调整，得到调整后的特征/>，即：And to the disturbance characteristics Make adjustments to get the adjusted features/> ,Right now:
； ;
具体地，获取原始特征在训练批次层面的统计参数（例如原始特征的统计分布的均值和方差等参数），也就是获取某个批次的原始特征数据的统计参数，或者是获取多个批次的原始特征数据的统计参数平均值，或者是将多个批次的原始特征数据进行融合获取融合后数据的统计参数，然后通过改变原始特征的统计参数进行特征扰动。本实施例通过调整原始特征的高斯分布的方差参数对分布进行缩放，并利用缩放后的原始特征的统计分布的统计参数中的均值向量和标准差向量/>来产生扰动样本，即：Specifically, obtain the statistical parameters of the original features at the training batch level (such as the mean and variance of the statistical distribution of the original features), that is, obtain the statistical parameters of the original feature data of a certain batch, or obtain multiple batches. The average statistical parameter of the original feature data of times, or fusion of multiple batches of original feature data to obtain the statistical parameters of the fused data, and then perform feature perturbation by changing the statistical parameters of the original features. This embodiment scales the distribution by adjusting the variance parameters of the Gaussian distribution of the original features, and uses the mean vector in the statistical parameters of the statistical distribution of the scaled original features. and standard deviation vector/> To generate perturbation samples, that is:
， ,
其中，和/>是原始特征/>分布的批次层面的均值和标准差。in, and/> is the original feature/> The batch-level mean and standard deviation of the distribution.
调整的目的是，使得调整后的特征能够和未扰动的原始特征在基础分类网络上能有相同的分类结果，即：The purpose of adjustment is to enable the adjusted features to be compared with the unperturbed original features in the basic classification network. can have the same classification results, that is:
。 .
步骤S150：将调整后的特征输入到基础分类网络，获得分类结果。Step S150: Input the adjusted features into the basic classification network to obtain the classification result.
基于本实施例的执行器网络进行仿真实验，得到对扰动特征调整前、后的特征分布对比图，如图6所示，其中，图6中的（a）是对扰动特征调整前的特征分布图，图6中的（b）是对扰动特征调整后的特征分布图，图6中的（a）和（b）的横坐标和纵坐标均表示调整后的特征值，每一个灰色区域均表示一个分类结果。很显然，调整后的特征的可区分性更高，也就是更容易被准确地分类。A simulation experiment was performed based on the actuator network of this embodiment, and a comparison chart of the characteristic distribution before and after adjusting the disturbance characteristics was obtained, as shown in Figure 6, where (a) in Figure 6 is the characteristic distribution before adjusting the disturbance characteristics. Figure, (b) in Figure 6 is the feature distribution map after adjusting the disturbance feature. The abscissa and ordinate of (a) and (b) in Figure 6 both represent the adjusted feature value. Each gray area Represents a classification result. Obviously, the adjusted features are more distinguishable, that is, they are easier to classify accurately.
步骤S160：基于分类结果和图像类别标签计算联合损失，重复步骤S130-步骤S150，直至联合损失达到预设的联合损失阈值，获得训练好的域泛化图像分类模型。Step S160: Calculate the joint loss based on the classification result and the image category label, repeat steps S130 to S150 until the joint loss reaches the preset joint loss threshold, and obtain the trained domain generalization image classification model.
具体地，基于结构约束损失函数和调整后的特征，生成调整后的特征的结构约束损失函数；基于调整后特征的结构约束损失函数和调整后特征的分类性能，构建联合损失函数。Specifically, based on the structural constraint loss function and the adjusted features, a structural constraint loss function of the adjusted features is generated; based on the structural constraint loss function of the adjusted features and the classification performance of the adjusted features, a joint loss function is constructed.
联合训练所用损失函数由交叉熵损失和结构约束损失组成，即：The loss function used in joint training consists of cross-entropy loss and structural constraint loss, namely:
， ,
其中，表示联合损失，/>为调整后的特征的结构约束损失，且相比于原始特征，调整后的特征能够更好地满足结构约束；/>为调整后的特征的结构约束损失的权重，/>，其取值可以根据实际需求灵活设定；/>为交叉熵损失，即：in, Represents joint loss,/> is the structural constraint loss of the adjusted features, and compared with the original features, the adjusted features can better satisfy the structural constraints;/> is the weight of the structural constraint loss of the adjusted features,/> , its value can be flexibly set according to actual needs;/> is the cross entropy loss, that is:
， ,
其中，表示交叉熵函数，/>表示调整后特征的分类结果，y表示类别标签，N表示图像类别标签的数量，交叉熵损失/>用于描述调整后特征/>的分类性能，也就是基于预设的基础分类条件和调整后的特征，获得调整后的特征的分类结果；获取源域数据的分类标签，判断调整后的特征的分类结果和分类标签之间的相似度，以评价调整后特征/>的分类性能。in, Represents the cross entropy function, /> Represents the classification result of the adjusted features, y represents the category label, N represents the number of image category labels, cross entropy loss/> Used to describe adjusted characteristics/> Classification performance, that is, based on the preset basic classification conditions and adjusted features, obtain the classification results of the adjusted features; obtain the classification labels of the source domain data, and determine the difference between the classification results of the adjusted features and the classification labels Similarity to evaluate adjusted features/> classification performance.
需要声明的是，在训练阶段，基础特征提取网络和基础分类网络/>被冻结，不参与模型更新。并且在结构约束网络/>训练完成后，在接下来的分布约束网络/>、数据转换网络/>和执行器网络/>的联合训练过程中被冻结，不参与模型更新。It should be stated that during the training phase, the basic feature extraction network and basic classification network/> It is frozen and does not participate in model updates. And in the structural constraint network/> After the training is completed, the next distribution constraint network/> , data conversion network/> and actuator network/> is frozen during the joint training process and does not participate in model updates.
利用训练好的域泛化图像分类模型进行分类的过程，如图7所示，具体如下：The classification process using the trained domain generalization image classification model is shown in Figure 7, as follows:
首先，利用基础特征提取网络提取原始特征/>并计算源域的类中心，对原始特征/>施加扰动得到扰动特征/>；其次，通过结构约束网络/>利用结构约束/>获取扰动特征/>的结构约束偏差/>，分布约束网络/>利用源域数据的类中心/>获取扰动特征/>的分布约束偏差/>；然后，利用数据转换网络/>将感知到的结构约束偏差/>和分布约束偏差/>转换为指导信号/>，在指导信号/>的引导下，利用执行器网络/>对扰动特征/>进行调整得到调整后的特征/>；最后，将调整后的特征/>传入基础分类网络/>，获取分类结果。First, use the basic feature extraction network Extract original features/> And calculate the class center of the source domain , for original features/> Apply perturbation to obtain perturbation characteristics/> ;Secondly, constrain the network through structure/> Leverage structural constraints/> Get disturbance features/> Structural constraint deviation/> , distribution constraint network/> Class center that leverages source domain data/> Get disturbance features/> Distribution constraint deviation/> ;Then, use the data conversion network/> Convert perceived structural constraint deviations/> and distribution constraint deviation/> Convert to guidance signal/> , in guidance signal/> Under the guidance of the executor network/> For disturbance characteristics/> Make adjustments to get adjusted features/> ;Finally, the adjusted features/> Incoming basic classification network/> , obtain the classification results.
综上所述，本实施例的方法通过引入域偏移传感器来感知由域偏移引起的约束偏差，根据源域图像特征的统计分布特点对源域图像特征的统计参数进行缩放得到扰动后的源域图像特征，并根据约束偏差对缩放扰动后的源域特征进行调整优化，获得调整后的特征，然后基于对调整后的特征的分类性能的评估，训练出满足分类结果精度的各个子网络，从而提高图片分类模型的精度与泛化能力，并且整个泛化过程涉及到的参数较少，消耗的计算资源和时间较少，分类效率较高，而且无需依赖复杂的数据生成和增强，使得模型在不同的环境下都能够保持良好的性能，具有良好的鲁棒性和稳健性。To sum up, the method of this embodiment introduces a domain shift sensor to sense the constraint deviation caused by the domain shift, and scales the statistical parameters of the source domain image features according to the statistical distribution characteristics of the source domain image features to obtain the perturbed Source domain image features, and adjust and optimize the source domain features after scaling perturbation according to the constraint deviation to obtain the adjusted features. Then, based on the evaluation of the classification performance of the adjusted features, train each sub-network that meets the accuracy of the classification results. , thereby improving the accuracy and generalization ability of the image classification model, and the entire generalization process involves fewer parameters, consumes less computing resources and time, has higher classification efficiency, and does not need to rely on complex data generation and enhancement, making The model can maintain good performance in different environments and has good robustness and robustness.
示例性系统Example system
如图8所示，对应于上述基于特征调整的域泛化图像分类方法，本发明实施例还提供一种基于特征调整的域泛化分类系统，上述基于特征调整的域泛化分类系统包括：As shown in Figure 8, corresponding to the above-mentioned domain generalization image classification method based on feature adjustment, embodiments of the present invention also provide a domain generalization classification system based on feature adjustment. The above-mentioned domain generalization classification system based on feature adjustment includes:
数据获取模块810，用于获取目标域的图像及图像类别标签，并获取训练好的域泛化图像分类模型，所述训练好的域泛化图像分类模型包括训练好的基础特征提取网络、训练好的域偏移感知网络、训练好的执行器网络和训练好的基础分类网络；The data acquisition module 810 is used to obtain images and image category labels of the target domain, and obtain a trained domain generalized image classification model. The trained domain generalized image classification model includes a trained basic feature extraction network, a trained domain generalized image classification model, and a trained domain generalized image classification model. A good domain shift aware network, a well-trained actuator network and a well-trained basic classification network;
域泛化分类模块820，用于利用所述目标域的图像构建目标集，并将所述目标集输入至所述域泛化图像分类模型，对所述目标域的图像进行分类，获得最终的分类结果。The domain generalization classification module 820 is used to construct a target set using images in the target domain, input the target set into the domain generalization image classification model, classify the images in the target domain, and obtain the final Classification results.
具体的，本实施例中，上述基于特征调整的域泛化分类系统的具体功能还可以参照上述基于特征调整的域泛化图像分类方法中的对应描述，在此不再赘述。Specifically, in this embodiment, the specific functions of the domain generalization classification system based on feature adjustment can also refer to the corresponding description in the domain generalization image classification method based on feature adjustment, which will not be described again here.
基于上述实施例，本发明还提供了一种智能终端，其原理框图可以如图9所示。上述智能终端包括通过系统总线连接的处理器、存储器、网络接口以及显示屏。其中，该智能终端的处理器用于提供计算和控制能力。该智能终端的存储器包括非易失性存储介质、内部存储器。该非易失性存储介质存储有操作系统和基于特征调整的域泛化分类程序。该内部存储器为非易失性存储介质中的操作系统和基于基于特征调整的域泛化分类程序的运行提供环境。该智能终端的网络接口用于与外部的终端通过网络连接通信。该基于特征调整的域泛化分类程序被处理器执行时实现上述任意一种基于特征调整的域泛化图像分类方法的步骤。该智能终端的显示屏可以是液晶显示屏或者电子墨水显示屏。Based on the above embodiments, the present invention also provides an intelligent terminal, the functional block diagram of which can be shown in Figure 9 . The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the smart terminal is used to provide computing and control capabilities. The memory of the smart terminal includes non-volatile storage media and internal memory. The non-volatile storage medium stores an operating system and a domain generalization classification program based on feature adjustment. This internal memory provides an environment for the operation of an operating system and a domain generalization classification program based on feature tuning in a non-volatile storage medium. The network interface of the smart terminal is used to communicate with external terminals through network connections. When the feature adjustment-based domain generalization classification program is executed by the processor, the steps of any of the above feature adjustment-based domain generalization image classification methods are implemented. The display screen of the smart terminal may be a liquid crystal display screen or an electronic ink display screen.
本领域技术人员可以理解，图9中示出的原理框图，仅仅是与本发明方案相关的部分结构的框图，并不构成对本发明方案所应用于其上的智能终端的限定，具体的智能终端可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。Those skilled in the art can understand that the principle block diagram shown in Figure 9 is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the smart terminal to which the solution of the present invention is applied. The specific smart terminal More or fewer components may be included than shown in the figures, or certain components may be combined, or may have a different arrangement of components.
在一个实施例中，提供了一种智能终端，上述智能终端包括存储器、处理器以及存储在上述存储器上并可在上述处理器上运行的基于特征调整的域泛化分类程序，上述基于特征调整的域泛化分类程序被上述处理器执行时实现本发明实施例提供的任意一种基于特征调整的域泛化图像分类方法的步骤。In one embodiment, a smart terminal is provided. The smart terminal includes a memory, a processor, and a domain generalization classification program based on feature adjustment that is stored in the memory and can be run on the processor. The feature adjustment-based domain generalization classification program is When the domain generalization classification program is executed by the above-mentioned processor, the steps of any domain generalization image classification method based on feature adjustment provided by the embodiments of the present invention are implemented.
本发明实施例还提供一种计算机可读存储介质，上述计算机可读存储介质上存储有基于特征调整的域泛化分类程序，上述基于特征调整的域泛化分类程序被处理器执行时实现本发明实施例提供的任意一种基于特征调整的域泛化图像分类方法的步骤。Embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a domain generalization classification program based on feature adjustment. When the domain generalization classification program based on feature adjustment is executed by a processor, the present invention is implemented. The steps of any domain generalization image classification method based on feature adjustment provided by the embodiments of the invention.
应理解，上述实施例中各步骤的序号大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。It should be understood that the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将上述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本发明的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional units and modules is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs. Module completion means dividing the internal structure of the above device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be hardware-based. It can also be implemented in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of distinguishing each other and are not used to limit the scope of the present invention. For the specific working processes of the units and modules in the above system, please refer to the corresponding processes in the foregoing method embodiments, and will not be described again here.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。In the above embodiments, each embodiment is described with its own emphasis. For parts that are not detailed or documented in a certain embodiment, please refer to the relevant descriptions of other embodiments.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各实例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟是以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered to be beyond the scope of the present invention.
在本发明所提供的实施例中，应该理解到，所揭露的装置/终端设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/终端设备实施例仅仅是示意性的，例如，上述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以由另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。In the embodiments provided by the present invention, it should be understood that the disclosed apparatus/terminal equipment and methods can be implemented in other ways. For example, the apparatus/terminal equipment embodiments described above are only illustrative. For example, the division of the above modules or units is only a logical function division. In actual implementation, it can be divided in other ways, such as multiple units or units. Components may be combined or may be integrated into another system, or some features may be ignored, or not implemented.
以上所述实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解；其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不是相应技术方案的本质脱离本发明各实施例技术方案的精神和范围，均应包含在本发明的保护范围之内。The above-described embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that they can still implement the above-mentioned implementations. Modifications are made to the technical solutions described in the examples, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not mean that the essence of the corresponding technical solutions deviates from the spirit and scope of the technical solutions of each embodiment of the present invention, and they should all be included in this document. within the scope of protection of the invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311371704.5ACN117115567B (en) | 2023-10-23 | 2023-10-23 | Domain generalization image classification method, system, terminal and medium based on feature adjustment |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311371704.5ACN117115567B (en) | 2023-10-23 | 2023-10-23 | Domain generalization image classification method, system, terminal and medium based on feature adjustment |
| Publication Number | Publication Date |
|---|---|
| CN117115567Atrue CN117115567A (en) | 2023-11-24 |
| CN117115567B CN117115567B (en) | 2024-03-26 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311371704.5AActiveCN117115567B (en) | 2023-10-23 | 2023-10-23 | Domain generalization image classification method, system, terminal and medium based on feature adjustment |
| Country | Link |
|---|---|
| CN (1) | CN117115567B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118096769A (en)* | 2024-04-29 | 2024-05-28 | 中国科学院宁波材料技术与工程研究所 | A method and device for analyzing retinal OCT images |
| CN118212461A (en)* | 2024-03-27 | 2024-06-18 | 南方科技大学 | Model domain adaptive method, device, terminal and medium based on dual-layer vision |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023030281A1 (en)* | 2021-09-03 | 2023-03-09 | 北京字节跳动网络技术有限公司 | Training method and apparatus, image processing method, electronic device and storage medium |
| US20230104127A1 (en)* | 2021-10-04 | 2023-04-06 | Samsung Electronics Co., Ltd. | Systems, methods, and apparatus for image classification with domain invariant regularization |
| CN116342938A (en)* | 2023-03-10 | 2023-06-27 | 西安理工大学 | Domain generalization image classification method based on mixture of multiple potential domains |
| CN116452862A (en)* | 2023-03-30 | 2023-07-18 | 华南理工大学 | Image classification method based on domain generalization learning |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023030281A1 (en)* | 2021-09-03 | 2023-03-09 | 北京字节跳动网络技术有限公司 | Training method and apparatus, image processing method, electronic device and storage medium |
| US20230104127A1 (en)* | 2021-10-04 | 2023-04-06 | Samsung Electronics Co., Ltd. | Systems, methods, and apparatus for image classification with domain invariant regularization |
| CN116342938A (en)* | 2023-03-10 | 2023-06-27 | 西安理工大学 | Domain generalization image classification method based on mixture of multiple potential domains |
| CN116452862A (en)* | 2023-03-30 | 2023-07-18 | 华南理工大学 | Image classification method based on domain generalization learning |
| Title |
|---|
| YI ZHANG ET AL.: "Cross-Modal Concept Learning and Inference for Vision-Language Models", 《ARXIV:2307.15460V1》, pages 1 - 13* |
| ZHEHAN KAN ET AL.: "Self-Correctable and Adaptable Inference for Generalizable Human Pose Estimation", 《ARXIV:2303.11180V2》, pages 1 - 10* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118212461A (en)* | 2024-03-27 | 2024-06-18 | 南方科技大学 | Model domain adaptive method, device, terminal and medium based on dual-layer vision |
| CN118212461B (en)* | 2024-03-27 | 2025-06-27 | 南方科技大学 | Model field self-adaption method, device, terminal and medium based on double-layer vision |
| CN118096769A (en)* | 2024-04-29 | 2024-05-28 | 中国科学院宁波材料技术与工程研究所 | A method and device for analyzing retinal OCT images |
| Publication number | Publication date |
|---|---|
| CN117115567B (en) | 2024-03-26 |
| Publication | Publication Date | Title |
|---|---|---|
| CN117115567A (en) | Domain generalization image classification method, system, terminal and medium based on feature adjustment | |
| CN110472675B (en) | Image classification method, image classification device, storage medium and electronic equipment | |
| CN113435546A (en) | Migratable image recognition method and system based on differentiation confidence level | |
| CN106991047B (en) | Method and system for predicting object-oriented software defects | |
| CN105913450A (en) | Tire rubber carbon black dispersity evaluation method and system based on neural network image processing | |
| CN109344713B (en) | A pose-robust face recognition method | |
| CN112116019B (en) | Multi-sensor Vine Copula heterogeneous information decision fusion method | |
| CN110781970A (en) | Method, device and equipment for generating classifier and storage medium | |
| CN116402777B (en) | Power equipment detection method and system based on machine vision | |
| CN113541985A (en) | Internet of things fault diagnosis method, model training method and related device | |
| CN114912489A (en) | Signal modulation identification method | |
| CN115659254A (en) | A Method for Analysis of Power Quality Disturbance in Distribution Network Based on Dual-modal Feature Fusion | |
| CN117056902A (en) | Password management method and system for Internet of things | |
| US20230409422A1 (en) | Systems and Methods for Anomaly Detection in Multi-Modal Data Streams | |
| CN114882534A (en) | Pedestrian re-identification method, system and medium based on counterfactual attention learning | |
| CN119830149A (en) | Radiation source open set identification method and device based on dynamic group constant-change network and generation countermeasure network | |
| CN110728292A (en) | Self-adaptive feature selection algorithm under multi-task joint optimization | |
| CN119128720A (en) | Gas classification and recognition method based on graph neural network based on node labeling framework | |
| CN119128721A (en) | Gas classification and recognition method based on graph neural network based on edge labeling framework | |
| CN119167146A (en) | A method and device for automatic modulation recognition of small sample signals | |
| CN114067350A (en) | Model updating method, device and computer program product | |
| CN117743916A (en) | Model training method, abnormal information detection method and device | |
| CN116824219A (en) | Bridge construction environment protection process and system thereof | |
| CN112507137B (en) | Small sample relation extraction method based on granularity perception in open environment and application | |
| CN113341391B (en) | Radar target multi-frame joint detection method in unknown environment based on deep learning |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |