CN106127247B - Image classification method based on the more example support vector machines of multitask - Google Patents

Abstract

Translated fromChinese

本发明公开了一种基于多任务多示例支持向量机的图像分类方法。该方法包括：为T组图像建立T个学习任务；对T个学习任务的图像进行多示例化；为T个任务中的每个类别的图像构建一个类包；建立类包中的示例到多示例包的欧式距离公式；构建类包到多示例包的示例距离向量；建立类包到多示例包的加权欧式距离公式；约束多示例包到所属类别的距离小于到其他类别的距离；建立多任务多示例支持向量机的优化问题；转换优化问题为传统单任务单示例支持向量机问题；求解支持向量机优化问题。本发明涉及了一种最优化加权欧式距离公式的方法，通过把图像示例化，建立多任务多示例支持向量机学习问题，从而最优化出理想权值，从而提高图像分类器的性能。

The invention discloses an image classification method based on a multi-task and multi-example support vector machine. The method includes: establishing T learning tasks for T groups of images; performing multi-instantiation on the images of the T learning tasks; building a class bag for each category of images in the T tasks; establishing the examples in the class bag to Euclidean distance formula of multi-instance package; construct example distance vector from class package to multi-instance package; establish weighted Euclidean distance formula from class package to multi-instance package; constrain the distance from multi-instance package to the category it belongs to is less than the distance to other categories; establish The optimization problem of multi-task multi-instance support vector machine; convert the optimization problem to the traditional single-task single-instance support vector machine problem; solve the support vector machine optimization problem. The invention relates to a method for optimizing a weighted Euclidean distance formula. By instantiating an image, a multi-task and multi-example support vector machine learning problem is established, so as to optimize the ideal weight and improve the performance of the image classifier.

Description

Translated fromChinese

基于多任务多示例支持向量机的图像分类方法Image classification method based on multi-task and multi-instance support vector machine

技术领域technical field

本发明涉及图像分类技术领域，特别是涉及基于多任务多示例支持向量机的图像分类方法。The invention relates to the technical field of image classification, in particular to an image classification method based on a multi-task and multi-instance support vector machine.

背景技术Background technique

随着信息技术的进步与社交网络的长久发展，互联网上面已经存在着海量的图像，并且每天新上传到互联网上的图像数量也呈指数上升，图像所包含的场景也越来越丰富，虽然社交网站得到了长久的发展，但是网站上海量的图片却没有得到充分的利用，并且每天都会有大量新的图像上传到网站上面，如何识别出未被标记的图像，并且准确分类到对应的类别中以更好地服务网站用户，是大部分互联网公司都在研究的一个问题。With the advancement of information technology and the long-term development of social networks, there are already a large number of images on the Internet, and the number of new images uploaded to the Internet every day has also increased exponentially, and the scenes contained in the images have become more and more abundant. The website has been developed for a long time, but the large amount of pictures on the website has not been fully utilized, and a large number of new images are uploaded to the website every day. How to identify unmarked images and accurately classify them into corresponding categories In order to better serve website users, it is a problem that most Internet companies are studying.

一方面，由于在拍摄图像的时候可能会包含各种各样的背景元素，则会导致图像不仅仅包含一个场景，如果采用传统的单示例图像识别方法，如单示例支持向量机，可能会导致误分类。例如，在动物园拍摄景物的时候，可能会同时把不同物种拍到同一张图像，如人、马、小鸟等动物都可能会在同一张图像中。On the one hand, since the image may contain various background elements, the image will not contain only one scene. If traditional single-instance image recognition methods, such as single-instance support vector machine are used, it may misclassified. For example, when shooting a scene in a zoo, different species may be captured in the same image at the same time, such as people, horses, birds and other animals may be in the same image.

另一方面，由于互联网的开放性以及拍摄设备的多样性，同一个人的照片可能会出现在不同的社交网站上面，或者由不同设备所拍摄，或者由不同的视频所剪辑而来，把这些图片混合一起进行识别，显然是不合理的；再者，为了提高图像分类器的性能，需要大量的有标记的图像来进行分类器的训练，如果训练样本不足，则会导致分类器的性能下降，从而影响图像分类的效果。早期的图像分类都是通过人工标记的形式进行分类，但是这种方法的人工成功会非常高，在少量图像下，或许还可行，但是以互联网现在的图像产生速度，则不太可取。On the other hand, due to the openness of the Internet and the diversity of shooting devices, photos of the same person may appear on different social networking sites, or taken by different devices, or edited from different videos. It is obviously unreasonable to mix them together for recognition. Furthermore, in order to improve the performance of the image classifier, a large number of labeled images are needed for the training of the classifier. If the training samples are insufficient, the performance of the classifier will be degraded. Thereby affecting the effect of image classification. Early image classification was done in the form of manual labeling, but the artificial success of this method would be very high. It may be feasible with a small number of images, but it is not desirable at the current image generation speed of the Internet.

发明内容SUMMARY OF THE INVENTION

虽然同一类型的已标注的图像在互联网上面的数量很多，但是由于来源方式不同，例如，拍摄的设备或者储存的社交网站不同，把这些图片混合一起进行分类器的训练是不合理的，但是根据来源形式来进行分组训练，则可能会遇到训练样本不足从而导致分类器的精度下降等问题，所以可以采用多任务的形式，把若干组图片同时进行训练，并且利用每组图片的相关性来提高每组图片分类器的性能。而且由于图像含有多个场景，把图像看成单示例来进行处理，则会忽略掉多个场景的相关性，此时可以采用多示例学习方法，把一个图像看成多个示例。Although there are many labeled images of the same type on the Internet, it is unreasonable to mix these images together for classifier training due to different sources, such as different shooting devices or different social networking sites. If you use the source form for group training, you may encounter problems such as insufficient training samples, which will lead to a decrease in the accuracy of the classifier. Therefore, you can use the form of multi-tasking to train several groups of pictures at the same time, and use the correlation of each group of pictures to Improve the performance of each group of image classifiers. Moreover, since the image contains multiple scenes, if the image is treated as a single example, the correlation of multiple scenes will be ignored. At this time, the multi-instance learning method can be used to treat an image as multiple examples.

本发明的基于多任务多示例支持向量机的图像分类方法包括如下步骤：The image classification method based on the multi-task multi-instance support vector machine of the present invention comprises the following steps:

(1)获取若干组的图像，并且保证每组图像的数量不多，以组为单位，建立若干个学习任务，并且以人工标记的形式，进行图像的人工分类。(1) Obtain several groups of images, and ensure that the number of images in each group is small, establish several learning tasks in groups, and perform manual classification of images in the form of manual labels.

(2)把所有学习任务的所有图像，转换为多示例数据。(2) Convert all images of all learning tasks to multi-instance data.

(3)在每个多示例学习任务中，为每个图像类别构建一个相关联的多示例包，该多示例包在本发明中称为类包，并且建立类包中的示例到多示例包的欧式距离公式。(3) In each multi-instance learning task, build an associated multi-instance bag for each image category, which is called a class bag in the present invention, and build the examples in the class bag to the multi-instance bag The Euclidean distance formula.

(4)构建类包到多示例包的示例距离向量，从而建立类包到多示例包的加权欧式距离公式。(4) Construct the example distance vector from the class bag to the multi-instance bag, so as to establish the weighted Euclidean distance formula from the class bag to the multi-instance bag.

(5)建立约束，保证多示例包到所属类别的距离要远远小于到其他类别的距离。(5) Establish constraints to ensure that the distance between the multi-instance package and the category to which it belongs is much smaller than the distance to other categories.

(6)建立多任务多示例支持向量机的优化问题。(6) Establish the optimization problem of multi-task multi-instance support vector machine.

(7)转换步骤(6)的多任务多示例支持向量机优化问题为一个类似单任务单示例支持向量机的优化问题。(7) Convert the multi-task multi-instance SVM optimization problem of step (6) into an optimization problem similar to a single-task single-instance SVM.

(8)求解步骤(7)的支持向量机优化问题，可以获得最优化的权值，从而训练出一个基于多任务多示例支持向量机的图像分类器，进行图像的分类。(8) Solving the support vector machine optimization problem in step (7), the optimal weights can be obtained, so as to train an image classifier based on a multi-task and multi-instance support vector machine to classify images.

附图说明Description of drawings

图1为本发明的基于最大间距多任务多示例学习的网页分类方法的流程图。FIG. 1 is a flow chart of a webpage classification method based on maximum-spacing multi-task multi-instance learning according to the present invention.

具体实施方式Detailed ways

本发明的基于多任务多示例支持向量机的图像分类方法包括如下步骤：The image classification method based on the multi-task multi-instance support vector machine of the present invention comprises the following steps:

第一步，获取若干组的图像，并且保证每组图像的数量不多，以组为单位，建立若干个学习任务，并且以人工标记的形式，进行图像的人工分类。例如，如果存在T组图像，则建立T个图像分类器学习任务，而且由于T个任务的图像数量都不多，可以进行人工标记。The first step is to obtain several groups of images, and ensure that the number of images in each group is not large, establish several learning tasks in groups, and perform manual classification of images in the form of manual labels. For example, if there are T groups of images, T image classifier learning tasks are established, and since the number of images in the T tasks is not large, manual labeling can be performed.

第二步，把所有学习任务的所有图像，转换为多示例数据。由于图像含有多个场景，而在分类的时候，只需要其中的一个关键场景，所以此时把整个图像转换为一个单示例来进行分类，可能会忽略掉多个场景的相关性，导致分类效果变差，所以此时可以采用多示例学习方法来进行图像分类。采用多示例学习方法之前，需要对图像进行多示例数据化，可以采用经典的图像切割方法，如本发明采用的Blobworld System，来进行图像的区域化，此时对每个图像区域进行特征提取，从而使该图像区域转换为一个示例。一个图像含有多个区域，则可以转换为多个示例，此时一张图像可以称为一个多示例包。The second step is to convert all images of all learning tasks into multi-instance data. Since the image contains multiple scenes, and only one of the key scenes is required for classification, converting the entire image into a single example for classification may ignore the correlation of multiple scenes, resulting in a classification effect. Therefore, the multi-instance learning method can be used for image classification at this time. Before adopting the multi-instance learning method, it is necessary to digitize the image with multiple instances. The classical image cutting method, such as the Blobworld System adopted in the present invention, can be used to regionalize the image. At this time, feature extraction is performed for each image area, Thus turning that image area into an example. If an image contains multiple regions, it can be converted into multiple examples. In this case, an image can be called a multi-instance package.

第三步，在每个多示例学习任务中，为每个图像类别构建一个相关联的多示例包，该多示例包在本发明中称为类包，并且建立类包中的示例到多示例包的欧式距离公式。不像传统的多示例方法，本发明不直接关注图像与图像之间的距离，而是把每个类别的所有图像组合在一起，建立一个类级别的多示例包，简称为类包，并且建立类包中的示例到多示例包的欧式距离公式，如下：The third step, in each multi-instance learning task, builds an associated multi-instance bag for each image category, which is called a class bag in the present invention, and builds the examples in the class bag to the multi-instance The Euclidean distance formula for the package. Unlike the traditional multi-instance method, the present invention does not directly focus on the distance between images, but combines all images of each category together to establish a class-level multi-instance package, referred to as a class package for short, and establishes a class-level multi-instance package. The Euclidean distance formula from the example in the class package to the multi-example package is as follows:

在上式中，示例是类包C_kt的第j个示例，是多示例包B_it的中心。n_kt是类包C_kt的示例个数。In the above formula, the example is the jth example of class package C_kt , Is the center of the multi-example package_Bit . n_kt is the number of instances of the class package C_kt .

第四步，构建类包到多示例包的示例距离向量，从而建立类包到多示例包的加权欧式距离公式。在第三步中，可以求得每个类包示例到多示例包的距离大小，以该距离大小为向量元素，建立类包到多示例包的示例距离向量，则第t个任务的第k个类别到第i个多示例包的示例距离向量如下：The fourth step is to construct the example distance vector from the class package to the multi-instance package, so as to establish the weighted Euclidean distance formula from the class package to the multi-instance package. In the third step, the distance from each class bag example to the multi-example bag can be obtained, and the distance size is used as the vector element to establish the example distance vector from the class bag to the multi-example bag, then the kth of the t-th task example distance vector for the ith class to the ith multi-example bag as follows:

建立一个与示例距离向量等长度的权值向量w_kt，该权值向量定义如下：build a distance vector from the example The weight vector w_kt of equal length is defined as follows:

将示例距离向量与权值向量w_kt想乘，则可以得到类包到多示例包的加权欧式距离公式：the example distance vector If you want to multiply it with the weight vector w_kt , you can get the weighted Euclidean distance formula from the class bag to the multi-instance bag:

第五步，建立约束，保证多示例包到所属类别的距离要远远小于到其他类别的距离。建立以下约束：The fifth step is to establish constraints to ensure that the distance between the multi-example package and the category to which it belongs is much smaller than the distance to other categories. Establish the following constraints:

上式中，P_t(B_it)为多示例包B_it所属的类别集合，N_t(B_it)为与多示例包B_it无关的类别集合，为误差项，该约束保证了类别n到多示例包B_it的距离要大于类别p到多示例包B_it的距离。In the above formula, P_t (B_it ) is the category set to which the multi-instance package B_it belongs, N_t (B_it ) is the category set irrelevant to the multi-instance package B_it , As the error term, this constraint ensures that the distance from category n to the multi-instance package B_it is greater than the distance from category p to the multi-instance package B_it .

第六步，建立多任务多示例支持向量机的优化问题。在第t个任务中，把所有类别的权值向量组成一个向量w_t，如下：The sixth step is to establish the optimization problem of multi-task multi-instance support vector machine. In the t-th task, the weight vectors of all categories are formed into a vector w_t , as follows:

相应的，构建一个等长的向量向量由和-组成，该向量的其他位置填充0，所以可以把第五步中所建立的约束转换为如下的形式：Correspondingly, construct a vector of equal length vector Depend on and- The other positions of the vector are filled with 0, so the constraints established in the fifth step can be converted into the following form:

基于该约束，把w_t转换为多任务学习的形式，即w_t＝w₀+v_t，w₀被认为是所有任务共享的公共权值系数，而v_t是每个任务所则独享的权值系数，为此建立一个多任务多示例支持向量机的优化问题，如下：Based on this constraint, wt is converted into the form of multi-task learning, that_{is, wt = w 0 +v t,w0is} considered to be the common weight coefficient shared by all tasks, and v_t is exclusive to each task The weight coefficient of , establishes a multi-task multi-instance support vector machine optimization problem, as follows:

上式中，C_w用来控制误差项的大小，正则化参数γ₀和γ₁用来控制多示例学习任务间的相似性。如果γ₀趋向于无穷大，则每个多示例学习任务所训练出来的分类器是不相关的。相反的，如果γ₁趋向于无穷大，则所有多示例学习任务训练出来的分类器是相同或者类似的。In the above formula, C_w is used to control the error term The size of the regularization parameters γ₀ and γ₁ are used to control the similarity between multi-instance learning tasks. If γ₀ tends to infinity, the classifiers trained for each multi-instance learning task are irrelevant. Conversely, if_γ1 tends to infinity, the classifiers trained by all multi-instance learning tasks are the same or similar.

第七步，转第六步的多任务多示例支持向量机优化问题为一个类似单任务单示例支持向量机的优化问题。为了使用二次规划等数值求解技术来解决该多任务多示例支持向量机问题，需要把该问题转换为一个类似传统支持向量机优化问题的形式，因此建立两个向量如下：In the seventh step, the multi-task multi-instance support vector machine optimization problem in the sixth step is turned into an optimization problem similar to a single-task and single-instance support vector machine. In order to solve the multi-task multi-instance SVM problem using numerical solution techniques such as quadratic programming, the problem needs to be transformed into a form similar to the traditional SVM optimization problem, so two vectors are established as follows:

根据以上两个向量，可以把第六步的多任务多示例支持向量机转换为标准的支持向量机优化问题形式，如下：According to the above two vectors, the multi-task multi-instance support vector machine in the sixth step can be converted into a standard support vector machine optimization problem form, as follows:

第八步，求解第七步的支持向量机优化问题，可以获得最优化的权值，从而训练出一个基于多任务多示例支持向量机的图像分类器，进行图像的分类。The eighth step is to solve the support vector machine optimization problem in the seventh step, and the optimized weights can be obtained, so as to train an image classifier based on the multi-task and multi-instance support vector machine to classify the images.

在不脱离本发明精神或必要特性的情况下，可以其它特定形式来体现本发明。应将所述具体实施例各方面仅视为解说性而非限制性。因此，本发明的范畴如随附申请专利范围所示而非如前述说明所示。所有落在申请专利范围的等效意义及范围内的变更应视为落在申请专利范围的范畴内。The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. The aspects of the specific embodiments described are to be considered illustrative only and not restrictive. Accordingly, the scope of the present invention is indicated by the appended claims rather than by the foregoing description. All changes within the equivalent meaning and scope of the patent application shall be deemed to fall within the scope of the patent application.

Claims (4)

1. a kind of image classification method based on the more example support vector machines of multitask, which comprises the steps of:

The first step, the image for obtaining several groups establish several learning tasks, and as unit of group with the shape of handmarkingFormula carries out the manual sort of image；

Second step, all images all learning tasks, are converted to more sample datas；

Third step, in each multi-instance learning task, for each image category construct an associated more example packet set,More example packet collection are collectively referred to as class packet, and establish the example in class packet to more example packets Euclidean distance formula；

The example distance vector of 4th step, building class packet to more example packets, so that the weighting for establishing class packet to more example packets is EuropeanRange formula；

5th step establishes constraint, guarantees that more example packets are less than the distance of other classifications to the distance of generic；

6th step, the optimization problem for establishing the more example support vector machines of multitask；

7th step, convert the more example support vector machines optimization problems of multitask of the 6th step into a single task list example support toThe optimization problem of amount machine；

8th step, the support vector machines optimization problem for solving the 7th step, can obtain the weight of optimization, to train oneBased on the Image Classifier of the more example support vector machines of multitask, the classification of image is carried out；

In 6th step, the optimization problem of the more example support vector machines of multitask is established；In t-th of task, all classesOther weight vector forms a vector w_t, it is as follows:

Correspondingly, one isometric vector of buildingVectorByWithComposition, the other positions filling of the vector0, it is possible to the constraints conversion established in the 5th step be following form:

Based on the constraint, w_tBe converted to the form of multi-task learning, i.e. w_t=w₀+v_t, w₀It is considered as all task sharingsPublic weight coefficient, and v_tIt is the weight coefficient that each task is exclusively enjoyed, establishes the more example supporting vectors of a multitask thusThe optimization problem of machine is as follows:

In above formula, T is task number, C_wFor controlling error termSize, regularization parameter γ₀And γ₁It is more for controllingSimilitude between learn-by-example task, if γ₀It is intended to infinity, then each multi-instance learning task trains point comeClass device is incoherent；Opposite, if γ₁It is intended to infinity, then all multi-instance learning tasks train the classification comeDevice is same or similar；

In 7th step, the more example support vector machines optimization problems of multitask for turning the 6th step are a single task list example branchThe optimization problem of vector machine is held, solves the more example supporting vectors of the multitask to use the numerical technologies such as quadratic programmingMachine problem, needs the problem to be converted to the form of a similar traditional support vector machine optimization problem, thus establish two toIt measures as follows:

In above formula,According to two above vector, the more example support vector machines of multitask of the 6th step can be convertedIt is as follows for the support vector machines optimization problem form of standard:

2. the image classification method according to claim 1 based on the more example support vector machines of multitask, which is characterized in thatIn third step, in each multi-instance learning task, an associated more example packets are constructed for each image category, this showsExample packet be known as class packet, and establish the example in class packet to more example packets Euclidean distance formula specifically: each classificationAll images are combined, and establish more example packets of a class rank, referred to as class packet, and the example established in class packet to showing moreThe Euclidean distance formula of example packet, as follows:

Wherein, exampleIt is class packet C_ktJ-th of example,It is more example packet B_itCenter, n_ktIt is class packet C_ktExampleNumber.

3. the image classification method according to claim 2 based on the more example support vector machines of multitask, which is characterized in thatIn 4th step, building class packet to more example packets example distance vector, thus the weighting for establishing class packet to more example packets it is European away fromFrom formula, in the third step, can in the hope of each class packet example to more example packets apart from size, using this apart from size as vectorElement establishes class packet to the example distance vector of more example packets, then k-th of class of t-th of task is clipped to the packet of example more than i-thExample distance vectorIt is as follows:

Establish one and example distance vectorThe weight vector w of equal length_kt, which is defined as follows:

By example distance vectorWith weight vector w_ktIt is multiplied, then weighted euclidean distance of the available class packet to more example packetsFormula:

4. the image classification method according to claim 3 based on the more example support vector machines of multitask, which is characterized in thatIn 5th step, constraint is established, guarantees that more example packets are less than the distance of other classifications to the distance of generic, is established followingConstraint:

In above formula, P_t(B_it) it is more example packet B_itAffiliated category set, N_t(B_it) be and more example packet B_itUnrelated classification collectionIt closes,For error term, which ensure that classification n to more example packet B_itDistance be greater than classification p to more example packet B_it'sDistance.