CN112784879A - Medical image segmentation or classification method based on small sample domain self-adaption - Google Patents

Abstract

The invention discloses a small sample domain self-adaptive medical image segmentation or classification method, which comprises the steps of downloading medical image data and clinical image data from a public data set, taking the processed medical image data as a source domain training data set with known labels, taking the processed clinical image data as a target domain data set to be classified or segmented, marking a very small amount of clinical image data by a doctor, constructing a small sample domain self-adaptive model by a feature extractor and a classifier which are realized by a convolutional neural network, and obtaining the trained small sample domain self-adaptive model by training; and inputting the target domain data set to be classified or segmented into the small sample domain adaptive model with the best classification effect to obtain the classification or segmentation result of the focus of the target domain data set. The invention adopts different objective functions according to whether the data contains labels or not, realizes cross-domain migration of the small sample domain self-adaptive model, and is applied to classification or segmentation of image data such as clinic, pathology, ultrasound and the like.

Description

Medical image segmentation or classification method based on small sample domain self-adaption

Technical Field

The invention belongs to the technical field of medical image classification, and particularly relates to a medical image segmentation or classification method based on small sample domain self-adaptation.

Background

In recent years, with the continuous development and progress of medical imaging technology and computer technology, medical image analysis has become an indispensable tool and technological means in medical research, clinical disease diagnosis and treatment. The traditional medical image analysis is mainly performed by detailed screening and screening by doctors so as to make corresponding diagnosis, and the method comprises edge detection, texture feature, morphological filtering, template matching and the like. However, the daily medical image data volume in the hospital is huge, and the repetitive work makes the doctor possibly generate misjudgment due to over fatigue.

Due to the ever-increasing computing power and the ever-increasing amount of available data, deep learning, in particular deep convolutional neural networks, has rapidly evolved into a research focus for medical image analysis, which is currently the main application area for medical image analysis. The classification is that a clinician needs to assist in diagnosing whether a human body has a focus by means of a medical image and quantitatively grades the degree of the focus, so that the automatic identification of a focus region and a normal tissue organ in the image is a basic task of medical image analysis. Segmentation, which is the key task of clinical surgical image navigation and image-guided tumor radiotherapy, is to identify the internal voxels as well as the external contours of the target region of interest in the image. The model can be trained using the public data set, and then the data to be recognized is input into the trained model to obtain a classification or segmentation result, which can effectively improve the diagnosis efficiency compared with the unaided visual inspection.

However, there are two main challenges in the classification or segmentation research of clinical medical images, on one hand, the obtained clinical data set has few labeled data, and the labeling cost of the doctor is high, and the doctor wastes time and labor. On the other hand, deep learning models mostly depend on a specific data set, and when a model trained based on the specific data set migrates to other data sets, the diagnosis result may be greatly deviated. For different medical imaging devices, due to different factors such as imaging principles, parameters and the like, different characterization degrees are provided for focuses, so that the diagnosis precision cannot be ensured.

Disclosure of Invention

In order to solve the problems in the prior art, the present invention provides a medical image segmentation or classification method based on small sample domain adaptation, which realizes the classification or segmentation of clinical medical image data.

In order to achieve the purpose, the invention provides the following technical scheme: a small sample domain self-adaptive medical image segmentation or classification method comprises the following specific steps:

s1, acquiring medical image data with classification or segmentation labels, and processing the acquired medical image data to obtain a source domain training data set with known labels;

s2, acquiring clinical image data to be classified or segmented, and processing the acquired clinical image data to obtain a target domain data set, wherein the target domain data set comprises a training set with known labels, a verification set and a test set and a training set with unknown labels;

s3, constructing a small sample domain self-adaptive model, wherein the small sample domain self-adaptive model comprises a feature extractor and a classifier which are realized by a convolutional neural network, inputting the source domain training data set with known labels in the step S1, the training set with known labels in the step S2 and the training set with unknown labels into the small sample domain self-adaptive model to train until the model converges, and obtaining the small sample domain self-adaptive model which is trained;

s4, adjusting the hyper-parameters of the trained small sample domain self-adaptive model, repeating the step S3 to obtain a plurality of small sample domain self-adaptive models, and selecting the small sample domain self-adaptive model with the best classification or segmentation effect on the verification set;

s5, the test set of the target domain data set in step S2 is input to the small sample domain adaptive model obtained in step S4, and the small sample domain adaptive model outputs the classification or segmentation result of the lesion in the clinical image data.

Further, in step S1, data enhancement is performed on the medical image data and the size of the medical image data is adjusted to conform to the network input standard of the small sample domain adaptive model, so as to obtain a source domain training data set.

Further, in step S2, the clinical image data includes pathological image data and ultrasound image data, and the size of the clinical image data is adjusted to conform to the network input standard of the small sample domain adaptive model, so as to obtain the target domain data set.

Further, the data volume ratio of the training set, the verification set and the test set in the target domain data set is 8:1: 1.

Further, in step S3, a small sample domain adaptive model is constructed based on a classical deep convolutional neural network, which includes ResNet, inclusion, and densnet.

Further, in step S3, the feature extractor is constructed by a deep convolutional neural network that removes the last one or several linear layers, and the classifier includes an n-way linear classification layer and a Softmax layer.

Further, a maximum correlation entropy function is adopted as a target function for a source domain training data set and a training set with known labels, and a feature extractor and a classifier are trained; the training set with unknown labels adopts a maximum minimum conditional entropy objective function, and trains a feature extractor and a classifier.

Further, in step S3, the iteration is continuously updated by using a back propagation algorithm and a random gradient descent method until the small sample domain adaptive model converges.

Further, an output layer of the small sample domain adaptive model is a Softmax layer, and the Softmax layer outputs a classification or segmentation result to which the clinical image belongs.

The invention discloses a device for applying a medical image segmentation or classification method based on small sample domain self-adaptation, which comprises the following steps:

the system comprises an acquisition module, a comparison module and a comparison module, wherein the acquisition module is used for acquiring a source domain training data set and a target domain data set with known labels, and the target domain data set comprises a training set with known labels, a verification set and a test set and a training set with unknown labels;

the training module is used for training the small sample self-adaptive model by using a source domain training data set with known labels, a training set with known labels and a training set with unknown labels to obtain a small sample domain self-adaptive model after training;

the verification module is used for verifying the trained small sample domain self-adaptive model to obtain the small sample domain self-adaptive model with the best classification or segmentation effect;

and the classification or segmentation module is used for classifying or segmenting the test set to obtain a classification or segmentation result.

Compared with the prior art, the invention has at least the following beneficial effects:

the invention provides a small sample domain self-adaptive medical image segmentation or classification method, which adopts a deep learning method and has the capability of self-learning effective information from medical image data, so that differences among different categories and the relation between an image and a segmentation mark are mined in the medical image data. The small sample domain self-adaptive method used by the invention has strong expandability, and under the condition of utilizing the public medical image data, only a very small amount of image data with label clinical acquisition needs to be provided, and a deep convolutional neural network is input to construct a model, and the accuracy of network classification or segmentation is improved along with the increase of the data volume, so that the high-precision classification or segmentation of a large amount of label-free clinical image data is realized.

The invention provides a small sample domain self-adaptive medical image segmentation or classification method, which takes public medical image data as a source domain with a known label, takes medical image data acquired in actual clinic as a target domain to be classified or segmented, marks a very small amount of clinical data through a doctor, establishes a feature extractor and a classifier realized by a convolutional neural network, and constructs a small sample domain self-adaptive model. The model introduces a maximum correlation entropy criterion to improve the classification or segmentation precision and generalization capability, and extracts distinctive features between different classes and simultaneously reduces the distribution difference between a source domain and a target domain by alternating maximum minimum conditional entropy under the condition of only a few labeled target domain samples, thereby improving the accuracy of a classifier on the target domain and realizing a cross-domain migration method between different data sets.

The invention provides a small sample domain self-adaptive medical image segmentation or classification method which can be applied to the classification or segmentation of pathological image data, ultrasonic image data and other image data.

Drawings

FIG. 1 is a flow chart of the present invention.

Fig. 2 is a model architecture diagram.

Detailed Description

The invention is further described with reference to the following figures and detailed description.

The method comprises the steps of constructing a small sample domain self-adaptive model based on a deep convolutional neural network, inputting the small sample domain self-adaptive model to a source domain training data set and a target domain data set which comprise labels, wherein the target domain data set comprises a training set with known labels and a training set with unknown labels, and an output layer of the small sample domain self-adaptive model is a Softmax layer. The small sample domain adaptive model is used as a black box, a certain focus characteristic is not required to be specified for classification, a computer learns the classification or segmentation process, and meanwhile, the whole image is input into the small sample domain adaptive model instead of the certain characteristic of the image, so that the loss of information can be reduced. When the model training is finished, target domain data is input into the model, and a classification or segmentation result can be obtained.

The small sample domain adaptive model comprises two parts, one part is used for feature extraction, and the other part is used for a classifier for classification or segmentation. For feature extraction, a feature extractor in the model is constructed by removing the last one or more linear layers by a basic deep convolutional neural network, and the feature extractor can automatically learn implicit multi-level disease diagnosis features;

preferably, the deep convolutional neural network includes ResNet, Incepton, DenseNet, and the like.

For the classifier, the classifier is composed of an n-way linear classification layer and a Softmax layer, the features extracted by the feature extractor are input into the n-way linear classification layer, the output result of the n-way linear classification layer is input into the Softmax layer, and the Softmax layer outputs the probability that the source domain training data set and the target domain data set are classified into each class.

According to the method, different target functions are designed according to whether a source domain training data set and a target domain data set contain labels or not, the different target functions are propagated reversely, the partial derivatives of the target functions to the weights of the neurons are calculated layer by layer and serve as the basis for updating the weights, and the learning of the convolutional neural network is completed in the weight modification process.

For labeled data comprising a source domain training data set and a training set with known labels, a feature extractor and a classifier are trained by using the maximum correlation entropy as an objective function, wherein the correlation entropy is used for quantifying the similarity between two random variables, A represents the probability of corresponding prediction, and B represents the corresponding real label, so that the feature extractor can be ensured to extract distinctive features. The objective function herein includes, but is not limited to, using a maximum correlation entropy function, and also using a cross entropy loss function, etc.

For a training set with unknown labels, alternately training by using a maximum minimum conditional entropy objective function to enable the conditional entropy of the classifier to be maximum and the conditional entropy of the feature extractor to be minimum: by means of a classifier framework based on cosine similarity, cosine similarity between specific weight vectors of n types of focuses and output of a feature extractor is calculated, n represents the number of classes of the focuses of 1, 2, 3 and … …, then a Softmax function is used for obtaining probability vectors of the n types of focuses, and the weight vector of each class is a representative point of the class. By maximizing the conditional entropy, the representative point does not depend too much on the source instance, so that a domain-invariant representative is obtained, and the domain-invariant representative can be used as the representative point of the source domain and the target domain; and then, by minimizing the conditional entropy, the target features extracted from the source domain and the target domain are close to the representative point to obtain the distinctive features.

Referring to fig. 1, the medical image segmentation or classification method based on small sample domain adaptation of the present invention can implement classification or segmentation of clinical images according to a data set used for training, and the specific process is as follows:

step 1, data acquisition: a large amount of labeled public data and clinical image data to be classified or segmented are collected. Downloading medical image data from the public data set, wherein the medical image data are marked with classification of the focus or have segmentation marks, and the medical image data can be pathological image data, ultrasonic image data and the like;

step 2, data processing: the method supports two-dimensional image data as input.

a. The downloaded medical image data is subjected to data enhancement through operations such as rotation, random cutting and turning, and all data are subjected to size adjustment to be in accordance with the input standard of the small sample domain self-adaptive model, so that a source domain training data set is obtained, and richer data can be obtained by performing data enhancement on the medical image data.

b. The method comprises the steps that clinical image data acquired in actual clinical are used as a target domain data set to be classified or segmented, the target domain data set is divided into a training set with known labels, a verification set and a test set, and the training set with unknown labels is divided into the training set with unknown labels, the verification set and the test set, wherein the data volume ratio of the training set with unknown labels to the test set in the target domain data set is 8:1: 1.

Step 3, model training: constructing a small sample domain self-adaptive model based on a classical convolutional neural network and training;

a. referring to fig. 2, a small sample domain adaptive model is constructed, which includes a feature extractor and a classifier implemented by a convolutional neural network, wherein the feature extractor is constructed by removing the last one or several linear layers through a deep convolutional neural network such as ResNet34, and the feature extractor automatically learns implicit multi-level disease diagnosis features.

The classifier is composed of an n-way linear classification layer and a Softmax layer, the features extracted by the feature extractor are used as the input of the classifier, the output result of the n-way linear classification layer is sent to the Softmax layer, and the Softmax layer outputs the probability that each sample is classified into each class. And (3) inputting the small sample domain self-adaptive model into the data set obtained in the step (2), wherein the data set comprises a source domain training data set, a small amount of labeled training sets and a large amount of unlabeled training sets, and the output layer of the small sample domain self-adaptive model is a Softmax layer.

Preferably, the feature extractor of the present invention employs a ResNet34 network, and a ResNet34 network can introduce jump connections, so that the back propagation of the gradient is easier, and a deeper convolutional neural network can be trained effectively.

According to the invention, different objective functions are used for calculation according to the obtained data whether a label exists or not: for a source domain training data set with known labels and a training set with known labels, a maximum correlation entropy function is used as an objective function, and a feature extractor and a classifier are trained.

Objective function H₁The definition is as follows:

in the formula (1), y_iIs the true value; the parameter sigma influences the experimental result and cannot be too large or too small; p (x)_i) For classification or segmentation results, the formula is: m is the number of labeled samples, σ is the kernel width, exp is an exponential function, representing the power of e

p(x_i)＝softmax(W^TF(x_i)) (2)

Wherein F (x)_i) For the features extracted by the feature extractor, W is the weight vector T is transposed, W is^TIs the transpose of the weight vector.

For training sets for which the labels are unknown, the maximum minimum conditional entropy objective function, objective function H, is used₂The definition is as follows:

in formula (3), n is the number of classes, p (y ═ k | x) represents the probability that a sample x with an unknown label is predicted as the kth class, and E represents the averaging; k denotes the kth class and log is the logarithm.

In general, the invention can be expressed as a competing learning between a feature extractor and a classifier. The classifier is trained by maximizing the conditional entropy and the feature extractor is trained by minimizing the conditional entropy. By designing such a network structure and objective function, the classification or segmentation model trained on the source domain can be well migrated into a classification or segmentation that owns the clinical data target domain.

b. And (4) putting the data of the training set obtained by the processing in the step (2) into the small sample domain adaptive model constructed in the step (a) in the step (3) in batches for training. And continuously updating and iterating until the deep learning model converges by adopting a back propagation algorithm and a random gradient descent method, and storing the deep learning model at the moment.

Preferably, the model convergence is performed by adopting a back propagation algorithm and a random gradient descent method, so that derivative calculation is simplified, and final network parameters are obtained more effectively.

c. And (c) adjusting the hyper-parameters of the deep learning model obtained in the step (b) in the step (3), repeating the step (b) for a plurality of times to obtain a plurality of models, and selecting the model with the best classification or segmentation effect on the verification set in the step (2).

And 4, inputting the test set into the model trained in the step 3, and obtaining a classification or segmentation result of the clinical image through model operation.

The invention discloses a device for applying a medical image segmentation or classification method based on small sample domain self-adaptation, which comprises the following steps:

the system comprises an acquisition module, a comparison module and a comparison module, wherein the acquisition module is used for acquiring a source domain training data set and a target domain data set with known labels, and the target domain data set comprises a training set with known labels, a verification set and a test set and a training set with unknown labels;

the training module is used for training the small sample self-adaptive model by using a source domain training data set with known labels, a training set with known labels and a training set with unknown labels to obtain a small sample domain self-adaptive model after training;

the verification module is used for verifying the trained small sample domain self-adaptive model to obtain the small sample domain self-adaptive model with the best classification or segmentation effect;

and the classification or segmentation module is used for classifying or segmenting the test set to obtain a classification or segmentation result.

Claims (10)

1. A small sample domain self-adaptive medical image segmentation or classification method is characterized by comprising the following specific steps:

s1, acquiring medical image data with classification or segmentation labels, and processing the acquired medical image data to obtain a source domain training data set with known labels;

s2, acquiring clinical image data to be classified or segmented, and processing the acquired clinical image data to obtain a target domain data set, wherein the target domain data set comprises a training set with known labels, a verification set and a test set and a training set with unknown labels;

s3, constructing a small sample domain self-adaptive model, wherein the small sample domain self-adaptive model comprises a feature extractor and a classifier which are realized by a convolutional neural network, inputting the source domain training data set with known labels in the step S1, the training set with known labels in the step S2 and the training set with unknown labels into the small sample domain self-adaptive model to train until the model converges, and obtaining the small sample domain self-adaptive model which is trained;

s4, adjusting the hyper-parameters of the trained small sample domain self-adaptive model, repeating the step S3 to obtain a plurality of small sample domain self-adaptive models, and selecting the small sample domain self-adaptive model with the best classification or segmentation effect on the verification set;

s5, the test set of the target domain data set in step S2 is input to the small sample domain adaptive model obtained in step S4, and the small sample domain adaptive model outputs the classification or segmentation result of the lesion in the clinical image data.

2. The method for segmenting or classifying medical images based on small sample domain adaptation according to claim 1, wherein in step S1, the medical image data is subjected to data enhancement and the size of the medical image data is adjusted to conform to the network input standard of the small sample domain adaptation model, so as to obtain the source domain training data set.

3. The method for small sample domain adaptive medical image segmentation or classification as claimed in claim 1, wherein in step S2, the size of the clinical image data is adjusted to meet the network input standard of the small sample domain adaptive model, so as to obtain the target domain data set.

4. The method for small-sample-domain-adaptive-based medical image segmentation or classification as claimed in claim 3, wherein the data volume ratio of the training set, the validation set and the test set in the target domain data set is 8:1: 1.

5. The method for small sample domain adaptive medical image segmentation or classification as claimed in claim 1, wherein in step S3, the small sample domain adaptive model is constructed based on a classical deep convolutional neural network, which includes ResNet, inclusion and densneet.

6. The small sample domain adaptive-based medical image segmentation or classification method as claimed in claim 1, wherein in step S3, the feature extractor is constructed by a deep convolutional neural network that removes the last one or several linear layers, and the classifier comprises an n-way linear classification layer and a Softmax layer.

7. The small sample domain adaptive-based medical image segmentation or classification method according to claim 6, wherein a maximum correlation entropy function is adopted as an objective function for a source domain training data set and a training set with known labels, and a feature extractor and a classifier are trained; the training set with unknown labels adopts a maximum minimum conditional entropy objective function, and trains a feature extractor and a classifier.

8. The method for segmenting or classifying medical images based on small sample domain adaptation as claimed in claim 1, wherein in step S3, iteration is continuously updated by using back propagation algorithm and stochastic gradient descent method until the small sample domain adaptation model converges.

9. The method for segmenting or classifying medical images based on small sample domain adaptation according to claim 1, wherein the output layer of the small sample domain adaptation model is a Softmax layer, and the Softmax layer outputs the classification or segmentation result of clinical images.

10. The method according to claim 1, wherein the device for applying the segmentation or classification method comprises:

the system comprises an acquisition module, a comparison module and a comparison module, wherein the acquisition module is used for acquiring a source domain training data set and a target domain data set with known labels, and the target domain data set comprises a training set with known labels, a verification set and a test set and a training set with unknown labels;

the training module is used for training the small sample self-adaptive model by using a source domain training data set with known labels, a training set with known labels and a training set with unknown labels to obtain a small sample domain self-adaptive model after training;

the verification module is used for verifying the trained small sample domain self-adaptive model to obtain the small sample domain self-adaptive model with the best classification or segmentation effect;

and the classification or segmentation module is used for segmenting or classifying the test set to obtain a segmentation or classification result.

Images