技术领域technical field
本发明属于人工智能技术领域，主要是一种基于深度学习算法，并使用了注意力机制增强效果的胃癌病理切片计算机辅助识别模型构建方法。The invention belongs to the technical field of artificial intelligence, and mainly relates to a method for constructing a computer-aided recognition model of gastric cancer pathological slices based on a deep learning algorithm and using an attention mechanism to enhance the effect.
背景技术Background technique
胃癌是世界上第四大常见癌症，其死亡率在所有癌症中更是高居第二。因此胃癌逐渐成为了全民普遍关注的一个公共卫生问题。而如果在胃癌肿瘤产生的早期,患者就能被确诊，那么对于胃癌患者的治疗将取得显著的效果，极大降低胃癌疾病的死亡率。胃癌病理切片图像就是通过穿刺活检取得胃部组织之后，对组织切片用苏木精&伊红(Hematoxylin&Eosin,H&E)染色，再用数码相机通过显微镜拍摄取得的图像。胃癌病理切片可以为临床医师的治疗与诊断提供重要的指导作用。然而在病理医师的分析与诊断过程中，医生需要结合自己长期积累的临床诊断经验来判断胃癌病理切片中是否有癌变，这种人工的诊断方法，对医生的专业知识和从业经验都有极高的要求，而且也易受阅片者主观情绪和疲劳阅片等因素的影响，所以这种高度依赖人为因素的诊断过程具有主观差异性，医生的一小点失误都可能会给病人带来严重的后果。Gastric cancer is the fourth most common cancer in the world, and its death rate is the second highest among all cancers. Therefore, gastric cancer has gradually become a public health problem of general concern. And if patients can be diagnosed in the early stage of gastric cancer tumors, then the treatment of gastric cancer patients will achieve significant results and greatly reduce the mortality of gastric cancer diseases. The images of gastric cancer pathological sections are the images obtained by obtaining gastric tissue through biopsy, staining the tissue sections with Hematoxylin & Eosin (H&E), and then taking pictures with a digital camera through a microscope. Gastric cancer pathological slides can provide important guidance for clinicians in the treatment and diagnosis. However, in the analysis and diagnosis process of pathologists, doctors need to combine their long-term accumulated clinical diagnosis experience to judge whether there is cancer in gastric cancer pathological sections. requirements, and is also easily affected by factors such as the reader's subjective emotions and fatigue reading, so this diagnostic process that is highly dependent on human factors has subjective differences, and even a small mistake by the doctor may bring serious serious problems to the patient. s consequence.
计算机辅助诊断(Computer-Aided Diagnosis,CAD)是当前人工智能在医学领域重要应用，已经成为当前的一大研究热点。近几年来，随着大数据技术的不断发展，人们可以轻松地获取有效的医学信息，并且现如今计算能力大幅提升，计算机对病理图像有着非常强大的处理与分析能力。病理学家已经认识到CAD对病理组织图像的评价分析具有很高的鲁棒性与高效性，这些结果可以很好地支持病理学家对疾病的判断，并为进一步治疗提供了更准确的参考。因此，借助现代计算机的强大计算能力，构建一个高鲁棒性的胃癌病理切片CAD模型是可行的，并且具有重要意义。Computer-Aided Diagnosis (CAD) is an important application of artificial intelligence in the medical field, and has become a major research hotspot. In recent years, with the continuous development of big data technology, people can easily obtain effective medical information, and now the computing power has been greatly improved, and computers have very powerful processing and analysis capabilities for pathological images. Pathologists have realized that the evaluation and analysis of CAD on pathological tissue images is highly robust and efficient, and these results can well support pathologists in their judgment of the disease and provide a more accurate reference for further treatment . Therefore, with the powerful computing power of modern computers, it is feasible and of great significance to construct a highly robust CAD model of pathological slices of gastric cancer.
目前，CAD在该领域上的研究还非常稀少。现存方法中主要是使用支持向量机(SupportVector Machine,SVM)进行分类。Cosatto等人基于感兴趣区域(Regions-of-Interest,ROI)对每个胃癌病理切片的不同区域进行分割、标注，根据切片的颜色手动提取特征，构建训练集，然后使用多示例的方式进行半监督训练，使用SVM对样本进行分类。这些传统方法还是拥有一定的缺陷。首先，在进行病理图像分割以及特征提取时，会要求研究人员拥有相关领域的专业知识，否则很难对样本切片中的的形态学特征、纹理特征以及相关结构进行描述。同时，病理图像往往十分复杂，传统方法可能难以提取出有区分性的高质量特征，导致识别精度较为一般。At present, the research on CAD in this field is still very rare. The existing methods mainly use Support Vector Machine (SVM) for classification. Cosatto et al. segmented and labeled different regions of each gastric cancer pathological slice based on Regions-of-Interest (ROI), manually extracted features according to the color of the slice, constructed a training set, and then used multiple examples for semi-analysis. Supervised training, using SVM to classify samples. These traditional methods still have certain defects. First of all, when performing pathological image segmentation and feature extraction, researchers are required to have professional knowledge in related fields, otherwise it is difficult to describe the morphological features, texture features and related structures in the sample slices. At the same time, pathological images are often very complex, and traditional methods may be difficult to extract distinguishing high-quality features, resulting in relatively mediocre recognition accuracy.
发明内容Contents of the invention
本发明运用人工智能技术，实现基于深度学习的胃癌病理切片的识别，辅助医生临床诊断。首先收集胃癌病理切片真实数据集，经由专业医师进行数据标注，分析数据集质量，并进行数据增强与数据预处理。然后构建出高效提取出胃癌病理切片图像特征的卷积神经网络(Convolutional NeuralNetwork,CNN)，最终使用建立好的数据集对模型进行训练并测试，完成对胃癌病理切片的识别。The invention uses artificial intelligence technology to realize the recognition of gastric cancer pathological slices based on deep learning, and assist doctors in clinical diagnosis. First, the real data set of gastric cancer pathological slices is collected, and the data is marked by professional doctors, the quality of the data set is analyzed, and data enhancement and data preprocessing are performed. Then a convolutional neural network (Convolutional Neural Network, CNN) that efficiently extracts the image features of gastric cancer pathological slices was constructed, and finally the established data set was used to train and test the model to complete the recognition of gastric cancer pathological slices.
本发明主要面对的问题有如下几点：The main problems faced by the present invention are as follows:
1、高质量的胃癌病理图像十分珍贵，图像内容极其复杂，数据少、难度大，在训练过程中模型很容易出现过拟合的状况；1. High-quality gastric cancer pathological images are very precious, and the content of the images is extremely complex, with little data and great difficulty. During the training process, the model is prone to overfitting;
2、图像尺寸较大，都在百万级像素，难以计算，直接进行分析识别会占用大量的计算资源，考虑到实际使用，模型必须有较高的识别效率以及较低的资源占用；2. The size of the image is large, with millions of pixels, which is difficult to calculate. Direct analysis and recognition will take up a lot of computing resources. Considering the actual use, the model must have high recognition efficiency and low resource consumption;
3、一幅胃癌病例切片图像中可能有很大片的区域都是正常区域，这可能会干扰模型对患病特征的提取；3. There may be a large area in a gastric cancer case slice image that is a normal area, which may interfere with the model's extraction of disease features;
4、在实际诊断过程中，医生与患者会更希望诊断时对患病样本有更细致的检测，以减少医疗事故，将病人误诊为正常人的损失远大于将正常人误诊为病人，因此模型要对患病的胃癌病理切片，即正样本有较高的敏感度。4. In the actual diagnosis process, doctors and patients would prefer to have more detailed detection of diseased samples during diagnosis to reduce medical accidents. The loss of misdiagnosing a patient as a normal person is much greater than misdiagnosing a normal person as a patient. Therefore, the model It is necessary to have high sensitivity to pathological sections of gastric cancer, that is, positive samples.
针对上述问题，本发明使用了一个121层稠密连接卷积神经网络(DenselyConnected Convolutional Network,DenseNet)进行图像的识别。DenseNet中的稠密块(Dense Block)结构可以让网络的高层部分获取到浅层特征，很好地减轻过拟合现象。同时该模型层数较多，能拟合出更为复杂、更加光滑的决策函数。尽管层数很多，但该模型的参数数量并不多，很好地节约了资源占用。为进一步避免过拟合，本发明还采用了迁移学习的训练机制。模型会先在ImageNet数据集上进行预训练，让模型获得很强的图像特征提取能力，在正式训练时模型的主要优化就能更好地集中在如何提取患病区域的特征，极大地提高数据的利用效率。In view of the above problems, the present invention uses a 121-layer densely connected convolutional neural network (DenselyConnected Convolutional Network, DenseNet) for image recognition. The dense block (Dense Block) structure in DenseNet allows the high-level part of the network to obtain shallow features, which can well alleviate the phenomenon of over-fitting. At the same time, the model has more layers, which can fit a more complex and smoother decision function. Although there are many layers, the number of parameters of the model is not many, which saves resource consumption very well. In order to further avoid over-fitting, the present invention also adopts a training mechanism of transfer learning. The model will be pre-trained on the ImageNet dataset first, so that the model can obtain a strong ability to extract image features. During the formal training, the main optimization of the model can better focus on how to extract the features of the diseased area, which greatly improves the data quality. utilization efficiency.
得力于该模型强大地特征提取能力，将输入图像压缩到了224×224的尺寸，大幅减少模型参数数量，加快计算。同时由于在ImageNet数据集上的预训练，胃癌切片数据集的数据分布越接近ImageNet数据集，则会有越好的训练成果，因此所有数据在进入模型前都使用了ImageNet数据集的平均值与方差进行标准化处理。训练时也对数据进行了随机的水平、竖直翻转以及15°的旋转进行数据增强。Thanks to the powerful feature extraction capability of the model, the input image is compressed to a size of 224×224, which greatly reduces the number of model parameters and speeds up calculation. At the same time, due to the pre-training on the ImageNet data set, the closer the data distribution of the gastric cancer slice data set is to the ImageNet data set, the better the training results will be. Therefore, all the data use the average value of the ImageNet data set before entering the model. The variance is standardized. During training, the data is also randomly flipped horizontally, vertically, and rotated by 15° for data enhancement.
为了进一步强调图像中患病区域中所包含的信息，本方法将注意力机制引入DenseNet之中。医生在对病理切片进行诊断时也过多关注正常部分，而是会将更多精力集中于有患病嫌疑的区域。因此在模型的第一个、第二个、第三个稠密块之间的过渡层(Transition Layer)后加入了注意力模块(Attention Module)。进入该模块的特征图(Feature Map)通过两次连续的上采样与下采样，即通过训练时的上下文自发地让模型学习特征图中每一点代表患病特征的概率，之后再根据概率大小加强特征图中的患病相关信息。两次上采样与下采样即先通过两次卷积操作以及池化操作增大感受野，压缩特征图，再通过两次反卷积操作以及上池化操作还原输入尺寸。在第一次上采样之后，其输出会通过一个跳跃式连接(Skip Connection)与第一次下采样的结果相加，这会促使网络从多个感受野提取信息。由于特征图在经过DenseNet的卷积结构之后会与原输入进行拼接，因此数据每经过一次卷积就会膨胀一次，很不利于特征图的压缩。因此本发明在注意力模块中改用ResNet的残差结构进行卷积，以此来继续保留底层部分中提取出的特征。In order to further emphasize the information contained in the diseased area in the image, this method introduces the attention mechanism into DenseNet. Doctors also pay too much attention to normal parts when making diagnosis on pathological slides, but will concentrate more energy on suspected diseased areas. Therefore, the attention module (Attention Module) is added after the transition layer (Transition Layer) between the first, second, and third dense blocks of the model. The feature map (Feature Map) entering this module undergoes two consecutive upsampling and downsampling, that is, through the context during training, the model can spontaneously learn the probability that each point in the feature map represents a diseased feature, and then strengthen it according to the probability Disease-related information in the feature map. The two up-sampling and down-sampling operations first increase the receptive field through two convolution operations and pooling operations, compress the feature map, and then restore the input size through two deconvolution operations and upper pooling operations. After the first upsampling, its output will be added to the result of the first downsampling through a skip connection (Skip Connection), which will prompt the network to extract information from multiple receptive fields. Since the feature map will be spliced with the original input after passing through the convolution structure of DenseNet, the data will expand every time it passes through convolution, which is not conducive to the compression of the feature map. Therefore, the present invention uses the residual structure of ResNet to perform convolution in the attention module, so as to continue to retain the features extracted from the bottom layer.
本发明在训练过程中使用了交叉熵函数，为了增强模型对患病切片的敏感度，本发明在交叉熵函数中正样本的部分加上了一个大于1的权重。The present invention uses the cross-entropy function in the training process. In order to enhance the sensitivity of the model to the diseased slice, the present invention adds a weight greater than 1 to the part of the positive sample in the cross-entropy function.
综上所述，基于深度学习胃癌病理切片的计算机辅助模型构建方法，该方法包括以下步骤：In summary, the computer-aided model construction method based on deep learning of gastric cancer pathological slices, the method includes the following steps:
步骤1、构建一个121层的DenseNet模型，如图8所示，该模型的主干部分是由4个逐渐加深的稠密结构与4个过渡层交替拼接而成。其中的过渡层结构以及组成稠密结构的基本卷积结构分布如图2与图1。每个稠密结构内，每次卷积操作开始前，都会将之前每一次卷积的结果在通道方向上拼接，实现跳跃式的特征传递。模型的最后一层为一个Sigmoid单输出的全连接层，输出模型分类的结果。除最后一层外，所有层的参数均初始化为该模型结构在ImageNet数据集上预训练好的参数；Step 1. Build a 121-layer DenseNet model, as shown in Figure 8. The backbone of the model is composed of 4 gradually deepening dense structures and 4 transition layers alternately spliced. The distribution of the transition layer structure and the basic convolutional structure that constitutes the dense structure is shown in Figure 2 and Figure 1. In each dense structure, before each convolution operation starts, the results of each previous convolution will be spliced in the direction of the channel to achieve skip-style feature transfer. The last layer of the model is a fully connected layer with a Sigmoid single output, which outputs the results of the model classification. Except for the last layer, the parameters of all layers are initialized to the pre-trained parameters of the model structure on the ImageNet dataset;
步骤2、对胃癌病理切片数据集进行相关预处理；具体包括以下步骤：Step 2. Perform relevant preprocessing on the gastric cancer pathological slice data set; specifically, the following steps are included:
步骤2.1、将每张胃癌病理切片的图像使用二线性插值的方式压缩至224×224的尺寸；Step 2.1, compressing the image of each pathological slice of gastric cancer to a size of 224×224 by means of bilinear interpolation;
步骤2.2、使用ImageNet数据集的平均值与方差对胃癌病理切片数据集进行标准化处理；Step 2.2, using the mean value and variance of the ImageNet data set to standardize the gastric cancer pathological slice data set;
步骤2.3、将胃癌病理切片数据集中胃癌病理切片图像随机地分为三组：训练集、验证集以及测试集，作为优选，训练集中的数据占所有数据的80％，验证集与测试集中的数据占所有数据的80％；Step 2.3, the gastric cancer pathological slice images in the gastric cancer pathological slice data set are randomly divided into three groups: a training set, a verification set and a test set. As a preference, the data in the training set accounts for 80% of all data, and the data in the verification set and the test set 80% of all data;
步骤3、使用预处理好的数据集对模型进行训练。Step 3. Use the preprocessed data set to train the model.
作为优选，步骤3具体包括以下步骤：As preferably, step 3 specifically includes the following steps:
步骤3.1、模型的训练算法调为使用标准的Adam优化算法对模型进行训练，损失函数设为在正样本部分添加了值1.5权重的交叉熵函数，训练时batch size为10；Step 3.1, the training algorithm of the model is adjusted to use the standard Adam optimization algorithm to train the model, the loss function is set to a cross-entropy function with a weight of 1.5 added to the positive sample part, and the batch size is 10 during training;
步骤3.2、训练时，胃癌病理切片图像进入模型前，有20％的几率会进行如下操作中的其中一种以进行数据增强：随机地进行水平、竖直翻转，以及15°的正时针或逆时针旋转，其中超出边界的点会被替换为白色(RGB(255,255,255)),即胃癌病理切片的背景颜色；Step 3.2. During training, before gastric cancer pathological slice images enter the model, there is a 20% chance that one of the following operations will be performed for data enhancement: random horizontal and vertical flips, and 15° clockwise or counterclockwise Rotate clockwise, and the points beyond the boundary will be replaced with white (RGB(255,255,255)), which is the background color of gastric cancer pathological slices;
步骤3.3、模型在胃癌病理切片数据集上训练2个epoch，使其获得提取胃癌病理切片特征的基本能力；Step 3.3, the model is trained for 2 epochs on the gastric cancer pathological slice data set, so that it can obtain the basic ability to extract the features of gastric cancer pathological slices;
步骤3.4、在模型的第一个、第二个、第三个稠密块之间，添加注意力模块。继续训练60个epoch，让模型在接下来的训练过程中自发地将更多注意力集中于患病部分。在每训练一个epoch之后，让模型对验证集进行预测，同时记录模型预测的正确率。最终选择验证集损失值最小的模型作为最终结果。Step 3.4. Add an attention module between the first, second, and third dense blocks of the model. Continue to train for 60 epochs, allowing the model to spontaneously focus more on the diseased part in the next training process. After each training epoch, let the model predict the verification set, and record the accuracy of the model prediction. Finally, the model with the smallest loss value in the validation set is selected as the final result.
步骤3.5、保存最优模型，并使用测试集数据测量该模型分类的准确率。Step 3.5, save the optimal model, and use the test set data to measure the classification accuracy of the model.
作为优选，模型的整体训练平台是基于云端的，Keras框架搭建于Linux系统上,后端使用TensorFlow。训练的GPU为GTX1080，使用CUDA作为显卡计算的运算驱动，并使用cuDNN对深度学习进行加速。As a preference, the overall training platform of the model is based on the cloud, the Keras framework is built on the Linux system, and the backend uses TensorFlow. The GPU used for training is GTX1080, CUDA is used as the calculation driver for graphics card calculations, and cuDNN is used to accelerate deep learning.
与现有技术相比，本发明具有以下明显优势：Compared with the prior art, the present invention has the following obvious advantages:
1、在深度学习算法的帮助下，本CAD模型自动地提取出胃癌病理切片中的特征，完全摆脱了相关医学专业知识的束缚；1. With the help of deep learning algorithm, this CAD model automatically extracts the features of gastric cancer pathological slices, completely getting rid of the shackles of relevant medical professional knowledge;
2、本发明所构建出的模型识别精度较高，明显高于现有的传统方法；2. The recognition accuracy of the model constructed by the present invention is higher, obviously higher than that of the existing traditional methods;
3、对数据量的依赖大幅减少，由于DenseNet本身模型结构的优势以及迁移学习的应用，该模型在数据量较少的情况下，也可以避免过拟合，训练出高精度的模型；3. The dependence on the amount of data is greatly reduced. Due to the advantages of DenseNet's own model structure and the application of transfer learning, the model can also avoid overfitting and train a high-precision model when the amount of data is small;
4、模型收敛速度较快，训练时的资源占用较低。4. The model converges faster, and the resource usage during training is lower.
5、模型参数较少，在进行样本识别时运算效率高，资源占用低，可以很好地投入实际实用之中；5. The model parameters are few, the calculation efficiency is high when performing sample identification, and the resource occupation is low, which can be well put into practical use;
6、模型对患病切片有较高的敏感度，降低了医疗事故出现的可能性。6. The model has high sensitivity to diseased slices, which reduces the possibility of medical accidents.
附图说明：Description of drawings:
图1为识别模型中的稠密卷积结构(Dense Conv Block)；Figure 1 shows the dense convolution structure (Dense Conv Block) in the recognition model;
图2为识别模型中稠密卷积结构间的过渡层(Transition Layer)；Figure 2 is the transition layer (Transition Layer) between dense convolution structures in the recognition model;
图3为识别模型中用于大幅压缩输入图像的输入处理结构(Input Block)；Fig. 3 is the input processing structure (Input Block) used to greatly compress the input image in the recognition model;
图4为识别模型中的残差卷积结构(Residual Conv Block)；Figure 4 shows the residual convolution structure (Residual Conv Block) in the recognition model;
图5为注意力模块中的下采样结构(Down Sample Block)；Figure 5 shows the downsampling structure (Down Sample Block) in the attention module;
图6为注意力模块中的上采样结构(Up Sample Block)；Figure 6 shows the upsampling structure (Up Sample Block) in the attention module;
图7为注意力模块(Attention Module)的基本结构；Figure 7 shows the basic structure of the Attention Module;
图8为本方法识别模型添加注意力模型之前的结构；Fig. 8 is the structure before the attention model is added to the recognition model of this method;
图9为本方法识别模型添加注意力模型之后的结构。Figure 9 shows the structure after adding the attention model to the identification model of this method.
具体实施方式Detailed ways
以下结合具体实施例，并参照附图，对本发明进一步详细说明。The present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.
本发明所用到的硬件为一台可以进行深度学习的工作站。所使用的辅助工具为深度学习训练框架Keras。The hardware used in the present invention is a workstation capable of deep learning. The auxiliary tool used is the deep learning training framework Keras.
本发明所提供的基于深度学习的胃癌病理切片的计算机辅助诊断模型构建方法主要包括以下步骤：The computer-aided diagnosis model construction method of gastric cancer pathological slices based on deep learning provided by the present invention mainly includes the following steps:
步骤1，构建一个121层的DenseNet模型，如图8所示，该模型的主干部分是由4个逐渐加深的稠密结构与4个过渡层交替拼接而成。其中的过渡层结构以及组成稠密结构的基本卷积结构分布如图2与图1。每个稠密结构内，每次卷积操作开始前，都会将之前每一次卷积的结果在通道方向上拼接，实现跳跃式的特征传递。模型的最后一层为一个Sigmoid单输出的全连接层，输出模型分类的结果。除最后一层外，所有层的参数均初始化为该模型结构在ImageNet数据集上预训练好的参数Step 1, build a 121-layer DenseNet model, as shown in Figure 8, the backbone of the model is composed of 4 gradually deepening dense structures and 4 transition layers alternately spliced. The distribution of the transition layer structure and the basic convolutional structure that constitutes the dense structure is shown in Figure 2 and Figure 1. In each dense structure, before each convolution operation starts, the results of each previous convolution will be spliced in the direction of the channel to achieve skip-style feature transfer. The last layer of the model is a fully connected layer with a Sigmoid single output, which outputs the results of the model classification. Except for the last layer, the parameters of all layers are initialized to the pre-trained parameters of the model structure on the ImageNet dataset
步骤2，对胃癌病理切片数据集进行相关预处理。Step 2, perform relevant preprocessing on the gastric cancer pathological slice data set.
步骤2.1，将每张胃癌病理切片的图像使用二线性插值的方式压缩至224×224的尺寸。In step 2.1, the image of each gastric cancer pathological section is compressed to a size of 224×224 by means of bilinear interpolation.
步骤2.2，使用ImageNet数据集的平均值与方差对胃癌病理切片数据集进行标准化处理，即对于图像中第i个点的原像素值xi有：Step 2.2, use the mean and variance of the ImageNet data set to standardize the gastric cancer pathological slice data set, that is, for the original pixel value xi of the i-th point in the image:
其中μ和σ2分别代表ImageNet数据集的均值与方差。where μ and σ2 represent the mean and variance of the ImageNet dataset, respectively.
步骤2.3，将胃癌病理切片数据集中胃癌病理切片图像随机地分为三组：训练集、验证集以及测试集。训练集中的数据占所有数据的80％，验证集与测试集中的数据占所有数据的80％。In step 2.3, the gastric cancer pathological slice images in the gastric cancer pathological slice dataset are randomly divided into three groups: a training set, a verification set, and a test set. The data in the training set accounts for 80% of all the data, and the data in the verification set and test set account for 80% of all the data.
步骤2.4，训练集数据进入模型前，会有随机地进行水平、竖直翻转，以及15°的正时针或逆时针旋转，其中超出边界的点会被替换为白色(RGB(255,255,255)),即胃癌病理切片的背景颜色。Step 2.4, before the training set data enters the model, it will be randomly flipped horizontally and vertically, and rotated clockwise or counterclockwise by 15°, and the points beyond the boundary will be replaced with white (RGB(255,255,255)), that is The background color of gastric cancer pathology slides.
步骤3，使用处理好的数据集对模型进行训练。Step 3, use the processed data set to train the model.
步骤3.1、模型的训练算法调为使用标准的Adam优化算法对模型进行训练，batchsize设为10，损失函数设为在正样本部分添加了值1.5权重的交叉熵函数，最终的交叉熵函数为：Step 3.1, the training algorithm of the model is adjusted to use the standard Adam optimization algorithm to train the model, the batchsize is set to 10, and the loss function is set to a cross-entropy function with a weight of 1.5 added to the positive sample part. The final cross-entropy function is:
loss(y,y′)＝wy·-log(y′)+(1-y)·-log(1-y′)，loss(y,y′)=wy·-log(y′)+(1-y)·-log(1-y′),
其中y为期望输出，y’为实际输出，w为权重且值为1.5。Where y is the desired output, y' is the actual output, w is the weight and the value is 1.5.
步骤3.2、训练时，胃癌病理切片图像进入模型前，有20％的几率会进行如下操作中的其中一种以进行数据增强：随机地进行水平、竖直翻转，以及15°的正时针或逆时针旋转，其中超出边界的点会被替换为白色(RGB(255,255,255)),即胃癌病理切片的背景颜色；Step 3.2. During training, before gastric cancer pathological slice images enter the model, there is a 20% chance that one of the following operations will be performed for data enhancement: random horizontal and vertical flips, and 15° clockwise or counterclockwise Rotate clockwise, and the points beyond the boundary will be replaced with white (RGB(255,255,255)), which is the background color of gastric cancer pathological slices;
步骤3.3，模型在胃癌病理切片数据集上训练2个epoch，使其获得提取胃癌病理切片特征的基本能力。In step 3.3, the model is trained for 2 epochs on the gastric cancer pathological slice data set, so that it can acquire the basic ability to extract features of gastric cancer pathological slices.
步骤3.4，如图8所示，在模型的第一个、第二个、第三个稠密块之间，添加注意力模块，如图中虚线部分所示。注意力模块的结构如图7所示。继续训练60个epoch，让模型在接下来的训练过程中自发地将更多注意力集中于患病部分。在每训练一个epoch之后，让模型对验证集进行预测，同时记录模型预测的正确率。最终选择验证集损失值最小的模型作为最终结果。Step 3.4, as shown in Figure 8, add an attention module between the first, second, and third dense blocks of the model, as shown in the dotted line in the figure. The structure of the attention module is shown in Fig. 7. Continue to train for 60 epochs, allowing the model to spontaneously focus more on the diseased part in the next training process. After each training epoch, let the model predict the verification set, and record the accuracy of the model prediction. Finally, the model with the smallest loss value in the validation set is selected as the final result.
步骤3.5、保存最优模型，并使用测试集数据测量该模型分类的准确率。Step 3.5, save the optimal model, and use the test set data to measure the classification accuracy of the model.
以上实施例仅为本发明的示例性实施例，不用于限制本发明，本发明的保护范围由权利要求书限定。本领域技术人员可以在本发明的实质和保护范围内，对本发明做出各种修改或等同替换，这种修改或等同替换也应视为落在本发明的保护范围内。The above embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and the protection scope of the present invention is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present invention within the spirit and protection scope of the present invention, and such modifications or equivalent replacements should also be deemed to fall within the protection scope of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810665639.XACN108898175B (en) | 2018-06-26 | 2018-06-26 | A computer-aided model construction method based on deep learning for gastric cancer pathological slices |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810665639.XACN108898175B (en) | 2018-06-26 | 2018-06-26 | A computer-aided model construction method based on deep learning for gastric cancer pathological slices |
| Publication Number | Publication Date |
|---|---|
| CN108898175Atrue CN108898175A (en) | 2018-11-27 |
| CN108898175B CN108898175B (en) | 2020-11-20 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810665639.XAExpired - Fee RelatedCN108898175B (en) | 2018-06-26 | 2018-06-26 | A computer-aided model construction method based on deep learning for gastric cancer pathological slices |
| Country | Link |
|---|---|
| CN (1) | CN108898175B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109685141A (en)* | 2018-12-25 | 2019-04-26 | 哈工大机器人（合肥）国际创新研究院 | A kind of robotic article sorting visible detection method based on deep neural network |
| CN109727238A (en)* | 2018-12-27 | 2019-05-07 | 贵阳朗玛信息技术股份有限公司 | The recognition methods of x-ray chest radiograph and device |
| CN109784347A (en)* | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN110378463A (en)* | 2019-07-15 | 2019-10-25 | 北京智能工场科技有限公司 | A kind of artificial intelligence model standardized training platform and automated system |
| CN110472676A (en)* | 2019-08-05 | 2019-11-19 | 首都医科大学附属北京朝阳医院 | Stomach morning cancerous tissue image classification system based on deep neural network |
| CN110633385A (en)* | 2019-09-24 | 2019-12-31 | 天津医科大学 | A method for retrieval and compression of medical images |
| CN110706793A (en)* | 2019-09-25 | 2020-01-17 | 天津大学 | Attention mechanism-based thyroid nodule semi-supervised segmentation method |
| CN110738231A (en)* | 2019-07-25 | 2020-01-31 | 太原理工大学 | An Improved S-DNet Neural Network Model for Classification of Mammary X-ray Images |
| CN110766670A (en)* | 2019-10-18 | 2020-02-07 | 厦门粉红思黛医学科技有限公司 | Mammary gland molybdenum target image tumor localization algorithm based on deep convolutional neural network |
| CN111126175A (en)* | 2019-12-05 | 2020-05-08 | 厦门大象东方科技有限公司 | Facial image recognition algorithm based on deep convolutional neural network |
| CN111680553A (en)* | 2020-04-29 | 2020-09-18 | 北京联合大学 | A method and system for pathological image recognition based on depthwise separable convolution |
| CN112488234A (en)* | 2020-12-10 | 2021-03-12 | 武汉大学 | End-to-end histopathology image classification method based on attention pooling |
| CN112767329A (en)* | 2021-01-08 | 2021-05-07 | 北京安德医智科技有限公司 | Image processing method and device and electronic equipment |
| CN113011306A (en)* | 2021-03-15 | 2021-06-22 | 中南大学 | Method, system and medium for automatic identification of bone marrow cell images in continuous maturation stage |
| CN113109327A (en)* | 2021-03-09 | 2021-07-13 | 杭州市林业科学研究院 | Prediction method of dry rot of hickory nut |
| CN113112459A (en)* | 2021-03-26 | 2021-07-13 | 四川大学 | Oral squamous cell carcinoma differentiation degree prediction system |
| CN113192633A (en)* | 2021-05-24 | 2021-07-30 | 山西大学 | Stomach cancer fine-grained classification method based on attention mechanism |
| CN113628754A (en)* | 2021-08-12 | 2021-11-09 | 武剑 | Artificial intelligence-based dynamic prediction model construction method and system for cerebrovascular disease |
| CN113947607A (en)* | 2021-09-29 | 2022-01-18 | 电子科技大学 | A deep learning-based method for building a survival prognosis model for cancer pathological images |
| CN114548380A (en)* | 2022-03-04 | 2022-05-27 | 桂林医学院 | A ConvNeXt-based Recognition Method of Gastric Cancer Pathological Sections |
| CN114862770A (en)* | 2022-04-18 | 2022-08-05 | 华南理工大学 | Gastric cancer pathological section image segmentation prediction method based on SEnet |
| CN115131279A (en)* | 2021-03-12 | 2022-09-30 | 香港大学 | Disease classification via deep learning models |
| CN115690056A (en)* | 2022-11-03 | 2023-02-03 | 中国科学院自动化研究所 | Gastric cancer pathological image classification method and system based on HER2 gene detection |
| CN116492048A (en)* | 2023-04-03 | 2023-07-28 | 中国人民解放军总医院第一医学中心 | Intraoperative pancreatic duct position prediction method and device based on deep learning |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060285743A1 (en)* | 2005-06-20 | 2006-12-21 | Shih-Jong J. Lee | Object based boundary refinement method |
| CN104834943A (en)* | 2015-05-25 | 2015-08-12 | 电子科技大学 | Brain tumor classification method based on deep learning |
| CN108021916A (en)* | 2017-12-31 | 2018-05-11 | 南京航空航天大学 | Deep learning diabetic retinopathy sorting technique based on notice mechanism |
| CN108038519A (en)* | 2018-01-30 | 2018-05-15 | 浙江大学 | A kind of uterine neck image processing method and device based on dense feature pyramid network |
| CN108108757A (en)* | 2017-12-18 | 2018-06-01 | 深圳市唯特视科技有限公司 | A kind of diabetic foot ulcers sorting technique based on convolutional neural networks |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060285743A1 (en)* | 2005-06-20 | 2006-12-21 | Shih-Jong J. Lee | Object based boundary refinement method |
| CN104834943A (en)* | 2015-05-25 | 2015-08-12 | 电子科技大学 | Brain tumor classification method based on deep learning |
| CN108108757A (en)* | 2017-12-18 | 2018-06-01 | 深圳市唯特视科技有限公司 | A kind of diabetic foot ulcers sorting technique based on convolutional neural networks |
| CN108021916A (en)* | 2017-12-31 | 2018-05-11 | 南京航空航天大学 | Deep learning diabetic retinopathy sorting technique based on notice mechanism |
| CN108038519A (en)* | 2018-01-30 | 2018-05-15 | 浙江大学 | A kind of uterine neck image processing method and device based on dense feature pyramid network |
| Title |
|---|
| YUEXIANG LI: "Deep Learning Based Gastric Cancer Indentification", 《2018 IEEE 15TH INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING》* |
| 于观贞: "人工智能在肿瘤病理诊断和评估中的应用与思考", 《第二军医大学学报》* |
| 黄奕晖,冯前进: "基于三维全卷积DenseNet的脑胶质瘤MRI分割", 《南方医科大学学报》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784347A (en)* | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN109784347B (en)* | 2018-12-17 | 2022-04-26 | 西北工业大学 | Image classification method based on multi-scale dense convolutional neural network and spectral attention mechanism |
| CN109685141A (en)* | 2018-12-25 | 2019-04-26 | 哈工大机器人（合肥）国际创新研究院 | A kind of robotic article sorting visible detection method based on deep neural network |
| CN109685141B (en)* | 2018-12-25 | 2022-10-04 | 合肥哈工慧拣智能科技有限公司 | Robot article sorting visual detection method based on deep neural network |
| CN109727238A (en)* | 2018-12-27 | 2019-05-07 | 贵阳朗玛信息技术股份有限公司 | The recognition methods of x-ray chest radiograph and device |
| CN110378463A (en)* | 2019-07-15 | 2019-10-25 | 北京智能工场科技有限公司 | A kind of artificial intelligence model standardized training platform and automated system |
| CN110378463B (en)* | 2019-07-15 | 2021-05-14 | 北京智能工场科技有限公司 | Artificial intelligence model standardization training platform and automatic system |
| CN110738231A (en)* | 2019-07-25 | 2020-01-31 | 太原理工大学 | An Improved S-DNet Neural Network Model for Classification of Mammary X-ray Images |
| CN110738231B (en)* | 2019-07-25 | 2022-12-27 | 太原理工大学 | Method for classifying mammary gland X-ray images by improving S-DNet neural network model |
| CN110472676A (en)* | 2019-08-05 | 2019-11-19 | 首都医科大学附属北京朝阳医院 | Stomach morning cancerous tissue image classification system based on deep neural network |
| CN110633385B (en)* | 2019-09-24 | 2023-05-12 | 天津医科大学 | Medical image retrieval and compression method |
| CN110633385A (en)* | 2019-09-24 | 2019-12-31 | 天津医科大学 | A method for retrieval and compression of medical images |
| CN110706793A (en)* | 2019-09-25 | 2020-01-17 | 天津大学 | Attention mechanism-based thyroid nodule semi-supervised segmentation method |
| CN110766670A (en)* | 2019-10-18 | 2020-02-07 | 厦门粉红思黛医学科技有限公司 | Mammary gland molybdenum target image tumor localization algorithm based on deep convolutional neural network |
| CN111126175A (en)* | 2019-12-05 | 2020-05-08 | 厦门大象东方科技有限公司 | Facial image recognition algorithm based on deep convolutional neural network |
| CN111680553A (en)* | 2020-04-29 | 2020-09-18 | 北京联合大学 | A method and system for pathological image recognition based on depthwise separable convolution |
| CN112488234A (en)* | 2020-12-10 | 2021-03-12 | 武汉大学 | End-to-end histopathology image classification method based on attention pooling |
| CN112488234B (en)* | 2020-12-10 | 2022-04-29 | 武汉大学 | End-to-end histopathology image classification method based on attention pooling |
| CN112767329B (en)* | 2021-01-08 | 2021-09-10 | 北京安德医智科技有限公司 | Image processing method and device and electronic equipment |
| CN112767329A (en)* | 2021-01-08 | 2021-05-07 | 北京安德医智科技有限公司 | Image processing method and device and electronic equipment |
| CN113109327B (en)* | 2021-03-09 | 2023-11-17 | 杭州市林业科学研究院 | Method for predicting dry rot of hickory |
| CN113109327A (en)* | 2021-03-09 | 2021-07-13 | 杭州市林业科学研究院 | Prediction method of dry rot of hickory nut |
| CN115131279A (en)* | 2021-03-12 | 2022-09-30 | 香港大学 | Disease classification via deep learning models |
| CN113011306A (en)* | 2021-03-15 | 2021-06-22 | 中南大学 | Method, system and medium for automatic identification of bone marrow cell images in continuous maturation stage |
| CN113112459A (en)* | 2021-03-26 | 2021-07-13 | 四川大学 | Oral squamous cell carcinoma differentiation degree prediction system |
| CN113192633B (en)* | 2021-05-24 | 2022-05-31 | 山西大学 | Attention-based fine-grained classification of gastric cancer |
| CN113192633A (en)* | 2021-05-24 | 2021-07-30 | 山西大学 | Stomach cancer fine-grained classification method based on attention mechanism |
| CN113628754A (en)* | 2021-08-12 | 2021-11-09 | 武剑 | Artificial intelligence-based dynamic prediction model construction method and system for cerebrovascular disease |
| CN113628754B (en)* | 2021-08-12 | 2022-04-08 | 武剑 | Cerebrovascular disease dynamic prediction model construction method and system based on artificial intelligence |
| CN113947607B (en)* | 2021-09-29 | 2023-04-28 | 电子科技大学 | Cancer pathological image survival prognosis model construction method based on deep learning |
| CN113947607A (en)* | 2021-09-29 | 2022-01-18 | 电子科技大学 | A deep learning-based method for building a survival prognosis model for cancer pathological images |
| CN114548380A (en)* | 2022-03-04 | 2022-05-27 | 桂林医学院 | A ConvNeXt-based Recognition Method of Gastric Cancer Pathological Sections |
| CN114862770A (en)* | 2022-04-18 | 2022-08-05 | 华南理工大学 | Gastric cancer pathological section image segmentation prediction method based on SEnet |
| CN114862770B (en)* | 2022-04-18 | 2024-05-14 | 华南理工大学 | SENet-based gastric cancer pathological section image segmentation prediction method |
| CN115690056A (en)* | 2022-11-03 | 2023-02-03 | 中国科学院自动化研究所 | Gastric cancer pathological image classification method and system based on HER2 gene detection |
| CN116492048A (en)* | 2023-04-03 | 2023-07-28 | 中国人民解放军总医院第一医学中心 | Intraoperative pancreatic duct position prediction method and device based on deep learning |
| Publication number | Publication date |
|---|---|
| CN108898175B (en) | 2020-11-20 |
| Publication | Publication Date | Title |
|---|---|---|
| CN108898175B (en) | A computer-aided model construction method based on deep learning for gastric cancer pathological slices | |
| CN109272048B (en) | A pattern recognition method based on deep convolutional neural network | |
| WO2020151536A1 (en) | Brain image segmentation method, apparatus, network device and storage medium | |
| CN106408001B (en) | A fast detection method for regions of interest based on deep kernel hashing | |
| CN113420826B (en) | Liver focus image processing system and image processing method | |
| CN110991254B (en) | Ultrasound image video classification prediction method and system | |
| CN111563897A (en) | Breast nuclear magnetic image tumor segmentation method and device based on weak supervised learning | |
| CN113658151B (en) | Breast lesion magnetic resonance image classification method, equipment and readable storage medium | |
| CN113782184B (en) | A stroke-assisted assessment system based on pre-learning of facial key points and features | |
| CN105976393A (en) | Cerebrum focal lesion automatic detecting method and cerebrum focal lesion automatic detecting device | |
| Al-Masni et al. | A deep learning model integrating FrCN and residual convolutional networks for skin lesion segmentation and classification | |
| JP2023540950A (en) | Multi-arm machine learning model with attention for lesion segmentation | |
| CN117078697A (en) | Fundus disease seed detection method based on cascade model fusion | |
| CN116739988A (en) | Deep learning cerebral hemorrhage classification method based on multi-difficulty course learning | |
| Zhang et al. | Retinal vessel segmentation by a transformer-u-net hybrid model with dual-path decoder | |
| Jahanifar et al. | Automatic recognition of the supraspinatus tendinopathy from ultrasound images using convolutional neural networks | |
| Juang et al. | Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images | |
| Gairola et al. | MLFF-Net: A multi-model late feature fusion network for skin disease classification | |
| Pranav et al. | Comparative study of skin lesion classification using dermoscopic images | |
| Guo et al. | Computer-aided diagnosis of pituitary microadenoma on dynamic contrast-enhanced MRI based on spatio-temporal features | |
| Ye et al. | Improved nested U-structure for accurate nailfold capillary segmentation | |
| Alshalman et al. | Skin Cancer Detection by Using Deep Learning Approach | |
| Yu et al. | Multi-task learning for calcaneus fracture diagnosis of X-ray images | |
| CN115375632A (en) | Lung nodule intelligent detection system and method based on CenterNet model | |
| CN114359308A (en) | Aortic dissection method based on edge response and nonlinear loss |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20201120 |