CN105825509A

Movatterモバイル変換

Info

Publication number: CN105825509A
Application number: CN201610154198.8A
Authority: CN
Inventors: 秦臻; 杨晓明; 蓝天; 秦志光; 陈圆; 徐路路; 陈浩; 肖哲; 王飞; 李雪瑞
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2016-03-17
Filing date: 2016-03-17
Publication date: 2016-08-03

Abstract

Translated fromChinese

本发明公开了一种基于3D卷积神经网络的脑血管分割方法，该方法涉及机器学习、模式识别、图像处理等领域。该方法首先将标记好的脑血管造影图像按序堆叠成3维矩阵，对于其中的血管点取25×25×25大小的patch，然后再随机相同数量、相同大小的非血管点patch，得到训练数据。之后将训练数据输入到3D卷积神经网络用于训练，得到训练模型。然后将实际的血管造影序列图像的每个像素点取25×25×25大小的patch，输入到模型中，得到分类标签，在将分类标签拉伸为相同大小的图像，即为分割血管图像。该方法具有准确率高、泛型程度好的效果。

The invention discloses a method for segmenting cerebral blood vessels based on a 3D convolutional neural network, which relates to the fields of machine learning, pattern recognition, image processing and the like. In this method, the marked cerebral angiography images are firstly stacked into a 3-dimensional matrix, and a patch of 25×25×25 size is taken for the blood vessel points, and then the non-vascular point patches of the same number and size are randomized to obtain training data. Then input the training data into the 3D convolutional neural network for training to obtain the training model. Then, each pixel of the actual angiography sequence image is taken as a patch of 25×25×25 size, and input into the model to obtain the classification label. After stretching the classification label into an image of the same size, it is the segmented blood vessel image. This method has the effect of high accuracy and good generic degree.

Description

Translated fromChinese

基于3D卷积神经网络的脑血管分割方法Cerebrovascular Segmentation Method Based on 3D Convolutional Neural Network

技术领域technical field

本发明属于计算机视觉领域，更为具体地讲，涉及一种基于3D卷积神经网络，针对血管造影图像的血管分割方法。The invention belongs to the field of computer vision, and more specifically relates to a blood vessel segmentation method for angiographic images based on a 3D convolutional neural network.

背景技术Background technique

随着人们生活水平的提高，血管疾病已成为危害人们健康的首要疾病之一。血管是人体中非常重要的器官，一旦发生病变将严重影响着人们的正常生活。因此，血管疾病的早期预防、诊断和治疗凸显重要，借助于现代医学影像手段对血管进行检查、分析以及辅助治疗也成为该领域研究的热点。With the improvement of people's living standards, vascular disease has become one of the primary diseases that endanger people's health. Blood vessels are very important organs in the human body, once pathological changes occur, it will seriously affect people's normal life. Therefore, the early prevention, diagnosis and treatment of vascular diseases are of great importance, and the inspection, analysis and adjuvant treatment of blood vessels with the help of modern medical imaging methods have also become research hotspots in this field.

目前，用于血管疾病检查和诊断的临床技术主要有：基于射线的数字减影血管造影(DigitalSubtractionAngiography,DSA)，基于超声的彩色经颅多普勒成像(ColorTranscranialDoppler,CTD)，磁共振血管造影(MagneticResonanceAngiography,MRA)，计算机断层扫描血管造影(ComputedTomographyAngiography)等以上几种。At present, the clinical techniques used for the examination and diagnosis of vascular diseases mainly include: ray-based digital subtraction angiography (Digital Subtraction Angiography, DSA), ultrasound-based color transcranial Doppler imaging (Color Transcranial Doppler, CTD), magnetic resonance angiography ( Magnetic Resonance Angiography, MRA), computed tomography angiography (Computed Tomography Angiography) and more.

而卷积神经网络在自然图像的分类及分割任务中取得了巨大成功，因此近年来将深度学习的方法应用到医学图像的研究也越来越多。卷积神经网络是人工神经网络的一种，已成为当前语音分析和图像识别领域的研究热点，它的权值共享网络结构使之更类似于生物神经网络，降低了网络模型的复杂度，减少了权值的数量。该优点在网络的输入是多维图像时表现的更为明显，使图像可以直接作为网络的输入，避免了传统识别算法中复杂的特征提取和数据重建过程。卷积网络是为识别二维形状而特殊设计的一个多层感知器，这种网络结构对平移、比例缩放、倾斜或者其他形式的变形具有高度不变性。Convolutional neural networks have achieved great success in the classification and segmentation of natural images. Therefore, in recent years, there have been more and more researches on applying deep learning methods to medical images. Convolutional neural network is a kind of artificial neural network, which has become a research hotspot in the field of speech analysis and image recognition. Its weight sharing network structure makes it more similar to biological neural network, which reduces the complexity of the network model and reduces the the number of weights. This advantage is more obvious when the input of the network is a multi-dimensional image, so that the image can be directly used as the input of the network, avoiding the complicated feature extraction and data reconstruction process in the traditional recognition algorithm. The convolutional network is a multi-layer perceptron specially designed to recognize two-dimensional shapes. This network structure is highly invariant to translation, scaling, tilting, or other forms of deformation.

发明内容Contents of the invention

本发明的目的在于设计一种能够从3个维度提取图像信息的卷积神经网络，用于分割脑血管图像，该方法对传统的2维卷积神经网络做了3D扩展，能够提取相似血管造影图像中更多的隐含信息。The purpose of the present invention is to design a convolutional neural network capable of extracting image information from three dimensions for segmenting cerebrovascular images. This method expands the traditional 2D convolutional neural network in 3D and can extract similar angiography More hidden information in images.

为实现上述目的，本发明一种基于3D卷积神经网络的分类方法，主要包括两个阶段：训练阶段和预测阶段；训练阶段是对带有标签的3维脑血管造影图像输入进3D卷积神经网络，训练网络参数，得到训练模型。预测阶段是根据该训练好的模型预测测试数据，分割脑血管。In order to achieve the above object, a classification method based on a 3D convolutional neural network of the present invention mainly includes two stages: a training stage and a prediction stage; the training stage is to enter a 3D convolutional image input with a label Neural network, train the network parameters, and get the training model. The prediction stage is to predict the test data according to the trained model and segment the cerebral blood vessels.

训练阶段如图1所示，具体技术流程如下：The training phase is shown in Figure 1, and the specific technical process is as follows:

步骤一：首先对脑血管造影图像中的血管点做好标签，然后将标记好的图像堆叠成3维矩阵，对每一个血管点取25×25×25的patch即为正样本，然后在3维矩阵中随机取相同数量的非血管点的path即为负样本；Step 1: First, label the blood vessel points in the cerebral angiography image, then stack the marked images into a 3-dimensional matrix, and take a 25×25×25 patch for each blood vessel point as a positive sample, and then in 3 The path that randomly takes the same number of non-vascular points in the dimensional matrix is a negative sample;

步骤二：输入训练样本，对该样本进行归一化处理，然后进行神经网络的训练；Step 2: Input training samples, normalize the samples, and then train the neural network;

步骤三：在第一层设置20个卷积核，每个卷积核大小为6×6×6，并采用全连接方式与输入层相连进行卷积，得到20个大小为20×20×20的特征map；Step 3: Set 20 convolution kernels in the first layer, each convolution kernel size is 6×6×6, and use the full connection method to connect to the input layer for convolution, and get 20 convolution kernels with a size of 20×20×20 The feature map;

步骤四：对第一层的各个特征map进行空间上的下采样，采样单元为2，得到20个大小为10×10×10的特征map，即为第二层；Step 4: Perform spatial downsampling on each feature map of the first layer, the sampling unit is 2, and obtain 20 feature maps with a size of 10×10×10, which is the second layer;

步骤五：对第二层的各个特征map采用5×5×5大小的3D卷积核进行3D卷积，输出为40个大小为6×6×6的特征map，连接方式采用全连接，此为第三层；Step 5: 3D convolution is performed on each feature map of the second layer with a 3D convolution kernel of size 5×5×5, and the output is 40 feature maps with a size of 6×6×6, and the connection method adopts full connection. for the third layer;

步骤六：对第三层的各个特征map进行空间上的下采样，采样单元为2，得到40个大小为3×3×3的特征map，即为第四层；Step 6: Perform spatial downsampling on each feature map of the third layer, the sampling unit is 2, and obtain 40 feature maps with a size of 3×3×3, which is the fourth layer;

步骤七：对第四层的各个特征map采用3×3×3大小的3D卷积核进行3D卷积，输出为80个大小为1×1×1的特征map，连接方式采用全连接，此为第五层；Step 7: Use a 3×3×3 size 3D convolution kernel to perform 3D convolution on each feature map of the fourth layer, and the output is 80 feature maps with a size of 1×1×1, and the connection method adopts full connection. is the fifth floor;

步骤八：将第五层的各个特征map拉伸成维度为80的特征向量，然后对该特征向量左乘一个128×80的随机参数矩阵，得到一个128维的特征向量，此为第六层；Step 8: Stretch each feature map of the fifth layer into a feature vector with a dimension of 80, and then multiply the feature vector to the left by a 128×80 random parameter matrix to obtain a 128-dimensional feature vector, which is the sixth layer ;

步骤九：将第六层得到的特征向量输入进一个LogisticRegression分类器，输出为一个0到1的浮点数，表示输入样本中心是血管点的概率，此为第七层；Step 9: Input the feature vector obtained in the sixth layer into a LogisticRegression classifier, and the output is a floating-point number from 0 to 1, indicating the probability that the center of the input sample is a blood vessel point, which is the seventh layer;

步骤十：通过BP(反向传播)算法对每一层的计算参数进行调整，使最终预测标签和训练标签的误差函数最小，当误差满足收敛条件时，迭代结束，得到训练模型。Step 10: Adjust the calculation parameters of each layer through the BP (Back Propagation) algorithm to minimize the error function between the final prediction label and the training label. When the error meets the convergence condition, the iteration ends and the training model is obtained.

预测阶段forecast stage

步骤十一：对测试图像的每个像素点进行步骤一中的取25×25×25大小的patch的操作，得到测试样本，将测试样本输入进训练模型，得到预测标签；Step 11: Perform the patch operation of 25×25×25 in step 1 for each pixel of the test image to obtain a test sample, input the test sample into the training model, and obtain the predicted label;

步骤十二：设置一个阈值(如0.8)，将大于等于阈值的预测标签设置为1，将小于阈值的预测标签设置为0，然后将所有的预测标签根据对应位置回复为原本图像大小，即为得到的血管分割图像。Step 12: Set a threshold (such as 0.8), set the predicted labels greater than or equal to the threshold to 1, set the predicted labels smaller than the threshold to 0, and then restore all the predicted labels to the original image size according to the corresponding position, which is The obtained blood vessel segmentation image.

附图说明Description of drawings

图1是3D卷积的示意图；Figure 1 is a schematic diagram of 3D convolution;

图2是3D卷积神经网络的总体结构图。Figure 2 is an overall structure diagram of a 3D convolutional neural network.

下面结合附图对本发明的具体实施方式进行描述，以便本领域的技术人员更好地理解本发明。需要特别提醒注意的是，在以下的描述中，可能淡化本发明主要内容的已知功能和设计的详细描述将被忽略。Specific embodiments of the present invention will be described below in conjunction with the accompanying drawings, so that those skilled in the art can better understand the present invention. It should be noted that in the following description, detailed descriptions of known functions and designs that may dilute the main content of the present invention will be omitted.

在本实施方案中，本发明一种基于3D卷积神经网络的脑血管分割方法主要包括以下环节：1.前向传播、2.反向传播。In this embodiment, a method for cerebrovascular segmentation based on a 3D convolutional neural network in the present invention mainly includes the following links: 1. forward propagation, 2. backpropagation.

其中前向传播过程中3D卷积操作实现如下公式：The 3D convolution operation in the forward propagation process implements the following formula:

${v v}_{i i j j}^{x x y the y z z} = = tanh tanh (({b b}_{i i j j} + + \underset{m m}{Σ Σ} {Σ Σ}_{p p = = 00}^{{P P}_{i i} - - 11} {Σ Σ}_{q q = = 00}^{{Q Q}_{i i} - - 11} {Σ Σ}_{r r = = 00}^{{R R}_{i i} - - 11} {ω ω}_{i i j j m m}^{p p q q r r} {v v}_{((i i - - 11)) m m}^{((x x + + p p)) ((y the y + + q q)) ((z z + + r r))}))$

其中P_i,Q_i,R_i为卷积核的大小，是卷积核连接到前层第m个特征map中坐标为(i,j,m)的参数。Where P_i , Q_i , R_i are the size of the convolution kernel, It is the parameter whose coordinates are (i, j, m) in the mth feature map of the convolution kernel connected to the front layer.

反向传播更新权值使用的是BP算法，3D卷积神经网络使用的BP算法与传统的BP算法不同，且由于卷积神经网络中卷积层和下采样层交替出现，故卷积层和下采样层的误差惩罚项δ的计算是不同的；Backpropagation uses the BP algorithm to update the weights. The BP algorithm used by the 3D convolutional neural network is different from the traditional BP algorithm. Since the convolutional layer and the downsampling layer appear alternately in the convolutional neural network, the convolutional layer and the downsampling layer appear alternately. The calculation of the error penalty term δ for the downsampling layer is different;

对于输出层神经元的误差惩罚项δ为：The error penalty term δ for the output layer neurons is:

δ^L＝f′(u^L)ο(yⁿ-tⁿ)δ^L ＝f′(u^L )ο(yⁿ -tⁿ )

其中，yⁿ表示神经网络实际的输出向量，tⁿ为样本对应的实际标签，L代表最后一层分类层，ο表示点乘。Among them, yⁿ represents the actual output vector of the neural network, tⁿ is the actual label corresponding to the sample, L represents the last classification layer, and ο represents the dot product.

第l层的误差惩罚项如下：The error penalty term of layer l is as follows:

δ^l＝(W^l+1)^Tδ^l+1οf′（u^l)δ^l ＝(W^l+1 )^T δ^l+1 οf′（u^l ）

然后由计算得到的误差惩罚项来计算权值参数的梯度。在向量模式中可以由计算输入向量的内积得到：Then the gradient of the weight parameter is calculated by the calculated error penalty term. In vector mode it can be obtained by computing the inner product of the input vectors:

$\frac{\partial \partial E E.}{\partial \partial {W W}^{l l}} = = {x x}^{l l - - 11} {(({δ δ}^{l l}))}^{T T}$

${ΔW ΔW}^{l l} = = - - η η \frac{\partial \partial E E.}{\partial \partial {W W}^{l l}}$

由此可看出前一层的误差依赖于后一层的误差，即计算梯度是由后层逐步向前层计算的。It can be seen that the error of the previous layer depends on the error of the latter layer, that is, the calculated gradient is calculated by the latter layer step by step.

针对3D卷积神经网络，其具体的卷积层的误差惩罚项的计算如下：For the 3D convolutional neural network, the calculation of the error penalty term of the specific convolutional layer is as follows:

其中，C为常数，表示下采样的尺度，up为上采样函数，即将矩阵每个维度都扩展C倍。Among them, C is a constant, indicating the scale of downsampling, and up is an upsampling function, that is, expanding each dimension of the matrix by C times.

下采样层的误差惩罚项的计算如下：The error penalty term for the downsampling layer is calculated as follows:

其中，conv3表示3维卷积，rot180表示将卷积核旋转180度，full表示卷积边界的方式。Among them, conv3 means 3-dimensional convolution, rot180 means to rotate the convolution kernel by 180 degrees, and full means the way of convolution boundary.

得到每一层的误差惩罚项后，就可以计算参数的梯度：After obtaining the error penalty term of each layer, the gradient of the parameter can be calculated:

$\frac{\partial \partial E E.}{\partial \partial {k k}_{i i j j}^{l l}} = = r r o o t t 180180 ((c c o o n no v v 33 (({x x}_{i i}^{l l - - 11},, r r o o t t 180180 (({δ δ}_{j j}^{l l})) {,,}^{' '} {valid valid}^{' '}))))$

$\frac{\partial \partial E E.}{\partial \partial {b b}_{j j}} = = \underset{u u,, v v}{Σ Σ} {(({δ δ}_{j j}^{l l}))}_{u u v v}$

其中，valid为卷积边界的方式，表示不对边界做任何处理。Among them, valid is the method of convolving the boundary, which means that no processing is done on the boundary.

相比于其他传统的脑血管分割方法如阈值分割、区域生长、主动轮廓，我们的方法能够提取脑血管图像的三维特征，可以达到更好的分割效果。且由于神经网络包含大量参数，故得到的训练模型具有良好的泛化能力，对各种脑血管造影图像(如CTA，MRA)都与较高的分割准确率。Compared with other traditional cerebrovascular segmentation methods such as threshold segmentation, region growing, and active contours, our method can extract 3D features of cerebrovascular images and achieve better segmentation results. And because the neural network contains a large number of parameters, the obtained training model has good generalization ability, and has high segmentation accuracy for various cerebral angiography images (such as CTA, MRA).

Claims

Translated fromChinese

1.本发明一种基于3D卷积神经网络的分类方法，主要包括两个阶段：训练阶段和预测阶段。1. A classification method based on a 3D convolutional neural network of the present invention mainly includes two stages: a training stage and a prediction stage.

步骤一：首先对脑血管造影图像中的血管点做好标签，然后将标记好的图像堆叠成3维矩阵，取25×25×25的patch作为样本；Step 1: First, label the blood vessel points in the cerebral angiography image, then stack the marked images into a 3-dimensional matrix, and take a 25×25×25 patch as a sample;

步骤十：通过BP(反向传播)算法对每一层的计算参数进行调整，使最终预测标签和训练标签的误差函数最小，得到训练模型。Step 10: Adjust the calculation parameters of each layer through the BP (Back Propagation) algorithm to minimize the error function between the final prediction label and the training label, and obtain the training model.

2.根据步骤十得到的训练模型，进行样本预测：2. According to the training model obtained in step 10, perform sample prediction:

步骤十一：对测试图像的每个像素点进行步骤一中的取25×25×25大小的patch的操作，得到测试样本，将测试样本输入进训练模型，得到预测标签；Step 11: Perform the patch operation of 25×25×25 in step 1 on each pixel of the test image to obtain a test sample, input the test sample into the training model, and obtain the predicted label;

步骤十二：设置一个阈值，将大于等于阈值的预测标签设置为1，将小于阈值的预测标签设置为0，然后将所有的预测标签根据对应位置回复为原本图像大小，即为得到的血管分割图像。Step 12: Set a threshold, set the prediction labels greater than or equal to the threshold to 1, set the prediction labels less than the threshold to 0, and then restore all the prediction labels to the original image size according to the corresponding position, which is the obtained blood vessel segmentation image.

3.如权利要求1所述的基于3D卷积神经网络的分类方法，主要是卷积层和池化层的交互堆叠，并通过BP算法训练参数，得到特征表述用于分类。3. the classification method based on 3D convolutional neural network as claimed in claim 1, mainly is the interactive stacking of convolutional layer and pooling layer, and by BP algorithm training parameter, obtains characteristic expression and is used for classification.