CN108765369B - Method, apparatus, computer device and storage medium for detecting lung nodule - Google Patents

Abstract

The application provides a method, a device, a computer device and a storage medium for detecting lung nodules, wherein the method comprises the following steps: dividing the lung CT image by a three-dimensional convolutional neural network division model to obtain a lung region image; detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model; and classifying the suspicious nodules through a three-dimensional classification network, and removing the false nodules. According to the method, the device, the computer equipment and the storage medium for detecting the lung nodule, the lung CT image is segmented through the three-dimensional convolutional neural network segmentation model, and the lung region image is segmented, so that the segmentation speed is high, and the subsequent detection speed of the lung nodule is increased; meanwhile, the method is applicable to the segmentation of lung areas of all lung CT images.

Description

Method, apparatus, computer device and storage medium for detecting lung nodule

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for detecting a lung nodule, a computer device, and a storage medium.

Background

Pulmonary sarcoidosis (sarcoidosis) is a granulomatous disease of multiple systems and organs with unknown etiology, and has recently attracted widespread domestic attention. Pulmonary sarcoidosis often invades organs such as the lung, bilateral pulmonary lymph nodes, eyes, skin, etc. Pulmonary cell proliferation or foreign bodies can lead to the development of pulmonary nodules that occur in an increasing number of human lungs in an increasingly worse environment. Today, lung nodules are a very common symptom, and many young people need to go to the hospital to remove the lung nodules as early as possible.

The lung nodule part is required to be separated from the lung CT image before the lung nodule is removed, and is usually separated by the clinical experience of doctors at present, or the diagnosis speed is low, more time is required for the doctors, and the accuracy is low; meanwhile, the existing detection method can not divide lung areas for some special lung CT, and the subsequent detection is seriously hindered.

Disclosure of Invention

The main object of the present application is to provide a method, an apparatus, a computer device and a storage medium for detecting lung nodules, which are suitable for segmenting lung areas of all lung CT images, and overcome the defect of slow detection speed in the prior art.

In order to achieve the above object, the present application provides a method for detecting a lung nodule, comprising the steps of:

dividing the lung CT image by a three-dimensional convolutional neural network division model to obtain a lung region image;

detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model;

and classifying the suspicious nodules through a three-dimensional classification network, and removing the false nodules.

Further, the step of segmenting the lung CT image by the three-dimensional convolutional neural network segmentation model to segment the lung region image includes:

the lung CT image is preprocessed to remove image noise.

Further, the step of segmenting the lung CT image by the three-dimensional convolutional neural network segmentation model includes:

and extracting the lung region characteristics in the lung CT image by using a plurality of groups of convolution layers, adding a batch normalization method into each group of convolution layers to carry out convolution on each group of convolution layers, and carrying out up-sampling processing on a convolution result to obtain a lung region image with the same size as the original size of the lung CT image.

Further, the three-dimensional U-Net detection model uses a loss function that is a focal loss function and a regression loss function.

Further, the step of detecting suspicious nodules from the lung region image with a three-dimensional U-Net detection model includes:

sequentially carrying out four times of con-volume and max-volume on the lung region image, and carrying out two times of deconvolution calculation to obtain a first probability map; before the lung region image is subjected to deconvolution calculation for two times, a branch is added, and the two branches are subjected to deconvolution calculation respectively to obtain a corresponding second probability map;

simultaneously inputting the two second probability maps and the first probability map into a back propagation algorithm for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image;

and detecting suspicious nodules from the lung region image according to the final probability map.

Further, the step of classifying the suspicious nodules through a three-dimensional classification network to remove the false nodules includes:

inputting the suspicious nodes into an input layer of a three-dimensional two-classification network, and sequentially passing through five convolution layers to obtain a feature map;

sequentially inputting the feature images to three full-connection layers, inputting the feature images to a softmax layer through the full-connection layers, and finally outputting the feature images through the softmax layer to obtain the probability of suspicious nodule classification, wherein the probability of suspicious nodule classification is the confidence degree of suspicious nodule detection as a lung nodule;

And removing false nodules, wherein the false nodules are suspicious nodules with confidence below a set value.

Further, the step of sequentially inputting the feature map to three full connection layers specifically includes:

the feature map is input to batch normalization layers and sequentially input to three fully connected layers via batch normalization layers.

The application also provides a detection device of lung nodule, include:

the segmentation unit is used for segmenting the lung CT image through a three-dimensional convolutional neural network segmentation model to obtain a lung region image;

the detection unit is used for detecting suspicious nodules from the lung region image through a three-dimensional U-Net detection model;

and the classification unit is used for classifying the suspicious nodules through a three-dimensional classification network and removing the false nodules.

The present application also provides a computer device comprising a memory storing a computer program and a processor implementing the steps of any of the methods described above when the computer program is executed by the processor.

The present application also provides a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of the preceding claims.

The lung nodule detection method, the lung nodule detection device, the computer equipment and the storage medium provided by the application have the following beneficial effects:

according to the method, the device, the computer equipment and the storage medium for detecting the lung nodule, the lung CT image is segmented through the three-dimensional convolutional neural network segmentation model, and the lung region image is segmented, so that the segmentation speed is high, and the subsequent detection speed of the lung nodule is increased; meanwhile, the method is applicable to the segmentation of lung areas of all lung CT images; and detecting suspicious nodules from the lung region image through a three-dimensional U-Net detection model, classifying the suspicious nodules through a three-dimensional two-classification network, removing false nodules, improving the effect of detecting lung nodules, and improving the detection accuracy.

Drawings

FIG. 1 is a schematic diagram of steps in a method for detecting lung nodules in one embodiment of the present application;

FIG. 2 is a schematic diagram of steps in a method for detecting lung nodules in another embodiment of the present application;

FIG. 3 is a schematic view of a lung nodule detection apparatus according to an embodiment of the present application;

FIG. 4 is a block diagram of a detection unit in an embodiment of the present application;

FIG. 5 is a block diagram of a classification unit in another embodiment of the present application;

Fig. 6 is a block diagram schematically illustrating a structure of a computer device according to an embodiment of the present application.

The realization, functional characteristics and advantages of the present application will be further described with reference to the embodiments, referring to the attached drawings.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

Referring to fig. 1, in an embodiment of the present application, a method for detecting a lung nodule is provided, including the following steps:

step S1, segmenting a lung CT image through a three-dimensional convolutional neural network segmentation model to obtain a lung region image;

s2, detecting suspicious nodules from the lung region image through a three-dimensional U-Net detection model;

and step S3, classifying the suspicious nodules through a three-dimensional classification network, and removing the false nodules.

In this embodiment, a hospital uses medical equipment to take CT images of the patient's lungs, which are three-dimensional images. The lung CT image includes not only the lung region image, but also some other tissue or medical equipment image around the lung, which may interfere with the subsequent lung nodule detection process, and also easily enlarges the search area when detecting a lung nodule.

Therefore, as described in step S1 above, in detecting a lung nodule from a patient' S lung CT image, it is generally necessary to first segment the lung CT image and segment the lung region image therefrom in order to reduce interference with other tissues and unnecessary search regions. In the prior art, an image segmentation algorithm (such as threshold segmentation, region segmentation, edge segmentation, histogram method and the like) is generally used for segmenting a lung region image from a lung CT image, but the image segmentation algorithm has low segmentation speed and cannot be suitable for segmenting all lung CT images.

Segmentation of the lung region image can also be achieved by a two-dimensional segmentation model, but the segmentation effect is not ideal. Therefore, in step S1 of the present embodiment, the three-dimensional full convolutional neural network segmentation model is used to segment the lung CT image to realize three-dimensional segmentation, and the lung region image is segmented from the lung CT image, so as to reduce interference of other tissues and unnecessary search regions. The three-dimensional convolutional neural network segmentation model is different from the two-dimensional segmentation model in that the two-dimensional segmentation model segments the three-dimensional lung CT image into two-dimensional images, so that incomplete data expression is necessarily caused, the segmentation effect is influenced, and the two-dimensional segmentation needs to be carried out from multiple dimensions, so that the segmentation speed is lower; and the three-dimensional convolutional neural network segmentation model is used for directly realizing three-dimensional segmentation, and the segmented lung region image is also a three-dimensional image, so that the data expression is comprehensive, the segmentation effect is good, and the segmentation speed is high.

Therefore, in S1 of this step, the segmentation of the lung region image using the three-dimensional convolutional neural network segmentation model has two major advantages over the segmentation using the existing image segmentation algorithm:

(1) the segmentation speed is faster, the traditional image algorithm needs to use 2-8 minutes for segmenting each lung CT image, and the traditional image algorithm only needs 5-10 seconds for segmenting the model by using a three-dimensional convolutional neural network.

(2) The traditional image segmentation algorithm can not segment the lung region of all the lung CT images (for example, some special lung CT images can not be segmented), so that the subsequent detection process is seriously hindered, and the segmentation processing of all the lung CT images can be ensured by using a three-dimensional convolutional neural network segmentation model.

After the step S1, segmenting a lung region image from the lung CT image; suspicious nodules, which are features suspected to be lung nodules, are detected from the lung region images by a three-dimensional U-Net detection model as described in step S2. The two-dimensional U-Net is an existing segmentation model for two-dimensional images, is a semantic segmentation network based on FCN, is suitable for segmenting medical images, can segment three-dimensional images, and has poor segmentation effect, so that the three-dimensional U-Net detection model is used for detecting suspicious nodules in the embodiment. There are other two-dimensional detection models currently available, such as faster-rcnn, ssd, etc., for detecting suspicious nodules; because the lung region image is three-dimensional data, it is obvious that the effect of detecting suspicious nodules by using a three-dimensional detection model is best; thus, the three-dimensional U-Net detection model is used in this embodiment to detect suspicious nodules therefrom. Through experimental comparison, the detection of suspicious nodules by using the three-dimensional U-Net detection model in the application is expected to have better effect on other detection models.

The suspicious nodules detected in the above step S2 include many false nodules (false positives), and if these false nodules are not removed, a lot of unnecessary work is brought to the doctor, so in order to ensure a higher lung nodule detection rate, further screening of the suspicious nodules is required.

In step S3 of this embodiment, the suspicious nodules are further classified by using a three-dimensional classification network, so as to obtain the confidence coefficient of each suspicious nodule, the false nodule with lower confidence coefficient is removed (i.e. the false nodule is removed), and only the true nodule with higher confidence coefficient is reserved, so that the purpose of suppressing false positive is achieved, and the lung nodule detection rate is improved. In step S3, classification may be performed using other two-dimensional classification models, but the classification effect is poor, and classification using a three-dimensional classification network is good.

Referring to fig. 2, in an embodiment, before the step S1 of segmenting the lung region image by using the three-dimensional convolutional neural network segmentation model to segment the lung CT image, the method includes:

step S101, preprocessing the lung CT image to remove image noise.

Since a patient's lung CT image photographed by a hospital using medical equipment has much noise such as bright spots of bones, metal lines of a CT bed, etc., it is necessary to remove noise from the lung CT image before dividing the lung CT image. The pretreatment process is a conventional method, for example, in a specific embodiment, the pretreatment process may be: using 600HU as threshold value to make binarization treatment on lung CT image, because upper and lower regions of lung CT image are connected with external portion, it needs to be removed; thus, the final image pixel value is clipped to [ -1200, 600], and then scaled to [0, 255]; wherein the pixel point of the non-lung area is set at 170.

In other embodiments, the preprocessing described above may also adjust pixel spacing, image contrast, etc. of the CT image of the lung.

In one embodiment, the step S1 of segmenting the lung CT image by using the three-dimensional convolutional neural network segmentation model includes:

and extracting the lung region characteristics in the lung CT image by using a plurality of groups of convolution layers, adding a batch normalization method into each group of convolution layers to carry out convolution on each group of convolution layers, and carrying out up-sampling processing on a convolution result to obtain a lung region image with the same size as the original size of the lung CT image.

In this embodiment, the three-dimensional convolutional neural network partition model structurally has multiple groups of convolutional layers similar to a VGG network (a convolutional neural network), and a batch normalization (batch normalization) layer is added to roll and upsample each convolutional layer. Wherein upsampling is the inverse of downsampling, both upsampling and downsampling are resampling of the digital signal, the resampling sample rate being compared to the sample rate at which the digital signal was originally obtained (e.g., sampled from the analog signal), being greater than the sample rate of the original signal, and smaller than the sample rate, being referred to as downsampling, the upsampling being substantial, i.e., interpolation or interpolation.

In a specific embodiment, 5 groups of convolution layers are used for extracting lung region features in the lung CT image, a batch normalization method is added to each group of convolution layers to carry out convolution of 1X1X1 on each group of convolution layers, and after up-sampling processing is carried out on convolution results, a lung region image with the same size as the original size of the lung CT image is obtained.

In one embodiment, the three-dimensional U-Net detection model in step S2 uses a focal loss function and a regression loss function. The purpose of using the focal loss function is to solve the problem of serious imbalance of positive and negative samples in the training process of the detection model. The simple negative sample has the same contribution to the model as the negative sample of the difficult case, and the focal loss can lead the model to focus on the negative sample distinction of the difficult case, so that the model result has better performance on the FROC curve.

In one embodiment, the step S2 of detecting suspicious nodules from the lung region image through the three-dimensional U-Net detection model includes:

a. the lung region image is subjected to four times of convolution and max pooling, and two times of deconvolution are calculated to obtain a first probability map; before the lung region image is subjected to deconvolution calculation for two times, a branch is added, and the two branches are subjected to deconvolution calculation respectively to obtain a corresponding second probability map;

b. Simultaneously inputting the two second probability maps and the first probability map into a back propagation algorithm for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image;

c. and detecting suspicious nodules from the lung region image according to the final probability map.

The three-dimensional U-Net detection model used in this embodiment is limited to GPU (graphics processor) memory size, requiring sample processing of the segmented lung region images, and in particular, when inputting images, a cube of 128 x 128 is selected, at this time, 70% of the input cubes have a nodule, and the rest 30% are randomly cut, so that the samples comprise background samples. Large lung nodules are fewer in number than small lung nodules, so on removal of the samples, the 40mm diameter nodule samples are expanded 2-6 times for larger than 30mm diameter. At the same time, all samples described above need to be scaled with probability rollover to eliminate the over-fitting problem.

As described in the above step a, in this embodiment, the 128 x 128 cube image subjected to the sample processing is input into a three-dimensional U-Net detection model, after 4 times of con-volume and max-volume calculation, obtaining a first probability map of 32 x 32 by 2 times of deconvolution calculation; simultaneously adding new branches before each deconvolution calculation, and outputting a second probability map of 32 x 32 by each branch through the corresponding deconvolution layer calculation; b, inputting two second probability maps of 32 x 32 and a first probability map of 32 x 32 into a back propagation algorithm at the same time for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image; detecting suspicious nodules from the lung region image according to the final probability map; and if the probability exceeds the preset value, the probability is taken as a suspicious node.

Specifically, when detecting suspicious nodules, it is necessary to identify the foreground and the background, in this embodiment, an IOU (interaction-over-Union) index is used to determine the foreground and the background, where the IOU is an evaluation index of the segmentation model, and is the ratio of the Intersection of the detection area and the real area to the Union, and a larger value represents that the segmentation result of the segmentation model is closer to the real value. Here foreground refers to the target, i.e., lung nodule, with an IOU greater than 0.5 being foreground and less than 0.02 being background. When training the segmentation model, we use this index of the IOU to determine if the current detection window contains lung nodules, i.e. the IOU of the current detection window and the real lung nodules, and consider that the current detection window contains lung nodules when above 0.5, and that the samples between 0.02-0.5 are discarded as negative samples, i.e. background, of less than 0.02. Since the size of the lung nodules is distributed over a larger interval of 3mm-30mm, it is necessary to use different size detection window sizes to detect the lung nodules. Therefore, in the present embodiment, 5 anchors (anchor points) are used in the detection model: 4. 6, 12, 20, 30. The value of Anchor is used to define the size of the detection window, for example Anchor=4 is the detection of suspicious nodules with a detection window size of 4x4x4.

In an embodiment, the step S3 of classifying the suspicious nodes through the three-dimensional classification network and removing the false nodes specifically includes:

s301, inputting the suspicious nodules to an input layer of a three-dimensional two-class network, and sequentially passing through five convolution layers to obtain a feature map;

s302, sequentially inputting the feature map to three full-connection layers, inputting the feature map to a softmax layer through the full-connection layers, and outputting the feature map to obtain the probability of suspicious nodule classification through the softmax layer, wherein the probability of suspicious nodule classification is the confidence degree of suspicious nodule detection as a lung nodule;

s303, removing the false nodules, wherein the false nodules are suspicious nodules with confidence degrees lower than a set value.

Specifically, the step of sequentially inputting the feature map to three full connection layers specifically includes:

the feature map is input to batch normalization layers and sequentially input to three fully connected layers via batch normalization layers.

In this embodiment, the three-dimensional two-class network includes 5 convolutional layers, three fully-connected layers, and one batch normalization layer and softmax layer. In this embodiment, as described in S301, the suspicious nodules detected in step S2 are input into a three-dimensional bisectional network in a cube with a size of 36X20, a feature map is obtained through 5 convolution layers, as described in S302, after the feature map is input into batch normalization layers, the feature map is sequentially input into three full-connected layers, and finally the probability of classification of the suspicious nodules is obtained through softmax output, wherein the probability of classification of the suspicious nodules is used as the confidence that the suspicious nodules are detected as lung nodules. A set value is preset, and when the confidence coefficient is higher than the set value, the lung nodule is judged; if the confidence is lower than the set value, the false node is judged to be the false node, namely the false positive, and the false positive is required to be removed, so that the aim of inhibiting the false positive is fulfilled, the detection rate of the lung node is ensured, and the workload of doctors is reduced.

In another embodiment, after detecting the lung nodule in the lung CT image through the detection process of the above embodiment, the position of the lung nodule may be determined, the three-dimensional shape of the lung nodule may be depicted in the three-dimensional image, the size of the lung nodule may be calculated, and the monitored information may be sent to a medical computer device for display, so that a doctor may make a reasonable treatment plan according to the information. Because the size, shape, position and other information of the lung nodule are different, the doctor will have different treatment schemes, so that the information is acquired, which is beneficial to the doctor to make a reasonable treatment scheme. Finally, the patient's above-mentioned conditions and the corresponding treatment regimen may also be recorded in a database for storage.

In yet another embodiment, after detecting a lung nodule, determining the location, shape and size of the lung nodule, and matching similar/similar cases in a database of historic diagnosis based on the above information, so as to refer to the database for accurate diagnosis by a doctor, analyze similar cases, and also facilitate analysis of the lung nodule disease.

In summary, in the method for detecting a lung nodule provided in the embodiments of the present application, a three-dimensional convolutional neural network segmentation model is used to segment a lung CT image, so as to segment a lung region image, so that the segmentation speed is fast, and the subsequent detection speed of the lung nodule is fast; meanwhile, the method is applicable to the segmentation of lung areas of all lung CT images; and detecting suspicious nodules from the lung region image through a three-dimensional U-Net detection model, classifying the suspicious nodules through a three-dimensional two-classification network, removing false nodules, improving the effect of detecting lung nodules, and improving the detection accuracy.

Referring to fig. 3, there is further provided in an embodiment of the present application a device for detecting a lung nodule, including:

thesegmentation unit 10 is used for segmenting the lung CT image through a three-dimensional convolutional neural network segmentation model to obtain a lung region image;

adetection unit 20 for detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model;

and theclassification unit 30 is used for classifying the suspicious nodules through a three-dimensional classification network and removing the false nodules.

In this embodiment, a hospital uses medical equipment to take CT images of the patient's lungs, which are three-dimensional images. The lung CT image includes not only the lung region image, but also some other tissue or medical equipment image around the lung, which may interfere with the subsequent lung nodule detection process, and also easily enlarges the search area when detecting a lung nodule. Therefore, in detecting a lung nodule from a lung CT image of a patient, in order to reduce interference of other tissues and unnecessary search areas, thesegmentation unit 10 generally needs to segment the lung CT image first, from which a lung region image is segmented. In the prior art, an image segmentation algorithm (such as threshold segmentation, region segmentation, edge segmentation, histogram method and the like) is generally used for segmenting a lung region image from a lung CT image, but the image segmentation algorithm has low segmentation speed and cannot be suitable for segmenting all lung CT images.

Segmentation of the lung region image can also be achieved by a two-dimensional segmentation model, but the segmentation effect is not ideal. Therefore, thesegmentation unit 10 of the present embodiment uses the segmentation model of the three-dimensional full convolutional neural network to segment the lung CT image to realize three-dimensional segmentation, and segments the lung region image from the lung CT image, thereby reducing interference of other tissues and unnecessary search regions. The three-dimensional convolutional neural network segmentation model is different from the two-dimensional segmentation model in that the two-dimensional segmentation model segments the three-dimensional lung CT image into two-dimensional images, so that incomplete data expression is necessarily caused, the segmentation effect is influenced, and the two-dimensional segmentation needs to be carried out from multiple dimensions, so that the segmentation speed is lower; and the three-dimensional convolutional neural network segmentation model is used for directly realizing three-dimensional segmentation, and the segmented lung region image is also a three-dimensional image, so that the data expression is comprehensive, the segmentation effect is good, and the segmentation speed is high.

Therefore, the segmentation of the lung region image using the three-dimensional convolutional neural network segmentation model in thesegmentation unit 10 has two major advantages over segmentation using the existing image segmentation algorithm:

(1) the segmentation speed is faster, the traditional image algorithm needs to use 2-8 minutes for segmenting each lung CT image, and the traditional image algorithm only needs 5-10 seconds for segmenting the model by using a three-dimensional convolutional neural network.

(2) The traditional image segmentation algorithm can not segment the lung region of all the lung CT images (for example, some special lung CT images can not be segmented), so that the subsequent detection process is seriously hindered, and the segmentation processing of all the lung CT images can be ensured by using a three-dimensional convolutional neural network segmentation model.

After the segmentation by thesegmentation unit 10, a lung region image is segmented from the lung CT image; the detectingunit 20 detects suspicious nodules, which are features suspected to be lung nodules, from the lung region images by using a three-dimensional U-Net detection model. U-Net is an existing segmentation model for two-dimensional images, and other two-dimensional detection models such as a master-rcnn, ssd and the like can be used for detecting suspicious nodules; because the lung region image is three-dimensional data, it is obvious that the effect of detecting suspicious nodules by using a three-dimensional detection model is best; thus, the three-dimensional U-Net detection model is used in this embodiment to detect suspicious nodules therefrom. Through experimental comparison, the detection of suspicious nodules by using the three-dimensional U-Net detection model in the application is expected to have better effect on other detection models.

The suspicious nodules detected by thedetection unit 20 include many false nodules (false positives), and if these false nodules are not removed, a lot of unnecessary work is brought to the doctor, so in order to ensure a high lung nodule detection rate, further screening of the suspicious nodules is required.

Theclassification unit 30 of the embodiment further classifies the suspicious nodules by using a three-dimensional classification network to obtain the confidence coefficient of each suspicious nodule, removes the false nodule with lower confidence coefficient (i.e. removes the false nodule), and only retains the true nodule with higher confidence coefficient, thereby achieving the purpose of suppressing false positive and improving the detection rate of the lung nodule. In step S3, classification may be performed using other two-dimensional classification models, but the classification effect is poor, and classification using a three-dimensional classification network is good.

In an embodiment, the device for detecting a lung nodule further includes:

apreprocessing unit 101, configured to perform preprocessing on the lung CT image to remove image noise.

Since a patient's lung CT image photographed by a hospital using medical equipment has much noise such as bright spots of bones, metal lines of a CT bed, etc., it is necessary to remove noise from the lung CT image before dividing the lung CT image. The pretreatment process is a conventional method, for example, in a specific embodiment, the pretreatment process may be: using 600HU as threshold value to make binarization treatment on lung CT image, because upper and lower regions of lung CT image are connected with external portion, it needs to be removed; thus, the final image pixel value is clipped to [ -1200, 600], and then scaled to [0, 255]; wherein the pixel point of the non-lung area is set at 170.

In other embodiments, thepreprocessing unit 101 may also adjust the pixel interval, the image contrast, etc. of the lung CT image during the preprocessing.

In one embodiment, the dividingunit 10 is specifically configured to:

and extracting the lung region characteristics in the lung CT image by using a plurality of groups of convolution layers, adding a batch normalization method into each group of convolution layers to carry out convolution on each group of convolution layers, and carrying out up-sampling processing on a convolution result to obtain a lung region image with the same size as the original size of the lung CT image.

In this embodiment, the three-dimensional convolutional neural network partition model structurally has multiple groups of convolutional layers similar to the VGG network, and a batch normalization (batch normalization) layer is added to roll and upsample each convolutional layer. Wherein upsampling is the inverse of downsampling, both upsampling and downsampling are resampling of the digital signal, the resampling sample rate being compared to the sample rate at which the digital signal was originally obtained (e.g., sampled from the analog signal), being greater than the sample rate of the original signal, and smaller than the sample rate, being referred to as downsampling, the upsampling being substantial, i.e., interpolation or interpolation.

In a specific embodiment, 5 groups of convolution layers are used for extracting lung region features in the lung CT image, a batch normalization method is added to each group of convolution layers to carry out convolution of 1X1X1 on each group of convolution layers, and after up-sampling processing is carried out on convolution results, a lung region image with the same size as the original size of the lung CT image is obtained.

In one embodiment, the three-dimensional U-Net detection model in thedetection unit 20 uses a focal loss function and a regression loss function. The purpose of using the focal loss function is to solve the problem of serious imbalance of positive and negative samples in the training process of the detection model. The simple negative sample has the same contribution to the model as the negative sample of the difficult case, and the focal loss can lead the model to focus on the negative sample distinction of the difficult case, so that the model result has better performance on the FROC curve.

Referring to fig. 4, in an embodiment, the detectingunit 20 includes:

afirst calculation module 201, configured to sequentially perform four times of accounting and max accounting on the lung region image, and perform two times of deconvolution to obtain a first probability map; before the lung region image is subjected to deconvolution calculation for two times, a branch is added, and the two branches are subjected to deconvolution calculation respectively to obtain a corresponding second probability map;

thesecond calculation module 202 is configured to input the two second probability maps and the first probability map into a back propagation algorithm at the same time to perform iterative calculation, so as to obtain a final probability map, where the final probability map represents the probability of a lung nodule in the lung region image;

And thedetection module 203 is used for detecting suspicious nodules from the lung region image according to the final probability map.

The three-dimensional U-Net detection model used in this embodiment is limited to the GPU memory size, and requires sample processing of the segmented lung region images, specifically, when inputting images, a cube of 128 x 128 is selected, at this time, 70% of the input cubes have a nodule, and the rest 30% are randomly cut, so that the samples comprise background samples. Large lung nodules are fewer in number than small lung nodules, so on removal of the samples, the 40mm diameter nodule samples are expanded 2-6 times for larger than 30mm diameter. At the same time, all samples described above need to be scaled with probability rollover to eliminate the over-fitting problem.

In this embodiment, thefirst calculation module 201 inputs the 128 x 128 cube image subjected to the sample processing to a three-dimensional U-Net detection model, after 4 times of con-volume and max-volume calculation, obtaining a first probability map of 32 x 32 by 2 times of deconvolution calculation; simultaneously adding new branches before each deconvolution calculation, and outputting a second probability map of 32 x 32 by each branch through the corresponding deconvolution layer calculation; thesecond calculation module 202 inputs the two second probability maps of 32 x 32 and the first probability map of 32 x 32 into the back propagation algorithm for iterative calculation, obtaining a final probability map, the final probability map representing the probability of a lung nodule in the lung region image; and detecting suspicious nodules from the lung region image according to the final probability map.

Specifically, when thedetection module 203 detects suspicious nodules, it needs to identify the foreground and the background, and in this embodiment, the foreground and the background are determined using the IOU index, which is an evaluation index of the segmentation model, and is the ratio of the intersection and the union of the detection area and the real area, and the larger the obtained value is the closer the segmentation result of the segmentation model is to the true value. Here foreground refers to the target, i.e., lung nodule, with an IOU greater than 0.5 being foreground and less than 0.02 being background. When training the segmentation model, we use this index of the IOU to determine if the current detection window contains lung nodules, i.e. the IOU of the current detection window and the real lung nodules, and consider that the current detection window contains lung nodules when above 0.5, and that the samples between 0.02-0.5 are discarded as negative samples, i.e. background, of less than 0.02. Since the size of the lung nodules is distributed over a larger interval of 3mm-30mm, it is necessary to use different size detection window sizes to detect the lung nodules. Therefore, in the present embodiment, 5 anchors (anchor points) are used in the detection model: 4. 6, 12, 20, 30. The value of Anchor is used to define the size of the detection window, for example Anchor=4 is the detection of suspicious nodules with a detection window size of 4x4x4.

Referring to fig. 5, in an embodiment, theclassification unit 30 includes:

theconvolution module 301 is configured to input the suspicious nodule to an input layer of a three-dimensional two-class network, and sequentially obtain a feature map through five convolution layers;

theoutput module 302 is configured to sequentially input the feature map to three full-connection layers, input the feature map to a softmax layer through the full-connection layers, and finally output the feature map through the softmax layer to obtain a probability of classifying suspicious nodules, where the probability of classifying suspicious nodules is a confidence level that suspicious nodules are detected as lung nodules;

the removingmodule 303 is configured to remove a false nodule, where the false nodule is a suspicious nodule with a confidence level lower than a set value.

Specifically, theoutputting module 302 inputs the feature map to three fully connected layers in sequence specifically includes:

the feature map is input to batch normalization layers and sequentially input to three fully connected layers via batch normalization layers.

In this embodiment, the three-dimensional two-class network includes 5 convolutional layers, three fully-connected layers, and one batch normalization layer and softmax layer. In this embodiment, theconvolution module 301 inputs the detected suspicious nodules into a three-dimensional two-class network in a cube with a size of 36X20, obtains a feature map through 5 convolution layers, and theoutput module 302 inputs the feature map into batch normalization layers, sequentially inputs the feature map into three full-connection layers, and finally outputs the probability of classification of the suspicious nodules through softmax, wherein the probability of classification of the suspicious nodules is used as the confidence of the suspicious nodules being detected as lung nodules. A set value is preset, and when the confidence coefficient is higher than the set value, the lung nodule is judged; if the confidence level is lower than the set value, the lung nodule is judged to be a false nodule, namely the false positive, and the false positive is required to be removed through theremoval module 303, so that the purpose of suppressing the false positive is achieved, the detection rate of the lung nodule is ensured, and the workload of a doctor is reduced.

In another embodiment, after detecting the lung nodule in the lung CT image through the detection process of the above embodiment, the position of the lung nodule may be determined, the three-dimensional shape of the lung nodule may be depicted in the three-dimensional image, the size of the lung nodule may be calculated, and the monitored information may be sent to a medical computer device for display, so that a doctor may make a reasonable treatment plan according to the information. Because the size, shape, position and other information of the lung nodule are different, the doctor will have different treatment schemes, so that the information is acquired, which is beneficial to the doctor to make a reasonable treatment scheme. Finally, the patient's above-mentioned conditions and the corresponding treatment regimen may also be recorded in a database for storage.

In yet another embodiment, after detecting a lung nodule, determining the location, shape and size of the lung nodule, and matching similar/similar cases in a database of historic diagnosis based on the above information, so as to refer to the database for accurate diagnosis by a doctor, analyze similar cases, and also facilitate analysis of the lung nodule disease.

In summary, in the lung nodule detection device provided in the embodiment of the present application, thesegmentation unit 10 segments the lung CT image through the three-dimensional convolutional neural network segmentation model, so as to segment the lung region image, so that the segmentation speed is fast, and the subsequent detection speed of the lung nodule is fast; meanwhile, the method is applicable to the segmentation of lung areas of all lung CT images; thedetection unit 20 detects suspicious nodules from the lung region image through a three-dimensional U-Net detection model, and theclassification unit 30 classifies the suspicious nodules through a three-dimensional classification network, so that false nodules are removed, the effect of detecting lung nodules is improved, and the detection accuracy is improved.

Referring to fig. 6, a computer device is further provided in the embodiment of the present application, where the computer device may be a server, and the internal structure of the computer device may be as shown in fig. 6. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer equipment is used for storing data such as a three-dimensional convolutional neural network segmentation model and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of detecting lung nodules.

The processor executes the steps of the method for detecting a lung nodule: dividing the lung CT image by a three-dimensional convolutional neural network division model to obtain a lung region image; detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model; and classifying the suspicious nodules through a three-dimensional classification network, and removing the false nodules.

In one embodiment, the processor segments the lung CT image with a three-dimensional convolutional neural network segmentation model, and before the step of segmenting the lung region image, the processor comprises:

the lung CT image is preprocessed to remove image noise.

In one embodiment, the step of segmenting the lung CT image by the processor through a three-dimensional convolutional neural network segmentation model includes:

and extracting the lung region characteristics in the lung CT image by using a plurality of groups of convolution layers, adding a batch normalization method into each group of convolution layers to carry out convolution on each group of convolution layers, and carrying out up-sampling processing on a convolution result to obtain a lung region image with the same size as the original size of the lung CT image.

In one embodiment, the three-dimensional U-Net detection model uses a loss function that is a focal loss function and a regression loss function.

In one embodiment, the step of the processor detecting suspicious nodules from the lung region images using a three-dimensional U-Net detection model includes:

sequentially carrying out four times of con-volume and max-volume on the lung region image, and carrying out two times of deconvolution calculation to obtain a first probability map; before the lung region image is subjected to deconvolution calculation for two times, a branch is added, and the two branches are subjected to deconvolution calculation respectively to obtain a corresponding second probability map;

Simultaneously inputting the two second probability maps and the first probability map into a back propagation algorithm for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image;

and detecting suspicious nodules from the lung region image according to the final probability map.

In one embodiment, the step of classifying the suspicious nodules by the processor over a three-dimensional classification network to remove false nodules comprises:

inputting the suspicious nodes into an input layer of a three-dimensional two-classification network, and sequentially passing through five convolution layers to obtain a feature map;

sequentially inputting the feature images to three full-connection layers, inputting the feature images to a softmax layer through the full-connection layers, and finally outputting the feature images through the softmax layer to obtain the probability of suspicious nodule classification, wherein the probability of suspicious nodule classification is the confidence degree of suspicious nodule detection as a lung nodule;

and removing false nodules, wherein the false nodules are suspicious nodules with confidence below a set value.

In an embodiment, the step of sequentially inputting the feature map to three fully connected layers by the processor specifically includes:

the feature map is input to batch normalization layers and sequentially input to three fully connected layers via batch normalization layers.

Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of a portion of the architecture in connection with the present application and is not intended to limit the computer device to which the present application is applied.

An embodiment of the present application further provides a computer storage medium, on which a computer program is stored, where the computer program when executed by a processor implements a method for detecting a lung nodule, specifically: dividing the lung CT image by a three-dimensional convolutional neural network division model to obtain a lung region image; detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model; and classifying the suspicious nodules through a three-dimensional classification network, and removing the false nodules.

In one embodiment, the processor segments the lung CT image with a three-dimensional convolutional neural network segmentation model, and before the step of segmenting the lung region image, the processor comprises:

the lung CT image is preprocessed to remove image noise.

In one embodiment, the step of segmenting the lung CT image by the processor through a three-dimensional convolutional neural network segmentation model includes:

and extracting the lung region characteristics in the lung CT image by using a plurality of groups of convolution layers, adding a batch normalization method into each group of convolution layers to carry out convolution on each group of convolution layers, and carrying out up-sampling processing on a convolution result to obtain a lung region image with the same size as the original size of the lung CT image.

In one embodiment, the three-dimensional U-Net detection model uses a loss function that is a focal loss function and a regression loss function.

In one embodiment, the step of the processor detecting suspicious nodules from the lung region images using a three-dimensional U-Net detection model includes:

sequentially carrying out four times of con-volume and max-volume on the lung region image, and carrying out two times of deconvolution calculation to obtain a first probability map; before the lung region image is subjected to deconvolution calculation for two times, a branch is added, and the two branches are subjected to deconvolution calculation respectively to obtain a corresponding second probability map;

simultaneously inputting the two second probability maps and the first probability map into a back propagation algorithm for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image;

and detecting suspicious nodules from the lung region image according to the final probability map.

In one embodiment, the step of classifying the suspicious nodules by the processor over a three-dimensional classification network to remove false nodules comprises:

inputting the suspicious nodes into an input layer of a three-dimensional two-classification network, and sequentially passing through five convolution layers to obtain a feature map;

Sequentially inputting the feature images to three full-connection layers, inputting the feature images to a softmax layer through the full-connection layers, and finally outputting the feature images through the softmax layer to obtain the probability of suspicious nodule classification, wherein the probability of suspicious nodule classification is the confidence degree of suspicious nodule detection as a lung nodule;

and removing false nodules, wherein the false nodules are suspicious nodules with confidence below a set value.

In an embodiment, the step of sequentially inputting the feature map to three fully connected layers by the processor specifically includes:

the feature map is input to batch normalization layers and sequentially input to three fully connected layers via batch normalization layers.

In summary, in the method, the device, the computer equipment and the storage medium for detecting the lung nodule provided in the embodiments of the present application, the three-dimensional convolutional neural network segmentation model is used to segment the lung CT image, so as to segment the lung region image, and the segmentation speed is fast, so that the speed of subsequently detecting the lung nodule is increased; meanwhile, the method is applicable to the segmentation of lung areas of all lung CT images; and detecting suspicious nodules from the lung region image through a three-dimensional U-Net detection model, classifying the suspicious nodules through a three-dimensional two-classification network, removing false nodules, improving the effect of detecting lung nodules, and improving the detection accuracy.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by hardware associated with a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium provided herein and used in embodiments may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the claims, and all equivalent structures or equivalent processes using the descriptions and drawings of the present application, or direct or indirect application in other related technical fields are included in the scope of the claims of the present application.

Claims (9)

1. A method for detecting a lung nodule, comprising the steps of:

dividing the lung CT image by a three-dimensional convolutional neural network division model to obtain a lung region image;

Detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model;

classifying the suspicious nodules through a three-dimensional classification network to remove false nodules;

the step of detecting suspicious nodules from the lung region images by a three-dimensional U-Net detection model includes:

sequentially carrying out four convolution and maximum pooling on the lung region image, and carrying out two deconvolution calculation to obtain a first probability map; before the lung region image is subjected to two times of deconvolution calculation, a branch is added, and the two branches are subjected to corresponding deconvolution calculation to obtain a corresponding second probability map;

simultaneously inputting the two second probability maps and the first probability map into a back propagation algorithm for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image;

and detecting suspicious nodules from the lung region image according to the final probability map.

2. The method according to claim 1, wherein the step of segmenting the lung CT image by using a three-dimensional convolutional neural network segmentation model to segment the lung region image comprises:

The lung CT image is preprocessed to remove image noise.

3. The method of claim 1, wherein the step of segmenting the lung CT image using a three-dimensional convolutional neural network segmentation model comprises:

and extracting the lung region characteristics in the lung CT image by using a plurality of groups of convolution layers, adding a batch normalization method into each group of convolution layers to carry out convolution on each group of convolution layers, and carrying out up-sampling processing on a convolution result to obtain a lung region image with the same size as the original size of the lung CT image.

4. The method of claim 1, wherein the three-dimensional U-Net detection model uses a loss function that is a focal loss function and a regression loss function.

5. The method of claim 1, wherein the step of classifying the suspicious nodules by a three-dimensional classification network to remove false nodules comprises:

inputting the suspicious nodes into an input layer of a three-dimensional two-classification network, and sequentially passing through five convolution layers to obtain a feature map;

sequentially inputting the feature images to three full-connection layers, inputting the feature images to a softmax layer through the full-connection layers, and finally outputting the feature images through the softmax layer to obtain the probability of suspicious nodule classification, wherein the probability of suspicious nodule classification is the confidence degree of suspicious nodule detection as a lung nodule;

And removing false nodules, wherein the false nodules are suspicious nodules with confidence below a set value.

6. The method according to claim 5, wherein the step of sequentially inputting the feature map to three fully connected layers comprises:

and inputting the characteristic map to a batch standardization layer, and sequentially inputting the characteristic map to three full-connection layers through the batch standardization layer.

7. A lung nodule detection apparatus, comprising:

the segmentation unit is used for segmenting the lung CT image through a three-dimensional convolutional neural network segmentation model to obtain a lung region image;

the detection unit is used for detecting suspicious nodules from the lung region image through a three-dimensional U-Net detection model;

the classification unit is used for classifying the suspicious nodules through a three-dimensional classification network and removing false nodules;

the detection of suspicious nodules from the lung region images by a three-dimensional U-Net detection model includes:

sequentially carrying out four convolution and maximum pooling on the lung region image, and carrying out two deconvolution calculation to obtain a first probability map; before the lung region image is subjected to two times of deconvolution calculation, a branch is added, and the two branches are subjected to corresponding deconvolution calculation to obtain a corresponding second probability map;

Simultaneously inputting the two second probability maps and the first probability map into a back propagation algorithm for iterative computation to obtain a final probability map, wherein the final probability map represents the probability of a lung nodule in the lung region image;

and detecting suspicious nodules from the lung region image according to the final probability map.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.

9. A computer storage medium having stored thereon a computer program, which when executed by a processor realizes the steps of the method according to any of claims 1 to 6.

Images