Disclosure of Invention
The invention aims to provide a fetal image recognition method and system based on deep learning, which are used for solving the problems of low model training efficiency and high cost in the prior art.
In order to solve the technical problems, the invention specifically provides the following technical scheme:
a fetal image recognition method based on deep learning comprises the following steps:
step S1: acquiring an ultrasonic fetal image with a diagnosis conclusion, preprocessing the ultrasonic fetal image to obtain a preprocessed data set x1 ；
Step S2: for the data set x1 Performing first dimension reduction to obtain a data set x2 The method comprises the following specific steps of:
step S21: calculating the data set x1 A potential spatial dimension, the potential spatial dimension calculation formula being:
；
wherein,for potential spatial dimension, +.>In order to compress the activation parameters of the path,Win order to compress the weight matrix,be is a compressed path deviation;
step S22: according to the potential space dimensionFor the data set x1 Performing first dimension reduction to obtain a data set x2 The calculation formula is as follows:
；
wherein x is2 For the data set obtained by the first dimension reduction,it is the activation of the extension path that,WT transpose of the compression weight matrix;
step S3: for the data set x2 Data processing is carried out to obtain a data set x3 ；
Step S4: for the data set x3 Performing second dimension reduction to obtain a data set x4 ；
Step S5: employing the data set x4 Training a deep learning model to obtain a trained fetal image recognition model;
step S6: step S6: and inputting the image to be predicted into the fetal image recognition model, and outputting a recognition result of the fetal image.
Further, in the step S1, the preprocessing of the ultrasonic fetal image specifically includes: and removing speckle noise of the ultrasonic fetal image by using a smoothing filter to obtain a smoothed denoising image.
Further, in the step S3, the data set x is acquired2 Data processing is carried out to obtain a data set x3 The method comprises the following steps:
step S31, for said dataset x2 Performing Fourier transform;
step S32, selecting a radiation group characteristic, and screening the data set after Fourier transformation according to the radiation group characteristic to obtain a data set x3 。
Further, the radiation group feature includes: first order statistics, gray level co-occurrence matrix, shape-based representation.
Further, in the step S4, the data set x is acquired3 Performing second dimension reduction to obtain a data set x4 The method specifically comprises the following steps: the data set x is subjected to the Laplace eigenmap method3 Dimension reduction is carried out to obtain a data set x4 。
Further, in the step S5, the deep learning model is a UNet deep segmentation network model.
Further, in the step S5, the specific training steps of the deep learning model are as follows:
step S51: integrating the data set x4 Randomly dividing the training group and the testing group;
step S52: training the deep learning model using the training set;
step S53: and inputting the test group into the deep learning model, and testing whether the deep learning model meets the preset precision requirement, if so, completing training to obtain a trained fetal image recognition model, and if not, returning to the step S52, and training the deep learning model again.
A deep learning-based fetal image recognition system using the deep learning-based fetal image recognition method of any one of the above, comprising the following modules:
and a data acquisition module: for obtaining a plurality of ultrasonic fetal images with diagnosis conclusions, preprocessing the ultrasonic fetal images to obtain a preprocessed data set x1 ；
The dimension reduction processing module is used for: is connected with the data acquisition module and is used for acquiring the data set x1 Performing first dimension reduction to obtain a data set x2 For the data set x2 Data processing is carried out to obtain a data set x3 For the data set x3 Performing second dimension reduction to obtain a data set x4 ；
The fetal image recognition model training module: is connected with the dimension reduction processing module and is used for adopting the data set x4 Training a deep learning model to obtain a trained fetal image recognition model;
and a result output module: the fetal image recognition model training module is connected with the fetal image recognition model training module and is used for inputting an image to be predicted into the fetal image recognition model and realizing the recognition of the fetal shape by the fetal image recognition model.
Compared with the prior art, the invention has the following beneficial effects:
firstly, the method performs dimension reduction on the data set twice, and calculates the potential space dimension of the data set through the first pass; reducing the dimension of the data set according to the calculated potential space dimension; related information such as an activation expanding path, a compression weight matrix and the like is considered in the dimension reduction process, and compared with the dimension reduction method in the prior art, the dimension of a data set can be effectively reduced, so that the efficiency of subsequent model training is greatly improved;
secondly, performing Fourier transformation and data screening on the low-dimensional data set to obtain conventional radiological characteristic data when performing second dimension reduction on the data set, and performing dimension reduction processing on the radiological characteristic data set by using a Laplacian eigenmap method to obtain a dimension-reduced data set; then inputting the training data into a deep learning model, so that the training precision of the model is greatly improved;
thirdly, the training data containing the comprehensive information of fetal image recognition is obtained through dimension reduction of the data, and the model is trained, so that the obtained model can recognize each body part of the fetal ultrasonic image, namely one model realizes the comprehensive recognition of the image, and the cost of model training is reduced.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The concepts related to the present application will be described with reference to the accompanying drawings. It should be noted that the following descriptions of the concepts are only for making the content of the present application easier to understand, and do not represent a limitation on the protection scope of the present application; meanwhile, the embodiments and features in the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Example 1
The fetal ultrasound image helps to assess the growth and development of the fetus. In the prior art, a doctor estimates the growth condition of a fetus by visually estimating the shape of the fetus, and the estimation mode has large error. In the embodiment, the fetal shape in the ultrasonic image of the fetus is segmented through image recognition and segmentation of the ultrasonic image of the fetus, so that a doctor is helped to evaluate the growth condition of the fetus.
As shown in fig. 1, the invention provides a fetal image recognition method based on deep learning, which comprises the following steps:
step S1: acquiring an ultrasonic fetal image with a diagnosis conclusion, preprocessing the ultrasonic fetal image to obtain a preprocessed data set x1 。
Specifically, in the step S1, the step of preprocessing the ultrasonic fetal image includes:
the step S11: removing speckle noise of the ultrasonic fetal image by using a smoothing filter to obtain a smoothed denoising image;
the step S12: selecting filters with sizes of 3×3 and 7×7 to apply to the direction smoothing filter;
the S13: denoising all the denoising images through the direction smoothing filter to obtain a data set x1 。
By the denoising step of this step, the visual appearance of the ultrasound image can be improved, and the ultrasound image can be made smoother by applying a smoothing filter to the ultrasound image.
Step S2: for the data set x1 Performing first dimension reduction to obtain a data set x2 。
The method comprises the following specific steps:
first the dataset x is calculated1 Potential spatial dimensions;
in particular, the data set x1 Having a spatial dimension of 512 x 512, the data set x1 To a potential spatial dimension, the potential spatial dimension calculation formula is:
wherein,representing a potential dimension of space that is to be considered,
is an activation parameter of the compressed path,
w represents a compression weight matrix and,
representing a compression path deviation;
then, according to the calculated potential space dimensionFor the data set x1 Performing first dimension reduction to obtain a data set x2 ；
Specifically, from the calculated potential spatial dimensionsFor the data set x1 Performing first dimension reduction to obtain a data set x2 The formula of (2) is:
wherein x is2 For the data set obtained by the first dimension reduction,is an active extension path, +.>Is the transpose of the compression weight matrix.
In contrast to the prior art dimension reduction method, in the present embodiment, the potential spatial dimension is calculated by first calculatingThe data set is then reduced in dimension according to the potential spatial dimensions, and there may beThe dimensionality of the data set is effectively reduced, and the lost image characteristic information is less, so that the efficiency of subsequent model training is greatly improved.
Step S3: for the data set x2 Data processing is carried out to obtain a data set x3 。
For ultrasound images, the radiation features may be classified into different categories, e.g., first order features, including tissue density, shape features (i.e., volume and surface area), and texture features.
Specifically, in the step S3, the low-dimensional dataset x2 Fourier transforming to obtain a conventional radiological characteristic data set x3 The method comprises the following steps: for the low-dimensional dataset x2 Performing Fourier transform, selecting three types of radiological group features, namely first order statistics, gray level co-occurrence matrix and shape-based expression, obtaining 354 radiological features, and storing the 354 radiological features into a specific array to obtain a conventional radiological feature data set x3 。
The first order statistic is a characteristic value calculated directly based on the pixel gray level distribution of the original image;
the gray level co-occurrence matrix is a statistical method for describing the texture features of the image, captures the texture features of the image by analyzing the spatial relationship among pixels in the image and the statistical distribution of gray levels, and can be used for a plurality of computer vision tasks such as image classification, texture recognition, image segmentation and the like;
the shape-based representation is identified from each body part of the fetal ultrasound image.
When the method for extracting the characteristics of the radioactive group is selected, the invention considers that the structure of the fetal ultrasonic image is complex, the fetal ultrasonic image is easy to be influenced by the condition of a parent, the position and the form of the fetus, the connection area between the fetus and the parent is fuzzy and is not easy to be divided, and the difficulty of image analysis is increased.
Step S4: for the data set x3 Performing second dimension reduction to obtain a data set x4 。
Specifically, in the step S4, the data set x is subjected to3 The mode of dimension reduction through the dimension reduction path is as follows: the method of Laplacian eigenmap is adopted for the radiological characteristic data set x3 Dimension reduction is performed to obtain the radiological characteristic data set x with 354 radiological groups3 Reduced to data set x with 12 radial groups4 ；
Step S5: employing the data set x4 Training the deep learning model to obtain a trained fetal image recognition model.
The deep learning model is a UNet deep segmentation network model.
In the neural network structure, the UNet architecture is suitable for dividing various targets or organs in a plurality of medical imaging modes, pixel point types can be predicted through a small number of training pictures, and the data size of medical images is matched with the UNet model in size, so that overfitting can be effectively avoided; thus, the present invention selects the original UNet depth-segmentation network to identify the shape of the fetus.
Specifically, in the step S5, the specific steps of training the deep learning model are as follows:
step S51: integrating the data set x4 Randomly dividing the training group and the testing group;
step S52: training the deep learning model using the training set;
step S53: and inputting the test group into the deep learning model, and testing whether the deep learning model meets the preset precision requirement, if so, completing training to obtain a trained fetal image recognition model, and if not, returning to the step S52, and training the deep learning model again.
Further, the model parameters are training step length, learning rate and other parameters.
Step S6: and inputting the image to be predicted into the fetal image recognition model, and outputting a recognition result of the fetal image.
According to the invention, the original data set is subjected to preprocessing, data set dimension reduction, feature extraction and feature dimension reduction in sequence, so that the data dimension reduction is realized under the condition of less influence on the data, the data volume of the training set used as a training UNet depth segmentation network model is greatly reduced, the calculation cost is reduced, and the calculation speed is accelerated.
Example 2
As shown in fig. 2, the present invention further proposes a fetal image recognition system based on deep learning, using the fetal image recognition method based on deep learning as described in any one of embodiment 1, comprising the following modules:
and a data acquisition module: for obtaining a plurality of ultrasonic fetal images with diagnosis conclusions, preprocessing the ultrasonic fetal images to obtain a preprocessed data set x1 ；
The dimension reduction processing module is used for: is connected with the data acquisition module and is used for acquiring the data set x1 Performing first dimension reduction to obtain a data set x2 For the data set x2 Data processing is carried out to obtain a data set x3 For the data set x3 Performing second dimension reduction to obtain a data set x4 ；
The fetal image recognition model training module: is connected with the dimension reduction processing module and is used for adopting the data set x4 Training a deep learning model to obtain a trained fetal image recognition model;
and a result output module: the fetal image recognition model training module is connected with the fetal image recognition model training module and is used for inputting an image to be predicted into the fetal image recognition model and realizing the recognition of the fetal shape by the fetal image recognition model.
Example 3
This embodiment includes a computer-readable storage medium having stored thereon a data processing program that is executed by a processor to perform the deep learning-based fetal image recognition method of embodiment 1.
It will be apparent to one of ordinary skill in the art that embodiments herein may be provided as a method, apparatus (device), or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Including but not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer, and the like. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The description herein is with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices) and computer program products according to embodiments herein. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the present application. As used in this specification, the terms "a," "an," "the," and/or "the" are not intended to be special references, but rather are intended to include the singular as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method or apparatus comprising such elements.
It should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," and the like indicate an orientation or a positional relationship based on that shown in the drawings, and are merely for convenience of description and simplification of the description, and do not indicate or imply that the apparatus or element in question must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present application. Unless specifically stated or limited otherwise, the terms "mounted," "connected," and the like are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the terms in this application will be understood by those of ordinary skill in the art in a specific context.
The above examples and/or embodiments are merely for illustrating the preferred embodiments and/or implementations of the present technology, and are not intended to limit the embodiments and implementations of the present technology in any way, and any person skilled in the art should be able to make some changes or modifications to the embodiments and/or implementations without departing from the scope of the technical means disclosed in the present disclosure, and it should be considered that the embodiments and implementations are substantially the same as the present technology.
Specific examples are set forth herein to illustrate the principles and embodiments of the present application, and the description of the examples above is only intended to assist in understanding the methods of the present application and their core ideas. The foregoing is merely a preferred embodiment of the present application, and it should be noted that, due to the limited text expressions, there is virtually no limit to the specific structure, and that, for a person skilled in the art, modifications, alterations and combinations of the above described features may be made in an appropriate manner without departing from the principles of the present application; such modifications, variations and combinations, or the direct application of the concepts and aspects of the invention in other applications without modification, are intended to be within the scope of this application.