Detailed Description
The invention is further illustrated by the following specific figures and examples.
As shown in fig. 1, the cardiacparameter acquisition apparatus 100 may include: a first acquisition module 110, a second acquisition module 120, a first processing module 130, and a second processing module 140. In some embodiments, the cardiacparameter acquisition method 200 may be implemented by the cardiacparameter acquisition apparatus 100.
A first obtaining module 110, configured to obtain an ultrasound image sequence, where content corresponding to the ultrasound image sequence includes a heart region.
A second obtaining module 120, configured to obtain preset processing parameters, where the processing parameters at least include: envelope type parameter, scanning type parameter, initial frame parameter.
A first processing module 130, configured to process the ultrasound image sequence according to at least the envelope type parameter and the scanning type parameter to obtain segmented cardiac contour data, where the cardiac contour data at least includes: endocardial curve data, epicardial curve data, and myocardial centerline data.
A second processing module 140, configured to process the segmented cardiac contour data based on at least a preset feature search method and the initial frame parameter, so as to obtain a plurality of measurement points.
The second processing module 140 is further configured to process the plurality of measurement points based on a preset tracking method to obtain a tracking result;
the second processing module 140 is further configured to obtain a cardiac evaluation parameter based on at least the tracking result.
In some embodiments, the first processing module 130 is further configured to process the ultrasound image sequence by using a corresponding preset contour identification method based on the preset envelope type parameter, so as to obtain heart contour data; and processing the heart contour data by adopting a corresponding preset segmentation method based on the preset scanning type parameters to obtain segmented heart contour data.
In some embodiments, the first processing module 130 includes a first segmentation module 131, and the first segmentation module 131 is configured to process the ultrasound image sequence with a segmentation model to obtain cardiac contour data when the envelope type parameter is an automatic envelope type.
In some embodiments, the segmentation model is a machine learning model.
In some embodiments, the first processing module 130 includes a training module 132, the training module 132 configured to obtain a training set, the training set including a sequence of sample ultrasound images and marker data, the marker data including at least one of endocardial curve data, epicardial curve data, and myocardial centerline data; and training the initial model by using the training set to obtain a segmentation model.
In some embodiments, the second processing module 140 is configured to process each segment of the cardiac contour data of an initial frame in the ultrasound image sequence by using a preset feature search method, so as to obtain a plurality of measurement points of the initial frame.
In some embodiments, the second processing module 140 is configured to divide the ultrasound image sequence into an initial frame and a plurality of subsequent frames based on a preset initial frame parameter; based on each measuring point of an initial frame, obtaining corresponding displacement change information of each measuring point of the initial frame in a subsequent frame by using a preset tracking method; and obtaining the speed information of each measuring point of the initial frame based on the time difference between the initial frame and the corresponding subsequent frame.
In some embodiments, the second processing module 140 is configured to obtain a cardiac evaluation parameter based on the corresponding displacement change information and velocity information of each measurement point of the initial frame in a subsequent frame; the cardiac evaluation parameters include at least: myocardial strain information, myocardial strain rate information.
The embodiment of thecardiac analysis apparatus 100 is related to the embodiment of thecardiac analysis method 200, and details thereof are not repeated here, and refer to the related description of fig. 2 to fig. 4.
As shown in fig. 2, the cardiacparameter acquisition method 200 is implemented by the cardiacparameter acquisition apparatus 100. Themethod 200 may include:
step 210: a sequence of ultrasound images may be acquired, the content corresponding to the sequence of ultrasound images including a region of the heart. Acquiring preset processing parameters, wherein the processing parameters can include but are not limited to: envelope type parameter, scan type parameter, initial frame parameter, etc.
In some embodiments, the first acquisition module 110 may acquire a sequence of ultrasound images. In some embodiments, the first obtaining module 110 may obtain the sequence of ultrasound images by way of an ultrasound device (e.g., a color ultrasound device, a black and white ultrasound device, a palm ultrasound device, etc.), a database (e.g., a PACS system), or the like. The corresponding content of the sequence of ultrasound images comprises at least a cardiac region, which may be, for example, an echocardiographic sequence. In some embodiments, the content to which the sequence of ultrasound images corresponds may include at least a left ventricular region. In some embodiments, the first acquisition module 110 may acquire ultrasound video data in which the content corresponding to the sequence of images includes a cardiac region. The first obtaining module 110 may split the ultrasound video data to obtain an ultrasound image sequence. In some embodiments, the first obtaining module 110 may perform sampling processing on the ultrasound image sequence to obtain a sampled ultrasound image sequence. For example, the first acquisition module 110 performs sampling processing on the acquired 1000 frames of ultrasound image sequences by taking 5 as a unit, so as to obtain 200 frames of ultrasound image sequences. In some embodiments, the first acquisition module 110 may acquire electrocardiogram data of an ultrasound image sequence synchronously while acquiring the ultrasound image sequence.
In some embodiments, the second obtaining module 120 may obtain preset processing parameters. The processing parameters are used to set and/or indicate subsequent processing operations. In some embodiments, the envelope type parameters may include at least: an automatic envelope type, a semi-automatic envelope type, a manual envelope type, etc. When the type is an automatic envelope type, thecardiac analysis apparatus 100 automatically performs envelope processing on the ultrasound image sequence. When the envelope type is semi-automatic, thecardiac analysis apparatus 100 sets a plurality of points by a user, and performs envelope processing on an ultrasound image sequence by combining a preset segmentation algorithm. When the envelope is of the manual envelope type, theanalysis apparatus 100 performs envelope processing on the ultrasound image sequence through points set by the user. In some embodiments, the scan type parameter may include at least: apical long-axis pattern type (ALAX), apical two-chamber pattern type (A2C), apical four-chamber pattern type (A4C), PSAXB, PSAXM, PSAXAP, etc. In some embodiments, the corresponding segmentation method may be determined from the scan type parameter. In some embodiments, the processing parameters may also include a segmentation method parameter. The segmentation method parameters may include at least: a first staging method, a second staging method, and a third staging method. Thecardiac analysis apparatus 100 may perform segmentation processing on the envelope processed ultrasound image sequence according to different segmentation methods. In some embodiments, the initial frame parameter may be set automatically by theapparatus 100 or manually by a user. For example, the initial frame parameter may be set to an ultrasound image corresponding to an R-wave of an electrocardiogram. It should be noted that the initial frame parameter may be set according to the actual application, and the present invention is not limited to this.
Step 220: the ultrasound image sequence may be processed at least according to the envelope type parameter and the scanning type parameter to obtain segmented cardiac contour data, where the cardiac contour data at least includes: endocardial curve data, epicardial curve data, and myocardial centerline data.
In some embodiments, the first processing module 130 may envelope the sequence of ultrasound images. In some embodiments, the first processing module 130 may process the ultrasound image sequence by using a corresponding preset contour identification method based on the preset envelope type parameter, so as to obtain heart contour data. In some embodiments, the first processing module 130 may include a first segmentation module 131. In some embodiments, when the envelope type parameter is an automatic envelope type, the first processing module 131 may process the ultrasound image sequence by using a segmentation model to obtain cardiac contour data. In some embodiments, the segmentation model may be a machine learning model. The machine learning model may be a deep learning network model. The deep learning network model may include, but is not limited to: GAN model, VAE model, VGG16 model, VGG19 model, R-CNN model, Fast R-CNN model, Faster R-CNN model, FCN model, etc.
In some embodiments, the first processing module 130 may include a training module 132. In some embodiments, the segmentation model may be obtained by: the training module 132 obtains a training set comprising a sample ultrasound image sequence and marker data comprising at least endocardial curve data, epicardial curve data, and/or myocardial centerline data; the training module 132 trains the initial model using the training set to obtain a segmentation model. In some embodiments, the training module 132 may acquire the sample ultrasound image sequence by way of an ultrasound device (e.g., a color ultrasound device, a black and white ultrasound device, a palm ultrasound device, etc.), a database (e.g., a PACS system), or the like. The sequence of ultrasound images includes at least heart region data, for example, the sequence of ultrasound images may be echocardiographic data. In some embodiments, the training module 132 may acquire sample ultrasound video data in which a sequence of images includes a cardiac region. The training module 132 may perform splitting processing on the sample ultrasound video data to obtain a sample ultrasound image sequence. In some embodiments, the sample ultrasound image sequence may be subjected to preprocessing operations including, but not limited to: denoising processing, enhancing processing, normalizing processing, scaling, rotating or translating the sample and the like. In some embodiments, the marking data includes at least: endocardial curve data, epicardial curve data, and/or myocardial centerline data. The training module 132 generates a training set based on the sample ultrasound image sequence and the labeled image sequence, and trains an initial model using the training set to obtain a segmentation model.
In some embodiments, when the envelope type parameter is a manual envelope type, the first processing module 130 may generate an endocardial envelope curve by a user manually inputting a number of points (for example, inputting at least six points), and the first processing module 130 generates an epicardial curve and a myocardial midline according to the generated endocardial curve. In some embodiments, the first processing module 130 may output and display the generated endocardial curve, epicardial curve and myocardium centerline, and adjust the generated curve points by the user (including the positions of the points on the curve and the thickness of the myocardium between the epicardial curve and the epicardial curve, etc.), so as to make the generated heart contour data more accurate.
In some embodiments, when the envelope type parameter is a semi-automatic envelope type, the first processing module 130 may generate an endocardium envelope curve by a preset segmentation algorithm by inputting a number of points (e.g., inputting three points, which may include a point selected from a cardiac apex point and two cardiac base points) manually by a user, and the first processing module 130 generates an epicardium curve and a myocardial centerline according to the generated endocardium curve. The preset segmentation algorithm may include a morphology algorithm, a level set algorithm, and the like. The segmentation algorithm can evolve according to the energy function of the curve to obtain the heart contour data. In some embodiments, the first processing module 130 may output and display the generated endocardial curve, epicardial curve and myocardium centerline, and adjust the generated curve points by a user (e.g., a doctor) (including the positions of the points on the curve and the thickness of the myocardium between the epicardial curve and the like) so as to make the generated cardiac contour data more accurate.
Step 230: the segmented cardiac contour data may be processed based on at least a preset feature search method and the initial frame parameter to obtain a plurality of measurement points.
In some embodiments, the first processing module 130 may process the cardiac contour data by using a preset segmentation method based on a preset scanning type parameter, so as to obtain segmented cardiac contour data. In some embodiments, the first processing module 130 may select a corresponding segmentation type parameter according to a preset scan type parameter, and perform segmentation processing on the cardiac contour data (e.g., divide the cardiac contour data into several segments). In some embodiments, when the scan type parameter is: when ALAX, A2C, A4C is used, the second processing module 140 selects the first segment type, and can symmetrically divide the heart contour data into seven segments from the left heart base point to the right heart base point. That is, the endocardial curve, epicardial curve and the myocardium midline are symmetrically divided into seven segments for treatment. In some embodiments, the scan type parameter is: when PSAXB or PSAXM is used, the second processing module 140 selects the second segment type, and may divide the heart contour data into six segments symmetrically from the left cardinal point to the right cardinal point. In some embodiments, the scan type parameter is: in PSAXAP, the second processing module 140 selects the third segment type, and may divide the heart contour data into four segments symmetrically from the left cardinal point to the right cardinal point.
In some embodiments, the second processing module 140 may process the segmented cardiac contour data to obtain a plurality of measurement points based on at least a preset feature search method.
In some embodiments, the second processing module 140 may divide the ultrasound image sequence into an initial frame and a subsequent frame based on a preset initial frame parameter. In some embodiments, the second processing module 140 may process each segment of the cardiac contour data of the initial frame in the ultrasound image sequence by using the presetfeature search method 300 to obtain a plurality of measurement points of the initial frame. For the contents of thefeature search method 300, reference may be made to the related description of fig. 3, which is not described herein again.
Step 240: the plurality of measurement points may be processed based on a preset tracking method to obtain a tracking result.
In some embodiments, the second processing module 140 may obtain, based on each measurement point of an initial frame, corresponding displacement change information of each measurement point of the initial frame in a subsequent frame by using a preset tracking method; and obtaining the speed information of each measuring point of the initial frame based on the time difference between the initial frame and the corresponding subsequent frame. In some embodiments, the predetermined tracking method may include, but is not limited to, a block matching method, an optical flow tracking algorithm, and the like. In some embodiments, the second processing module 140 may track, by using an optical flow tracking algorithm, corresponding measurement points of measurement points in a subsequent frame in an initial frame, and separately track, according to the difference in the number of segments in which the measurement points are located, a measurement point pair set in different segments (i.e., a point pair between each measurement point in the initial frame and a corresponding measurement point on each subsequent frame), so as to obtain measurement point displacement variation information in each segment region; and dividing the displacement change information by the time difference between the two image frames of the subsequent frame and the initial frame to obtain the speed information of each measuring point of the initial frame based on the time difference between the initial frame and the corresponding subsequent frame.
Step 250: a cardiac evaluation parameter may be obtained based at least on the tracking results.
In some embodiments, the second processing module 140 may obtain the cardiac evaluation parameter based on the corresponding displacement change information and velocity information of each measurement point of the initial frame for the subsequent frame. In some embodiments, the cardiac evaluation parameters may include at least: myocardial strain information, myocardial strain rate information. In some embodiments, the cardiac evaluation parameters may include: myocardial strain information, myocardial strain rate information, displacement change information of a measurement point, velocity information of a measurement point, and the like. In some embodiments, the second processing module 140 may obtain the myocardial strain information by comparing the length of the whole segment on the cardiac contour data, and divide the myocardial strain information by the time interval to obtain the myocardial strain rate information.
In particular, myocardial strain information may be obtained by relatively varying the distance between different regions of the myocardium. For example, at time t0, there are two arbitrary points P1 and P2 on the heart, the distance between P1 and P2 is L0, at time t1, due to the beating of the heart, the positions of P1 and P2 change, and the positions move to P1 'and P2', respectively, and the distance between P1 'and P2' is L1, so that the corresponding strain information is (L1-L0)/L0, and the strain rate information can be obtained by dividing the strain information by the time interval, that is, the corresponding strain rate information is (L1-L0)/(L0x (t1-t 0)).
In some embodiments, after all the frames of the ultrasound image sequence are processed iteratively, the second processing module 140 performs a smoothing operation on the obtained displacement change information of the measurement point, the velocity information, the myocardial strain information, and the myocardial strain rate information to obtain corresponding curves, and calculates image frame values (corresponding to time values on the electrocardiogram) at peak time of each curve.
As shown in fig. 3, thefeature search method 300 may include:
step 310: the second processing module 140 performs gaussian low-pass filtering on the ultrasound image sequence to obtain a low-pass filtered ultrasound image sequence.
In some embodiments, the second processing module 140 may perform a low-pass filtering process on the ultrasound image sequence by using a gaussian convolution kernel, and the size of the gaussian convolution kernel may be set according to the actual application, for example, the size of the gaussian convolution kernel is 5 × 5.
Step 320: the second processing module 140 performs convolution processing on the ultrasound image sequence after low-pass filtering by using a laplacian operator to obtain an ultrasound image sequence after laplacian processing.
In some embodiments, the second processing module 140 may perform filtering processing on the ultrasound image sequence by using a laplacian, and the size of the laplacian may be set according to an actual application, for example, the size of the laplacian is 5 × 5.
Step 330: the second processing module 140 traverses all positions in the laplacian-processed ultrasound image sequence where the pixel values are greater than the preset threshold, and uses the points corresponding to the obtained positions as measurement points.
In some embodiments, the preset threshold may be set according to practical applications, for example, the preset threshold is set to 30, 50 or the preset threshold is set to be in a range of 60 to 70.
As shown in fig. 4, a cardiacparameter acquisition method 400 is shown below, which cardiacparameter acquisition method 400 may be implemented by thecardiac analysis apparatus 100. The cardiacparameter acquisition method 400 may include:
step 401, the user selects input processing parameters. Instep 401, thecardiac analysis device 100 may acquire the processing parameters input by the user. The processing parameters may include: envelope type parameter, initial frame parameter, scan type parameter, etc. The envelope type parameter may include: including manual envelope types, semi-automatic envelope types, and automatic envelope types.
Step 402, the user loads the left ventricular ultrasound image. Instep 402, thecardiac analysis device 100 may acquire an ultrasound image, for example, a left ventricular ultrasound image. When the left ventricular ultrasound video is acquired, thecardiac analysis device 100 may split the left mr ultrasound video to obtain a left ventricular ultrasound image. In some embodiments, a user may select a specific image frame in the entire video cycle as an initial frame (for example, select an image frame corresponding to an R-wave point obtained from an electrocardiogram as an initial frame), and if a complete cardiac cycle is included from the selected initial frame to a last frame of the video, select an ultrasound image in the complete cardiac cycle from the initial frame; if the selected initial frame to the last frame of the video does not contain a complete cardiac cycle image, all the ultrasonic images from the designated initial frame to the last frame of the video are selected; if the user does not specify, theheart analysis device 100 specifies an initial frame based on a preset setting. For example, thecardiac analysis device 100 determines, from the current video sequence, an image frame corresponding to the first R-wave point obtained from the electrocardiogram as an initial frame.
Instep 403, different envelope processing is performed according to the envelope type parameter. Specifically, instep 403, thecardiac analysis apparatus 100 may perform manual envelope processing, semi-automatic envelope processing, or automatic envelope processing according to different envelope type parameters.
Atstep 404, when the envelope type parameter is a manual envelope type, at least 6 points are input by a user for envelope processing. Theheart analysis device 100 acquires at least 6 points input by the user, and generates an endocardial curve, an epicardial curve, and a myocardial centerline using a segmentation module.
Instep 405, when the envelope type parameter is a semi-automatic envelope type, three points (one apex point and two cardinal points) are input by a user for envelope processing. Theheart analysis device 100 acquires three points (one apex point and two cardinal points) input by the user, and generates endocardial and epicardial curves and a myocardial centerline using a segmentation module.
In theabove step 404 or 405, the segmentation module employs a segmentation algorithm to generate an endocardium curve, an epicardium curve, and a myocardium midline; the segmentation algorithm includes, but is not limited to, morphology, level set, etc., and the segmentation algorithm evolves according to the energy function of the curve to obtain the final segmentation contour.
Instep 406, when the envelope type parameter is an automatic envelope type, thecardiac analysis apparatus 100 performs envelope processing using the segmentation model. Theheart analysis apparatus 100 generates endocardial and epicardial curves and myocardial centerlines using a segmentation model.
Specifically, the segmentation model is implemented instep 414. Instep 414, thecardiac analysis apparatus 100 trains a segmentation model network for generating an endocardial curve, an epicardial curve, and a myocardial centerline of the left ventricle by collecting data. The training process of the segmentation model comprises the following steps: the method comprises the steps of obtaining an ultrasonic image scanned by the left ventricle as sample data, using a left ventricle endocardium curve, an epicardium curve and a myocardium center line which are marked with the sample data by a marking tool as marking data, and dividing the sample data with the marking data into a training set, a verification set and a test set which are respectively used for training, optimizing and testing a segmentation model. The segmentation model uses a left ventricle ultrasonic image as input, outputs a single-channel image with the same size as the input image after a series of operations of forming a network by basic artificial neurons such as convolution, maximum pooling, linear rectification filtering, transposed convolution and the like, and on the output image, the pixel value of the position of an endocardium curve can be set to be 1, the pixel value of the position of an epicardium curve can be set to be 3, the pixel value of the position of a myocardial centerline can be set to be 2, and the rest non-curve parts are 0; the output image is judged by pixel values to obtain the endocardium curve, the epicardium curve and the myocardium central line which correspond to the output image;
instep 407, theheart analysis apparatus 100 outputs heart contour data including endocardial curve, epicardial curve, and myocardium centerline.
Instep 408, thecardiac analysis device 100 performs different segmentation modes according to the scan type parameters. Specifically, instep 408, the cardiac contour data may be divided into a plurality of segments according to the scan type parameter, and the specific number of segments may be determined by the scan type parameter.
Instep 409, when the scanning mode is ALAX, A2C, A4C, theheart analysis device 100 divides the endocardial curve, epicardial curve, and myocardial midline into seven segments.
Instep 410, when the scanning mode is PSAXB or PSAXM, theheart analysis apparatus 100 divides the endocardial curve, the epicardial curve, and the myocardium centerline into six segments.
Instep 411, when the scanning mode is PSAXAP, theheart analysis apparatus 100 divides the endocardial curve, the epicardial curve, and the myocardium midline into four segments.
Instep 412, thecardiac analysis apparatus 100 performs segmentation processing on the endocardium and epicardium curves and the myocardium centerline using different segmentation models, searches all the measurement points in all the segments for the initial frame using a feature search method, tracks all the corresponding measurement points for the subsequent frame using an optical flow tracking algorithm, and calculates corresponding displacement change information, velocity information, myocardium strain information, and myocardium strain rate information.
And step 413, smoothing the corresponding curves, and outputting all the curves and the heart evaluation parameters.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to examples, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.