技术领域Technical Field
本发明涉及图像处理技术领域，尤其是涉及一种腕部模型的重建方法、装置、电子设备及存储介质。The present invention relates to the field of image processing technology, and in particular to a wrist model reconstruction method, device, electronic equipment and storage medium.
背景技术Background Art
针对膝关节、髋关节等相对结构简单、目标较大的解剖对象，目前相关技术提出面向核磁图像(MRI，Magnetic Resonance Imaging)中相关解剖结构进行三维重建的技术方案，然而对于腕关节中体积较小、形态复杂的精细结构，利用上述技术方案对腕关节进行三维重建的效果较差。For anatomical objects with relatively simple structures and large targets, such as the knee joint and hip joint, current related technologies have proposed technical solutions for three-dimensional reconstruction of related anatomical structures in magnetic resonance imaging (MRI). However, for the small and complex fine structures in the wrist joint, the effect of three-dimensional reconstruction of the wrist joint using the above technical solutions is poor.
发明内容Summary of the invention
有鉴于此，本发明的目的在于提供一种腕部模型的重建方法、装置、电子设备及存储介质，可以显著提高腕部子结构模型的重建精度和重建质量。In view of this, an object of the present invention is to provide a method, device, electronic device and storage medium for reconstructing a wrist model, which can significantly improve the reconstruction accuracy and reconstruction quality of the wrist substructure model.
第一方面，本发明实施例提供了一种腕部模型的重建方法，包括：获取目标腕部结构的腕部图像集合；其中，所述腕部图像集合包括多个图像采集方位处采集的原始腕部图像；对所述腕部图像集合中的多个原始腕部图像进行图像融合处理，得到所述目标腕部结构对应的目标腕部图像；通过预先训练得到的腕部分割网络对所述目标腕部图像进行分割处理，得到所述目标腕部结构对应的目标分割结果；其中，所述目标分割结果包括多个腕部子结构；基于所述目标分割结果重建所述目标腕部结构中各个所述腕部子结构对应的腕部子结构模型。In a first aspect, an embodiment of the present invention provides a method for reconstructing a wrist model, comprising: acquiring a wrist image set of a target wrist structure; wherein the wrist image set includes original wrist images acquired at multiple image acquisition positions; performing image fusion processing on multiple original wrist images in the wrist image set to obtain a target wrist image corresponding to the target wrist structure; performing segmentation processing on the target wrist image by a pre-trained wrist segmentation network to obtain a target segmentation result corresponding to the target wrist structure; wherein the target segmentation result includes multiple wrist substructures; and reconstructing a wrist substructure model corresponding to each of the wrist substructures in the target wrist structure based on the target segmentation result.
在一种实施方式中，所述对所述腕部图像集合中的多个原始腕部图像进行图像融合处理，得到所述目标腕部结构对应的目标腕部图像，包括：针对所述腕部图像集合中的每个原始腕部图像，对所述原始腕部图像进行预处理，得到中间腕部图像；对各个所述中间腕部图像进行图像融合处理，得到所述目标腕部结构对应的目标腕部图像。In one embodiment, the image fusion processing is performed on multiple original wrist images in the wrist image set to obtain the target wrist image corresponding to the target wrist structure, including: for each original wrist image in the wrist image set, preprocessing the original wrist image to obtain an intermediate wrist image; performing image fusion processing on each of the intermediate wrist images to obtain the target wrist image corresponding to the target wrist structure.
在一种实施方式中，所述对所述原始腕部图像进行预处理，包括对所述原始腕部图像执行以下至少一种预处理操作：对所述原始腕部图像的灰度直方图进行调整，以调整所述原始腕部图像的对比度；利用B样条插值算法对所述原始腕部图像进行上采样，以调整所述原始腕部图像的分辨率；从所述原始腕部图像中确定参考图像，并将除所述参考图像之外的其余原始腕部图像向所述参考图像对齐。In one embodiment, the preprocessing of the original wrist image includes performing at least one of the following preprocessing operations on the original wrist image: adjusting the grayscale histogram of the original wrist image to adjust the contrast of the original wrist image; upsampling the original wrist image using a B-spline interpolation algorithm to adjust the resolution of the original wrist image; determining a reference image from the original wrist image, and aligning the remaining original wrist images except the reference image to the reference image.
在一种实施方式中，所述原始腕部图像包括冠位图像、矢位图像和轴位图像；所述从所述原始腕部图像中确定参考图像，并将除所述参考图像之外的其余原始腕部图像向所述参考图像对齐，包括：将所述冠位图像确定为参考图像；根据最小二乘法和所述冠位图像，确定第一变换矩阵和第二变换矩阵；基于所述第一变换矩阵对所述矢位图像进行形变，以使所述矢位图像向所述冠位图像对齐；以及，基于所述第二变换矩阵对所述轴位图像进行形变，以使所述轴位图像向所述冠位图像对齐。In one embodiment, the original wrist image includes a coronal image, a sagittal image and an axial image; determining a reference image from the original wrist image and aligning the remaining original wrist images except the reference image to the reference image includes: determining the coronal image as the reference image; determining a first transformation matrix and a second transformation matrix based on the least squares method and the coronal image; deforming the sagittal image based on the first transformation matrix to align the sagittal image to the coronal image; and deforming the axial image based on the second transformation matrix to align the axial image to the coronal image.
在一种实施方式中，所述对各个所述中间腕部图像进行图像融合处理，得到所述目标腕部结构对应的目标腕部图像，包括：提取每个所述中间腕部图像的多个维度的初始小波分量；对于每个维度的初始小波分量，根据预设评价函数确定对所述维度的初始小波分量进行融合时，基于各个所述中间腕部图像对应的权重系数，对所述维度的初始小波分量进行加权求和，得到所述维度的融合后小波分量；对每个所述维度的融合后小波分量进行小波反变换处理，得到所述目标腕部结构对应的目标腕部图像。In one embodiment, the image fusion processing is performed on each of the intermediate wrist images to obtain the target wrist image corresponding to the target wrist structure, including: extracting initial wavelet components of multiple dimensions of each of the intermediate wrist images; for the initial wavelet components of each dimension, determining to fuse the initial wavelet components of the dimension according to a preset evaluation function, performing weighted summation on the initial wavelet components of the dimension based on the weight coefficients corresponding to each of the intermediate wrist images to obtain the fused wavelet components of the dimension; and performing inverse wavelet transform processing on the fused wavelet components of each dimension to obtain the target wrist image corresponding to the target wrist structure.
在一种实施方式中，所述腕部分割网络包括二维图像分割子网络和分割结果三维优化子网络，所述目标腕部图像包括多个二维切片；所述通过预先训练得到的腕部分割网络对所述目标腕部图像进行分割处理，得到所述目标腕部结构对应的目标分割结果，包括：通过所述二维图像分割子网络对每个二维切片进行分割处理，得到初始分割结果；通过所述分割结果三维优化子网络，基于所述二维切片在所述目标腕部图像中的边缘轮廓，修正所述初始分割结果得到目标分割结果；其中，所述腕部子结构包括尺骨远端分割结果、桡骨远端分割结果和TFCC结构分割结果中的一种或多种。In one embodiment, the wrist segmentation network includes a two-dimensional image segmentation subnetwork and a three-dimensional optimization subnetwork for segmentation results, and the target wrist image includes multiple two-dimensional slices; the wrist segmentation network obtained through pre-training performs segmentation processing on the target wrist image to obtain a target segmentation result corresponding to the target wrist structure, including: performing segmentation processing on each two-dimensional slice through the two-dimensional image segmentation subnetwork to obtain an initial segmentation result; through the three-dimensional optimization subnetwork for segmentation results, based on the edge contour of the two-dimensional slice in the target wrist image, correcting the initial segmentation result to obtain the target segmentation result; wherein, the wrist substructure includes one or more of the distal ulna segmentation result, the distal radius segmentation result and the TFCC structure segmentation result.
在一种实施方式中，所述腕部分割网络的训练步骤，包括：获取训练图像；其中，所述训练图像包括多个训练二维切片；从所述训练二维切片中选取第一切片和第二切片；确定所述第一切片对应的第一标签，并基于所述第一切片和所述第一标签对所述二维图像分割子网络进行训练；通过训练后的所述二维图像分割子网络对所述第二切片进行分割处理，得到所述第二切片的初始分割结果；基于所述第二切片的初始分割结果确定所述第二切片对应的第二标签，并基于所述第二切片的初始分割结果和所述第二标签对所述分割结果三维优化子网络进行训练。In one embodiment, the training steps of the wrist segmentation network include: acquiring a training image; wherein the training image includes multiple training two-dimensional slices; selecting a first slice and a second slice from the training two-dimensional slices; determining a first label corresponding to the first slice, and training the two-dimensional image segmentation subnetwork based on the first slice and the first label; performing segmentation processing on the second slice by the trained two-dimensional image segmentation subnetwork to obtain an initial segmentation result of the second slice; determining a second label corresponding to the second slice based on the initial segmentation result of the second slice, and training the segmentation result three-dimensional optimization subnetwork based on the initial segmentation result of the second slice and the second label.
第二方面，本发明实施例还提供一种腕部模型的重建装置，包括：图像获取模块，用于获取目标腕部结构的腕部图像集合；其中，所述腕部图像集合包括多个图像采集方位处采集的原始腕部图像；融合模块，用于对所述腕部图像集合中的多个原始腕部图像进行图像融合处理，得到所述目标腕部结构对应的目标腕部图像；分割模块，用于通过预先训练得到的腕部分割网络对所述目标腕部图像进行分割处理，得到所述目标腕部结构对应的目标分割结果；其中，所述目标分割结果包括多个腕部子结构；重建模块，用于基于所述目标分割结果重建所述目标腕部结构中各个所述腕部子结构对应的腕部子结构模型。In a second aspect, an embodiment of the present invention further provides a device for reconstructing a wrist model, comprising: an image acquisition module, used to acquire a wrist image set of a target wrist structure; wherein the wrist image set includes original wrist images acquired at multiple image acquisition positions; a fusion module, used to perform image fusion processing on multiple original wrist images in the wrist image set to obtain a target wrist image corresponding to the target wrist structure; a segmentation module, used to perform segmentation processing on the target wrist image through a pre-trained wrist segmentation network to obtain a target segmentation result corresponding to the target wrist structure; wherein the target segmentation result includes multiple wrist substructures; and a reconstruction module, used to reconstruct a wrist substructure model corresponding to each of the wrist substructures in the target wrist structure based on the target segmentation result.
第三方面，本发明实施例还提供一种电子设备，包括处理器和存储器，所述存储器存储有能够被所述处理器执行的计算机可执行指令，所述处理器执行所述计算机可执行指令以实现第一方面提供的任一项所述的腕部模型的重建方法。In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the wrist model reconstruction methods provided in the first aspect.
第四方面，本发明实施例还提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机可执行指令，所述计算机可执行指令在被处理器调用和执行时，计算机可执行指令促使处理器实现第一方面提供的任一项所述的腕部模型的重建方法。In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the wrist model reconstruction methods provided in the first aspect.
本发明实施例提供的一种腕部模型的重建方法、装置、电子设备及存储介质，首先获取目标腕部结构的腕部图像集合，该腕部图像集合包括多个图像采集方位处采集的原始腕部图像，然后对腕部图像集合中的多个原始腕部图像进行图像融合处理，得到目标腕部结构对应的目标腕部图像，再通过预先训练得到的腕部分割网络对目标腕部图像进行分割处理，得到目标腕部结构对应的目标分割结果(包括多个腕部子结构)，最后基于目标分割结果重建目标腕部结构中各个腕部子结构对应的腕部子结构模型。上述方法对腕部图像集合中多个图像采集方位处采集的原始腕部图像进行图像融合处理，可以得到分辨率较高、结构连续性更强的目标腕部图像，然后利用腕部分割模型可以更为精确地分割目标腕部图像得到目标分割结果，最后基于该目标分割结果即可得到精度和质量均较高的腕部子结构模型，从而显著提高了腕部子结构模型的重建精度和重建质量。The embodiment of the present invention provides a wrist model reconstruction method, device, electronic device and storage medium, firstly obtains a wrist image set of a target wrist structure, the wrist image set includes original wrist images collected at multiple image collection positions, then performs image fusion processing on multiple original wrist images in the wrist image set to obtain a target wrist image corresponding to the target wrist structure, then performs segmentation processing on the target wrist image through a wrist segmentation network obtained by pre-training to obtain a target segmentation result (including multiple wrist substructures) corresponding to the target wrist structure, and finally reconstructs a wrist substructure model corresponding to each wrist substructure in the target wrist structure based on the target segmentation result. The above method performs image fusion processing on the original wrist images collected at multiple image collection positions in the wrist image set, and can obtain a target wrist image with higher resolution and stronger structural continuity, and then uses the wrist segmentation model to more accurately segment the target wrist image to obtain a target segmentation result, and finally based on the target segmentation result, a wrist substructure model with high accuracy and quality can be obtained, thereby significantly improving the reconstruction accuracy and reconstruction quality of the wrist substructure model.
本发明的其他特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本发明而了解。本发明的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
图1为本发明实施例提供的一种腕部模型的重建方法的流程示意图；FIG1 is a schematic flow chart of a method for reconstructing a wrist model provided by an embodiment of the present invention;
图2为本发明实施例提供的一种原始腕部图像的示意图；FIG2 is a schematic diagram of an original wrist image provided by an embodiment of the present invention;
图3为本发明实施例提供的另一种原始腕部图像的示意图；FIG3 is a schematic diagram of another original wrist image provided by an embodiment of the present invention;
图4为本发明实施例提供的一种离散小波变换算法的原理示意图；FIG4 is a schematic diagram of the principle of a discrete wavelet transform algorithm provided by an embodiment of the present invention;
图5为本发明实施例提供的一种腕部分割网络的结构示意图；FIG5 is a schematic diagram of the structure of a wrist segmentation network provided by an embodiment of the present invention;
图6为本发明实施例提供的另一种腕部模型的重建方法的流程示意图；FIG6 is a schematic flow chart of another wrist model reconstruction method provided by an embodiment of the present invention;
图7为本发明实施例提供的一种腕部模型的重建装置的结构示意图；FIG7 is a schematic structural diagram of a wrist model reconstruction device provided by an embodiment of the present invention;
图8为本发明实施例提供的一种电子设备的结构示意图。FIG8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.
具体实施方式DETAILED DESCRIPTION
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合实施例对本发明的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
目前，相关技术提出面向核磁图像中相关解剖结构进行三维重建的技术方案，主要包括图像预处理、基于传统图像处理方法或手动描绘方法的图像分割、三维重建及物体表面网格化等几个关键技术步骤。但是，该技术方案无法实现对腕关节解剖结构的三维重建，特别是腕部一些体积较小、形态复杂的精细结构，诸如三角纤维软骨复合体(TFCC，triangular fibrocartilage complex)等，而精确的腕部解剖结构如TFCC的三维重建结果直接影响到相关疾病的诊断与治疗。At present, the relevant technology proposes a technical solution for 3D reconstruction of relevant anatomical structures in nuclear magnetic resonance images, which mainly includes several key technical steps such as image preprocessing, image segmentation based on traditional image processing methods or manual drawing methods, 3D reconstruction and object surface meshing. However, this technical solution cannot achieve 3D reconstruction of wrist anatomical structures, especially some small and complex fine structures in the wrist, such as the triangular fibrocartilage complex (TFCC). The accurate 3D reconstruction results of wrist anatomical structures such as TFCC directly affect the diagnosis and treatment of related diseases.
上述技术方案至少存在以下问题：The above technical solution has at least the following problems:
(1)由于腕关节区域骨骼与韧带等解剖结构体积较小，受MRI成像机制限制，Z轴分辨率较低，一般是XY方向分辨率的十分之一(甚至更小)，导致精细结构的提取与三维结构的重建变得困难。(1) Due to the small size of anatomical structures such as bones and ligaments in the wrist joint area and the limitations of the MRI imaging mechanism, the Z-axis resolution is low, generally one-tenth (or even smaller) of the XY-axis resolution, making it difficult to extract fine structures and reconstruct three-dimensional structures.
(2)受MRI图像噪声影响，如运动伪影等，基于传统图像处理技术的分割算法很难精确识别MRI图像中的腕关节解剖结构，如尺骨、桡骨、TFCC等。(2) Affected by MRI image noise, such as motion artifacts, segmentation algorithms based on traditional image processing technology are difficult to accurately identify wrist joint anatomical structures in MRI images, such as the ulna, radius, and TFCC.
(3)采用人工描绘的方式提取腕关节解剖结构成本极高，一般情况下分割一幅256x256x128个像素的三维图像，需要花费一位专业大夫1小时时间，另一方面，长时间的人工分割将引入大量主观分割误差，降低重建质量。(3) It is extremely costly to extract the anatomical structure of the wrist joint by manual delineation. Generally speaking, it takes a professional doctor one hour to segment a 256x256x128 pixel three-dimensional image. On the other hand, long manual segmentation will introduce a large number of subjective segmentation errors, reducing the reconstruction quality.
基于此，本发明实施提供了一种腕部模型的重建方法、装置、电子设备及存储介质，可以显著提高腕部模型的重建精度和重建质量。Based on this, the present invention provides a wrist model reconstruction method, device, electronic device and storage medium, which can significantly improve the reconstruction accuracy and reconstruction quality of the wrist model.
为便于对本实施例进行理解，首先对本发明实施例所公开的一种腕部模型的重建方法进行详细介绍，参见图1所示的一种腕部模型的重建方法的流程示意图，该方法主要包括以下步骤S102至步骤S108：To facilitate understanding of this embodiment, a wrist model reconstruction method disclosed in an embodiment of the present invention is first described in detail. Referring to a flowchart of a wrist model reconstruction method shown in FIG. 1 , the method mainly includes the following steps S102 to S108:
步骤S102，获取目标腕部结构的腕部图像集合。其中，腕部图像集合包括多个图像采集方位处采集的原始腕部图像，原始腕部图像可以采用MRI图像，为三维图像。示例性的，图像采集方位也即采集视角，包括冠位(coronal)、矢位(sagittal)和轴位(axial)，原始腕部图像将包括冠位图像、矢位图像和轴位图像，冠位图像也即在矢位视角下采集的原始腕部图像，矢位图像也即在轴位视角下采集的原始腕部图像，轴位图像也即在冠位视角下采集的原始腕部图像。Step S102, obtaining a wrist image set of the target wrist structure. The wrist image set includes original wrist images collected at multiple image collection orientations, and the original wrist images can be MRI images, which are three-dimensional images. Exemplarily, the image collection orientation, that is, the collection viewing angle, includes coronal, sagittal, and axial. The original wrist images will include coronal images, sagittal images, and axial images. The coronal image is the original wrist image collected at the sagittal viewing angle, the sagittal image is the original wrist image collected at the axial viewing angle, and the axial image is the original wrist image collected at the coronal viewing angle.
在一种实施方式中，可以采用相关的MRI设备采集目标腕部结构的冠位图像、矢位图像和轴位图像。In one embodiment, a related MRI device may be used to acquire coronal images, sagittal images, and axial images of the target wrist structure.
步骤S104，对腕部图像集合中的多个原始腕部图像进行图像融合处理，得到目标腕部结构对应的目标腕部图像。其中，目标腕部图像也为三维图像。Step S104: performing image fusion processing on a plurality of original wrist images in the wrist image set to obtain a target wrist image corresponding to the target wrist structure, wherein the target wrist image is also a three-dimensional image.
在一种实施方式中，可以预先对原始腕部图像进行对比度调整处理、分辨率调整处理或图像配准处理等预处理，通过对原始腕部图像进行分辨率调整处理，可以得到各向同性的三维图像，然后将预处理后的冠位图像、矢位图像和轴位图像进行图像融合处理，即可得到在三维空间各个方向上分辨率高、细节丰富的高质量目标腕部图像，该目标腕部图像可以清晰呈现各处形状细节，以便于腕部分割网络可以分割出目标腕部图像中个精细解剖结构(也即，腕部子结构)。In one embodiment, the original wrist image may be preprocessed in advance by contrast adjustment processing, resolution adjustment processing, or image registration processing. By performing resolution adjustment processing on the original wrist image, an isotropic three-dimensional image may be obtained. Then, the preprocessed coronal image, sagittal image, and axial image are fused to obtain a high-quality target wrist image with high resolution and rich details in all directions of the three-dimensional space. The target wrist image can clearly present the shape details in various places, so that the wrist segmentation network can segment out the fine anatomical structures (i.e., wrist substructures) in the target wrist image.
步骤S106，通过预先训练得到的腕部分割网络对目标腕部图像进行分割处理，得到目标腕部结构对应的目标分割结果。其中，目标分割结果包括多个腕部子结构，腕部分割网络包括二维图像分割子网络和分割结果三维优化子网络，二维图像分割子网络对二维切片进行分割处理，其输入为目标腕部图像的多个二维切片，输出为各个二维切片的初始分割结果，分割结果三维优化子网络用于对初始分割结果进行修正，其输入为初始分割结果，输出为目标分割结果。Step S106, segmenting the target wrist image through the wrist segmentation network obtained by pre-training, and obtaining the target segmentation result corresponding to the target wrist structure. The target segmentation result includes multiple wrist substructures, the wrist segmentation network includes a two-dimensional image segmentation subnetwork and a segmentation result three-dimensional optimization subnetwork, the two-dimensional image segmentation subnetwork performs segmentation processing on the two-dimensional slices, and its input is multiple two-dimensional slices of the target wrist image, and its output is the initial segmentation result of each two-dimensional slice, and the segmentation result three-dimensional optimization subnetwork is used to correct the initial segmentation result, and its input is the initial segmentation result, and its output is the target segmentation result.
在一种实施方式中，可以利用二维图像分割子网络对二维切片进行分割处理得到相应的初始分割结果，再利用分割结果三维优化子网络对初始分割结果进行修正，即可得到目标腕部结构对应的目标分割结果。In one embodiment, the two-dimensional slices can be segmented using a two-dimensional image segmentation subnetwork to obtain a corresponding initial segmentation result, and then the initial segmentation result can be corrected using a three-dimensional optimization subnetwork of the segmentation result to obtain a target segmentation result corresponding to the target wrist structure.
步骤S108，基于目标分割结果重建目标腕部结构各个腕部子结构对应的腕部子结构模型。Step S108: reconstructing the wrist substructure model corresponding to each wrist substructure of the target wrist structure based on the target segmentation result.
在一种实施方式中，可以将所有目标分割结果进行堆叠，然后使用中值滤波操作在三维空间上对所有分割结果进行平滑，最后通过Marching Cubes算法实现腕关节各结构三维闭合表面的提取与网格化，即可得到目标腕部结构的腕部子结构模型。In one implementation, all target segmentation results may be stacked, and then all segmentation results may be smoothed in three-dimensional space using a median filter operation, and finally the three-dimensional closed surfaces of the wrist joint structures may be extracted and meshed using a Marching Cubes algorithm to obtain a wrist substructure model of the target wrist structure.
本发明实施例提供的腕部模型的重建方法，对腕部图像集合中多个图像采集方位处采集的原始腕部图像进行图像融合处理，可以得到分辨率较高、结构连续性更强的目标腕部图像，然后利用腕部分割模型可以更为精确地分割目标腕部图像得到目标分割结果，最后基于该目标分割结果即可得到精度和质量均较高的腕部子结构模型，从而显著提高了腕部子结构模型的重建精度和重建质量。The wrist model reconstruction method provided by the embodiment of the present invention performs image fusion processing on the original wrist images collected at multiple image collection positions in the wrist image set, so as to obtain a target wrist image with higher resolution and stronger structural continuity. Then, the wrist segmentation model can be used to more accurately segment the target wrist image to obtain a target segmentation result. Finally, based on the target segmentation result, a wrist substructure model with higher accuracy and quality can be obtained, thereby significantly improving the reconstruction accuracy and reconstruction quality of the wrist substructure model.
考虑到受MRI成像机制限制，原始腕部图像在Z轴分辨率较低，一般是XY方向分辨率的十分之一甚至更小，不利于精细结构的提取和三维结构的重建，因此在对原始腕部图像进行图像融合处理之前，需要先对其进行预处理，以归一化各原始腕部图像间像素分布与各方向分辨率，得到各向同性的三维图像，基于此，本发明实施例提供了一种前述步骤S104的实施方式，可以参见如下步骤1至步骤2：Considering that the original wrist image is limited by the MRI imaging mechanism, the Z-axis resolution is low, generally one tenth of the XY-direction resolution or even smaller, which is not conducive to the extraction of fine structures and the reconstruction of three-dimensional structures. Therefore, before the original wrist image is fused, it is necessary to pre-process it to normalize the pixel distribution and the resolution in each direction between the original wrist images to obtain an isotropic three-dimensional image. Based on this, the embodiment of the present invention provides an implementation of the aforementioned step S104, which can be referred to as follows: Steps 1 to 2:
步骤1，针对腕部图像集合中的每个原始腕部图像，对原始腕部图像进行预处理，得到中间腕部图像。其中，预处理包括对比度调整处理、分辨率调整处理和图像配准处理中的一种或多种。在一种实施方式中，可以对原始腕部图像执行以下至少一种预处理操作：Step 1: for each original wrist image in the wrist image set, preprocess the original wrist image to obtain an intermediate wrist image. The preprocessing includes one or more of contrast adjustment processing, resolution adjustment processing and image registration processing. In one embodiment, at least one of the following preprocessing operations can be performed on the original wrist image:
(1)对原始腕部图像的灰度直方图进行调整，以调整原始腕部图像的对比度。参见图2所示的一种原始腕部图像的示意图，其中，冠位图像记为图像IC，矢位图像记为图像IS，轴位图像记为图像IA。在一种实施方式中，可以利用限制对比度自适应直方图均衡(CLAHE，Contrast Limited Adaptive Histogram Equalization)算法，调整原始腕部图像的灰度直方图分布，从而提高原始腕部图像的对比度。以冠位图像IC为例，利用CLAHE算法对冠位图像IC的灰度直方图的分布进行调整，即可增大冠位图像IC的对比度。(1) The grayscale histogram of the original wrist image is adjusted to adjust the contrast of the original wrist image. Referring to a schematic diagram of an original wrist image shown in FIG2 , the coronal image is recorded as imageIC , the sagittal image is recorded as imageIS , and the axial image is recorded as imageIA . In one embodiment, the grayscale histogram distribution of the original wrist image can be adjusted using a contrast-limited adaptive histogram equalization (CLAHE) algorithm to improve the contrast of the original wrist image. Taking the coronal imageIC as an example, the contrast of the coronal imageIC can be increased by adjusting the distribution of the grayscale histogram of the coronal imageIC using the CLAHE algorithm.
(2)利用B样条插值算法对原始腕部图像进行上采样，以调整原始腕部图像的分辨率，得到各向同性的三维图像。在一种实施方式中，通过B样条插值算法对原始腕部图像进行上采样，可以使原始腕部图像在Z方向上的分辨率与XY平面上的分辨率一致(～0.1mm)，从而使各个图像采集方位采集的原始腕部图像在该图像采集方位下的平面上的分辨率最高，且另外两个图像采集方位上的分辨率经过上采样后分辨率明显提升。(2) Upsampling the original wrist image using a B-spline interpolation algorithm to adjust the resolution of the original wrist image and obtain an isotropic three-dimensional image. In one embodiment, upsampling the original wrist image using a B-spline interpolation algorithm can make the resolution of the original wrist image in the Z direction consistent with the resolution on the XY plane (~0.1 mm), so that the original wrist image acquired at each image acquisition orientation has the highest resolution on the plane under the image acquisition orientation, and the resolutions at the other two image acquisition orientations are significantly improved after upsampling.
请继续参见图2，以冠位图像为例，冠位图像在XY平面很清晰，但是Z方向扫描图像的间隔较大，因此导致图像较为稀疏，造成冠位图像在Z方向的分辨率较低，利用B样条插值算法，对Z方向相邻的两个冠位图像直接进行插值，比如Z方向两个冠位图像之间的距离为1mm，利用B样条插值算法在两个冠位图像之间插入9副图像，即可使每个图像之间的距离为0.1mm，其效果如图3所示的另一种原始腕部图像的示意图，此时Z方向的分辨率与XY平面的分辨率一致，且Z方向上的图像内容更为连续。Please continue to refer to Figure 2. Taking the coronal image as an example, the coronal image is very clear in the XY plane, but the interval of the scanned image in the Z direction is large, so the image is relatively sparse, resulting in a low resolution of the coronal image in the Z direction. The B-spline interpolation algorithm is used to directly interpolate two adjacent coronal images in the Z direction. For example, the distance between the two coronal images in the Z direction is 1 mm. The B-spline interpolation algorithm is used to insert 9 images between the two coronal images, so that the distance between each image is 0.1 mm. The effect is shown in another schematic diagram of the original wrist image shown in Figure 3. At this time, the resolution in the Z direction is consistent with the resolution in the XY plane, and the image content in the Z direction is more continuous.
(3)从原始腕部图像中确定参考图像，并将除参考图像之外的其余原始腕部图像向参考图像对齐。在一种实施方式中，可以采用基于块匹配的全局配准方法，将其余原始腕部图像向参考图像对齐，具体的，可参见如下(3.1)至(3.3)：(3) Determine a reference image from the original wrist images, and align the remaining original wrist images except the reference image to the reference image. In one embodiment, a global registration method based on block matching can be used to align the remaining original wrist images to the reference image. For details, see (3.1) to (3.3) below:
(3.1)将冠位图像确定为参考图像。(3.1) The coronal image is determined as the reference image.
(3.2)根据最小二乘法和冠位图像，确定第一变换矩阵和第二变换矩阵。其中，第一变换矩阵TS→C和第二变换矩阵TA→C可以为7个自由度的刚性变换矩阵，即旋转(θ，φ，ω)、平移(t_1，t_2，t_3)、与尺度缩放因子(s)，刚性变换矩阵如下所示：(3.2) According to the least squares method and the coronal image, the first transformation matrix and the second transformation matrix are determined. The first transformation matrixTS→C and the second transformation matrixTA→C can be rigid transformation matrices with 7 degrees of freedom, namely rotation (θ, φ, ω), translation (t_1, t_2, t_3), and scale factor (s). The rigid transformation matrix is shown as follows:
在一种实施方式中，可以采用截尾最小二乘法(LTS，Least Trimmed Squares)以循环迭代的优化方式求解最优的第一变换矩阵和第二变换矩阵。In one implementation, the Least Trimmed Squares (LTS) method may be used to solve the optimal first transformation matrix and the second transformation matrix in a cyclic iterative optimization manner.
(3.3)基于第一变换矩阵对矢位图像进行形变，以使矢位图像向冠位图像对齐；以及，基于第二变换矩阵对轴位图像进行形变，以使轴位图像向冠位图像对齐。在一种实施方式中，可以将矢位图像IS和轴位图像IA同时向冠位图像IC对齐，即估计第一变换矩阵TS→C和第二变换矩阵TA→C，同时保证TA→C＝(TC→A)-1和TS→C＝(TC→S)-1，将矢位图像IS向冠位图像IC对齐：将轴位图像IA向冠位图像IC对齐：其中，IS′为形变后的矢位图像，IA′为形变后的轴位图像。(3.3) deforming the sagittal image based on the first transformation matrix so that the sagittal image is aligned with the coronal image; and deforming the axial image based on the second transformation matrix so that the axial image is aligned with the coronal image. In one embodiment, the sagittal imageIS and the axial imageIA can be aligned with the coronal imageIC at the same time, that is, estimating the first transformation matrixTS→C and the second transformation matrixTA→C , while ensuringTA→C = (TC→A )-1 andTS→C = (TC→S )-1 , and aligning the sagittal imageIS with the coronal imageIC : Align the axial imageIA to the coronal imageIC : Among them,IS′ is the sagittal image after deformation, andIA′ is the axial image after deformation.
步骤2，对各个中间腕部图像进行图像融合处理，得到目标腕部结构对应的目标腕部图像。在一种实施方式中，可以利用离散小波变换(DWT，Discrete Wavelet Transform)算法对各个中间腕部图像进行图像融合处理，参见图4所示的一种离散小波变换算法的原理示意图，将三个图像采集方位下采集的MRI图像(IA′，IS′，IC)进行融合，小波变换将MRI图像向不同尺度和不同方向的正交空间进行映射，其中各小波分量中的高频成分保留MRI图像在不同尺度下的细节结构，是重要的语义信息，本发明实施例通过将这些高频成分进行融合，重建各个方位下的细节成分，得到在三维空间各个方向上分辨率高、细节丰富的高质量目标腕部图像。Step 2, perform image fusion processing on each intermediate wrist image to obtain a target wrist image corresponding to the target wrist structure. In one embodiment, a discrete wavelet transform (DWT) algorithm can be used to perform image fusion processing on each intermediate wrist image. Referring to the principle schematic diagram of a discrete wavelet transform algorithm shown in FIG4 , MRI images (IA′ ,IS′ ,IC ) collected at three image acquisition orientations are fused. The wavelet transform maps the MRI image to orthogonal spaces of different scales and directions, wherein the high-frequency components in each wavelet component retain the detailed structure of the MRI image at different scales, which is important semantic information. The embodiment of the present invention fuses these high-frequency components to reconstruct the detailed components at each orientation, thereby obtaining a high-quality target wrist image with high resolution and rich details in all directions of three-dimensional space.
基于此，本发明实施例提供了一种图像融合处理的实施方式，参见如下步骤2.1至步骤2.3：Based on this, the embodiment of the present invention provides an implementation method of image fusion processing, see the following steps 2.1 to 2.3:
步骤2.1，提取每个中间腕部图像的多个维度的初始小波分量。其中，维度包括尺度和方向，例如，第m个尺度n方向。在一种实施方式中，可以使用3个尺度7个方向的Haar滤波器提取每个中间腕部图像的小波分量，以冠位图像IC为例，冠位图像IC的初始小波分量表达式如下所示：其中为冠位图像IC在第m个尺度n方向上的初始小波分量。Step 2.1, extracting the initial wavelet components of multiple dimensions of each intermediate wrist image. The dimension includes scale and direction, for example, the mth scale n direction. In one embodiment, a Haar filter with 3 scales and 7 directions can be used to extract the wavelet components of each intermediate wrist image. Taking the crown imageIC as an example, the initial wavelet component expression of the crown imageIC is as follows: in is the initial wavelet component of the coronal imageIC in the mth scale and n direction.
步骤2.2，对于每个维度的初始小波分量，根据预设评价函数确定对维度的初始小波分量进行融合时，基于各个中间腕部图像对应的权重系数，对维度的初始小波分量进行加权求和，得到维度的融合后小波分量。以第m个尺度n方向上的初始小波分量为例，评价函数如下所示：Step 2.2, for the initial wavelet components of each dimension, when the initial wavelet components of the dimension are fused according to the preset evaluation function, the initial wavelet components of the dimension are weighted summed based on the weight coefficients corresponding to each intermediate wrist image to obtain the fused wavelet components of the dimension. Taking the initial wavelet component in the mth scale n direction as an example, the evaluation function is as follows:
其中，threshold为预设阈值，为评价结果。基于该评价函数，可以确定出图像IS′的评价结果和图像IA′的评价结果根据各个评价结果确定第m个尺度n方向上各个图像的权重系数，公式如下所示：Among them, threshold is the preset threshold, Based on this evaluation function, the evaluation result of image IS′ can be determined Evaluation results of image IA′ The weight coefficients of each image in the mth scale n direction are determined according to each evaluation result. The formula is as follows:
其中，为冠位图像IC在第m个尺度n方向上的权重系数，为图像IS′在第m个尺度n方向上的权重系数，为图像IA′在第m个尺度n方向上的权重系数，则融合后小波分量如下所示：in, is the weight coefficient of the crown imageIC in the mth scale n direction, is the weight coefficient of the imageIS′ in the mth scale n direction, is the weight coefficient of imageIA′ in the mth scale n direction, then the fused wavelet components As shown below:
步骤2.3，对每个维度的融合后小波分量进行小波反变换处理，得到目标腕部结构对应的目标腕部图像。其中，目标腕部图像也可称之为时域图像或腕部三维MRI图像。在一种实施方式中，目标腕部图像IF的表达式如下所示：Step 2.3, perform inverse wavelet transform processing on the fused wavelet components of each dimension to obtain a target wrist image corresponding to the target wrist structure. The target wrist image may also be referred to as a time domain image or a three-dimensional MRI image of the wrist. In one embodiment, the expression of the target wrist imageIF is as follows:
对于前述步骤S106，可以基于半监督学习的腕部分割网络将上述目标腕部图像拆分为多个二维切片，该腕部分割网络包括二维图像分割子网络和分割结果三维优化子网络，然后使用少量人工标注通过半监督方式训练得到的轻量级的二维图像分割子网络，精准并高效地提取二维切片中的结构，包括尺骨远端、桡骨远端与TFCC结构，然后利用监督方式训练得到的分割结果三维优化子网络对二维图像分割子网络提取的结构进行修正，即可得到精度较高的目标分割结果。具体的，可参见如下步骤a1至步骤a2：For the aforementioned step S106, the above-mentioned target wrist image can be split into multiple two-dimensional slices based on the wrist segmentation network of semi-supervised learning. The wrist segmentation network includes a two-dimensional image segmentation subnetwork and a segmentation result three-dimensional optimization subnetwork. Then, a lightweight two-dimensional image segmentation subnetwork obtained by semi-supervised training with a small amount of manual annotations is used to accurately and efficiently extract the structures in the two-dimensional slices, including the distal end of the ulna, the distal end of the radius and the TFCC structure. Then, the three-dimensional optimization subnetwork of the segmentation result obtained by supervised training is used to correct the structure extracted by the two-dimensional image segmentation subnetwork, and a target segmentation result with high accuracy can be obtained. Specifically, please refer to the following steps a1 to a2:
步骤a1，通过二维图像分割子网络对每个二维切片进行分割处理，得到初始分割结果。其中，初始分割结果包括尺骨远端、桡骨远端与TFCC结构的图像。在一种实施方式中，二维图像分割子网络可以采用U-Net神经网络，其输入为每个二维切片，输出为尺骨远端、桡骨远端与TFCC结构的图像。Step a1, segment each two-dimensional slice through a two-dimensional image segmentation subnetwork to obtain an initial segmentation result. The initial segmentation result includes images of the distal end of the ulna, the distal end of the radius, and the TFCC structure. In one embodiment, the two-dimensional image segmentation subnetwork can use a U-Net neural network, whose input is each two-dimensional slice, and the output is an image of the distal end of the ulna, the distal end of the radius, and the TFCC structure.
步骤a2，通过分割结果三维优化子网络，基于二维切片在目标腕部图像中的边缘轮廓，修正初始分割结果得到目标分割结果。其中，腕部子结构包括尺骨远端分割结果、桡骨远端分割结果和TFCC结构分割结果中的一种或多种，尺骨远端分割结果也即修正后的尺骨远端图像，桡骨远端分割结果也即修正后的桡骨远端图像，TFCC结构分割结果也即修正后的TFCC结构图像。在一种实施方式中，分割结果三维优化子网络可基于空间连续型对初始分割结果进行修正，即可得到分割效果较好的目标分割结果。Step a2, through the segmentation result three-dimensional optimization sub-network, based on the edge contour of the two-dimensional slice in the target wrist image, correct the initial segmentation result to obtain the target segmentation result. Among them, the wrist substructure includes one or more of the distal ulna segmentation result, the distal radius segmentation result and the TFCC structure segmentation result. The distal ulna segmentation result is also the corrected distal ulna image, the distal radius segmentation result is also the corrected distal radius image, and the TFCC structure segmentation result is also the corrected TFCC structure image. In one embodiment, the segmentation result three-dimensional optimization sub-network can correct the initial segmentation result based on the spatial continuity type, so as to obtain a target segmentation result with better segmentation effect.
参见图5所示的一种腕部分割网络的结构示意图，基于此，本发明实施例还提供了一种腕部分割网络的训练步骤，参见如下步骤b1至步骤b5：Referring to the schematic diagram of the structure of a wrist segmentation network shown in FIG5 , based on this, an embodiment of the present invention further provides a training step of a wrist segmentation network, see the following steps b1 to b5:
步骤b1，获取训练图像。其中，训练图像包括多个训练二维切片。在一种实施方式中，训练图像中包括N个二维图像平面。Step b1, obtaining a training image. The training image includes a plurality of training two-dimensional slices. In one embodiment, the training image includes N two-dimensional image planes.
步骤b2，从训练二维切片中选取第一切片和第二切片；在一种实施方式中，在训练图像中选取n个第一切片，n<<N，并将其余训练二维切片作为第二切片。Step b2, selecting a first slice and a second slice from the training two-dimensional slices; in one embodiment, n first slices are selected from the training image, n<<N, and the remaining training two-dimensional slices are used as second slices.
步骤b3，确定第一切片对应的第一标签，并基于第一切片和第一标签对二维图像分割子网络进行训练。在一种实施方式中，人工标注尺骨远端、桡骨远端和TFCC结构，得到标签1(尺骨远端)、标签2(桡骨远端)与标签3(TFCC结构)，其中，背景标签为0，基于上述数据通过监督式学习训练二维图像分割子网络。Step b3, determine the first label corresponding to the first slice, and train the two-dimensional image segmentation subnetwork based on the first slice and the first label. In one embodiment, the distal end of the ulna, the distal end of the radius, and the TFCC structure are manually labeled to obtain label 1 (distal end of the ulna), label 2 (distal end of the radius), and label 3 (TFCC structure), wherein the background label is 0, and the two-dimensional image segmentation subnetwork is trained through supervised learning based on the above data.
步骤b4，通过训练后的二维图像分割子网络对第二切片进行分割处理，得到第二切片的初始分割结果。在一种实施方式中，使用前述训练得到的二维图像分割子网络对第二切片进行分割，得到相应的初始分割结果。Step b4, segmenting the second slice using the trained two-dimensional image segmentation subnetwork to obtain an initial segmentation result of the second slice. In one embodiment, the second slice is segmented using the trained two-dimensional image segmentation subnetwork to obtain a corresponding initial segmentation result.
步骤b5，基于第二切片的初始分割结果确定第二切片对应的第二标签，并基于第二切片的初始分割结果和第二标签对分割结果三维优化子网络进行训练。在一种实施方式中，可以使用3D形态学操作去除初始分割结果中的背景噪声、填补前景空洞等，以精炼各初始分割结果，将精炼后的初始分割结果作为第二切片对应的第二标签(也即，伪标签)，考虑到精炼后的初始分割结果的准确度可能依旧较差，因此可以对图像进行人工分割，以得到效果较好的第二标签。分割结果三维优化子网络输入通道为M(奇数)，即第二切片平面上下各(M-1)/2幅图像的分割结果，输出为第二切片的第二标签，该分割结果三维优化子网络通过结合第二切片在三维空间上的形状连续性修正初始分割结果。Step b5, determine the second label corresponding to the second slice based on the initial segmentation result of the second slice, and train the segmentation result three-dimensional optimization subnetwork based on the initial segmentation result and the second label of the second slice. In one embodiment, 3D morphological operations can be used to remove background noise in the initial segmentation results, fill foreground holes, etc., to refine each initial segmentation result, and use the refined initial segmentation result as the second label (that is, pseudo-label) corresponding to the second slice. Considering that the accuracy of the refined initial segmentation result may still be poor, the image can be manually segmented to obtain a better second label. The input channel of the segmentation result three-dimensional optimization subnetwork is M (odd number), that is, the segmentation result of each (M-1)/2 image above and below the second slice plane, and the output is the second label of the second slice. The segmentation result three-dimensional optimization subnetwork corrects the initial segmentation result by combining the shape continuity of the second slice in three-dimensional space.
可选的，二维图像分割子网络的监督式损失函数1使用交叉熵损失，分割结果三维优化子网络的无监督式损失函数2使用最小均方误差函数。Optionally, the supervised loss function 1 of the two-dimensional image segmentation subnetwork uses a cross entropy loss, and the unsupervised loss function 2 of the segmentation result three-dimensional optimization subnetwork uses a minimum mean square error function.
为便于理解，本发明实施例提供了一种腕部模型的重建方法的应用示例，参见图6所示的另一种腕部模型的重建方法的流程示意图，该方法主要包括以下步骤S602至步骤S608：For ease of understanding, an embodiment of the present invention provides an application example of a method for reconstructing a wrist model. Referring to FIG. 6 , a flow chart of another method for reconstructing a wrist model is shown. The method mainly includes the following steps S602 to S608:
步骤S602，对MRI图像进行预处理。Step S602: pre-processing the MRI image.
步骤S604，对预处理后的MRI图像进行融合得到腕部三维MRI图像。Step S604: fusing the preprocessed MRI images to obtain a three-dimensional MRI image of the wrist.
步骤S606，对腕部三维MRI图像进行分割得到目标分割结果。Step S606: segment the three-dimensional MRI image of the wrist to obtain a target segmentation result.
步骤S608，基于目标分割结果进行三维重建和网格化得到腕部子结构模型。Step S608: Perform three-dimensional reconstruction and meshing based on the target segmentation result to obtain a wrist substructure model.
综上所述，本发明实施例至少具有以下特点：In summary, the embodiments of the present invention have at least the following features:
(1)提出一种多方位核磁共振图像(MRI)融合方法，将腕关节冠位、矢位与轴位三个视角下拍摄的图像进行融合，提高三维图像在各个方向的分辨率，使腕关节中精细的解剖结构表现的更清晰，结构连续性更强。(1) A multi-directional magnetic resonance image (MRI) fusion method is proposed to fuse images taken from the coronal, sagittal and axial perspectives of the wrist joint, thereby improving the resolution of the three-dimensional image in all directions and making the fine anatomical structure of the wrist joint clearer and more continuous.
(2)提出一种基于半监督学习的MRI图像深度网络分割方法，输入一幅目标腕部图像，专家人工描绘几个关键切片上的腕关节解剖结构，输入腕部分割网络之后可将整个三维图像进行精确分割。(2) A deep network segmentation method for MRI images based on semi-supervised learning is proposed. An image of the target wrist is input, and experts manually depict the anatomical structure of the wrist joint on several key slices. After inputting the structure into the wrist segmentation network, the entire three-dimensional image can be accurately segmented.
对于前述实施例提供的腕部模型的重建方法，本发明实施例提供了一种腕部模型的重建装置，参见图7所示的一种腕部模型的重建装置的结构示意图，该装置主要包括以下部分：With respect to the wrist model reconstruction method provided in the above-mentioned embodiment, an embodiment of the present invention provides a wrist model reconstruction device. Referring to the structural schematic diagram of a wrist model reconstruction device shown in FIG. 7 , the device mainly includes the following parts:
图像获取模块702，用于获取目标腕部结构的腕部图像集合；其中，腕部图像集合包括多个图像采集方位处采集的原始腕部图像；An image acquisition module 702 is used to acquire a wrist image set of a target wrist structure; wherein the wrist image set includes original wrist images acquired at multiple image acquisition positions;
融合模块704，用于对腕部图像集合中的多个原始腕部图像进行图像融合处理，得到目标腕部结构对应的目标腕部图像；A fusion module 704 is used to perform image fusion processing on multiple original wrist images in the wrist image set to obtain a target wrist image corresponding to the target wrist structure;
分割模块706，用于通过预先训练得到的腕部分割网络对目标腕部图像进行分割处理，得到目标腕部结构对应的目标分割结果；其中，目标分割结果包括多个腕部子结构；The segmentation module 706 is used to perform segmentation processing on the target wrist image through the wrist segmentation network obtained by pre-training, and obtain a target segmentation result corresponding to the target wrist structure; wherein the target segmentation result includes multiple wrist substructures;
重建模块708，用于基于目标分割结果重建目标腕部结构各个腕部子结构对应的腕部子结构模型。The reconstruction module 708 is used to reconstruct the wrist substructure model corresponding to each wrist substructure of the target wrist structure based on the target segmentation result.
本发明实施例提供的腕部模型的重建装置，对腕部图像集合中多个图像采集方位处采集的原始腕部图像进行图像融合处理，可以得到分辨率较高、结构连续性更强的目标腕部图像，然后利用腕部分割模型可以更为精确地分割目标腕部图像得到目标分割结果，最后基于该目标分割结果即可得到精度和质量均较高的腕部子结构模型，从而显著提高了腕部子结构模型的重建精度和重建质量。The wrist model reconstruction device provided by the embodiment of the present invention performs image fusion processing on the original wrist images collected at multiple image collection positions in the wrist image set, so as to obtain a target wrist image with higher resolution and stronger structural continuity. Then, the wrist segmentation model can be used to more accurately segment the target wrist image to obtain a target segmentation result. Finally, based on the target segmentation result, a wrist substructure model with higher accuracy and quality can be obtained, thereby significantly improving the reconstruction accuracy and reconstruction quality of the wrist substructure model.
在一种实施方式中，融合模块704还用于：针对腕部图像集合中的每个原始腕部图像，对原始腕部图像进行预处理，得到中间腕部图像；对各个中间腕部图像进行图像融合处理，得到目标腕部结构对应的目标腕部图像。In one embodiment, the fusion module 704 is further used to: for each original wrist image in the wrist image set, preprocess the original wrist image to obtain an intermediate wrist image; and perform image fusion processing on each intermediate wrist image to obtain a target wrist image corresponding to the target wrist structure.
在一种实施方式中，融合模块704还用于：对原始腕部图像的灰度直方图进行调整，以调整原始腕部图像的对比度；利用B样条插值算法对原始腕部图像进行上采样，以调整原始腕部图像的分辨率；从原始腕部图像中确定参考图像，并将除参考图像之外的其余原始腕部图像向参考图像对齐。In one embodiment, the fusion module 704 is further used to: adjust the grayscale histogram of the original wrist image to adjust the contrast of the original wrist image; upsample the original wrist image using a B-spline interpolation algorithm to adjust the resolution of the original wrist image; determine a reference image from the original wrist image, and align the remaining original wrist images except the reference image to the reference image.
在一种实施方式中，原始腕部图像包括冠位图像、矢位图像和轴位图像；融合模块704还用于：将冠位图像确定为参考图像；根据最小二乘法和冠位图像，确定第一变换矩阵和第二变换矩阵；基于第一变换矩阵对矢位图像进行形变，以使矢位图像向冠位图像对齐；以及，基于第二变换矩阵对轴位图像进行形变，以使轴位图像向冠位图像对齐。In one embodiment, the original wrist image includes a coronal image, a sagittal image, and an axial image; the fusion module 704 is further used to: determine the coronal image as a reference image; determine a first transformation matrix and a second transformation matrix based on the least squares method and the coronal image; deform the sagittal image based on the first transformation matrix to align the sagittal image with the coronal image; and deform the axial image based on the second transformation matrix to align the axial image with the coronal image.
在一种实施方式中，融合模块704还用于：提取每个中间腕部图像的多个维度的初始小波分量；对于每个维度的初始小波分量，根据预设评价函数确定对维度的初始小波分量进行融合时，基于各个中间腕部图像对应的权重系数，对维度的初始小波分量进行加权求和，得到维度的融合后小波分量；对每个维度的融合后小波分量进行小波反变换处理，得到目标腕部结构对应的目标腕部图像。In one embodiment, the fusion module 704 is also used to: extract the initial wavelet components of multiple dimensions of each intermediate wrist image; for the initial wavelet components of each dimension, determine the fusion of the initial wavelet components of the dimension according to a preset evaluation function, and perform weighted summation on the initial wavelet components of the dimension based on the weight coefficients corresponding to each intermediate wrist image to obtain the fused wavelet components of the dimension; perform inverse wavelet transform processing on the fused wavelet components of each dimension to obtain a target wrist image corresponding to the target wrist structure.
在一种实施方式中，腕部分割网络包括二维图像分割子网络和分割结果三维优化子网络，目标腕部图像包括多个二维切片；分割模块706还用于：通过二维图像分割子网络对每个二维切片进行分割处理，得到初始分割结果；通过分割结果三维优化子网络，基于二维切片在目标腕部图像中的边缘轮廓，修正初始分割结果得到目标分割结果；其中，腕部子结构包括尺骨远端分割结果、桡骨远端分割结果和TFCC结构分割结果中的一种或多种。In one embodiment, the wrist segmentation network includes a two-dimensional image segmentation subnetwork and a three-dimensional optimization subnetwork for segmentation results, and the target wrist image includes multiple two-dimensional slices; the segmentation module 706 is also used to: perform segmentation processing on each two-dimensional slice through the two-dimensional image segmentation subnetwork to obtain an initial segmentation result; and correct the initial segmentation result based on the edge contour of the two-dimensional slice in the target wrist image through the three-dimensional optimization subnetwork for segmentation results to obtain a target segmentation result; wherein the wrist substructure includes one or more of the distal ulna segmentation result, the distal radius segmentation result, and the TFCC structure segmentation result.
在一种实施方式中，上述装置还包括模型训练模块，用于：获取训练图像；其中，训练图像包括多个训练二维切片；从训练二维切片中选取第一切片和第二切片；确定第一切片对应的第一标签，并基于第一切片和第一标签对二维图像分割子网络进行训练；通过训练后的二维图像分割子网络对第二切片进行分割处理，得到第二切片的初始分割结果；基于第二切片的初始分割结果确定第二切片对应的第二标签，并基于第二切片的初始分割结果和第二标签对分割结果三维优化子网络进行训练。In one embodiment, the above-mentioned device also includes a model training module, which is used to: obtain a training image; wherein the training image includes multiple training two-dimensional slices; select a first slice and a second slice from the training two-dimensional slices; determine a first label corresponding to the first slice, and train a two-dimensional image segmentation subnetwork based on the first slice and the first label; perform segmentation processing on the second slice through the trained two-dimensional image segmentation subnetwork to obtain an initial segmentation result of the second slice; determine a second label corresponding to the second slice based on the initial segmentation result of the second slice, and train a segmentation result three-dimensional optimization subnetwork based on the initial segmentation result of the second slice and the second label.
本发明实施例所提供的装置，其实现原理及产生的技术效果和前述方法实施例相同，为简要描述，装置实施例部分未提及之处，可参考前述方法实施例中相应内容。The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
本发明实施例提供了一种电子设备，具体的，该电子设备包括处理器和存储装置；存储装置上存储有计算机程序，计算机程序在被所述处理器运行时执行如上所述实施方式的任一项所述的方法。An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.
图8为本发明实施例提供的一种电子设备的结构示意图，该电子设备100包括：处理器80，存储器81，总线82和通信接口83，所述处理器80、通信接口83和存储器81通过总线82连接；处理器80用于执行存储器81中存储的可执行模块，例如计算机程序。Figure 8 is a structural diagram of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes: a processor 80, a memory 81, a bus 82 and a communication interface 83. The processor 80, the communication interface 83 and the memory 81 are connected via the bus 82; the processor 80 is used to execute an executable module stored in the memory 81, such as a computer program.
其中，存储器81可能包含高速随机存取存储器(RAM，Random Access Memory)，也可能还包括非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器。通过至少一个通信接口83(可以是有线或者无线)实现该系统网元与至少一个其他网元之间的通信连接，可以使用互联网，广域网，本地网，城域网等。The memory 81 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 83 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
总线82可以是ISA总线、PCI总线或EISA总线等。所述总线可以分为地址总线、数据总线、控制总线等。为便于表示，图8中仅用一个双向箭头表示，但并不表示仅有一根总线或一种类型的总线。The bus 82 may be an ISA bus, a PCI bus or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bidirectional arrow is used in FIG8 , but this does not mean that there is only one bus or one type of bus.
其中，存储器81用于存储程序，所述处理器80在接收到执行指令后，执行所述程序，前述本发明实施例任一实施例揭示的流过程定义的装置所执行的方法可以应用于处理器80中，或者由处理器80实现。Among them, the memory 81 is used to store programs, and the processor 80 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiments of the present invention can be applied to the processor 80 or implemented by the processor 80.
处理器80可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器80中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器80可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)等；还可以是数字信号处理器(Digital SignalProcessing，简称DSP)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)、现成可编程门阵列(Field-Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本发明实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本发明实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器81，处理器80读取存储器81中的信息，结合其硬件完成上述方法的步骤。The processor 80 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 80. The above processor 80 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 81, and the processor 80 reads the information in the memory 81 and completes the steps of the above method in combination with its hardware.
本发明实施例所提供的可读存储介质的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行前面方法实施例中所述的方法，具体实现可参见前述方法实施例，在此不再赘述。The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
最后应说明的是：以上所述实施例，仅为本发明的具体实施方式，用以说明本发明的技术方案，而非对其限制，本发明的保护范围并不局限于此，尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本发明实施例技术方案的精神和范围，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210238090.2ACN114581424B (en) | 2022-03-11 | 2022-03-11 | Reconstruction method and device of wrist model, electronic equipment and storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210238090.2ACN114581424B (en) | 2022-03-11 | 2022-03-11 | Reconstruction method and device of wrist model, electronic equipment and storage medium |
| Publication Number | Publication Date |
|---|---|
| CN114581424A CN114581424A (en) | 2022-06-03 |
| CN114581424Btrue CN114581424B (en) | 2024-10-29 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210238090.2AActiveCN114581424B (en) | 2022-03-11 | 2022-03-11 | Reconstruction method and device of wrist model, electronic equipment and storage medium |
| Country | Link |
|---|---|
| CN (1) | CN114581424B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118447038A (en)* | 2024-04-17 | 2024-08-06 | 中国科学院自动化研究所 | Interactive image segmentation method and device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018119766A1 (en)* | 2016-12-28 | 2018-07-05 | 上海联影医疗科技有限公司 | Multi-modal image processing system and method |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1581230A (en)* | 2004-05-20 | 2005-02-16 | 上海交通大学 | Remote-senstive image interfusion method based on image local spectrum characteristic |
| US10467767B2 (en)* | 2016-12-23 | 2019-11-05 | International Business Machines Corporation | 3D segmentation reconstruction from 2D slices |
| CN113850816B (en)* | 2020-06-28 | 2024-06-21 | 中国人民解放军空军军医大学 | A segmentation device and method for cervical cancer MRI images |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018119766A1 (en)* | 2016-12-28 | 2018-07-05 | 上海联影医疗科技有限公司 | Multi-modal image processing system and method |
| Publication number | Publication date |
|---|---|
| CN114581424A (en) | 2022-06-03 |
| Publication | Publication Date | Title |
|---|---|---|
| Yan et al. | Multi-scale dense networks for deep high dynamic range imaging | |
| CN110321920B (en) | Image classification method, apparatus, computer-readable storage medium, and computer device | |
| US20230386048A1 (en) | Ultrasound image segmentation method and apparatus, terminal device, and storage medium | |
| CN112435309A (en) | Method for enhancing quality and resolution of CT image based on deep learning | |
| CN110648337A (en) | Hip joint segmentation method, hip joint segmentation device, electronic apparatus, and storage medium | |
| CN110998602A (en) | Classification and 3D modeling of 3D dento-maxillofacial structures using deep learning methods | |
| CN111047629A (en) | Method, device, electronic device and storage medium for multimodal image registration | |
| CN110619635B (en) | Hepatocellular carcinoma magnetic resonance image segmentation system and method based on deep learning | |
| CN112215291A (en) | Medical Image Feature Extraction and Classification Based on Cascaded Neural Networks | |
| CN118485643B (en) | Medical image analysis processing system based on image analysis | |
| Gu et al. | Deep generative adversarial networks for thin-section infant MR image reconstruction | |
| CN110047075A (en) | A kind of CT image partition method based on confrontation network | |
| CN116823625B (en) | Cross-contrast magnetic resonance super-resolution method and system based on variational self-encoder | |
| WO2022088572A1 (en) | Model training method, image processing and alignment method, apparatus, device, and medium | |
| CN115546270A (en) | Image registration method, model training method and equipment for multi-scale feature fusion | |
| CN112102327B (en) | Image processing method, device and computer readable storage medium | |
| CN116503505A (en) | A method, device, equipment and medium for removing artifacts from CBCT images | |
| CN119919291B (en) | Structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and equipment | |
| CN114581424B (en) | Reconstruction method and device of wrist model, electronic equipment and storage medium | |
| CN115410032B (en) | OCTA image classification structure training method based on self-supervised learning | |
| CN113963037A (en) | Image registration method, apparatus, computer equipment and storage medium | |
| CN120374818A (en) | Texture mapping method and equipment for oral cavity three-dimensional model aiming at soft tissues | |
| CN116433976A (en) | Image processing method, device, equipment and storage medium | |
| CN120088475A (en) | An automatic segmentation method of mandibular nerve canal based on multi-view guidance | |
| CN115761226A (en) | Oral cavity image segmentation identification method and device, electronic equipment and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |