技术领域Technical field
本公开涉及低质量眼底图像增强及眼底血管分割技术领域，具体涉及一种基于扩散模型的低质量眼底图像的增强及分割方法及系统。The present disclosure relates to the technical field of low-quality fundus image enhancement and fundus blood vessel segmentation, and specifically relates to a diffusion model-based enhancement and segmentation method and system for low-quality fundus images.
背景技术Background technique
高质量的视网膜图像在各种眼病的辅助诊断中至关重要，包括糖尿病、视网膜撕裂和脱离、青光眼、年龄相关性黄斑孔和变性。研究人员证明，视网膜血管的变化可以作为某些大脑和心血管疾病的早期筛查方法。因此，视网膜血管分割在视网膜图像分析中起着至关重要的作用。High-quality retinal images are critical in aiding the diagnosis of a variety of eye diseases, including diabetes, retinal tears and detachments, glaucoma, age-related macular holes and degeneration. Researchers have demonstrated that changes in retinal blood vessels could serve as an early screening method for certain brain and cardiovascular diseases. Therefore, retinal vessel segmentation plays a crucial role in retinal image analysis.
基于高质量图像准确识别血管结构，使医生能够做出精确的疾病诊断。然而，由疾病、设备限制或环境因素引起的视网膜图像退化是一个常见的问题，病变和光照不均匀是退化的主要原因。医疗专业知识的复杂性和对专业设备的依赖性使手工分割血管成为一项繁重和耗时的任务。因此，提高低质量的视网膜图像是许多下游任务的基本要求，直接或间接地影响各种视网膜图像的诊断。Accurate identification of vascular structures based on high-quality images enables doctors to make precise disease diagnoses. However, retinal image degradation caused by disease, equipment limitations, or environmental factors is a common problem, with lesions and uneven illumination being the main causes of degradation. The complexity of medical expertise and reliance on specialized equipment makes manual segmentation of blood vessels a tedious and time-consuming task. Therefore, improving low-quality retinal images is an essential requirement for many downstream tasks, directly or indirectly affecting the diagnosis of various retinal images.
近年来，研究人员提出了许多基于深度学习的图像增强和血管分割算法，旨在减轻医生的负担。Hu等人提出了一种新的超级血管算法，该算法以低分辨率图像作为输入，以高分辨率和准确的血管分割作为输出。他们采用超分辨率作为其辅助分支，提供潜在的高分辨率的详细特征，以增强血管分割。Alimanov等人引入了一种周期持续生成对抗网络(CycleGAN)，带有卷积注意力模块和改进的UNet模型(CBAM-UNet)用于视网膜血管分割，以解决低质量视网膜图像增强和血管分割的问题。CycleGAN中的对抗性训练逐步引导模型走向现实呈现，并且注意力模块增强了特征提取能力。然而，当应用于视网膜图像生成时，CycleGAN有一些缺点：一方面，它需要昂贵的计算资源和同时训练一个生成器-鉴别器对；另一方面，存在丢失病理病变和引入不存在的血管的风险，从而破坏血管和病理结构。虽然有许多关于图像恢复的生成模型，但它们往往忽略了对原始图像的底层分布的建模。这一限制使它们无法有效地保存病变信息或保持血管结构与原始图像的一致性。In recent years, researchers have proposed many deep learning-based image enhancement and blood vessel segmentation algorithms aimed at reducing the burden on doctors. Hu et al. proposed a new super-vessel algorithm that takes low-resolution images as input and high-resolution and accurate blood vessel segmentation as output. They adopt super-resolution as its auxiliary branch, providing potentially high-resolution detailed features to enhance vessel segmentation. Alimanov et al. introduced a Cycle Continuous Generative Adversarial Network (CycleGAN) with a convolutional attention module and an improved UNet model (CBAM-UNet) for retinal blood vessel segmentation to solve the problem of low-quality retinal image enhancement and blood vessel segmentation. question. The adversarial training in CycleGAN gradually guides the model towards realistic presentation, and the attention module enhances feature extraction capabilities. However, CycleGAN has some disadvantages when applied to retinal image generation: on the one hand, it requires expensive computing resources and simultaneous training of a generator-discriminator pair; on the other hand, there is the problem of missing pathological lesions and introducing non-existent blood vessels. risk, thereby damaging blood vessels and pathological structures. Although there are many generative models on image restoration, they often ignore the modeling of the underlying distribution of the original image. This limitation prevents them from effectively preserving lesion information or maintaining consistency of vascular structure with the original image.
发明内容Contents of the invention
为解决上述现有技术的不足，本公开提供了一种基于扩散模型的低质量眼底图像的增强及分割方法及系统，所述方案利用条件扩散模型实现低质量眼底图像的图像增强，同时，为了细化血管掩膜并进行低质量的图像恢复，设计了眼底血管掩膜分割作为扩散发生器的辅助分支；所述方案采用退化模型，通过获得退化因子来提高图像的质量，并以此实现低质量眼底图像血管分割准确性的提高。In order to solve the above deficiencies in the prior art, the present disclosure provides a method and system for enhancement and segmentation of low-quality fundus images based on a diffusion model. The solution uses a conditional diffusion model to achieve image enhancement of low-quality fundus images. At the same time, in order to To refine the blood vessel mask and perform low-quality image restoration, the fundus blood vessel mask segmentation is designed as an auxiliary branch of the diffusion generator; the scheme uses a degradation model to improve the quality of the image by obtaining the degradation factor, and thereby achieve low-quality images. Improvement of blood vessel segmentation accuracy in quality fundus images.
根据本公开实施例的第一个方面，提供了一种基于扩散模型的低质量眼底图像的增强及分割方法，包括：According to a first aspect of an embodiment of the present disclosure, a diffusion model-based enhancement and segmentation method for low-quality fundus images is provided, including:
获取待处理的低质量眼底图像及其对应的照明图；Obtain the low-quality fundus image to be processed and its corresponding illumination map;
基于所述低质量眼底图像，通过匹配滤波获得对应的眼底血管掩膜；Based on the low-quality fundus image, obtain the corresponding fundus blood vessel mask through matching filtering;
基于所述低质量眼底图像及其对应的照明图，利用预先训练的UNet网络，获得低质量图像的退化因子，其中，所述UNet网络的输入与输出之间设置有跳跃连接，且UNet网络的对称扩展路径中设置有注意力机制；Based on the low-quality fundus image and its corresponding illumination map, the pre-trained UNet network is used to obtain the degradation factor of the low-quality image, where a skip connection is set between the input and output of the UNet network, and the UNet network An attention mechanism is provided in the symmetric expansion path;
以所述低质量眼底图像及其对应的眼底血管掩膜作为预先训练的扩散模型的输入，获得图像增强后的眼底图像及其对应的眼底血管掩膜；其中，所述扩散模型采用UNet网络，所述UNet网络最后一层去噪器之后设置有用于预测血管掩膜的预测头，且所述退化因子嵌入至所述扩散模型中，与获得图像增强后的眼底图像进行关联。Using the low-quality fundus image and its corresponding fundus blood vessel mask as the input of a pre-trained diffusion model, an image-enhanced fundus image and its corresponding fundus blood vessel mask are obtained; wherein, the diffusion model adopts the UNet network, A prediction head for predicting blood vessel masks is provided after the last layer of denoiser of the UNet network, and the degradation factor is embedded in the diffusion model and associated with the enhanced fundus image.
进一步的，在将低质量眼底图像输入扩散模型前，先利用匹配滤波获得对应的初始的血管掩膜，将低质量眼底图像和获得的初始血管掩膜同时作为输入，将图像增强和血管掩膜分割作为联合任务，利用训练好的扩散模型获得增强后的眼底图像和相对应的血管掩膜。Furthermore, before inputting the low-quality fundus image into the diffusion model, matching filtering is first used to obtain the corresponding initial blood vessel mask. The low-quality fundus image and the obtained initial blood vessel mask are used as input at the same time, and the image enhancement and blood vessel mask are combined. Segmentation is a joint task, using the trained diffusion model to obtain the enhanced fundus image and the corresponding blood vessel mask.
进一步的，所述退化因子的获取，通过将低质量的眼底图像及其对应的照明图连接起来作为输入，通过预先训练的UNet网络保留原始眼底图像的基本特征和细节，获得低质量眼底图像的退化因子，并嵌入扩散模型迭代过程中进一步恢复低质量图像。Further, the degradation factor is obtained by connecting low-quality fundus images and their corresponding illumination maps as input, retaining the basic features and details of the original fundus images through the pre-trained UNet network, and obtaining the low-quality fundus images. Degradation factor, and embedded diffusion model iterative process to further restore low-quality images.
进一步的，所述预测头具体包括顺序连接的卷积层和Sigmoid函数。Further, the prediction head specifically includes sequentially connected convolutional layers and sigmoid functions.
进一步的，所述用于预测眼底血管掩膜的预测头的训练过程为：以扩散模型迭代过程的每个时间步中的眼底图像为输入，以对应的眼底血管掩膜为输出，训练所述预测头。Further, the training process of the prediction head for predicting the fundus blood vessel mask is: taking the fundus image in each time step of the diffusion model iteration process as input, and using the corresponding fundus blood vessel mask as the output, training the Prediction header.
进一步的，所述扩散模型的训练步骤包括：构建训练样本集，所述训练样本集包括低质量的眼底图像和配对的高质量的眼底图像；利用训练样本集，对血管掩膜感知模块中扩散模型进行训练，以低质量眼底图像、配对的高质量眼底图像和相应的初始血管掩膜分割图为输入，当损失函数达到最小值或迭代次数满足设定要求时，停止训练，得到训练完成的扩散模型。Further, the training steps of the diffusion model include: constructing a training sample set, the training sample set includes low-quality fundus images and paired high-quality fundus images; using the training sample set, conduct diffusion in the blood vessel mask perception module The model is trained, taking low-quality fundus images, paired high-quality fundus images and corresponding initial blood vessel mask segmentation maps as inputs. When the loss function reaches the minimum value or the number of iterations meets the set requirements, the training is stopped and the trained results are obtained. Diffusion model.
进一步的，所述方法并未直接将低质量的图像输入扩散模型中来消除退化因子并获得增强的图像，而是通过单独的网络分支明确提取退化因子。Furthermore, the method does not directly input low-quality images into the diffusion model to eliminate degradation factors and obtain enhanced images, but explicitly extracts degradation factors through a separate network branch.
根据本公开实施例的第二个方面，提供了一种基于扩散模型的低质量眼底图像的增强及分割系统，包括：According to a second aspect of the embodiment of the present disclosure, a diffusion model-based enhancement and segmentation system for low-quality fundus images is provided, including:
数据获取单元，其用于获取待处理的低质量眼底图像及其对应的照明图；a data acquisition unit, which is used to acquire the low-quality fundus image to be processed and its corresponding illumination map;
掩膜图像获取单元，其用于基于所述低质量眼底图像，通过匹配滤波获得对应的眼底血管掩膜；A mask image acquisition unit configured to obtain a corresponding fundus blood vessel mask through matching filtering based on the low-quality fundus image;
退化因子获取单元，其用于基于所述低质量眼底图像及其对应的照明图，利用预先训练的UNet网络，获得低质量图像的退化因子，其中，所述UNet网络的输入与输出之间设置有跳跃连接，且UNet网络的对称扩展路径中设置有注意力机制；Degradation factor acquisition unit, which is used to obtain the degradation factor of the low-quality image based on the low-quality fundus image and its corresponding illumination map using the pre-trained UNet network, wherein the input and output of the UNet network are set between There are skip connections, and an attention mechanism is set up in the symmetric expansion path of the UNet network;
增强及分割单元，其用于以所述低质量眼底图像及其对应的眼底血管掩膜作为预先训练的扩散模型的输入，获得图像增强后的眼底图像及其对应的眼底血管掩膜；其中，所述扩散模型采用UNet网络，所述UNet网络最后一层去噪器之后设置有用于预测眼底血管掩膜的预测头，且所述退化因子嵌入至所述扩散模型中，与获得图像增强后的眼底图像进行关联。An enhancement and segmentation unit configured to use the low-quality fundus image and its corresponding fundus blood vessel mask as input to a pre-trained diffusion model to obtain an image-enhanced fundus image and its corresponding fundus blood vessel mask; wherein, The diffusion model adopts the UNet network. After the last layer of the denoiser of the UNet network, a prediction head for predicting the fundus blood vessel mask is provided, and the degradation factor is embedded in the diffusion model, and the image enhancement is obtained. Fundus images are associated.
根据本公开实施例的第三个方面，提供了一种电子设备，包括存储器、处理器及存储在存储器上运行的计算机程序，所述处理器执行所述程序时实现所述的一种基于扩散模型的低质量眼底图像的增强及分割方法。According to a third aspect of the embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored and run on the memory. When the processor executes the program, the diffusion-based method is implemented. Enhancement and segmentation methods for low-quality fundus images of models.
根据本公开实施例的第四个方面，提供了一种非暂态计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现所述的一种基于扩散模型的低质量眼底图像的增强及分割方法。According to a fourth aspect of the embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which implements the low-quality diffusion model-based method when executed by a processor. Fundus image enhancement and segmentation methods.
以上一个或多个技术方案存在以下有益效果：One or more of the above technical solutions have the following beneficial effects:
(1)本公开提供了一种基于扩散模型的低质量眼底图像的增强及分割方法及系统，所述方案利用条件扩散模型实现低质量眼底图像的图像增强，同时，为了细化血管掩膜并进行低质量的图像恢复，设计了眼底血管掩膜分割作为扩散发生器的辅助分支；所述方案采用退化模型，通过获得退化因子来提高图像的质量，并以此实现低质量眼底图像血管分割准确性的提高。(1) The present disclosure provides a method and system for enhancement and segmentation of low-quality fundus images based on a diffusion model. The solution uses a conditional diffusion model to achieve image enhancement of low-quality fundus images. At the same time, in order to refine the blood vessel mask and For low-quality image restoration, fundus blood vessel mask segmentation is designed as an auxiliary branch of the diffusion generator; the solution uses a degradation model to improve the quality of the image by obtaining degradation factors, and thereby achieves accurate blood vessel segmentation of low-quality fundus images. Sexual improvement.
(2)所述方案通过提取低质量眼底图像和对应的初始眼底血管掩膜的特征，利用扩散模型，从迭代过程的每个时间步中获取并共享与图像增强和眼底血管最相关的信息，以此实现低质量眼底图像恢复和眼底血管分割准确性的提高，辅助医生诊断各种疾病。(2) The described scheme extracts features of low-quality fundus images and corresponding initial fundus blood vessels masks, and utilizes a diffusion model to obtain and share the most relevant information for image enhancement and fundus blood vessels from each time step of the iterative process, In this way, low-quality fundus images can be restored and the accuracy of fundus blood vessel segmentation can be improved to assist doctors in diagnosing various diseases.
(3)所述方案中的退化因子提取网络应用于低质量眼底图像增强中，学习与眼底图像相关的特征，剔除不相关的特征，获得眼底图像的退化因子和更鲜明的特征。(3) The degradation factor extraction network in the above scheme is used in low-quality fundus image enhancement to learn features related to the fundus image, eliminate irrelevant features, and obtain the degradation factor and more distinctive features of the fundus image.
(4)所述方案通过将低质量眼底图像和相对应的初始的血管掩膜分割图像相结合，利用训练好的血管掩膜预测头可以更有效地实现图像增强和血管分割的联合任务，从而进一步提高与视网膜相关的疾病检测的准确性；同时，所述方案通过对提取的退化因子加以利用，可以更准确地把握低质量眼底图像的退化程度，有助于与眼底图像增强的特征提取。(4) The described scheme combines low-quality fundus images with corresponding initial blood vessel mask segmentation images, and uses the trained blood vessel mask prediction head to more effectively achieve the joint tasks of image enhancement and blood vessel segmentation, thus The accuracy of retina-related disease detection is further improved; at the same time, by utilizing the extracted degradation factors, the scheme can more accurately grasp the degree of degradation of low-quality fundus images, which is helpful for feature extraction with fundus image enhancement.
附图说明Description of drawings
构成本公开的一部分的说明书附图用来提供对本公开的进一步理解，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。The description drawings that form a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure.
图1为本公开实施例中所述的一种基于扩散模型的低质量视网膜图像分割方法流程图；Figure 1 is a flow chart of a low-quality retinal image segmentation method based on a diffusion model described in an embodiment of the present disclosure;
图2为本公开实施例中所述的改进的UNet的结构示意图；Figure 2 is a schematic structural diagram of the improved UNet described in the embodiment of the present disclosure;
图3(a)至图3(b)为本公开实施例中所述的通过退化网络得到的退化眼底图像；Figure 3(a) to Figure 3(b) are degraded fundus images obtained through the degradation network described in the embodiment of the present disclosure;
图4为本公开实施例中所述的眼底图像增强对比图；Figure 4 is an enhanced contrast diagram of the fundus image described in the embodiment of the present disclosure;
图5为本公开实施例中所述的眼底图像血管分割对比图；Figure 5 is a comparison diagram of blood vessel segmentation in fundus images described in the embodiment of the present disclosure;
图6(a)至图6(b)为本公开实施例中所述的检测结果图；Figure 6(a) to Figure 6(b) are detection result diagrams described in the embodiment of the present disclosure;
图7为本公开实施例中所述的对退化模型的消融实验对比图。Figure 7 is a comparison diagram of ablation experiments on the degradation model described in the embodiment of the present disclosure.
具体实施方式Detailed ways
本技术领域技术人员可以理解，除非另外定义，这里使用的所有术语(包括技术术语和科学术语)具有与本公开所属领域中的普通技术人员的一般理解相同的意义。It can be understood by one of ordinary skill in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
还应该理解的是，诸如通用字典中定义的那些术语应该被理解为具有与现有技术的上下文中的意义一致的意义，并且除非像这里一样定义，不会用理想化或过于正式的含义来解释。It should also be understood that terms such as those defined in general dictionaries are to be understood to have meanings consistent with their meaning in the context of the prior art, and are not to be taken in an idealized or overly formal sense unless defined as herein. explain.
为便于理解本公开，下面结合附图以具体实施例对本公开作进一步解释说明，且具体实施例并不构成对本公开实施例的限定。In order to facilitate understanding of the present disclosure, the present disclosure will be further explained with specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute limitations to the embodiments of the present disclosure.
本领域技术人员应该理解，附图只是实施例的示意图，附图中的部件并不一定是实施本公开所必须的。Those skilled in the art should understand that the drawings are only schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present disclosure.
实施例一Embodiment 1
本实施例的目的是提供一种基于扩散模型的低质量视网膜图像分割方法。The purpose of this embodiment is to provide a low-quality retinal image segmentation method based on a diffusion model.
一种基于扩散模型的低质量眼底图像的增强及分割方法，包括：A diffusion model-based enhancement and segmentation method for low-quality fundus images, including:
获取待处理的低质量眼底图像及其对应的照明图；Obtain the low-quality fundus image to be processed and its corresponding illumination map;
基于所述低质量眼底图像，通过匹配滤波获得对应的眼底血管掩膜；Based on the low-quality fundus image, obtain the corresponding fundus blood vessel mask through matching filtering;
基于所述低质量眼底图像及其对应的照明图，利用预先训练的UNet网络，获得低质量图像的退化因子，其中，所述UNet网络的输入与输出之间设置有跳跃连接，且UNet网络的对称扩展路径中设置有注意力机制；Based on the low-quality fundus image and its corresponding illumination map, the pre-trained UNet network is used to obtain the degradation factor of the low-quality image, where a skip connection is set between the input and output of the UNet network, and the UNet network An attention mechanism is provided in the symmetric expansion path;
以所述低质量眼底图像及其对应的眼底血管掩膜作为预先训练的扩散模型的输入，获得图像增强后的眼底图像及其对应的眼底血管掩膜；其中，所述扩散模型采用UNet网络，所述UNet网络最后一层去噪器之后设置有用于预测血管掩膜的预测头，且所述退化因子嵌入至所述扩散模型中，与获得图像增强后的眼底图像进行关联。Using the low-quality fundus image and its corresponding fundus blood vessel mask as the input of a pre-trained diffusion model, an image-enhanced fundus image and its corresponding fundus blood vessel mask are obtained; wherein, the diffusion model adopts the UNet network, A prediction head for predicting blood vessel masks is provided after the last layer of denoiser of the UNet network, and the degradation factor is embedded in the diffusion model and associated with the enhanced fundus image.
在具体实施中，在将低质量眼底图像输入扩散模型前，先利用匹配滤波获得对应的初始的血管掩膜，将低质量眼底图像和获得的初始血管掩膜同时作为输入，将图像增强和血管掩膜分割作为联合任务，利用训练好的扩散模型获得增强后的眼底图像和相对应的血管掩膜。In the specific implementation, before inputting the low-quality fundus image into the diffusion model, matching filtering is first used to obtain the corresponding initial blood vessel mask. The low-quality fundus image and the obtained initial blood vessel mask are used as input at the same time, and the image enhancement and blood vessel mask are combined. Mask segmentation is a joint task, using the trained diffusion model to obtain the enhanced fundus image and the corresponding blood vessel mask.
在具体实施中，所述退化因子的获取，通过将低质量的眼底图像及其对应的照明图连接起来作为输入，通过预先训练的UNet网络保留原始眼底图像的基本特征和细节，获得低质量眼底图像的退化因子，并嵌入扩散模型迭代过程中进一步恢复低质量图像。In a specific implementation, the degradation factor is obtained by connecting low-quality fundus images and their corresponding illumination maps as input, and retaining the basic features and details of the original fundus images through the pre-trained UNet network to obtain low-quality fundus images. The degradation factor of the image is embedded in the diffusion model iterative process to further restore the low-quality image.
在具体实施中，所述预测头具体包括顺序连接的卷积层和Sigmoid函数。In a specific implementation, the prediction head specifically includes sequentially connected convolutional layers and sigmoid functions.
在具体实施中，所述用于预测眼底血管掩膜的预测头的训练过程为：以扩散模型迭代过程的每个时间步中的眼底图像为输入，以对应的眼底血管掩膜为输出，训练所述预测头。In a specific implementation, the training process of the prediction head for predicting the fundus blood vessel mask is: taking the fundus image in each time step of the diffusion model iterative process as input, using the corresponding fundus blood vessel mask as the output, training The prediction header.
在具体实施中，所述扩散模型的训练步骤包括：构建训练样本集，所述训练样本集包括低质量的眼底图像和配对的高质量的眼底图像；利用训练样本集，对血管掩膜感知模块中扩散模型进行训练，以低质量眼底图像、配对的高质量眼底图像和相应的初始血管掩膜分割图为输入，当损失函数达到最小值或迭代次数满足设定要求时，停止训练，得到训练完成的扩散模型。In a specific implementation, the training steps of the diffusion model include: constructing a training sample set, the training sample set includes low-quality fundus images and paired high-quality fundus images; using the training sample set, the blood vessel mask perception module The medium diffusion model is trained, taking low-quality fundus images, paired high-quality fundus images and corresponding initial blood vessel mask segmentation maps as inputs. When the loss function reaches the minimum value or the number of iterations meets the set requirements, the training is stopped and the training is obtained. Completed diffusion model.
在具体实施中，所述方法并未直接将低质量的图像输入扩散模型中来消除退化因子并获得增强的图像，而是通过单独的网络分支明确提取退化因子。In a specific implementation, the method does not directly input low-quality images into the diffusion model to eliminate degradation factors and obtain enhanced images, but explicitly extracts degradation factors through a separate network branch.
具体的，为了便于理解，以下结合附图对本实施例所述方案进行详细说明：Specifically, for ease of understanding, the solution described in this embodiment is described in detail below with reference to the accompanying drawings:
为了解决现有技术存在的问题，本实施例提供了一种基于扩散模型的低质量视网膜图像分割方法，实现了视网膜图像增强和眼底血管分割，辅助疾病的检测，所述方法具体包括如下步骤：In order to solve the problems existing in the existing technology, this embodiment provides a low-quality retinal image segmentation method based on the diffusion model, which realizes retinal image enhancement and fundus blood vessel segmentation, and assists in the detection of diseases. The method specifically includes the following steps:
如图1所示，首先，利用Ye等人提出的照明模块方法来估计照明图。将低质量的眼底图像及其对应的照明图连接起来作为输入，输入改进的UNet网络架构，估计低质量眼底图像的退化因子。所述改进的UNet网络架构为：为了保留原始眼底图像的基本特征，加入了一个跳跃连接，以连接修改后的UNet体系结构的输入和输出。此外，除了保留精确的细节外，UNet中的对称扩展路径中还集成了照明注意机制。As shown in Figure 1, first, the illumination map is estimated using the illumination module method proposed by Ye et al. Low-quality fundus images and their corresponding illumination maps are connected as input, and the improved UNet network architecture is input to estimate the degradation factor of low-quality fundus images. The improved UNet network architecture is: in order to retain the basic characteristics of the original fundus image, a skip connection is added to connect the input and output of the modified UNet architecture. Furthermore, in addition to preserving precise details, a lighting attention mechanism is integrated in the symmetric expansion path in UNet.
其中，所述照明模块方法的来源于Ye L,Fu X,Liu A,et al.ADecomposition-based Network for Non-uniform Illuminated Retinal Image Enhancement[C]//202115th International Symposium on Medical Information and CommunicationTechnology(ISMICT).2021.DOI:10.1109/ISMICT51748.2021.9434912。其具体步骤不再赘述。Among them, the illumination module method is derived from Ye L, Fu .2021.DOI:10.1109/ISMICT51748.2021.9434912. The specific steps will not be described again.
对后期实验部分需要的数据进行图像预处理，具体包括：Perform image preprocessing on the data required for later experiments, including:
首先，对眼底视网膜图像利用退化网络进行退化，获取配对的低质量和高质量视网膜图像；First, the fundus retinal image is degraded using a degenerate network to obtain paired low-quality and high-quality retinal images;
然后，对眼底图像进行水平/垂直翻转以及旋转实现数据扩张；Then, perform horizontal/vertical flipping and rotation on the fundus image to achieve data expansion;
最后，将所有的眼底图像调整为统一大小。Finally, all fundus images were adjusted to a uniform size.
通过图像增强获得高质量的眼底图像是准确的血管分割的关键。这些增强的图像是精确识别血管位置的重要参考。为了解决这个问题，本实施例提出了一种新的联合任务方法，结合了图像增强和血管分割，旨在同时获得增强的图像和准确的分割结果。有别于以往依赖于条件扩散模型的图像增强方法，本实施例提出的方法利用了一个血管掩膜感知的扩散模型。该模型可以逐步生成高质量的眼底图像，通过改进血管掩膜，提高了图像的质量和血管分割的准确性。Obtaining high-quality fundus images through image enhancement is the key to accurate blood vessel segmentation. These enhanced images are an important reference for accurately identifying the location of blood vessels. In order to solve this problem, this embodiment proposes a new joint task method that combines image enhancement and blood vessel segmentation, aiming to obtain enhanced images and accurate segmentation results simultaneously. Different from previous image enhancement methods that rely on conditional diffusion models, the method proposed in this embodiment utilizes a blood vessel mask-aware diffusion model. This model can gradually generate high-quality fundus images, improving the quality of the image and the accuracy of blood vessel segmentation by improving the blood vessel mask.
在具体实施中，所述血管掩膜感知模块中，在将低质量眼底图像输入扩散模型前，先利用匹配滤波获得对应的初始的血管掩膜，将低质量眼底图像和获得的初始血管掩膜同时作为输入，将血管掩膜分割作为联合任务，利用训练好的掩膜预测头获得增强后的眼底图像和相对应的血管掩膜。In a specific implementation, in the blood vessel mask sensing module, before inputting the low-quality fundus image into the diffusion model, matching filtering is first used to obtain the corresponding initial blood vessel mask, and the low-quality fundus image and the obtained initial blood vessel mask are At the same time, as input, blood vessel mask segmentation is used as a joint task, and the trained mask prediction head is used to obtain the enhanced fundus image and the corresponding blood vessel mask.
首先，回顾一下条件扩散模型，这是一种通过重复细化来实现图像高分辨率的方法。First, let's review the conditional diffusion model, which is a method of achieving high image resolution through repeated refinement.
去噪扩散概率模型(DDPM)属于一类能够从高斯噪声中产生样本的生成模型。它通过马尔可夫过程获取关于数据分布的知识来实现。DDPM由两个过程组成：正向过程和反向过程。前向过程是一个通过高斯分布抽样得到的具有下一个状态的马尔可夫过程，反向过程表示数据的联合分布。前向扩散过程是一个马尔可夫链，将高斯噪声转换为x0，共T步：Denoising diffusion probabilistic model (DDPM) belongs to a class of generative models capable of generating samples from Gaussian noise. It does this by acquiring knowledge about the data distribution through a Markov process. DDPM consists of two processes: forward process and reverse process. The forward process is a Markov process with the next state obtained by sampling from the Gaussian distribution, and the backward process represents the joint distribution of the data. The forward diffusion process is a Markov chain that converts Gaussian noise into x0 for T steps:
在时间步长t处的状态也可以从初始状态x0中计算出来。这可以看作是通过在当前状态中添加一个带有方差调度βt的小高斯噪声而获得的下一个状态。因此，从初始状态x0开始的噪声目标xt分布表示为：The state at time step t can also be calculated from the initial state x0 . This can be seen as the next state obtained by adding a small Gaussian noise with variance scheduleβt to the current state. Therefore, the noise target xt distribution starting from the initial state x0 is expressed as:
其中，βt是噪声调度，βt∈[0,1]和∈～(0，I)。反向过程是一个生成步骤，也可以近似为高斯分布。它在t个时间步长上迭代地从标准高斯分布中进行去噪，生成相应的图像。然后，训练DDPM来近似于反向扩散过程：Among them, βt is the noise schedule, βt ∈[0,1] and ∈~(0,I). The reverse process is a generative step that can also be approximated as a Gaussian distribution. It iteratively denoises from a standard Gaussian distribution over t time steps, generating the corresponding image. Then, DDPM is trained to approximate the back-diffusion process:
最近的研究表明，反向过程的方差计划可以通过神经网络学习与正向过程相同的方差计划。反向步骤的平均值为：Recent research has shown that the variance plan of the reverse process can be learned by a neural network to the same variance plan as the forward process. The average of the reverse steps is:
为了实现去噪过程的学习，首先通过在x0中加入高斯噪声∈来生成样本xt～q(xt|x0)，然后训练一个模型∈θ(xt,t)来预测所添加的噪声。当分布的平均值相等时，训练目标可以进一步简化为：In order to realize the learning of the denoising process, samples xt ~ q (xt |x0 ) are first generated by adding Gaussian noise ∈ to x0 , and then a model ∈θ (xt ,t) is trained to predict the added noise. When the means of the distributions are equal, the training objective can be further simplified to:
本实施例提出的模型利用随机迭代去噪过程，该过程适应去噪扩散概率模型，允许它学习条件反向过程pθ(x0：T|y)，而不修改x的扩散过程q(x1：T|x0)。在训练阶段，从联合分布中采样三联体/>其中x表示高质量图像，y表示低质量图像，/>表示对应的初始血管掩膜。/>可以通过匹配滤波得到，有效地提取出血管的边缘，从而提取出整个血管结构。The model proposed in this embodiment utilizes a stochastic iterative denoising process that adapts to the denoising diffusion probability model, allowing it to learn the conditional inverse process pθ (x0: T |y) without modifying the diffusion process q(x1: T |x0 ). During the training phase, from the joint distribution Medium sampling triplet/> where x represents a high-quality image and y represents a low-quality image, /> Indicates the corresponding initial blood vessel mask. /> It can be obtained through matching filtering, which can effectively extract the edges of blood vessels and thereby extract the entire blood vessel structure.
算法1总结了所提方法的训练阶段，其中学习了血管掩膜感知扩散模型的反向过程：Algorithm 1 summarizes the training phase of the proposed method, where the inverse process of the vessel mask-aware diffusion model is learned:
算法1血管掩膜感知扩散模型的训练过程Algorithm 1 Training process of blood vessel mask-aware diffusion model
其中，表示在每个时间步长t的扩散过程中获得的中间图像。通过边缘化高斯扩散过程，利用/>直接从高质量图像x0中采样中间xt项。采用了一个类似于的去噪器∈θ的UNet体系结构。去噪器∈θ以低质量图像y、中间变量xt和时间步长t作为输入，预测噪声图nt和细化的容器掩膜vt如下：in, represents the intermediate image obtained during the diffusion process at each time step t. By marginalizing the Gaussian diffusion process, using/> Sample the intermediate xt terms directly from the high-quality image x0 . A UNet architecture similar to the denoiser ∈θ is adopted. The denoiser ∈θ takes as input a low-quality image y, an intermediate variable xt , and a time step t, and predicts the noise map nt and the refined container mask vt as follows:
由于血管掩膜信息对高质量视网膜图像生成的强烈依赖，以及低质量视网膜图像的增强与相应的血管掩膜之间的密切相关性，本实施例提出了一个联合进行高质量图像预测和血管掩膜细化的模型。在每个时间步长t中，将一个掩膜预测头合并到∈θ的体系结构中，这个附加的组件，由一个1×1的卷积层和一个Sigmoid函数组成，负责基于∈θ最后一层的输出来预测改进的眼底血管掩膜。Due to the strong dependence of blood vessel mask information on the generation of high-quality retinal images, and the close correlation between the enhancement of low-quality retinal images and the corresponding blood vessel masks, this embodiment proposes a joint method for high-quality image prediction and blood vessel masking. Model of membrane refinement. At each time step t, a mask prediction head is incorporated into the architecture of ∈θ . This additional component, consisting of a 1 × 1 convolutional layer and a sigmoid function, is responsible for the last prediction based on ∈θ . The output of the layer is used to predict the improved fundus vessel mask.
进一步地，条件去噪方法学习了对噪声的预测。扩散目标函数为：Further, conditional denoising methods learn predictions about noise. The diffusion objective function is:
此外，在训练阶段，通过利用高质量和低质量的图像对，本实施例可以使用地面真实值(GT)血管掩膜作为参考，对掩膜的细化过程进行约束，确保其相干性和准确性。In addition, during the training phase, by utilizing high-quality and low-quality image pairs, this embodiment can use the ground truth (GT) blood vessel mask as a reference to constrain the mask refinement process to ensure its coherence and accuracy. sex.
特别地，通过将低质量图像与相应的高质量图像之间的残差映射二值化，可以得到地面真实血管掩膜In particular, ground-truth blood vessel masks can be obtained by binarizing the residual mapping between low-quality images and corresponding high-quality images
通过结合上述损失，本实施例得到了混合目标函数Ltotal，它在我们的血管掩膜感知模块中指导去噪器∈θ的训练如下：By combining the above losses, this embodiment obtains the hybrid objective function Ltotal , which guides the training of the denoiser ∈θ in our blood vessel mask perception module as follows:
Ltotal＝Ldiff+λLmaskLtotal =Ldiff +λLmask
其中，λ表示用于平衡各项影响的加权系数。Among them, λ represents the weighting coefficient used to balance various influences.
进一步地，所述退化因子提取模块中，没有直接将低质量的图像放到网络中以消除退化因子并获得增强的图像，而是通过单独的网络分支设计来明确地提取退化因子，具体包括：Furthermore, in the degradation factor extraction module, low-quality images are not directly put into the network to eliminate degradation factors and obtain enhanced images. Instead, degradation factors are explicitly extracted through separate network branch designs, specifically including:
退化的彩色眼底图像可以被认为是干净的图像和退化因子的组合。因此，对退化的眼底图像的描述可以简化如下：Degraded color fundus images can be considered as a combination of clean images and degradation factors. Therefore, the description of the degraded fundus image can be simplified as follows:
y＝x+dyy＝x+dy
其中，x表示对应的干净图像，y表示退化的低质量图像，dy表示与y相关的退化因子。当获得退化的低质量图像y时，高质量图像x’的恢复过程变为：Among them, x represents the corresponding clean image, y represents the degraded low-quality image, and dy represents the degradation factor related to y. When a degraded low-quality image y is obtained, the recovery process of a high-quality image x' becomes:
x′＝y-dyx′=ydy
关键问题是识别隐藏在y中的降解因素。退化因子提取网络不是直接将低质量的图像输入网络以消除退化因子并获得增强的图像，而是设计了一个专门的分支来明确地提取退化因子。给定一个低质量的图像y，相反的退化因子-dy可以被提取为-dy＝U(y)，其中，U()表示如图2所示的修改后的UNet体系结构。整体恢复网络G的公式可以表示为：The key issue is to identify the degradation factors hidden in y. Instead of directly feeding low-quality images into the network to eliminate degradation factors and obtain enhanced images, the degradation factor extraction network designs a specialized branch to explicitly extract degradation factors. Given a low-quality image y, the opposite degradation factor -dy can be extracted as -dy =U(y), where U() represents the modified UNet architecture as shown in Figure 2. The formula of the overall recovery network G can be expressed as:
x′＝G(y)＝y+U(y)x′=G(y)=y+U(y)
其中，所述改进的UNet体系结构的训练过程为：以低质量眼底图像和照明图为输入，以低质量眼底图像相关特征为输出，训练改进的UNet体系结构。The training process of the improved UNet architecture is as follows: using low-quality fundus images and illumination images as inputs, and using low-quality fundus image related features as outputs to train the improved UNet architecture.
进一步地，所述改进的UNet体系架构具体包括：跳跃连接，存在于UNet的输入和输出之间，以保留原始眼底图像的大部分重要特征；五个照明注意力模块，存在于UNet中的对称扩展路径中，以保留更精确的眼底图像的细节。Further, the improved UNet architecture specifically includes: skip connections, which exist between the input and output of UNet, to retain most of the important features of the original fundus image; five illumination attention modules, which exist in the symmetry of UNet Expand the path to preserve details in more precise fundus images.
进一步地，所述退化模块中，将低质量的眼底图像及其对应的照明图连接起来作为输入，通过改进的UNet体系结构保留原始眼底图像的基本特征和精确地细节，获得低质量眼底图像的退化因子，嵌入扩散模型迭代过程中去进一步恢复低质量图像。Further, in the degradation module, low-quality fundus images and their corresponding illumination maps are connected as input, and the basic characteristics and precise details of the original fundus images are retained through the improved UNet architecture, and the low-quality fundus images are obtained. The degradation factor is embedded in the diffusion model iteration process to further restore low-quality images.
可以通过在输入y和U(y)的输出之间引入一个跳跃连接来实现，将这个过程纳入到扩散模型的迭代中在每个时间步长t上细化结果。因此，等式可以修改为：This can be achieved by introducing a skip connection between the input y and the output of U(y), incorporating this process into the iterations of the diffusion model to refine the results at each time step t. Therefore, the equation can be modified as:
其中，所获得的作为扩散过程中的下一个时间步骤t的输入。这种方法在提高图像的整体质量和保留视盘和黄斑等关键特征之间进行了谨慎的平衡。图像增强过程保持了这些重要元素的结构完整性和空间一致性，确保它们以最小的位移或扭曲被保留。增强的图像显示出更好的视觉清晰度和细节，同时保持了临床分析和诊断所需的关键解剖结构的准确表现。Among them, the obtained as input to the next time step t in the diffusion process. This approach strikes a careful balance between improving the overall quality of the image and preserving key features such as the optic disc and macula. The image enhancement process maintains the structural integrity and spatial consistency of these important elements, ensuring they are preserved with minimal displacement or distortion. Enhanced images display greater visual clarity and detail while maintaining accurate representation of key anatomical structures required for clinical analysis and diagnosis.
利用训练完成的血管掩膜感知模块和退化模块相结合，嵌入到扩散模型的每个迭代过程中，输出最终的高质量眼底图像和对应的高分辨率的眼底血管掩膜。The trained blood vessel mask perception module and degradation module are combined and embedded in each iteration of the diffusion model to output the final high-quality fundus image and the corresponding high-resolution fundus blood vessel mask.
本实施例中，由于缺乏配对的低质量的视网膜图像和高质量的视网膜图像及其相应的高分辨率血管掩膜来训练本实施例提出的模型，利用沈等人提出的预先训练的数据驱动退化模型来降低视网膜图像。数据集采用的是公开普遍使用的DRIVE，STARE和CHASE_DB1，退化前的图像如图3(a)所示，退化后的图像如图3(b)所示。其中，DRIVE数据集由40张视网膜图像组成，分辨率为565×584像素；CHASE_DB1数据集由28张视网膜图像组成，分辨率为999×960像素；STARE数据集总共包含20张图像，分辨率为700×605像素。为了解决训练数据有限的问题，本实施例采用了在所有三个数据集上进行水平和垂直翻转以及旋转的数据增强，得到的训练集分别为80、48和80张图像。测试集中共有32张图像，其中DRIVE数据集上有20张图像，STARE数据集上有4张图像，CHASE_DB1数据集上有8张图像。In this embodiment, due to the lack of paired low-quality retinal images and high-quality retinal images and their corresponding high-resolution blood vessel masks to train the model proposed in this embodiment, the pre-trained data driver proposed by Shen et al. Degenerate models to degrade retinal images. The data set uses the commonly used DRIVE, STARE and CHASE_DB1. The image before degradation is shown in Figure 3(a), and the image after degradation is shown in Figure 3(b). Among them, the DRIVE data set consists of 40 retinal images with a resolution of 565×584 pixels; the CHASE_DB1 data set consists of 28 retinal images with a resolution of 999×960 pixels; the STARE data set contains a total of 20 images with a resolution of 999×960 pixels. 700×605 pixels. In order to solve the problem of limited training data, this embodiment uses data enhancement of horizontal and vertical flipping and rotation on all three data sets, and the resulting training sets are 80, 48 and 80 images respectively. There are a total of 32 images in the test set, including 20 images on the DRIVE data set, 4 images on the STARE data set, and 8 images on the CHASE_DB1 data set.
EyeQ数据集由12534张训练图像和16249张测试图像组成，根据其质量分为三类：16817张高质量图像，6435张可用质量图像和5540张拒绝质量图像。所有图像的固定分辨率都是800×800。本实施例随机选择了1544张“拒绝”图像作为测试集。The EyeQ dataset consists of 12534 training images and 16249 test images, which are divided into three categories according to their quality: 16817 high-quality images, 6435 usable-quality images, and 5540 rejection-quality images. The fixed resolution of all images is 800×800. This embodiment randomly selects 1544 "rejected" images as the test set.
本实施例使用PyTorch实现，并在NVIDIA RTX A30 GPU上进行训练。为了进行训练和测试，将所有输入图像的大小调整为512×512像素。This embodiment is implemented using PyTorch and trained on NVIDIA RTX A30 GPU. For training and testing, all input images are resized to 512 × 512 pixels.
本实施例使用峰值信噪比(PSNR)和结构相似度指数度量(SSIM)评估了该模型的图像增强性能。此外，需要注意的是，定量指标PSNR和SSIM需要成对的图像。PSNR量化了图像受到噪声的影响程度，而SSIM测量或预测了图像相对于原始未压缩或未失真图像的质量。对于PSNR和SSIM指标，较高的值表示更好的结果。This embodiment evaluates the image enhancement performance of the model using peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). Furthermore, it should be noted that the quantitative metrics PSNR and SSIM require paired images. PSNR quantifies how much an image is affected by noise, while SSIM measures or predicts the quality of an image relative to the original uncompressed or undistorted image. For PSNR and SSIM metrics, higher values indicate better results.
本实施例使用准确性(Acc)、敏感性(Se)、特异性(Sp)、ROC曲线下面积(AUC)和F1-score等评价指标来评估血管分割任务模型的性能。Acc是一个度量正确分类的像素与数据集中像素总数的比率的度量标准。Se评估正确识别真阳性的准确性，而Sp衡量正确识别真阴性的准确性。F1-score分数越高，说明模型能够准确识别血管像素，同时最小化非血管区域作为血管的错误分类。This embodiment uses evaluation indicators such as accuracy (Acc), sensitivity (Se), specificity (Sp), area under the ROC curve (AUC), and F1-score to evaluate the performance of the blood vessel segmentation task model. Acc is a metric that measures the ratio of correctly classified pixels to the total number of pixels in the dataset. Se evaluates the accuracy of correctly identifying true positives, while Sp measures the accuracy of correctly identifying true negatives. The higher the F1-score, the model can accurately identify blood vessel pixels while minimizing the misclassification of non-blood vessel areas as blood vessels.
本实施例与7种最先进的方法进行定性和定量比较以展示提出的网络的优势。对退化的眼底图像进行了实验，对模型的图像增强性能和血管分割性能进行了定性和定量分析。实验结果表明，该方法能有效地恢复眼底图像，保留清晰的细节，并准确地分割血管。与竞争方法的图像增强性能的比较如图4所示，展示了提出的模型、SCRNet和CycleGAN图像增强后的结果，提出的方法保持了视盘和血管的局部结构的一致性，特别是增强了不均匀的照明。表1和图6表示了本公开实施例图像增强效果数据。This example performs qualitative and quantitative comparisons with 7 state-of-the-art methods to demonstrate the advantages of the proposed network. Experiments were conducted on degraded fundus images, and the image enhancement performance and blood vessel segmentation performance of the model were qualitatively and quantitatively analyzed. Experimental results show that this method can effectively restore fundus images, retain clear details, and accurately segment blood vessels. A comparison of the image enhancement performance with competing methods is shown in Figure 4, which shows the results of image enhancement by the proposed model, SCRNet and CycleGAN. The proposed method maintains the consistency of the local structure of the optic disc and blood vessels, especially enhancing the different Uniform lighting. Table 1 and Figure 6 show the image enhancement effect data of the embodiment of the present disclosure.
表1本公开实施例图像增强效果数据Table 1 Image enhancement effect data of embodiments of the present disclosure
图5显示了原始的UNet和本实施例提出的模型的结果，以及相应的地面真相。从结果中可以明显看出，本实施例在血管分割方面优于UNet，因为它有效地分割了视网膜血管的更精细的细节，显示出更少的假阳性血管，导致更少的噪声和更清晰的分割。Figure 5 shows the results of the original UNet and the model proposed in this example, as well as the corresponding ground truth. It is evident from the results that this embodiment outperforms UNet in vessel segmentation as it effectively segments retinal vessels in finer details, showing fewer false positive vessels, resulting in less noise and greater clarity of division.
本实施例所述系统在三个数据集上与CBAM-UNet和UNet的血管分割检测对比结果及相应数据如图7以及表2所示。The comparison results and corresponding data of blood vessel segmentation detection between the system described in this embodiment and CBAM-UNet and UNet on three data sets are shown in Figure 7 and Table 2.
表2本公开实施例检测效果数据Table 2 Detection effect data of embodiments of the present disclosure
本实施例选择在EyeQ数据集上进行了消融研究，以评估在所述框架内的图像增强设计模块的有效性。此外，通过在DRIVE、STARE和CHASE_DB1数据集上的消融研究来评估血管分割的有效性。由于血管分割与图像增强密切相关，更高质量的图像恢复可以导致更准确的分割。退化模型保留了原始眼底图像的大部分重要特征，并确定了低质量图像的退化因素。这些退化因素被纳入迭代过程中，以指导在每个时间步长t的中间输出的细化。退化模块的消融研究的定性比较如图7及表3所示。添加退化模型后，图像增强的结果可以更突出血管结构，可以分割更多的小血管。This embodiment chooses to conduct an ablation study on the EyeQ data set to evaluate the effectiveness of the image enhancement design module within the framework. Furthermore, the effectiveness of vessel segmentation is evaluated through ablation studies on DRIVE, STARE and CHASE_DB1 datasets. Since blood vessel segmentation is closely related to image enhancement, higher quality image restoration can lead to more accurate segmentation. The degradation model retains most of the important features of the original fundus image and identifies the degradation factors of low-quality images. These degradation factors are incorporated into the iterative process to guide the refinement of the intermediate output at each time step t. A qualitative comparison of ablation studies of degraded modules is shown in Figure 7 and Table 3. After adding the degradation model, the image enhancement results can highlight the vascular structure more and segment more small blood vessels.
表3本公开实施例对退化模块消融实验的对比数据Table 3 Comparative data of degraded module ablation experiments according to embodiments of the present disclosure
实施例二Embodiment 2
本实施例的目的是提供一种基于扩散模型的低质量眼底图像的增强及分割系统。The purpose of this embodiment is to provide a diffusion model-based enhancement and segmentation system for low-quality fundus images.
一种基于扩散模型的低质量眼底图像的增强及分割系统，包括：A diffusion model-based enhancement and segmentation system for low-quality fundus images, including:
数据获取单元，其用于获取待处理的低质量眼底图像及其对应的照明图；a data acquisition unit, which is used to acquire the low-quality fundus image to be processed and its corresponding illumination map;
掩膜图像获取单元，其用于基于所述低质量眼底图像，通过匹配滤波获得对应的眼底血管掩膜；A mask image acquisition unit configured to obtain a corresponding fundus blood vessel mask through matching filtering based on the low-quality fundus image;
退化因子获取单元，其用于基于所述低质量眼底图像及其对应的照明图，利用预先训练的UNet网络，获得低质量图像的退化因子，其中，所述UNet网络的输入与输出之间设置有跳跃连接，且UNet网络的对称扩展路径中设置有注意力机制；Degradation factor acquisition unit, which is used to obtain the degradation factor of the low-quality image based on the low-quality fundus image and its corresponding illumination map using the pre-trained UNet network, wherein the input and output of the UNet network are set between There are skip connections, and an attention mechanism is set up in the symmetric expansion path of the UNet network;
增强及分割单元，其用于以所述低质量眼底图像及其对应的眼底血管掩膜作为预先训练的扩散模型的输入，获得图像增强后的眼底图像及其对应的眼底血管掩膜；其中，所述扩散模型采用UNet网络，所述UNet网络最后一层去噪器之后设置有用于预测眼底血管掩膜的预测头，且所述退化因子嵌入至所述扩散模型中，与获得图像增强后的眼底图像进行关联。An enhancement and segmentation unit configured to use the low-quality fundus image and its corresponding fundus blood vessel mask as input to a pre-trained diffusion model to obtain an image-enhanced fundus image and its corresponding fundus blood vessel mask; wherein, The diffusion model adopts the UNet network. After the last layer of the denoiser of the UNet network, a prediction head for predicting the fundus blood vessel mask is provided, and the degradation factor is embedded in the diffusion model, and the image enhancement is obtained. Fundus images are associated.
本实施例所述系统与实施例一中所述方法相对应，其技术细节在实施例一中已经进行了详细说明，故此处不再赘述。The system described in this embodiment corresponds to the method described in Embodiment 1, and its technical details have been described in detail in Embodiment 1, so they will not be described again here.
实施例三Embodiment 3
本实施例的目的是提供一种电子设备。The purpose of this embodiment is to provide an electronic device.
一种电子设备，包括存储器、处理器及存储在存储器上运行的计算机程序，所述处理器执行所述程序时实现所述的一种基于扩散模型的低质量眼底图像的增强及分割方法。An electronic device includes a memory, a processor, and a computer program stored and run on the memory. When the processor executes the program, it implements the diffusion model-based enhancement and segmentation method of low-quality fundus images.
实施例四Embodiment 4
本实施例的目的是提供一种非暂态计算机可读存储介质。The purpose of this embodiment is to provide a non-transitory computer-readable storage medium.
一种非暂态计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现所述的一种基于扩散模型的低质量眼底图像的增强及分割方法。A non-transitory computer-readable storage medium on which a computer program is stored, which implements the diffusion model-based enhancement and segmentation method of low-quality fundus images when executed by a processor.
以上实施例二中所述系统以及实施例三和四中涉及的各步骤与方法实施例一相对应，具体实施方式可参见实施例一的相关说明部分。术语“计算机可读存储介质”应该理解为包括一个或多个指令集的单个介质或多个介质；还应当被理解为包括任何介质，所述任何介质能够存储、编码或承载用于由处理器执行的指令集并使处理器执行本公开中的任一方法。The system described in the above second embodiment and the steps involved in the third and fourth embodiments correspond to the first method embodiment. For specific implementation details, please refer to the relevant description part of the first embodiment. The term "computer-readable storage medium" shall be understood to include a single medium or multiple media that includes one or more sets of instructions; and shall also be understood to include any medium capable of storing, encoding, or carrying instructions for use by a processor. The set of instructions executed cause the processor to perform any of the methods of the disclosure.
本领域技术人员应该明白，上述本公开中的各个模块或步骤可利用通用计算设备来实现。作为可选方案，它们也可以使用计算设备可执行的程序代码进行实现。因此，这些模块或步骤可被存储于储存设备中，由计算设备来执行；或者，它们可分别被构建成独立的集成电路模块；或者，这些模块或步骤中的多个可被制作成单一的集成电路模块以达到实现的目的。值得强调的是，本公开并不受特定硬件与软件组合的限制。本公开不限制于任何特定的硬件和软件的结合。Those skilled in the art should understand that each module or step in the present disclosure described above can be implemented using general-purpose computing devices. Alternatively, they may be implemented using program code executable by a computing device. Therefore, these modules or steps can be stored in a storage device and executed by a computing device; or they can be each constructed as an independent integrated circuit module; or multiple of these modules or steps can be made into a single Integrated circuit modules to achieve the purpose. It is worth emphasizing that the present disclosure is not limited to a specific combination of hardware and software. This disclosure is not limited to any specific combination of hardware and software.
本公开的描述是基于根据本公开实施例所示的方法、设备(系统)以及计算机程序产品的流程图和/或方框图来呈现的。在这里，应理解每一个流程和/或方框，如同流程图和/或方框图中所示，都可以通过计算机程序指令来实现。这些计算机程序指令可被传递给通用计算机、专用计算机、嵌入式处理器或其他可编程数据处理设备中的处理器，从而生成一台机器。通过这些指令，由计算机或其他可编程数据处理设备的处理器执行，从而产生一个装置，用于实现流程图中的一个或多个流程以及/或方框图中的一个或多个方框所指定的功能。The description of the present disclosure is presented based on flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products illustrated according to embodiments of the disclosure. Here, it will be understood that each process and/or block, as shown in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be passed to a processor in a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine. These instructions, when executed by a processor of a computer or other programmable data processing device, produce an apparatus for implementing one or more processes in the flowchart illustrations and/or one or more blocks in the block diagrams. Function.
这些计算机程序指令还可以被储存于能够引导计算机或其他可编程数据处理设备以特定方式运作的计算机可读存储介质中。这样，这些存储在计算机可读存储介质中的指令就能够生成一个包含指令装置的制品，该指令装置能够实现流程图中一个或多个流程，以及/或者方框图中一个或多个方框所规定的功能。These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to operate in a particular manner. In this way, the instructions stored in the computer-readable storage medium can produce an article containing instruction means that can implement one or more processes in the flowchart and/or one or more blocks in the block diagram. function.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions may also be loaded onto a computer or other programmable data processing device, and a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processes, thereby causing the instructions to be executed on the computer or other programmable device. Provides steps for implementing the functionality specified in a process or processes in a flow diagram and/or in a block or blocks in a block diagram.
以上所述仅为本公开的优选实施例而已，并不用于限制本公开，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。The above descriptions are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and changes. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.
上述虽然结合附图对本公开的具体实施方式进行了描述，但并非对本公开保护范围的限制，所属领域技术人员应该明白，在本公开的技术方案的基础上，本领域技术人员不需要付出创造性劳动即可做出的各种修改或变形仍在本公开的保护范围以内。Although the specific embodiments of the present disclosure have been described above in conjunction with the accompanying drawings, they do not limit the scope of the present disclosure. Those skilled in the art should understand that on the basis of the technical solutions of the present disclosure, those skilled in the art do not need to make creative efforts. Various modifications or deformations can be made and still fall within the scope of the present disclosure.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311265391.5ACN117314935A (en) | 2023-09-27 | 2023-09-27 | Diffusion model-based low-quality fundus image enhancement and segmentation method and system |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311265391.5ACN117314935A (en) | 2023-09-27 | 2023-09-27 | Diffusion model-based low-quality fundus image enhancement and segmentation method and system |
| Publication Number | Publication Date |
|---|---|
| CN117314935Atrue CN117314935A (en) | 2023-12-29 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311265391.5APendingCN117314935A (en) | 2023-09-27 | 2023-09-27 | Diffusion model-based low-quality fundus image enhancement and segmentation method and system |
| Country | Link |
|---|---|
| CN (1) | CN117314935A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118379209A (en)* | 2024-06-25 | 2024-07-23 | 海纳云物联科技有限公司 | Night scene image enhancement method, device, equipment and medium |
| WO2025157199A1 (en)* | 2024-01-24 | 2025-07-31 | The Hong Kong University Of Science And Technology | Motion artifact reduction in optical coherence tomography angiography images |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113658097A (en)* | 2021-07-15 | 2021-11-16 | 南方科技大学 | A method and device for training a fundus image quality enhancement model |
| CN115829885A (en)* | 2022-10-21 | 2023-03-21 | 南方科技大学 | Method, system, device and medium for enhancing fundus image based on feature consistency |
| CN116071256A (en)* | 2023-01-09 | 2023-05-05 | 上海人工智能创新中心 | A method for unified image inpainting and enhancement that generates diffusion priors |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113658097A (en)* | 2021-07-15 | 2021-11-16 | 南方科技大学 | A method and device for training a fundus image quality enhancement model |
| CN115829885A (en)* | 2022-10-21 | 2023-03-21 | 南方科技大学 | Method, system, device and medium for enhancing fundus image based on feature consistency |
| CN116071256A (en)* | 2023-01-09 | 2023-05-05 | 上海人工智能创新中心 | A method for unified image inpainting and enhancement that generates diffusion priors |
| Title |
|---|
| LANQING GUO等人: "ShadowDiffusion:When Degradation Prior Meets Diffusion Model for Shadow Removal", HTTPS://ARXIV.ORG/ABS/2212.04711, 9 December 2022 (2022-12-09), pages 1 - 10* |
| ZHENDONG WANG等人: "Uformer: A General U-Shaped Transformer for Image Restoration", 《IN PROCEEDINGS OF THE IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》, 24 June 2022 (2022-06-24), pages 1 - 11* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025157199A1 (en)* | 2024-01-24 | 2025-07-31 | The Hong Kong University Of Science And Technology | Motion artifact reduction in optical coherence tomography angiography images |
| CN118379209A (en)* | 2024-06-25 | 2024-07-23 | 海纳云物联科技有限公司 | Night scene image enhancement method, device, equipment and medium |
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Iternet: Retinal image segmentation utilizing structural redundancy in vessel networks | |
| CN110992382B (en) | Fundus image optic cup optic disc segmentation method and system for assisting glaucoma screening | |
| Zhang et al. | Automatic segmentation of the cardiac MR images based on nested fully convolutional dense network with dilated convolution | |
| CN112102283A (en) | Retina fundus blood vessel segmentation method based on depth multi-scale attention convolution neural network | |
| CN115546570B (en) | A vascular image segmentation method and system based on three-dimensional deep network | |
| CN117314935A (en) | Diffusion model-based low-quality fundus image enhancement and segmentation method and system | |
| CN113012163A (en) | Retina blood vessel segmentation method, equipment and storage medium based on multi-scale attention network | |
| CN117132774B (en) | Multi-scale polyp segmentation method and system based on PVT | |
| CN108764342B (en) | A Semantic Segmentation Method for Optic Disc and Optic Cup in Fundus Map | |
| Xu et al. | Computer aided diagnosis of diabetic retinopathy based on multi-view joint learning | |
| CN118470031B (en) | A retinal vessel segmentation method based on a multi-level full-resolution feature selection network | |
| CN115035127A (en) | A Retinal Vessel Segmentation Method Based on Generative Adversarial Networks | |
| Zang et al. | Interpretable diabetic retinopathy diagnosis based on biomarker activation map | |
| Lin et al. | Blu-gan: Bi-directional convlstm u-net with generative adversarial training for retinal vessel segmentation | |
| Jana et al. | Liver fibrosis and nas scoring from ct images using self-supervised learning and texture encoding | |
| CN117036905A (en) | Capsule endoscope image focus identification method based on HSV color space color attention | |
| Haider et al. | Modified anam-net based lightweight deep learning model for retinal vessel segmentation. | |
| Alimanov et al. | A hybrid approach for retinal image super-resolution | |
| Bhandari et al. | Soft Attention Mechanism Based Network to Extract Blood Vessels From Retinal Image Modality | |
| Wang et al. | An efficient hierarchical optic disc and cup segmentation network combined with multi-task learning and adversarial learning | |
| Liu et al. | AGFA-Net: Attention-Guided Feature-Aggregated Network for Coronary Artery Segmentation Using Computed Tomography Angiography | |
| Patel et al. | DiaRet: a browser-based application for the grading of diabetic retinopathy with integrated gradients | |
| Zheng et al. | Image segmentation of intestinal polyps using attention mechanism based on convolutional neural network | |
| Song et al. | Spatial-frequency fusion for retinal vessel segmentation | |
| CN117523189A (en) | Choroidal blood vessel segmentation method, device, terminal equipment and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |