Part of the book series:Lecture Notes in Computer Science ((LNIP,volume 13003))
Included in the following conference series:
2028Accesses
14Citations
Abstract
This paper strives to generate a synthetic computed tomography (CT) image from a magnetic resonance (MR) image. The synthetic CT image is valuable for radiotherapy planning when only an MR image is available. Recent approaches have made large strides in solving this challenging synthesis problem with convolutional neural networks that learn a mapping from MR inputs to CT outputs. In this paper, we find that all existing approaches share a common limitation: reconstruction breaks down in and around the high-frequency parts of CT images. To address this common limitation, we introduce frequency-supervised deep networks to explicitly enhance high-frequency MR-to-CT image reconstruction. We propose a frequency decomposition layer that learns to decompose predicted CT outputs into low- and high-frequency components, and we introduce a refinement module to improve high-frequency reconstruction through high-frequency adversarial learning. Experimental results on a new dataset with 45 pairs of 3D MR-CT brain images show the effectiveness and potential of the proposed approach. Code is available athttps://github.com/shizenglin/Frequency-Supervised-MR-to-CT-Image-Synthesis.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 7435
- Price includes VAT (Japan)
- Softcover Book
- JPY 9294
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Berker, Y., et al.: MRI-based attenuation correction for hybrid PET/MRI systems: a 4-class tissue segmentation technique using a combined ultrashort-echo-time/Dixon MRI sequence. J. Nucl. Med.53(5), 796–804 (2012)
Burgos, N., Guerreiro, F., et al.: Iterative framework for the joint segmentation and CT synthesis of MR images: application to MRI-only radiotherapy treatment planning. Phys. Med. Biol.62, 4237–4253 (2017)
Burgos, N., et al.: Robust CT synthesis for radiotherapy planning: application to the head and neck region. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9350, pp. 476–484. Springer, Cham (2015).https://doi.org/10.1007/978-3-319-24571-3_57
Burgos, N., et al.: Joint segmentation and CT synthesis for MRI-only radiotherapy treatment planning. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 547–555. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46723-8_63
Burgos, N., et al.: Iterative framework for the joint segmentation and CT synthesis of MR images: application to MRI-only radiotherapy treatment planning. Phys. Med. Biol.62(11), 4237 (2017)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46723-8_49
Degen, J., Heinrich, M.P.: Multi-atlas based pseudo-CT synthesis using multimodal image registration and local atlas fusion strategies. In: CVPR Workshops, pp. 160–168 (2016)
Emami, H., Dong, M., Nejad-Davarani, S.P., Glide-Hurst, C.K.: Generating synthetic CTs from magnetic resonance images using generative adversarial networks. Med. Phys.45(8), 3627–3636 (2018)
Ge, Y., Wei, D., Xue, Z., Wang, Q., Zhou, X., Zhan, Y., Liao, S.: Unpaired MR to CT synthesis with explicit structural constrained adversarial learning. In: ISBI (2019)
Guerreiro, F., et al.: Evaluation of a multi-atlas CT synthesis approach for MRI-only radiotherapy treatment planning. Physica Med.35, 7–17 (2017)
Han, X.: MR-based synthetic CT generation using a deep convolutional neural network. Med. Phys.44(4), 1408–1419 (2017)
Hangartner, T.N.: Thresholding technique for accurate analysis of density and geometry in QCT, PQCT and muCT images. J. Musculoskelet. Neuronal Interact.7(1), 9 (2007)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)
Jin, C.B., et al.: Deep CT to MR synthesis using paired and unpaired data. Sensors19(10), 2361 (2019)
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. In: ICLR (2019)
Kläser, K., et al.: Improved MR to CT synthesis for PET/MR attenuation correction using imitation learning. In: SSMI Workshop, pp. 13–21 (2019)
Korhonen, J., Kapanen, M., et al.: A dual model HU conversion from MRI intensity values within and outside of bone segment for MRI-based radiotherapy treatment planning of prostate cancer. Med Phys41, 011704 (2014)
Largent, A., et al.: Pseudo-CT generation for MRI-only radiation therapy treatment planning: comparison among patch-based, atlas-based, and bulk density methods. Int. J. Radiat. Oncol.* Biol.* Phys.103(2), 479–490 (2019)
Largent, A., et al.: Comparison of deep learning-based and patch-based methods for pseudo-CT generation in MRI-based prostate dose planning. Int. J. Radiat. Oncol.* Biol.* Phys.105(5), 1137–1150 (2019)
Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, efficiency, and representation of 3D convolutional networks: brain parcellation as a pretext task. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 348–360. Springer, Cham (2017).https://doi.org/10.1007/978-3-319-59050-9_28
Lin, Z., Zhong, M., Zeng, X., Ye, C.: Frequency-selective learning for CT to MR synthesis. In: Burgos, N., Svoboda, D., Wolterink, J.M., Zhao, C. (eds.) SASHIMI 2020. LNCS, vol. 12417, pp. 101–109. Springer, Cham (2020).https://doi.org/10.1007/978-3-030-59520-3_11
Liu, F., Jang, H., et al.: Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology286(2), 676–684 (2017)
Mathieu, M., Couprie, C., LeCun, Y.: Deep multi-scale video prediction beyond mean square error. In: ICLR (2016)
Navalpakkam, B., Braun, H., et al.: Magnetic resonance-based attenuation correction for PET/MR hybrid imaging using continuous valued attenuation maps. Invest. Radiol.48, 323–332 (2013)
Nie, D., Cao, X., Gao, Y., Wang, L., Shen, D.: Estimating CT image from MRI data using 3D fully convolutional networks. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 170–178. Springer, Cham (2016).https://doi.org/10.1007/978-3-319-46976-8_18
Nie, D., et al.: Medical image synthesis with context-aware generative adversarial networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 417–425. Springer, Cham (2017).https://doi.org/10.1007/978-3-319-66179-7_48
Peng, C., Zhang, X., Yu, G., Luo, G., Sun, J.: Large kernel matters-improve semantic segmentation by global convolutional network. In: CVPR (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).https://doi.org/10.1007/978-3-319-24574-4_28
Roy, S., Butman, J.A., Pham, D.L.: Synthesizing CT from ultrashort echo-time MR images via convolutional neural networks. In: SSMI Workshop, pp. 24–32 (2017)
Sjölund, J., Forsberg, D., Andersson, M., Knutsson, H.: Generating patient specific pseudo-CT of the head from MR using atlas-based regression. Phys. Med. Biol.60(2), 825 (2015)
Wolterink, J.M., Dinkla, A.M., Savenije, M.H., Seevinck, P.R., van den Berg, C.A., Išgum, I.: MR-to-CT synthesis using cycle-consistent generative adversarial networks. In: NIPS (2017)
Author information
Authors and Affiliations
University of Amsterdam, Amsterdam, The Netherlands
Zenglin Shi, Pascal Mettes & Cees Snoek
Shanghai Jiao Tong University, Shanghai, China
Guoyan Zheng
- Zenglin Shi
Search author on:PubMed Google Scholar
- Pascal Mettes
Search author on:PubMed Google Scholar
- Guoyan Zheng
Search author on:PubMed Google Scholar
- Cees Snoek
Search author on:PubMed Google Scholar
Corresponding author
Correspondence toZenglin Shi.
Editor information
Editors and Affiliations
Universitätsklinikum Heidelberg, Heidelberg, Germany
Sandy Engelhardt
Istanbul Technical University, Istanbul, Turkey
Ilkay Oksuz
The University of Texas at Arlington, Arlington, TX, USA
Dajiang Zhu
University of Hong Kong, Hong Kong, Hong Kong
Yixuan Yuan
TU Darmstadt, Darmstadt, Germany
Anirban Mukhopadhyay
University of Minnesota, Minneapolis, MN, USA
Nicholas Heller
Pennsylvania State University, University Park, PA, USA
Sharon Xiaolei Huang
University of Houston, Houston, TX, USA
Hien Nguyen
University of Bern, Bern, Switzerland
Raphael Sznitman
Johns Hopkins University, Baltimore, MD, USA
Yuan Xue
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, Z., Mettes, P., Zheng, G., Snoek, C. (2021). Frequency-Supervised MR-to-CT Image Synthesis. In: Engelhardt, S.,et al. Deep Generative Models, and Data Augmentation, Labelling, and Imperfections. DGM4MICCAI DALI 2021 2021. Lecture Notes in Computer Science(), vol 13003. Springer, Cham. https://doi.org/10.1007/978-3-030-88210-5_1
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-030-88209-9
Online ISBN:978-3-030-88210-5
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative