Part of the book series:Lecture Notes in Computer Science ((LNIP,volume 12901))
Included in the following conference series:
14kAccesses
Abstract
In clinical practice, a desirable medical image segmentation model should be able to learn from sequential training data from multiple sites, as collecting these data together could be difficult due to the storage cost and privacy restriction. However, existing methods often suffer from catastrophic forgetting problem for previous sites when learning from images from a new site. In this paper, we propose a novel comprehensive importance-based selective regularization method for continual segmentation, aiming to mitigate model forgetting by maintaining both shape and reliable semantic knowledge for previous sites. Specifically, we define a comprehensive importance weight for each model parameter, which consists of shape-aware importance and uncertainty-guided semantics-aware importance, by measuring how a segmentation’s shape and reliable semantic information is sensitive to the parameter. When training model on a new site, we adopt a selective regularization scheme that penalizes changes of parameters with high comprehensive importance, avoiding the shape knowledge and reliable semantics related to previous sites being forgotten. We evaluate our method on prostate MRI data sequentially acquired from six institutes. Results show that our method outperforms many continual learning methods for relieving model forgetting issue. Code is available athttps://github.com/jingyzhang/CISR.
J. Zhang and R. Gu—The authors contributed equally to this work.
This is a preview of subscription content,log in via an institution to check access.
Access this chapter
Subscribe and save
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
- Chapter
- JPY 3498
- Price includes VAT (Japan)
- eBook
- JPY 14871
- Price includes VAT (Japan)
- Softcover Book
- JPY 18589
- Price includes VAT (Japan)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Before each round of continual learning, the encoder component is pretrained and consecutively fine-tuned with the coupled decoder component, by minimizing a reconstruction loss with ground truth mask inputs. It should be frozen [24] in the later to avoid being corrupted by incomplete shape predictions due to model forgetting.
References
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision, pp. 139–154 (2018)
Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3366–3375 (2017)
Bloch, N., et al.: NCI-ISBI 2013 challenge: automated segmentation of prostate structures. Cancer Imaging Arch. (2015)
Douillard, A., Chen, Y., Dapogny, A., Cord, M.: PLOP: learning without forgetting for continual semantic segmentation. arXiv preprintarXiv:2011.11390 (2020)
Karani, N., Chaitanya, K., Baumgartner, C., Konukoglu, E.: A lifelong learning approach to brain MR segmentation across scanners and protocols. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 476–484. Springer, Cham (2018).https://doi.org/10.1007/978-3-030-00928-1_54
Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? In: Advances in Neural Information Processing Systems, pp. 5574–5584 (2017)
Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci.114(13), 3521–3526 (2017)
Lemaître, G., Martí, R., Freixenet, J., Vilanova, J.C., Walker, P.M., Meriaudeau, F.: Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: A review. Comput. Biol. Med.60, 8–31 (2015)
Litjens, G., Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal.42, 60–88 (2017)
Litjens, G., et al.: Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Med. Image Anal.18(2), 359–373 (2014)
Liu, Q., Dou, Q., Yu, L., Heng, P.A.: MS-Net: multi-site network for improving prostate segmentation with heterogeneous MRI data. IEEE Trans. Med. Imaging39(9), 2713–2724 (2020)
Liu, Q., Dou, Q., Heng, P.-A.: Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 475–485. Springer, Cham (2020).https://doi.org/10.1007/978-3-030-59713-9_46
Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. In: Advances in Neural Information Processing Systems, pp. 6467–6476 (2017)
Ma, J., He, J., Yang, X.: Learning geodesic active contours for embedding object global information in segmentation CNNs. IEEE Trans. Med. Imaging40(1), 93–104 (2021)
McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: the sequential learning problem. In: Psychology of Learning and Motivation, vol. 24, pp. 109–165. Elsevier (1989)
Navarro, F., et al.: Shape-aware complementary-task learning for multi-organ segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 620–627. Springer, Cham (2019).https://doi.org/10.1007/978-3-030-32692-0_71
Oktay, O., et al.: Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging37(2), 384–395 (2017)
Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: a review. Neural Netw.113, 54–71 (2019)
Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: ICARL: incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).https://doi.org/10.1007/978-3-319-24574-4_28
Shin, H., Lee, J.K., Kim, J., Kim, J.: Continual learning with deep generative replay. In: Advances in Neural Information Processing Systems, pp. 2990–2999 (2017)
Wang, G., et al.: Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE Trans. Med. Imaging37(7), 1562–1573 (2018)
Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprintarXiv:1708.01547 (2017)
Yue, Q., Luo, X., Ye, Q., Xu, L., Zhuang, X.: Cardiac segmentation from LGE MRI using deep neural network incorporating shape and spatial priors. In: Medical Image Computing and Computer-Assisted Intervention, pp. 559–567 (2019)
Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. Proc. Mach. Learn. Res.70, 3987 (2017)
Acknowledgments
This research is partially supported by the National Key research and development program (No. 2016YFC0106200), Beijing Natural Science Foundation-Haidian Original Innovation Collaborative Fund (No. L192006), and the funding from Institute of Medical Robotics of Shanghai Jiao Tong University as well as the 863 national research fund (No. 2015AA043203).
Author information
Authors and Affiliations
School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
Jingyang Zhang & Lixu Gu
Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China
Jingyang Zhang & Lixu Gu
School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
Ran Gu & Guotai Wang
SenseTime Research, Shanghai, China
Jingyang Zhang, Ran Gu & Lixu Gu
- Jingyang Zhang
You can also search for this author inPubMed Google Scholar
- Ran Gu
You can also search for this author inPubMed Google Scholar
- Guotai Wang
You can also search for this author inPubMed Google Scholar
- Lixu Gu
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toLixu Gu.
Editor information
Editors and Affiliations
Erasmus MC - University Medical Center Rotterdam, Rotterdam, The Netherlands
Marleen de Bruijne
University of Basel, Allschwil, Switzerland
Philippe C. Cattin
Inria Nancy Grand Est, Villers-lès-Nancy, France
Stéphane Cotin
ICube, Université de Strasbourg, CNRS, Strasbourg, France
Nicolas Padoy
National Center for Tumor Diseases (NCT/UCC), Dresden, Germany
Stefanie Speidel
Tencent Jarvis Lab, Shenzhen, China
Yefeng Zheng
ICube, Université de Strasbourg, CNRS, Strasbourg, France
Caroline Essert
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, J., Gu, R., Wang, G., Gu, L. (2021). Comprehensive Importance-Based Selective Regularization for Continual Segmentation Across Multiple Sites. In: de Bruijne, M.,et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_37
Download citation
Published:
Publisher Name:Springer, Cham
Print ISBN:978-3-030-87192-5
Online ISBN:978-3-030-87193-2
eBook Packages:Computer ScienceComputer Science (R0)
Share this paper
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
