Advances in deep learning (DL) have resulted in impressive accuracy in some medical image classification tasks, but often deep models lack interpretability. The ability of these models to explain their decisions is important for fostering clinical trust and facilitating clinical translation. Furthermore, for many problems in medicine there is a wealth of existing clinical knowledge to draw upon, which may be useful in generating explanations, but it is not obvious how this knowledge can be encoded into DL models - most models are learnt either from scratch or using transfer learning from a different domain. In this paper we address both of these issues. We propose a novel DL framework for image-based classification based on a variational autoencoder (VAE). The framework allows prediction of the output of interest from the latent space of the autoencoder, as well as visualisation (in the image domain) of the effects of crossing the decision boundary, thus enhancing the interpretability of the classifier. Our key contribution is that the VAE disentangles the latent space based on ‘explanations’ drawn from existing clinical knowledge. The framework can predict outputs as well as explanations for these outputs, and also raises the possibility of discovering new biomarkers that are separate (or disentangled) from the existing knowledge. We demonstrate our framework on the problem of predicting response of patients with cardiomyopathy to cardiac resynchronization therapy (CRT) from cine cardiac magnetic resonance images. The sensitivity and specificity of the proposed model on the task of CRT response prediction are 88.43% and 84.39% respectively, and we showcase the potential of our model in enhancing understanding of the factors contributing to CRT response.

1. 1.

   An inward-outward motion of the septum in early systole (mainly during isovolumetric contraction).

This work was supported by the EPSRC (EP/R005516/1 and EP/P001009/1) and the Wellcome EPSRC Centre for Medical Engineering at the School of Biomedical Engineering and Imaging Sciences, King’s College London (WT 203148/Z/16/Z). This research has been conducted using the UK Biobank Resource under Application Number 17806.

Authors and Affiliations

1. School of Biomedical Engineering and Imaging Sciences, King’s College London, London, UK

   Esther Puyol-Antón, James R. Clough, Bram Ruijsink, Baldeep S. Sidhu, Justin Gould, Bradley Porter, Marc Elliott, Vishal Mehta, Christopher A. Rinaldi & Andrew P. King

2. Guy’s and St Thomas’ Hospital, London, UK

   Bram Ruijsink, Baldeep S. Sidhu, Justin Gould, Bradley Porter, Marc Elliott, Vishal Mehta & Christopher A. Rinaldi

3. BioMedIA Group, Department of Computing, Imperial College London, London, UK

   Chen Chen & Daniel Rueckert

Corresponding author

Correspondence toEsther Puyol-Antón.

Editors and Affiliations

1. University of Toronto, Toronto, ON, Canada

   Anne L. Martel

2. The University of British Columbia, Vancouver, BC, Canada

   Purang Abolmaesumi

3. University College London, London, UK

   Danail Stoyanov

4. École Centrale de Nantes, Nantes, France

   Diana Mateus

5. EURECOM, Biot, France

   Maria A. Zuluaga

6. Chinese Academy of Sciences, Beijing, China

   S. Kevin Zhou

7. Sorbonne University, Paris, France

   Daniel Racoceanu

8. The Hebrew University of Jerusalem, Jerusalem, Israel

   Leo Joskowicz

Reprints and permissions

© 2020 Springer Nature Switzerland AG

Policies and ethics