Correction 12 January 2023

The reproducibility issues that haunt health-care AI

Health-care systems are rolling out artificial-intelligence tools for diagnosis and monitoring. But how reliable are the models?

Emily Sohn⁰

Emily Sohn
1. Emily Sohn is a freelance journalist in Minneapolis, Minnesota.
View author publications
Search author on:PubMed Google Scholar

Each day, around 350 people in the United States die from lung cancer. Many of those deaths could be prevented by screening with low-dose computed tomography (CT) scans. But scanning millions of people would produce millions of images, and there aren’t enough radiologists to do the work. Even if there were, specialists regularly disagree about whether images show cancer or not. The 2017 Kaggle Data Science Bowl set out to test whether machine-learning algorithms could fill the gap.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

9,800 Yen / 30 days

cancel any time

Learn more

Subscription info for Japanese customers

We have a dedicated website for our Japanese customers. Please go tonatureasia.com to subscribe to this journal.

Go to natureasia.com

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature613, 402-403 (2023)

doi: https://doi.org/10.1038/d41586-023-00023-2

Updates & Corrections

Correction 12 January 2023: An earlier version of this feature erroneously stated that Sayash Kapoor discovered reproducibility failures and pitfalls in 329 studies across 17 fields. In fact, those studies had themselves reported reproducibility failures and pitfalls.

References

Yu, K. H.et al.J. Med. Internet Res.22, e16709 (2020).
Article PubMed Google Scholar
Roberts, M.et al.Nature Mach. Intell.3, 199–217 (2021).
Article Google Scholar
McDermott, M. B. A.et al.Sci. Transl. Med.13, eabb1655 (2021).
Article PubMed Google Scholar
Wong, A.et al.JAMA Intern. Med.181, 1065–1070 (2021).
Article PubMed Google Scholar
Mongan, J., Moy, L. & Kahn, C. E.Radiol. Artif. Intell.2, e200029 (2020).
Article PubMed Google Scholar

Download references

Subjects

Latest on:

Jobs

W3 Endowed Professorship for “Hemodynamic Modeling in Atherosclerosis- (f/m/d) KSB Foundation W3 end
The Medical Faculty Mannheim of Heidelberg University offers the position of a W3 Endowed Professorship for “Hemodynamic Modeling in Atherosclero...
Mannheim, Baden-Württemberg (DE)
Medizinische Fakultät Mannheim der Universität Heidelberg
Junior Professorship (W1) for Biomechanics (f/m/d)
At the Medical Faculty Mannheim at Heidelberg University the Junior Professorship (W1) for Biomechanics (f/m/d) is to be filled at the earlie...
Mannheim, Baden-Württemberg (DE)
Medizinische Fakultät Mannheim der Universität Heidelberg
Full-time tenure-track faculty positions
The Department of Molecular Biology and Genetics in the College of Sciences at Koç University invites applications for full-time tenure-track facul...
Istanbul (TR)
Koç University
Postdoctoral Positions in ‘Stem Cells, Regenerative Biology and Medicine’
Laboratory of Regenerative Biology and Medicine (Dr. Yuval Rinkevich), CIMR
Beijing (CN)
The Chinese Institutes for Medical Research (CIMR), Beijing
High-Level Talent at the Major Agricultural Microbiology Research Facility of HZAU
Join HZAU's global faculty team to advance research with competitive benefits.
No.1 Shizishan Street, Hongshan District, Wuhan, Hubei Province, China
Huazhong Agricultural University (HZAU)