Aggregating published prediction models with individual participant data: a comparison of different approaches
- PMID:22733546
- DOI: 10.1002/sim.5412
Aggregating published prediction models with individual participant data: a comparison of different approaches
Abstract
During the recent decades, interest in prediction models has substantially increased, but approaches to synthesize evidence from previously developed models have failed to keep pace. This causes researchers to ignore potentially useful past evidence when developing a novel prediction model with individual participant data (IPD) from their population of interest. We aimed to evaluate approaches to aggregate previously published prediction models with new data. We consider the situation that models are reported in the literature with predictors similar to those available in an IPD dataset. We adopt a two-stage method and explore three approaches to calculate a synthesis model, hereby relying on the principles of multivariate meta-analysis. The former approach employs a naive pooling strategy, whereas the latter accounts for within-study and between-study covariance. These approaches are applied to a collection of 15 datasets of patients with traumatic brain injury, and to five previously published models for predicting deep venous thrombosis. Here, we illustrated how the generally unrealistic assumption of consistency in the availability of evidence across included studies can be relaxed. Results from the case studies demonstrate that aggregation yields prediction models with an improved discrimination and calibration in a vast majority of scenarios, and result in equivalent performance (compared with the standard approach) in a small minority of situations. The proposed aggregation approaches are particularly useful when few participant data are at hand. Assessing the degree of heterogeneity between IPD and literature findings remains crucial to determine the optimal approach in aggregating previous evidence into new prediction models.
Copyright © 2012 John Wiley & Sons, Ltd.
Similar articles
- Meta-analysis and aggregation of multiple published prediction models.Debray TP, Koffijberg H, Nieboer D, Vergouwe Y, Steyerberg EW, Moons KG.Debray TP, et al.Stat Med. 2014 Jun 30;33(14):2341-62. doi: 10.1002/sim.6080. Epub 2014 Jan 14.Stat Med. 2014.PMID:24752993
- A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis.Debray TP, Moons KG, Ahmed I, Koffijberg H, Riley RD.Debray TP, et al.Stat Med. 2013 Aug 15;32(18):3158-80. doi: 10.1002/sim.5732. Epub 2013 Jan 11.Stat Med. 2013.PMID:23307585
- Assessment and statistical modeling of the relationship between remotely sensed aerosol optical depth and PM2.5 in the eastern United States.Paciorek CJ, Liu Y; HEI Health Review Committee.Paciorek CJ, et al.Res Rep Health Eff Inst. 2012 May;(167):5-83; discussion 85-91.Res Rep Health Eff Inst. 2012.PMID:22838153
- [The meta-analysis of data from individual patients].Rovers MM, Reitsma JB.Rovers MM, et al.Ned Tijdschr Geneeskd. 2012;156(31):A4743.Ned Tijdschr Geneeskd. 2012.PMID:22853768Review.Dutch.
- Bayesian ranking of sites for engineering safety improvements: decision parameter, treatability concept, statistical criterion, and spatial dependence.Miaou SP, Song JJ.Miaou SP, et al.Accid Anal Prev. 2005 Jul;37(4):699-720. doi: 10.1016/j.aap.2005.03.012. Epub 2005 Apr 12.Accid Anal Prev. 2005.PMID:15949462Review.
Cited by
- Synthesis of clinical prediction models under different sets of covariates with one individual patient data.Yoneoka D, Henmi M, Sawada N, Inoue M.Yoneoka D, et al.BMC Med Res Methodol. 2015 Nov 19;15:101. doi: 10.1186/s12874-015-0087-x.BMC Med Res Methodol. 2015.PMID:26585325Free PMC article.
- Continual updating and monitoring of clinical prediction models: time for dynamic prediction systems?Jenkins DA, Martin GP, Sperrin M, Riley RD, Debray TPA, Collins GS, Peek N.Jenkins DA, et al.Diagn Progn Res. 2021 Jan 11;5(1):1. doi: 10.1186/s41512-020-00090-3.Diagn Progn Res. 2021.PMID:33431065Free PMC article.
- Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review.Binuya MAE, Engelhardt EG, Schats W, Schmidt MK, Steyerberg EW.Binuya MAE, et al.BMC Med Res Methodol. 2022 Dec 12;22(1):316. doi: 10.1186/s12874-022-01801-8.BMC Med Res Methodol. 2022.PMID:36510134Free PMC article.
- Pre-procedural risk models for patients undergoing transcatheter aortic valve implantation.Martin GP, Sperrin M, Mamas MA.Martin GP, et al.J Thorac Dis. 2018 Nov;10(Suppl 30):S3560-S3567. doi: 10.21037/jtd.2018.05.67.J Thorac Dis. 2018.PMID:30505535Free PMC article.Review.
- External validation of multivariable prediction models: a systematic review of methodological conduct and reporting.Collins GS, de Groot JA, Dutton S, Omar O, Shanyinde M, Tajar A, Voysey M, Wharton R, Yu LM, Moons KG, Altman DG.Collins GS, et al.BMC Med Res Methodol. 2014 Mar 19;14:40. doi: 10.1186/1471-2288-14-40.BMC Med Res Methodol. 2014.PMID:24645774Free PMC article.Review.
Publication types
MeSH terms
LinkOut - more resources
Miscellaneous