Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models - Journal of Clinical Epidemiology

Aug 3, 2026

Upload by Hazel

This systematic review summarizes how clinical prediction models using supervised machine learning are designed, modeled, evaluated, and reported across published studies. Searches covered 2018–2019 publications describing model development alone or with external validation, without restrictions by study design, data source, or health outcome. Among 152 included studies, most reported development only, commonly targeted binary outcomes, and frequently omitted sample size calculations and calibration reporting, using algorithms such as support vector machines and random forests. Findings highlight needs for better missing-value handling, internal validation methods, and clearer calibration reporting.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Abstract

Background and Objectives

Methods

Results

Funding, Registration and Protocol

Conclusion

Journal of Clinical Epidemiology 154 (2023) 8e22 REVIEW Systematic review identiﬁes the design and methodological conduct of studies on machine learning-based prediction models Constanza L. Andaur Navarro, Doctoral Studenta,b, * , Johanna A.A. Damen, Assistant Professora,b, Maarten van Smeden, Associate Professora, Toshihiko Takada, Assistant Professora, Steven W.J. Nijman, Doctoral Studenta, Paula Dhiman, Research Fellowc,d, Jie Ma, Medical Statisticianc, Gary S. Collins, Professorc,d, Ram Bajpai, Research Fellowe, Richard D. Riley, Professore, Karel G.M. Moons, Professora,b, Lotty Hooft, Professora,b aJulius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands bCochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands cCenter for Statistics in Medicine, Nufﬁeld Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK dNIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK eCentre for Prognosis Research, School of Medicine, Keele University, Keele, UK Accepted 22 November 2022; Published online 25 November 2022 Abstract Background and Objectives: We sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques. Methods: We search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes. Results: We included 152 studies, 58 (38.2%[95% CI 30.8e46.1]) were diagnostic and 94 (61.8%[95% CI 53.9e69.2]) prognostic studies. Most studies reported only the development of prediction models (n 5 133, 87.5%[95% CI 81.3e91.8]), focused on binary outcomes (n 5 131, 86.2%[95% CI 79.8e90.8), and did not report a sample size calculation (n 5 125, 82.2%[95% CI 75.4e87.5]) . The most common algorithms used were support vector machine (n 5 86/522, 16.5%[95% CI 13.5e19.9]) and random forest (n 5 73/522, 14% Funding: GSC is funded by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC) and by Cancer Research UK program grant (C49297/A27294) . PD is funded by the NIHR Oxford BRC. RB is afﬁliated to the National Institute for Health and Care Research (NIHR) Applied Research Collaboration (ARC) West Midlands. The views expressed are those of the authors and not necessarily those of the NHS, NIHR, or Department of Health and Social Care. None of the funding sources had a role in the design, conduct, analyses, or reporting of the study or in the decision to submit the manuscript for publication. Registration and protocol: This review was registered in PROSPERO (CRD42019161764). The study protocol can be accessed in [https://doi](https://doi). org/10.1136/bmjopen-2020-038832 . Competing interests: There are no conﬂicts of interest to declare. Availability of data, code, and other materials: Articles that support our ﬁndings are publicly available. Template data collection forms, detailed data extraction on all included studies, and analytical code are available upon reasonable request. Ethical approval: Not required for this work. Declaration of interests: The authors declare that they have no known competing ﬁnancial interests or personal relationships that could have appeared to inﬂuence the work reported in this paper. Author Contributions: Constanza L. Andaur Navarro: Conceptualization, Methodology, Investigation, Data Curation, Formal analysis, Writing-original draft, Writing-review & editing; Johanna A.A. Damen: Conceptualization, Methodology, Investigation, Writing -review & editing, Superv

FAQs

What did the review aim to summarize about machine learning prediction models?

It summarized study design, modeling strategies, and performance measures reported in clinical prediction models developed using supervised machine learning techniques.

How were studies selected for the review?

PubMed was searched for articles published between 01/01/2018 and 31/12/2019 describing development or development with external validation of multivariable prediction models using supervised machine learning.

Which methodological reporting gaps were most emphasized in the review’s conclusion?

The review highlighted the need for improved handling of missing values, stronger internal validation methods, and more complete reporting of calibration.

Aug 3, 2026

Upload by Hazel