Machine Learning to Enhance Electronic Detection of Diagnostic Errors - Research Letter - Health Informatics

Aug 2, 2026

Diagnostic errors substantially contribute to patient harm, yet systems for monitoring them remain limited. Electronic trigger algorithms (e-triggers) use EHR data to flag potential missed opportunities in diagnosis (MODs), but they show low predictive value and rely on time-intensive manual record review. This study tests whether machine learning can improve e-trigger performance and scale review emulation. Using VA national EHR data and clinician-labeled MOD/no-MOD cases, ML-enhanced triggers achieve higher positive predictive value for both dizziness and abdominal-pain pathways.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages4

Reading Time10 min

AccessFree to download

Introduction

Methods

Results

Dizziness e-trigger performance

Abdominal pain e-trigger performance

Figure and Table

Research Letter | Health Informatics Machine Learning to Enhance Electronic Detection of Diagnostic Errors Andrew J. Zimolzak, MD, MMSc; Li Wei, MS; Usman Mir, MBBS, MPH; Ashish Gupta, MD, MBA; Viralkumar Vaghani, MBBS, MPH, MS; Devika Subramanian, PhD, MS; Hardeep Singh, MD, MPH Introduction Diagnostic errors contribute substantially to patient harm, but strategies to monitor them are underdeveloped.1 Electronic trigger algorithms (e-triggers) can identify patients with potential diagnostic errors2 using electronic health record (EHR) data. However, their predictive values are low, and this process requires time-consuming manual medical record review to confirm missed opportunities in diagnosis (MODs) .3 Because e-triggers are designed using a priori assumptions rather than empirical data patterns, they may not detect MOD signals comprehensively. We tested whether machine learning (ML) can enhance e-trigger performance and emulate human medical record reviewers at a larger scale.4 Methods Based on expert input and existing frameworks,2,5 we designed rules-based e-triggers to find possible MODs in emergency departments (ED). Using Veterans Affairs national EHR data covering more than 20 million unique individuals, we identified 2 high-risk cohorts: (1) patients with stroke risk factors discharged from ED after presenting with dizziness or vertigo who were subsequently hospitalized for stroke or TIA within 30 days; and (2) patients discharged from ED with abdominal pain and abnormal temperature who were subsequently hospitalized within 10 days. All ED visits occurred between 2016 and 2020. Trained clinicians used standardized data collection instruments (eFigure 1 in Supplement 1) to review a random sample of medical records flagged by each e-trigger and labeled each as MOD or no MOD. Baylor College of Medicine review board approved the study and granted waiver of informed consent because it would not be feasible to obtain consent for medical record reviews from the large number of patients that we studied. Analyses were conducted from April 2020 to May 2024 using Python version 3.7.4 (Python Software Foundation), with the packages scipy, numpy, and scikit-learn. Medical records with clear evidence of MOD or no MOD were divided into training and test sets (eFigure 2 in Supplement 1) . ML methods were regularized logistic regression and random forests (with limited maximum tree depth to mitigate overfitting) . The dizziness and abdominal pain algorithms had access to 148 and 153 variables potentially associated with the outcomes, respectively, extracted from structured EHR data. These included demographics, laboratory values, vital signs, orders, visit times, and risk factors (eTable in Supplement 1) . Because methods emulated retrospective medical record review evaluation, rather than prehospital6 or ED point-of-care evaluation, variables were drawn from index ED data and subsequent hospital data. Variables were preselected based on bivariate association with MOD by t test or χ2 test as appropriate, with a statistical significance threshold of 2-sided P = .10. Positive predictive values (PPV) are reported as pooled values (training and test set combined) due to the limited number of criterion standard records labeled by clinicians. CIs are 95% Wald intervals. + Invited Commentary + Supplemental content Author affiliations and article information are listed at the end of this article. Open Access. This is an open access article distributed under the terms of the CC-BY License. JAMA Network Open. 2024;7(9):e2431982 . doi:10.1001/jamanetworkopen.2024.31982 September 9, 2024 1/4 Downloaded [from jamanetwork.com](from jamanetwork.com) by Rice University user on 09/30/2024 JAMA Network Open | Health Informatics Machine Learning to Enhance Electronic Detection of Diagnostic Errors Results For the dizziness e-trigger, reviewers identified MODs in 39 of 82 flagged records (PPV, 48%[95% CI, 37%-58%]). The b

FAQs

Why are electronic trigger algorithms for diagnostic errors limited?

They can flag potential missed opportunities in diagnosis using EHR data, but their predictive values are low and confirmation typically requires time-consuming manual medical record review.

How was machine learning used to enhance the electronic triggers?

Clinician-labeled MOD/no-MOD records from high-risk ED cohorts were used to train regularized logistic regression and random forests. The ML models leveraged structured EHR variables extracted from the index ED visit and subsequent hospital data.

What improvements did ML-enhanced triggers show in results?

For the dizziness pathway, the best ML model increased positive predictive value to about 92%. For the abdominal pain pathway, the ML-enhanced approach raised positive predictive value to about 93%, alongside improved identification of true MODs.

Aug 2, 2026

Upload by River Wang