Performance of machine learning versus the national early warning score for predicting patient deterioration risk - a single-site study of emergency admissions

Aug 6, 2026

Increasing operational pressures on emergency departments require rapid, accurate identification of patients likely to deteriorate and need urgent intervention. With electronic health records providing rich structured and free-text data, this study evaluates transformer-based machine learning approaches for unplanned ED admissions. Using routine EHR-derived feature sets from a large UK teaching hospital, models predict mortality or critical care admission within 24 hours and are benchmarked against the National Early Warning Score.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Abstract

Introduction

Early warning scores and clinical urgency

What is already known on this topic

What this study adds

How this study might affect research, practice or policy

Open access Original research To cite: Watson M, Boulitsakis Logothetis S, Green D, et al. Performance of machine learning versus the national early warning score for predicting patient deterioration risk: a single-site study of emergency admissions. BMJ Health Care Inform 2024;31:e101088 . doi:10 . 1136/ bmjhci-2024-101088 ► Additional supplemental material is published online only. To view, please visit the journal online ([https://doi.org/10.1136/](https://doi.org/10.1136/)[ ](https://doi.org/10.1136/)[bmjhci-2024-101088](bmjhci-2024-101088)) . Received 14 May 2024 Accepted 14 November 2024 © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY. Published by BMJ. For numbered affiliations see end of article. Correspondence to Dr Noura Al Moubayed; [noura.al-moubayed@durham](noura.al-moubayed@durham). [ac.uk](ac.uk) Performance of machine learning versus the national early warning score for predicting patient deterioration risk: a single-site study of emergency admissions Matthew Watson ,1 Stelios Boulitsakis Logothetis,2 Darren Green,3,4 Mark Holland ,5 Pinkie Chambers,6 Noura Al Moubayed 1,7 ABSTRACT Objectives Increasing operational pressures on emergency departments (ED) make it imperative to quickly and accurately identify patients requiring urgent clinical intervention. The widespread adoption of electronic health records (EHR) makes rich feature patient data sets more readily available. These large data stores lend themselves to use in modern machine learning (ML) models. This paper investigates the use of transformer-based models to identify critical deterioration in unplanned ED admissions, using free-text fields, such as triage notes, and tabular data, including early warning scores (EWS) . Design A retrospective ML study. Setting A large ED in a UK university teaching hospital. Methods We extracted rich feature sets of routine clinical data from the EHR and systematically measured the performance of tree-and transformer-based models for predicting patient mortality or admission to critical care within 24 hours of presentation to ED. We compared our proposed models to the National EWS (NEWS) . Results Models were trained on 174 393 admission records. We found that models including free-text triage notes outperform structured tabular data models, achieving an average precision of 0 .92, compared with 0.75 for tree-based models and 0.12 for NEWS. Conclusions Our findings suggests that machine learning models using free-text data have the potential to improve clinical decision-making in the ED; our techniques significantly reduce alert rate while detecting most highrisk patients missed by NEWS. INTRODUCTION Early recognition and intervention of deteriorating patients is vital to prevent avoidable hospital deaths.1 Track and trigger systems, such as early warning scores (EWS), were developed to meet this need, providing a single aggregated score from a patient’s vital signs. Score thresholds define recommended response levels and urgency. EWS are used throughout a patient’s hospital admission pathway, from initial evaluation WHAT IS ALREADY KNOWN ON THIS TOPIC ⇒ Increasing operational pressures on emergency departments (ED) make it imperative to quickly and accurately identify patients requiring urgent clinical intervention. Current track and trigger systems use relatively small amounts of parameters to identify physiologically unstable patients, but the widespread adoption of electronic health records (EHR) means that richer patient details are now available. However, the utility of this data, particularly free-text triage note data, for the use of early warning scores was unclear. WHAT THIS STUDY ADDS ⇒ Our study shows that, when used with transformerbased machine learning techniques, the rich patient data collected in EHR (including free-text triage notes) can significantly outperform the National Early Warning Score when predicting patient deterioratio

FAQs

What problem does this study address in emergency departments?

The study targets the need to quickly and accurately identify patients at high risk of deterioration who require urgent clinical intervention.

How did the researchers compare machine learning models with the National Early Warning Score?

They trained transformer-based and tree-based models on EHR features, including free-text triage notes and tabular data, then compared predictive performance against the National Early Warning Score (NEWS).

What was the key finding about using free-text triage notes?

Models incorporating free-text triage notes outperformed structured tabular approaches and achieved substantially better precision than NEWS, while reducing alert rates and still detecting most high-risk patients missed by NEWS.

Aug 6, 2026

Upload by Olivia Brown