Utilizing large language models and natural language processing to classify ischemia status from cardiac stress tests in a large multicenter healthcare system

Sep 30, 2026

Uploaded by Pentious

Myocardial ischemia documentation before invasive coronary angiography is recommended to reduce patient risk, yet quality-of-care assessment often depends on extracting clinical meaning from unstructured electronic medical record text. This study evaluates multiple NLP systems to classify cardiac stress test reports as ischemia or no ischemia, prioritizing the best balance of accuracy and feasibility across deployment needs. Four BERT LLMs and a rules-based system are assessed on annotated reports, demonstrating scalable text-driven ischemia detection for large-scale monitoring.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Abstract

Objective

Results

Keywords

Introduction

Cave et al. BMC Research Notes (2026) 19:10 [https://doi.org/10.1186/s13104-025-07586-5](https://doi.org/10.1186/s13104-025-07586-5) BMC Research Notes RESEARCH NOTE Open Access Utilizing large language models and natural language processing to classify ischemia status from cardiac stress tests in a large multicenter healthcare system Shayna Cave 1*, Kelly S. Peterson 1,4, Mary E. Plomondon 1 and Stephen W. Waldo 1,2,3 Abstract Objective Documentation of myocardial ischemia prior to invasive coronary angiography is recommended to minimize patient risk. However, obtaining this information for quality-of-care assessment often requires extracting clinical information from unstructured electronic medical records text. To this end, we sought to evaluate multiple natural language processing (NLP) systems in their ability to classify cardiac stress test reports as documenting ischemia or no ischemia, implementing the one with the best combination of accuracy and feasibility. Results Four BERT large language models (LLMs) were fine-tuned, and a rules-based system was designed by training, validating, and testing on an annotated sample of 654 stress test reports from a multisite and multiyear dataset from the Veterans Health Administration (VHA) . The LLM with the highest performance was a ClinicalBERT with precision, recall, and F1 of 86 .4%, 100%, and 92 . 7%, respectively. The rules-based NLP system achieved similar results of 88 . 1%, 97 .4%, and 92 . 5%, respectively. Stress test reports totaling 1,692,171 and representing 1,096,341 unique patients were classified using the rules-based system after ascertaining current technological limitations, and the system is presently operational for care quality evaluations. Utilizing NLP allows for accurate, high-throughput analysis of cardiac stress test text reports. Keywords Ischemia, NLP, LLM, Text classification, Rules-based, Cardiovascular *Correspondence: Shayna Cave [Shayna.Cave@va.gov](Shayna.Cave@va.gov) 1CART Program, Office of Quality and Patient Safety, Veterans Health Administration, Washington, DC, USA 2Rocky Mountain Regional Veterans Affairs Medical Center, 1700 N Wheeling St, Suite F2-166, Aurora, CO 80045, USA 3Division of Cardiology, Department of Medicine, University of Colorado, Aurora, CO, USA 4Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, UT, USA Introduction Reduced blood flow to the heart muscle results in myocardial ischemia, which can clinically manifest as chest pain, shortness of breath, or as a myocardial infarction, otherwise known as a heart attack. While coronary angiography serves as the gold standard to evaluate the presence of ischemia, directly visualizing flow through the coronary arteries that supply blood to the myocardium, it is invasive and increases patient risk. Therefore, professional societies have adopted appropriate-use criteria to determine when this test may be warranted [1–4]. Based on these recommendations, invasive coronary This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply 2025. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit [http://creativecomm

FAQs

What problem does the study address in assessing myocardial ischemia?

It addresses the lack of a discrete, readily available data field documenting ischemia status from non-invasive testing in the medical record, which limits population-level appropriateness monitoring for invasive procedures.

How were the NLP models evaluated in classifying ischemia status?

Four BERT large language models were fine-tuned, and a rules-based NLP system was trained, validated, and tested using an annotated sample of 654 stress test reports.

What classification system was ultimately used for large-scale care quality evaluations?

The highest-performing model and a rules-based system were compared; stress test reports totaling 1,692,171 were classified using the rules-based system in an operational workflow for care quality evaluations.

Sep 30, 2026

Uploaded by Pentious