Cardiac magnetic resonance imaging - Large language model Meta AI: afinetuned large language model for identifying findings and associated attributes in cardiac magnetic resonance imaging reports

Sep 29, 2026

Uploaded by Connor

Background: Cardiac magnetic resonance imaging (CMR) studies contain extensive information, yet free-text reports require manual review to extract clinical status. Automatic extraction could support clinical decision support and retrospective knowledge discovery. Existing approaches often have small samples and limited disease coverage, missing linguistic nuances such as uncertainty, severity, subtype, and anatomical location. Objective: build a broad named entity recognition model to extract common CMR findings and attributes from CMR reports. Methods, results, and conclusion follow.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Introduction

Methods

Results

Conclusion

Journal of Cardiovascular Magnetic Resonance 27 (2025) 101968 Contents lists available at ScienceDirect Journal of Cardiovascular Magnetic Resonance journal [homepage:](homepage: www.sciencedirect.com/journal/joc mr)[ www.sciencedirect.com/journal/joc mr](homepage: www.sciencedirect.com/journal/joc mr) Original Research Cardiac magnetic resonance imaging-large language model Meta AI: afinetuned large language model for identifying findings and associated attributes in cardiac magnetic resonance imaging reports Michelle Z. Fanga, Makiya Nakashimab,c, Kailash Singha, Eileen Galvanib, Xiaotan Sunb,c, Sharmeen Sorathiab, Kevin Dorocakb, Deborah Kwonb,c, Christopher Nguyenb,c, David Chenc,d,⁎ a Cleveland Clinic Lerner College of Medicine at Case Western, Cleveland, Ohio, USA b Heart Vascular and Thoracic Institute, Cleveland Clinic, Cleveland, Ohio, USAc Cardiovascular Innovation Research Center, Cleveland Clinic, Cleveland, Ohio, USA d Diagnostics Institute, Cleveland Clinic, Cleveland, Ohio, USA A R T I C L E I N F O Keywords: Transformers Natural language processing Ground-truth labeling Named entity recognition Diagnosis extraction Findings extraction Radiology report analysis A B S T R A C T Background: Cardiac magnetic resonance imaging (CMR) studies contain a wealth of information on a patient’s cardiovascular status. The ability to extract this data from free-text reports could serve to automate clinical decision support tools and generate data for retrospective clinical knowledge discovery, and clinical operational purposes. Few studies have examined the automatic extraction of data from free-text CMR reports, and the existing studies that do have key limitations, including small sample size and disease-specific data extraction. Existing studies also fail to extract features associated with the cardiovascular conditions that reflect nuances in natural language, such as uncertainty, severity, subtype, and anatomical locations of the condition. The goal of this study was to build a broad named entity recognition model to automatically extract a broad variety of common CMR findings and their associated attributes from CMR reports. Methods: We fine-tuned a Large Language Model Meta AI (LLaMA) model trained to identify 34 cardiovascular conditions and their associated attributes, including certainty, severity, location, and subtype of the condition. This model was trained on 1778 MRI reports and tested on 397 reports in an held-out test set and another 428 reports from another site in our hospital system with independent radiology practice and scanners. Results: Our model shows robust performance in predicting the mention of the 31 cardiovascular conditions (average F1 = 0.85). It also showed strong performance predicting attributes, including certainty (average F1 = 0.97) and severity (average F1 = 0.97). Model performance on the external validation set was generally slightly lower than the internal validation set, but performance was still strong (average F1 = 0.78 for mention, 0.97 for certainty, and 0.96 for severity). Conclusion: CMR-LLaMA has strong performance identifying a variety of concept mentions and moderate accuracies in extracting a selection of other associated attributes. NLP models can be used to automate the extraction of data from CMR reports to potentially assist with clinical and research workflow. 1. Introduction Cardiovascular magnetic resonance imaging (CMR) is an important imaging modality to evaluate many cardiovascular conditions [1]. CMR is the gold standard for evaluating a wide variety of morphologic, functional, and tissue parameters, such as cardiac chamber volumes, function and mass, and extent of fibrosis [1]. Such information is used to inform management for a wide variety of diseases, including heart Abbreviations: CMR, Cardiovascular magnetic resonance imaging; MRI, Magnetic Resonance Imaging; LLaMA, Large Language Model Meta AI; LA, left atrium; RA,

FAQs

What is the main goal of the CMR-LLaMA study?

To build a broad named entity recognition model that automatically extracts common CMR findings and their associated attributes from CMR free-text reports.

How was the model trained and evaluated?

The model was fine-tuned using 1778 MRI reports, then tested on a held-out set of 397 reports and externally on 428 reports from another hospital site with independent scanners.

What performance did the model achieve for condition mentions and attributes?

The model showed robust performance for predicting 31 cardiovascular conditions (average F1 = 0.85) and strong attribute extraction, including certainty (average F1 = 0.97) and severity (average F1 = 0.97), with slightly lower but still strong external results.

Sep 29, 2026

Uploaded by Connor