Multi-Agent AI System for Radiology Report Structuring and Quality Assurance - A Locally Deployed Approach

Sep 15, 2026

Upload by Aria

A locally deployed multi-agent AI system is developed to structure radiology reports and perform automated quality assurance without rewriting the original text. A retrospective dataset of 638 CT chest, abdomen, and pelvis reports from 15 board-certified radiologists (2023–2024) is used. The system assigns sentence-level anatomical sections and flags discrepancies across findings/impression, within-section inconsistencies, gender–anatomy conflicts, and critical findings lacking documented communication. A 45-report subset is independently evaluated by two radiologists, showing predominantly correct restructuring and strong QA ratings.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Abstract

Introduction

Materials and Methods

arXiv : 2608 . 18072v 1 [cs .CL] 18 Aug 2026 Multi-Agent AI System for Radiology Report Structuring and Quality Assurance with Independent Radiologist Evaluation Iryna Hartsock 1,∗ , Cesar Lam 2 , Christopher Otteni 2 , Aliya Qayyum 2 , Robert Gatenby 2 , Cyrillo Araujo 2 , and Ghulam Rasool 1,2,3,4 1 Department of Machine Learning, H. Lee Mofﬁtt Cancer Center & Research Institute, Tampa, FL, USA 2 Department of Diagnostic Imaging and Interventional Radiology, H. Lee Mofﬁtt Cancer Center & Research Institute, Tampa, FL, USA 3 Department of Neuro-Oncology, H. Lee Mofﬁtt Cancer Center & Research Institute, Tampa, FL, USA 4 Department of Translational Pathology, H. Lee Mofﬁtt Cancer Center & Research Institute, Tampa, FL, USA Correspondence*: Iryna Hartsock iryna.hartsock@mofﬁ[tt.org](tt.org) ABSTRACT Purpose: To develop and evaluate a locally deployed multi-agent AI system for radiology report structuring and quality assurance. Materials and Methods: This retrospective study included 638 radiology reports from CT examinations of the chest, abdomen, and pelvis dictated by 15 board-certiﬁed radiologists in 2023 and 2024 . A multi-agent AI pipeline was developed to perform report structuring and quality assurance (QA) . The system structured the report into standardized anatomical sections at the sentence level using regex rules and local large language models. It also detected mismatches between the Findings and Impression sections, or within sections; gender–anatomy conﬂicts; and undocumented communication of critical ﬁndings. Two board-certiﬁed radiologists independently evaluated a 45-report subset. Results: The multi-agent system structured the Findings sections of all reports (22,270 sentences) into apredeﬁned anatomical format while retaining the original report content. The system ﬂagged 90 (14.1%) reports, most commonly for section mismatches (80 reports, 12.5%) . In the radiologist evaluation, both reviewers agreed that 31 (69%) were correctly restructured, 2 reports (4%) were incorrectly restructured, and disagreed on the remaining 12 reports (27%) . Both reviewers agreed that no clinically important information was omitted and no fabricated content was introduced. Overall QA performance was rated as “excellent” or “good” in 84% of the evaluated reports, with the remaining reports rated as “fair”. Conclusion: A locally deployed multi-agent AI system combined radiology report structuring and quality assurance within a single workﬂow. The system demonstrated favorable performance in radiologist evaluation. Such systems may support standardization of reporting and quality assurance in radiology practice. Keywords: multi-agent artiﬁcial intelligence, radiology reports, structured reporting, quality assurance, large language models Hartsock et al. Multi-Agent AI System for Radiology Report Structuring and Quality Assurance 1 INTRODUCTION A challenge in radiology reporting is inconsistent or absent structure, which can lead referring physicians to overlook critical clinical information or spend time searching for it (1) . Radiology reports include sections such as ﬁndings and impressions, with ﬁndings sometimes described by anatomic regions or organs. However, the organization and placement of ﬁndings within reports can vary across radiologists, which may complicate communication between radiologists and referring clinicians (2, 3) . In addition, radiology reports without a consistent structure can limit their secondary use for research, quality monitoring, and clinical decision support systems (4) . Several approaches have been proposed to address these limitations. Structured reporting templates have been developed to standardize radiology reporting and improve consistency, although adoption has been variable due to concerns regarding workﬂow burden and reporting ﬂexibility (1, 5) . Recently, natural language processing (NLP) and large language model (LLM) approaches have been explored to extrac

FAQs

What problem does the multi-agent AI system address in radiology reporting?

It targets inconsistent or missing report structure that can cause missed clinical information, hinder communication, and limit secondary use for research and decision support. It also aims to detect discrepancies and communication errors that can affect clinical management.

How does the system structure radiology reports without altering content?

It assigns each sentence to pre-defined anatomical sections at the sentence level using regex rules and local large language models. This preserves the original report wording while enforcing standardized organization.

What types of quality assurance issues does the system flag?

It flags mismatches between Findings and Impression and discrepancies within sections, including gender–anatomy conflicts. It also detects critical findings that lack documented communication.

Sep 15, 2026

Upload by Aria