Enhancing Patient Understanding of Perianal Fistula MRI Findings Using ChatGPT - A Randomized, Single Centre Study

Q: What problem does this study address about perianal fistula MRI reports?

The study addresses that standard MRI fistula reporting can be complex and difficult for patients to understand, despite patients valuing MRI for accuracy and insight.

Q: How was GPT-4o evaluated in the study?

GPT-4o was evaluated in a three-phase design: prompt engineering and pilot testing, assessment of 250 consecutive MRI reports for hallucinations and themes, and a randomised patient review comparing original reports with AI-generated summaries.

Q: What were the main findings regarding readability and patient comprehension?

AI-generated summaries scored significantly higher for readability, comprehension, and usefulness than original reports, while trust ratings were equivalent and mean Flesch-Kincaid readability scores were substantially higher for the AI summaries.

Q: What risks were identified and how should safe implementation be handled?

Hallucinations occurred in a proportion of outputs and some unverified recommendations were identified. Safe implementation requires structured oversight, domain-specific refinement, and clinician validation, including use of standardised, clinician-approved lay summary templates.

Sep 30, 2026

Upload by Callum

Large Language Models (LLMs) may translate complex perianal fistula Magnetic Resonance Imaging (MRI) reports into patient-friendly summaries. This single-centre study assessed the clinical utility, safety, and patient acceptability of Generative Pre-trained Transformer (GPT-4o) summaries. A three-phase design tested feasibility, evaluated 250 consecutive MRI reports for hallucinations and thematic content, then randomised 61 patients to review original versus AI summaries. AI summaries improved readability, comprehension, and usefulness without reducing trust, while hallucinations and inconsistent terminology highlighted safety limits and the need for clinician-validated, structured templates.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

AccessFree to download

Abstract

Methods

Results

Conclusions

Introduction

Article Enhancing Patient Understanding of Perianal Fistula MRI Findings Using ChatGPT: A Randomized, Single Centre Study Easan Anand 1,2, *, Itai Ghersin 1, Gita Lingam 1,2, Katie Devlin 1, Theo Pelly 1,2, Daniel Singer 3, Chris Tomlinson 4, Robin E. J. Munro 5, Rachel Capstick 5, Anna Antoniou 5, Ailsa L. Hart 1,2, Phil Tozer 1,2, Kapil Sahnan 1,2 and Phillip Lung 1,2 Academic Editors: Takuji Tanaka and Consolato M. Sergi Received: 21 October 2025 Revised: 11 December 2025 Accepted: 22 December 2025 Published: 25 December 2025 Copyright: © 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. 1 Robin Phillips’ Fistula Research Unit, St Mark’s The National Bowel Hospital, London NW10 7NS, UK; [itai.ghersin@nhs.net](itai.ghersin@nhs.net) (I.G.); [gita.lingam@nhs.net](gita.lingam@nhs.net) (G.L.); [katie.devlin3@nhs.net](katie.devlin3@nhs.net) (K.D.); [h.pelly@nhs.net](h.pelly@nhs.net) (T.P.); [ailsa.hart@nhs.net](ailsa.hart@nhs.net) (A.L.H.); [philtozer@nhs.net](philtozer@nhs.net) (P.T.); [kapil.sahnan@nhs.net](kapil.sahnan@nhs.net) (K.S.); [philliplung@nhs.net](philliplung@nhs.net) (P.L.) 2 Department of Surgery & Cancer, Imperial College London, London SW7 2AZ, UK 3 Tenrec Analytics, St Albans AL1 4TJ, UK; [daniel.singer@tenrecanalytics.com](daniel.singer@tenrecanalytics.com) 4 Institute of Health Informatics, University College London, London WC1E 6BT, UK; [christopher.tomlinson@ucl.ac.uk](christopher.tomlinson@ucl.ac.uk) 5 St Mark’s The National Bowel Hospital, London NW10 7NS, UK; [rachelcapstick@hotmail.com](rachelcapstick@hotmail.com) (R.C.); [anna.antoniou@gmail.com](anna.antoniou@gmail.com) (A.A.) * Correspondence: [era24@ic.ac.uk](era24@ic.ac.uk); Tel.: +44-020-88643232 Abstract Background/Objectives: Large Language Models (LLMs) may help translate complex Magnetic Resonance Imaging (MRI) fistula reports into accessible, patient-friendly summaries. This study evaluated the clinical utility, safety, and patient acceptability of Generative Pre-trained Transformer (GPT-4o) in generating such reports. Methods: A three-phase study was conducted at a single centre. Phase I involved prompt engineering and pilot testing of GPT-4o outputs for feasibility. Phase II assessed 250 consecutive MRI fistula reports from September 2024 to November 2024, each reviewed by a multi-disciplinary panel to determine hallucinations and thematic content. Phase III randomised patients to review either a simple or complex fistula case, each containing an original report and an Artificial Intelligence (AI)-generated summary (order randomised, origin blinded), and rate readability, trustworthiness, usefulness and comprehension. Results: Sixteen patients participated in Phase I pilot testing. In Phase II, hallucinations occurred in 11% of outputs, with unverified recommendations also identified. In Phase III, 61 patients (mean age 48, 41% female) evaluated paired original and AI-generated summaries. AI summaries scored significantly higher for readability, comprehension, and usefulness than original reports (all p < 0.001), with equivalent trust ratings. Mean Flesch-Kincaid scores were markedly higher for AI-generated summaries (66 vs. 26; p < 0.001) . Clinicians highlighted improved anatomical structuring and accessible language, but emphasised risks of inaccuracies. A revised template incorporating Multi-Disciplinary Team (MDT)-focused action points and a lay summary section was co-developed. Conclusions: LLMs can enhance the readability and patient understanding of complex MRI reports but remain limited by hallucinations and inconsistent terminology. Safe implementation requires structured oversight, domain-specific refinement, and clinician validation. Future development should prioritise standardised reporting templates incorporating clinician-approved lay summaries. Keywor

FAQs

What problem does this study address about perianal fistula MRI reports?

The study addresses that standard MRI fistula reporting can be complex and difficult for patients to understand, despite patients valuing MRI for accuracy and insight.

How was GPT-4o evaluated in the study?

GPT-4o was evaluated in a three-phase design: prompt engineering and pilot testing, assessment of 250 consecutive MRI reports for hallucinations and themes, and a randomised patient review comparing original reports with AI-generated summaries.

What were the main findings regarding readability and patient comprehension?

AI-generated summaries scored significantly higher for readability, comprehension, and usefulness than original reports, while trust ratings were equivalent and mean Flesch-Kincaid readability scores were substantially higher for the AI summaries.

What risks were identified and how should safe implementation be handled?

Hallucinations occurred in a proportion of outputs and some unverified recommendations were identified. Safe implementation requires structured oversight, domain-specific refinement, and clinician validation, including use of standardised, clinician-approved lay summary templates.

Sep 30, 2026

Upload by Callum