Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences

Q: Why is faithfulness critical when using LLMs for clinical trial summaries?

Clinical trial summaries influence regulatory approvals, prescribing decisions, payer coverage determinations, and patient understanding. Hallucinated or distorted claims can mislead stakeholders and lead to harmful outcomes.

Q: What does the proposed benchmark evaluation framework include?

The framework uses 200 stratified trials from the AACT database, evaluates summaries for three stakeholder audiences with audience-specific prompt templates, and applies a six-dimension faithfulness annotation schema on a one-to-three ordinal scale.

Q: Which failure mode dominated across the evaluated LLMs, and how was improvement achieved?

Unsupported Claims was the dominant failure mode across GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash based on annotation scores. A knowledge-graph-augmented retrieval system grounded in PubMed Knowledge Graph improved NLI-based faithfulness scores with statistically significant gains, with improvement pathways differing by model.

Jul 16, 2026

Upload by Aria

Large language models are increasingly used to summarize clinical trial results for healthcare providers, patients, and payers, yet hallucinations create serious safety and decision risks. The study presents a benchmark framework to measure the faithfulness of LLM-generated trial summaries for three stakeholder audiences using 200 stratified trials, audience-specific prompts, and a six-dimension annotation scheme. Baselines are reported for GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash, and a knowledge-graph-augmented retrieval system grounded in PubMed improves NLI-based faithfulness with statistically significant gains.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Introduction

Motivation and stakes in clinical communication

Related work and limitations of existing metrics

Contributions and overview of the proposed framework

Faithful by Design: Evaluating and Improving LLM-Generated Clinical Trial Summaries for Multi-Stakeholder Audiences Robert Williams University of Texas at Austin Computer & Data Science Online Austin, Texas, USA [rgw@utexas.edu](rgw@utexas.edu) arXiv :2607 .09932v 1 [ cs .CL] 10 Jul 2026 Abstract Large language models are increasingly used to summarize clinical trial results for healthcare providers, patients, and payers, but their tendency to hallucinate poses significant risks in this high-stakes context. This study introduces a benchmark evaluation framework for measuring the faithfulness of LLM-generated clinical trial summaries across three stakeholder audiences. The framework consists of 200 stratified trials drawn from the Aggregate Analysis of ClinicalTrials.gov database, evaluated using audiencespecific prompt templates and a six-dimension faithfulness annotation schema. Baseline measurements were established for GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash across 1,800 generated summaries scored using a cross-encoder natural language inference (NLI) model. Unsupported Claims was identified as the dominant failure mode across all three models, with a mean annotation score of 1.55 out of three. A knowledge-graph-augmented retrieval system grounded in the PubMed Knowledge Graph was developed and evaluated against the baseline, producing statistically significant improvements in NLI-based faithfulness scores (entailment +0 . 0125, faithfulness +0 .0130, 􀀿 < 0. 0001) . Improvement pathways were model-dependent, with GPT-4o improving primarily through contradiction reduction while Claude Sonnet 4.6 and Gemini 2.5 Flash improved through increased entailment. CCS Concepts • Computing methodologies → Natural language processing; • Applied computing → Health informatics. Keywords clinical trial summarization, faithfulness evaluation, hallucination detection, knowledge graph, retrieval-augmented generation, NLI, large language models ACM Reference Format: Robert Williams. 2026. Faithful by Design: Evaluating and Improving LLMGenerated Clinical Trial Summaries for Multi-Stakeholder Audiences. In Proceedings of UT Austin AI in Health (AI in Health ’26) . ACM, New York, NY, USA, 8 pages. [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn) Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission [and/or a fee. Request permissions from permissions@acm.org](and/or a fee. Request permissions from permissions@acm.org). AI in Health ’26, Austin, TX © 2026 Copyright held by the owner/author(s) . Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn) 1 Introduction Clinical trial results are among the most consequential documents in healthcare. They underpin regulatory approvals, inform prescribing and treatment decisions, guide coverage determinations by payers, and must be communicated to patients in plain language to support informed consent. A single completed trial may require distinct summaries for healthcare providers, patients, and payers, each demanding a different framing of the same underlying evidence. ClinicalTrials.gov currently lists more than 400,000 registered trials, and this summarization workload currently falls to skilled medical writers who must translate complex findings for each audience without distorting them. Large language models (LLMs) are natural candidates for automating this

FAQs

Why is faithfulness critical when using LLMs for clinical trial summaries?

Clinical trial summaries influence regulatory approvals, prescribing decisions, payer coverage determinations, and patient understanding. Hallucinated or distorted claims can mislead stakeholders and lead to harmful outcomes.

What does the proposed benchmark evaluation framework include?

The framework uses 200 stratified trials from the AACT database, evaluates summaries for three stakeholder audiences with audience-specific prompt templates, and applies a six-dimension faithfulness annotation schema on a one-to-three ordinal scale.

Which failure mode dominated across the evaluated LLMs, and how was improvement achieved?

Unsupported Claims was the dominant failure mode across GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash based on annotation scores. A knowledge-graph-augmented retrieval system grounded in PubMed Knowledge Graph improved NLI-based faithfulness scores with statistically significant gains, with improvement pathways differing by model.

Jul 16, 2026

Upload by Aria