Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools

Sep 17, 2026

Uploaded by James

Legal practice increasingly relies on AI products for core tasks such as searching and summarizing caselaw and drafting legal documents. Because large language models can hallucinate—fabricating false information—their use remains risky in high-stakes settings. Some providers claim retrieval-augmented generation (RAG) eliminates hallucinations and guarantees “hallucination-free” legal citations, yet closed systems hinder systematic testing. This article presents a preregistered empirical evaluation and finds provider claims overstated.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages27

Reading Time1 h 8 min

AccessFree to download

Introduction

Background and motivation

Research gaps and objectives

Methods and evaluation approach

Preregistered empirical evaluation

Systems compared

Journal of Empirical Legal Studies ORIGINAL ARTICLE OPEN ACCESS Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools Varun Magesh1 | Faiz Surani1 | Matthew Dahl2 | Mirac Suzgun3 | Christopher D. Manning4 | Daniel E. Ho5 1Regulation, Evaluation, and Governance Lab, Stanford University, Stanford, California, USA | 2Yale University, New Haven, Connecticut, USA | 3Department of Computer Science, Stanford Law School, Regulation, Evaluation, and Governance Lab, Stanford University, Stanford, California, USA | 4Department of Linguistics, Department of Computer Science, Stanford Artificial Intelligence Laboratory, Stanford Institute for Human-Centered AI, Regulation, Evaluation, and Governance Lab, Stanford University, Stanford, California, USA | 5Stanford Law School, Department of Political Science, Department of Computer Science, Stanford Institute for Human-Centered AI, Stanford Institute for Economic Policy Research, Regulation, Evaluation, and Governance Lab, Stanford University, Stanford, California, USA Correspondence: Daniel E. Ho ([deho@stanford.edu](deho@stanford.edu)) Accepted: 14 March 2025 ABSTRACT Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI) . Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. However, the large language models used in these tools are prone to “hallucinate,” or make up false information, making their use risky in highstakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as“eliminating” or “avoid[ing]” hallucinations, or guaranteeing “hallucination-free” legal citations. Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introducesa comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law. 1 | Introduction In the legal profession, the recent integration of large language models (LLMs) into research and writing tools presents both unprecedented opportunities and significant challenges (KiteJackson 2023) . These systems promise to perform complex legal tasks, but their adoption remains hindered by a critical flaw: their tendency to generate incorrect or misleading information, a phenomenon generally known as “hallucination” (Dahl et al. 2024) . As some lawyers have learned the hard way, hallucinations are not merely a theoretical concern (Weiser and Bromwich 2023) . In one highly publicized case, a New York lawyer faced sanctions for citing ChatGPT-invented fictional cases in a legal brief (Weiser 2023); many similar incidents have since been | Varun Magesh and Faiz Surani contributed equally to this work. | | --- | | This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproducti

FAQs

What problem does the article address about AI legal research tools?

It addresses whether leading AI legal research tools truly avoid hallucinations, given that large language models can fabricate false information in legal contexts.

How did the authors evaluate the tools’ reliability?

The study reports the first preregistered empirical evaluation of AI-driven legal research tools, comparing performance across systems and against a general-purpose chatbot.

What do the results show about Lexis+ AI and Westlaw AI-Assisted Research?

The article finds that these tools still hallucinate frequently, with rates estimated between 17% and 33%, and also shows substantial differences in responsiveness and accuracy across systems.

Sep 17, 2026

Uploaded by James