SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

Jul 16, 2026

Upload by Levi

Uncertainty estimation is vital for trustworthy large language model (LLM) deployment and for self-refinement during generation. Existing methods use token-level or sequence-level granularities, which either lack semantic coherence or cannot localize erroneous parts. The work introduces Span-Level Uncertainty Estimation (SLUE) and SPANUQ, a lightweight ∼25M-parameter span probe that distills uncertainty knowledge into a single forward pass using a DETR-style span decoder and a Mixture of Beta uncertainty model. Experiments across five LLMs validate SPANUQ-BENCH and show superior quality and faster inference.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages27

Reading Time1 h 8 min

Abstract

Introduction

arXiv :2607 .0572 1v 1 [ cs .CL] 7 Jul 2026 SPANUQ: Span-Level Uncertainty Quantification for Large Language Model Generation Yimeng Zhang 1 Yingying Zhuang 1 , Ziyi Wang2 , Yuxuan Lu2 , Pei Chen 1 , Aman Gupta 1 , Zhe Su 1 , Ming Tan 1 , Zhilin Zhang 1 , Qun Liu 1 , Manikandarajan Ramanathan 1 , Rajashekar Maragoud 1 , Edward Vul 1 , Jing Huang 1 , Dakuo Wang2 1Amazon, 2Northeastern University Abstract Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors. We formalize Span-Level Uncertainty Estimation (SLUE), a new task that targets the natural granularity for uncertainty: semantically coherent text spans, each conveying a single assessable unit of meaning. To address this task, we introduce SPANUQ, a lightweight (∼25M parameter) probe that distills the uncertainty knowledge from expensive multi-sample inference into a single forward pass over LLM hidden states. SPANUQ employs a DETR-style span decoder to simultaneously detect spans and estimate their uncertainty via a Mixture of Beta distribution, trained with a principled combination of Beta NLL regression and contrastive ranking objectives. We construct SPANUQ-BENCH, the first span-level uncertainty benchmark comprising 20K prompts, ∼293K annotated spans, and continuous soft labels derived from multi-sample claim verification. Experiments on five LLM backbones show that SPANUQ consistently achieves the best span-level uncertainty quality (AUROC 0.908–0.944, MAE 0.110–0.129), outperforming the strongest probe baseline and all sampling-based methods while being 10–20× faster. Its DETR-based span detector attains 0.910 F1, surpassing the best heuristic by 39.4%, enabling precise error localization that sequence-level methods cannot provide. The framework generalizes across five LLMs spanning two model families (AUROC 0.908–0.944), and we additionally observe that sequence-level uncertainty is partially decomposable: the learned importance weighted span composition achieves ρseq = 0 .839, suggesting that span-level estimation subsumes sequence-level as a special case. The project page is available at [https://damon-demon.github.io/SpanUQ.html](https://damon-demon.github.io/SpanUQ.html). 1 Introduction Large language models (LLMs) generate fluent and coherent text across a wide range of tasks, yet their propensity to produce factually incorrect statements, commonly termed hallucinations, remains a critical barrier to deployment in high-stakes domains such as healthcare, legal analysis, and scientific research [1] . A fundamental step toward trustworthy LLM deployment is the ability to estimate how uncertain the model is about each piece of information it generates. Existing approaches to uncertainty estimation operate at two extremes of granularity. Token-level methods compute scores for individual tokens using predictive entropy or learned probes [2–4] . While computationally efficient, these scores are semantically incomplete: a single token rarely constitutes a verifiable ∗ Corresponding author Preprint. fact, and function words receive scores that carry no informational value. Sequence-level methods produce a single score for the entire response by sampling multiple outputs and measuring their semantic dispersion [5–7] . These methods capture meaningful semantic uncertainty but require 10–20× inference cost and cannot localize which parts of a response are unreliable. Consider the example in Fig. 1: an LLM generates “Marie Curie was a Polish physicist who won the Nobel Prize in 1901 .” A token-level method (a) assigns separate scores to every token; function words like “was” and “a” receive low but meaningless scores, while “p

FAQs

What problem does SpanUQ address in uncertainty estimation for LLMs?

It targets the mismatch between uncertainty granularity and meaningful semantic units, where token-level scores lack coherence and sequence-level scores cannot localize which text parts are unreliable.

How does SPANUQ estimate uncertainty at the span level?

SPANUQ uses a lightweight ∼25M-parameter DETR-style span decoder to detect semantically coherent spans and estimate each span’s uncertainty via a Mixture of Beta distribution.

What is SPANUQ-BENCH and how is it constructed?

SPANUQ-BENCH is the first span-level uncertainty benchmark with 20K prompts and about 293K annotated spans, using continuous soft labels derived from multi-sample claim verification.

Jul 16, 2026

Upload by Levi