Doomed from the Start Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

Jul 16, 2026

Upload by Rowan

Large language model (LLM) agents often follow trajectories that are destined to fail, but they still consume significant inference compute before failure becomes observable. This work predicts failure early using the agent’s internal representations: lightweight per-round probes on hidden activations anticipate eventual episode failure as early as the first round. The signal drives a calibrated abort cascade that jointly searches per-round recall budgets, providing an episode-level success recall guarantee. Experiments on TextCraft show compute savings of 47.1% and 37.2%, outperforming behavior-only monitoring, with additional analysis of sample complexity for certification.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

Abstract

Introduction

Problem: wasted compute in doomed episodes

Obstacles: scarce early signals, controllable harm bounds, and non-composition across rounds

Method: recall-controlled abort cascade with calibrated gates

Results: compute savings and effectiveness across models

Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade Kai Ruan 1 , Zihe Huang2 , Ziqi Zhou3 , Qianshan Wei4 , Xuan Wang5 , Hao Sun 1 ∗ 1 Gaoling School of Artificial Intelligence, Renmin University of China 2Institute of Computing Technology, Chinese Academy of Sciences 3Duke University 4Institute of Automation, Chinese Academy of Sciences 5 College of Computer Science, Zhejiang University [haosun@ruc.edu.cn](haosun@ruc.edu.cn) arXiv :2607 .06503v 1 [ cs .AI ] 7 Jul 2026 Abstract Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable. We show that failure is predictable early from the agent’s internal representations: lightweight per-round probes on hidden activations anticipate eventual episode failure as early as the first interaction round, where scorers reading only the agent’s observable behavior are barely better than chance. We turn this signal into a practical abort cascade: one distribution-free calibrated gate per round, with per-round recall budgets jointly searched so that eventually-successful episodes survive all gates at a userspecified global rate; this episode-level guarantee is the one that matters in deployment, since false-abort risk accumulates across gates. Across two agent models on TextCraft, the cascade meets every recall target from 90% to 97% and, atthe 90% target, saves 47. 1% ± 10.3%(Qwen-2 .5-7B) and 37.2% ± 8.8%(Llama-3 .2-3B) of inference compute, 1.6– 1.7× the best single-gate policy. An otherwise-identical cascade reading only behavior saves roughly half as much, and adding behavioral features to the probe yields no further gain: the hidden states capture what behavior reveals. Finally, we characterize the sample complexity of certifying high recall targets, telling practitioners which recall promises their data can, and provably cannot, back. The code will be released soon. Introduction LLM-based agents increasingly tackle long-horizon tasks (web navigation, tool use, embodied simulation) in which a single episode spans many rounds of interaction and consumes thousands of generated tokens. A large fraction of this compute is wasted: when an agent has misunderstood the task, entered an unrecoverable state, or begun to loop, the episode is already doomed long before it formally times out or returns a wrong answer. If we could detect such doomed episodes early and abort them, the saved compute could be reallocated to retries, sampling additional trajectories, or simply reducing serving cost. Three obstacles stand in the way. First, we need a signal that distinguishes doomed episodes from eventually-successful ones early in the trajectory, and early is precisely when be- ∗Corresponding author. episode continues while every gate passes round 1 round 2 round 3 · · · y ∈ {0 , 1} h1 h2 h3 f1 (h1 ) > τ1 ? f2 (h2 ) > τ2 ? f3 (h3 ) > τ3 ? yes abort: stop episode, save remaining compute C − cr τr : Clopper–Pearson-calibrated so gate r passes ≥ tr of successes (t1 , . . . , tR g): searched to maximize savings subject to global recall ≥ ρ⋆ Figure 1: Recall-controlled abort cascade. At each of the first Rg rounds, a linear probe fr reads the agent’s hidden state hr and a gate aborts the episode if the failure score exceedsa calibrated threshold τr ; an episode completes only if it survives every gate. Thresholds carry per-round distributionfree recall guarantees, and the per-round budgetstr are jointly searched so that the episode-level success recall meets a userchosen target ρ⋆ . havioral evidence is scarcest. We find that scorers reading only the agent’s observable behavior are barely better than chance in the first round and become informative only around rounds 3–4, by which time over a third of episodes have already finished and much of the useful

FAQs

How does the method predict doomed LLM agent episodes early?

It uses lightweight per-round probes that read the agent’s hidden activations and produce failure scores. These probes anticipate eventual episode failure as early as the first interaction round.

What is the recall-controlled abort cascade and how does it ensure safety?

At each of the first rounds, a gate compares the probe score to a calibrated threshold. Thresholds are set using Clopper–Pearson recall guarantees and per-round recall budgets are searched so global episode-level recall meets a user-specified target.

Why are behavior-only abort policies less effective than hidden-state probes?

The paper reports that scorers relying only on observable behavior are barely better than chance in the first round and become informative later. Hidden-state probes match or exceed the behavior scorer’s eventual peak much earlier, yielding stronger compute savings.

Jul 16, 2026

Upload by Rowan