When Failures Propagate: Causal Failure Attribution in Agentic Retrieval-Augmented Generation - Abstract

Sep 15, 2026

Agentic retrieval-augmented generation (RAG) interleaves retrieval, reasoning, and answer generation across multiple hops, creating a mismatch between visible end-to-end errors and their causes. A retrieval error at hop 1 can surface only as a wrong answer later, while later retrieval may also repair the trajectory. The paper introduces AgenticRAG-FP, an interventional benchmark that injects certified faults at chosen hops and re-executes downstream behavior, then evaluates causal diagnosers under propagation depth. Results show coverage-based diagnosis sharply degrades with depth, and counterfactual probes reveal missing post-hoc signal under constrained comparisons.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

1 Introduction

2 Related Work

When Failures Propagate: Causal Failure Attribution in Agentic Retrieval-Augmented Generation Lauren Pothuru Anote [laurenpothuru100@gmail.com](laurenpothuru100@gmail.com) arXiv : 2608 .20627v 1 [cs .CL] 20 Aug 2026 Abstract Agentic retrieval-augmented generation (RAG) interleaves retrieval, reasoning, and answer generation across multiple hops. A retrieval error at hop 1 can surface only as a wrong answer at hop 3, while later retrieval can also repair the trajectory. This paper introduces AgenticRAG-FP, an interventional benchmark for causal failure attribution in agentic RAG. The benchmark injects a certified fault at a specified hop, re-executes the downstream trajectory, and evaluates diagnosers against the known intervention. Its central question is whether a post-hoc trace still identifies the injected hop after the suffix changes. In the completed strict dense Claude Haiku 4.5 sweep on 80 three-hop MuSiQue questions, coverage-based diagnosis is 0.91 at hop 1 and 0.00 at hops 2 and 3 (n=43, 36 , 21 failed trajectories) . A smaller content-corruption study changes an answer-bearing or bridge fact in topically intact evidence. At depth 2, where 18 failed cases remain after filtering, coverage-based diagnosis is 0.00 and a frozen-hop counterfactual probe is 0.67 in an exploratory pooled comparison. Depth-3 content estimates are descriptive only because they contain three failed cases. These results make propagation depth an explicit evaluation axis for diagnosing agentic RAG failures while distinguishing broad evidence of post-hoc signal loss from small-sample method comparisons. 1 Introduction Retrieval-augmented generation (RAG) conditions generation on retrieved evidence rather than only model parameters (Lewis et al., 2020) . Agentic RAG extends this process across several decisions: a language model selects a sub-query, observes evidence, and decides whether to search again or answer, as in ReAct-style reasoning and acting (Yao et al., 2023) . This enables multi-hop question answering, but it also separates a visible end-to-end error from its cause. Consider a three-hop trajectory. If the evidence at hop 1 is irrelevant, the agent can form a drifted sub-query at hop 2, retrieve evidence on the wrong premise, and produce an ungrounded final answer. An evaluator that inspects only the final trace may identify a late low-coverage hop or label the failure as answer generation, while missing the earliest causal fault. A different agent may retrieve compensating evidence at a later hop and answer correctly. Standard answer accuracy, retrieval recall, and final-answer correctness conflate an injected fault, its propagation, and its recovery. Existing diagnostic methods can inspect traces or attempt local repair, but post-hoc traces do not provide a certified root-cause label. This leaves two evaluation questions unresolved: can a diagnoser recover a known intervention hop after the failure has propagated, and when do counterfactual repair probes supply information missing from the final trace? Answering either question requires an intervention that allows the agent to react and retains the intervention label. AgenticRAG-FP provides this setting. It corrupts a trace prefix at hop h, runs the remaining trajectory with the same agent, and scores diagnoses against the certified intervention. The paper centers on how propagation changes the information available for exact-hop attribution. Its three contributions are: 1. An interventional benchmark that supports structural retrieval faults and certified content corruption. The latter changes a known fact span in otherwise relevant evidence and records the span pair, enabling deterministic absorbed, resisted, and derailed outcome labels without an LLM judge. 2. A depth-conditioned analysis of post-hoc signal. In the completed strict dense MuSiQue sweep, coverage-based attribution falls from 0.91 at hop 1 to 0.00 at hops 2 and 3 . This pattern concerns

FAQs

Why can an early retrieval error appear only as a late wrong answer in agentic RAG?

Agentic RAG makes sequential decisions over multiple hops, so an incorrect evidence choice at hop 1 can alter later sub-queries and retrieval targets. The resulting trajectory can propagate the fault into the final answer while hiding the original cause.

What is AgenticRAG-FP and how does it support causal failure attribution?

AgenticRAG-FP injects a certified fault at a specified hop, re-executes the remaining trajectory with the same agent, and scores diagnosers against the known intervention label. This enables exact-hop attribution under propagation.

How does diagnosis performance change with propagation depth in the reported MuSiQue study?

In the completed strict dense Claude Haiku sweep, coverage-based diagnosis is 0.91 at hop 1 and drops to 0.00 at hops 2 and 3. The paper interprets this as loss of post-hoc signal available after suffix re-execution.

Sep 15, 2026

Uploaded by Clementine