What Survives Into Context: A Diagnostic for Budget-Constrained Multi-Hop RAG and When Submodular Evidence Packing Improves It

Jul 16, 2026

Upload by Anda

Retrieval-augmented generation (RAG) under a fixed reader-context budget turns evidence selection into a critical optimization problem, because only a fraction of retrieved passages can be shown. The work argues recall of the retrieved set is not the right target and proposes answer-in-context, measuring whether a gold answer survives as a contiguous span in the packed context. This predicts answer F1 better than recall, adds incremental predictive power, and guides a budgeted monotone submodular packer.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Introduction

Measurement gap: retrieval recall vs reader context

Proposed diagnostic and packed-context objective

Contributions and scope of results

Contributions

What Survives Into Context: A Diagnostic for Budget-Constrained Multi-Hop RAG and When Submodular Evidence Packing Improves It Ananto Nayan Bala Ahsanullah University of Science and Technology [nayan.ananto@gmail.com](nayan.ananto@gmail.com) arXiv :2607 .00725v 1 [ cs .CL] 1 Jul 2026 Abstract Retrieval-augmented generation (RAG) under a fixed reader-context budget forces a selection problem: of the evidence retrieved, only a fraction can be shown to the reader. We argue that document recall—the standard retrieval metric—is the wrong quantity to optimize in this regime, and we make two contributions. First, as a general contribution, we introduce answerin-context, a diagnostic that measures whether a gold answer survives as a contiguous span in the packed reader context (not the retrieved set) . It predicts answer F1 better than recall (r=0 .39–0.55 vs.∼0.31), separates answer quality roughly five-fold (0 .60 vs. 0.12 on HotpotQA), and carries information beyond retrieval: it adds ∆R2 =0 .17 over recall and shows a 4.6 × EM gap even among questions where all gold was retrieved. We also confirm it interventionally: on 2WikiMultiHopQAa packing change that raises coverage but not answer-in-context yields no accuracy gain. Second, as a conditional contribution, we cast reader-context construction as budgeted monotone submodular maximization and build a packer that jointly optimizes relevance, query coverage, representativeness, and diversity. On HotpotQA with a 160-token budget and a 3B reader it beats a strong focused heuristic, MMR, and naive packing—by up to +5 .1 F1 at equal-or-lower token cost, across three seeds. Crucially, we map the scope of this win honestly: it requires the conjunction of (i) multi-hop complementary structure,(ii) retrieval that surfaces the evidence,(iii) a binding but not extreme budget, and (iv) a reader weak enough that evidence density, not reading capacity, is the bottleneck. A quantization-controlled readerscale ladder (3B→7B→14B) shows the edge over the heuristic is absorbed by 7Band significantly reverses by 14B, while the diagnostic explains every boundary with a single variable. 1 Introduction A retrieval-augmented reader has a finite context window, and in practice an even smaller evidence budget: the share of that window allocated to retrieved passages. Once retrieval returns more relevant text than fits, the system must decide what to keep. This selection step is usually treated as an afterthought—concatenate the top-k, truncate to fit (Lewis et al., 2020 ; Ram et al., 2023)—yet under a tight budget it is the step that decides whether the reader ever sees the answer. The community’s default retrieval metric, recall@k, is computed on the retrieved document set. But the reader never consumes the retrieved set; it consumes the packed context. When packing discards evidence to fit a budget, recall and whatthe-reader-sees diverge. The divergence is acute for multi-hop questions (Yang et al., 2018 ; Trivediet al., 2022), where the answer depends on combining evidence from several documents: retrieving all of them is necessary but not sufficient, because the packer may keep a redundant pair and drop the bridge. Figure 1 makes the gap concrete. This paper starts from a measurement gap and ends with a method. We first ask: what property of the reader context actually predicts answer quality under a budget? We define answer-in-context—does a gold answer appear verbatim in the packed context—and show it predicts answer F1 far better than retrieval recall on every dataset we test (§3) . This reframes the budgeted-RAG objective from “retrieve the gold documents” to “pack so the answer survives.” We then ask: can a principled packer move that quantity? We formulate reader-context construction as budgeted monotone submodular maximization (§4) and show on HotpotQA it delivers a statistically clean win over heuristic packing, MMR, and naive concatenation across three seeds (§5) . A per-

FAQs

Why does retrieval recall@k fail to reflect answer quality under a fixed reader-context budget?

Because the reader never consumes the retrieved set; it consumes the packed context. When packing discards evidence to fit the budget, recall and what the reader actually sees can diverge, especially for multi-hop questions where missing bridges prevents the answer.

What is answer-in-context and what does it measure?

Answer-in-context is a diagnostic that checks whether a gold answer appears verbatim as a contiguous span in the packed reader context, not merely whether the answer-relevant documents were retrieved.

How does the submodular evidence packer improve performance, and when does it stop helping?

The packer formulates reader-context construction as budgeted monotone submodular maximization to balance relevance, coverage, representativeness, and diversity, improving HotpotQA up to around +5.1 F1 under tight token budgets. The gains depend on conditions such as multi-hop complementary structure, evidence being retrieved, a non-extreme budget, and a reader whose limitations make evidence density the bottleneck; increasing reader strength can absorb the packing advantage and reverse the trend at larger scales.

Jul 16, 2026

Upload by Anda