Equal Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

Jul 16, 2026

Upload by Oliver

Search APIs form the core retrieval layer for many tool-using agents and are often used most frequently. Traditional evaluations emphasize answer accuracy from URL, title, and snippet previews, while full-page retrieval remains token-intensive and motivates progressive disclosure. This work frames a commercial search API as a decision surface: the ranked evidence exposed before fetching shapes whether an agent answers, searches again, or spends tokens opening pages. Experiments with a fixed agent over SEALQA-HARD compare Brave, Tavily, and Firecrawl, showing similar accuracy but distinct evidence economies and contradiction-to-gold ratios, making provider choice a retrieval-budget and policy decision.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Abstract

Introduction

Related Work

Equal Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents Sriram Selvam [selvamsriram@gmail.com](selvamsriram@gmail.com) Anneswa Ghosh [anneswaghosh@gmail.com](anneswaghosh@gmail.com) arXiv :2607 . 10 198v 1 [ cs .CL] 11 Jul 2026 Abstract Search APIs are the fundamental retrieval layer for many agents and are often their most frequently used tool. Traditional search APIs provide URLs, titles, and snippets that preview website contents. Because full-page retrieval is token-intensive, agent retrieval architectures increasingly use progressive disclosure: the agent first sees snippets and then chooses whether to fetch full pages. In such systems, search API performance is often evaluated primarily by answer accuracy. We argue that a commercial search API is better understood as a decision surface: the ranked snippets, URLs, and metadata that determine whether an agent answers immediately, searches again, or spends tokens opening pages. We test this claim with one frozen GPT-5.4 agent, two tools (search_web and fetch_page), and 100 questions from SEALQA-HARD, varying only the search provider (Brave, Tavily, Firecrawl) . A Kimi-K2.6 oracle labels every content element visible to the agent (URL, title, snippet, and fetched page, when fetched), producing 6,869 valid per-URL judgments. We use an audited correct-answer label, semantic_match, which preserves exact matches while accepting harmless formatting and naming variants. Under this measure, the providers remain close (25, 25, 26 / 100), but their evidence economies differ sharply: Brave offers gold-answer-rich snippets, Tavily concentrates gold-supporting URLs at rank 1, and Firecrawl is associated with broader exploration under this fixed agent policy. We also introduce a surface contradiction-to-gold URL ratio, which varies from 0 .92 to 2 .59. Provider choice is therefore a retrieval-budget and policy decision, not merely a recall decision. 1 Introduction Early search-augmented agents often depended primarily on extracted page data provided by search API providers. As complex use cases have evolved, the agent landscape has shifted toward progressive disclosure: models receive top-n URLs, titles, snippets, and key metadata such as freshness, and then decide whether to fetch full pages. In this new landscape, engineering practice often treats search API providers as replaceable components. If two APIs return plausible top-k URLs and produce similar answer accuracy, the rest of the agent pipeline is assumed to be largely unaffected. This paper argues that the relevant object is the pre-fetch surface. Before a page is opened, an agent does not see a search index or a corpus; it relies on search API results. That surface is the evidence state on which the agent decides whether to answer, search again, or spend fetch tokens. We call commercial search APIs decision surfaces for tool-using language agents. For grounded language-model systems, the retrieval API is part of the grounding interface. It determines which evidence is exposed before generation, which contradictions enter context, and how much retrieval budget is spent before an answer is produced. Decision-surface evaluation therefore measures not only retrieval quality, but also the faithfulness and efficiency conditions under which grounded answers are generated. The distinction matters because final correctness can converge while the internal pipeline diverges. A provider may expose enough snippet evidence for immediate answering. Another may put the relevant URL at rank 1, making a top-result fetch policy effective. A third may expose sparse snippets and be associated with broader exploration under the same policy. These regimes can yield similar aggregate accuracy with different cost, latency, contamination, and failure modes. Under a controlled protocol, we freeze the answer model, prompt, tools, maximum iterations, judge, and page-fetch backend. The only ex

FAQs

为什么本文将搜索 API 称为“决策面（decision surface）”？

因为在页面被打开之前，代理看到的排序片段、URL与元数据会决定它是立刻作答、再次检索还是消耗代币去抓取页面。评估重点因此不只是检索好坏，还包括决策时的证据状态与效率条件。

实验中有哪些关键控制变量与变化因素？

本文冻结答案模型、提示、工具、最大迭代次数、评判器以及页面抓取后端。唯一实验条件是搜索 API 提供方在 Brave、Tavily 与 Firecrawl 之间变化。

结果显示三家搜索提供方的效果差异体现在哪里？

在 audited correct-answer 与 semantic_match 指标下，它们的答案正确性接近（约 25/25/26）。但证据经济差异显著：Brave 更偏向富含“金答案”片段，Tavily 更倾向把支持金答案的 URL 集中到第 1 名，而 Firecrawl 更关联到在固定策略下的更广探索，并伴随不同的表面矛盾到金 URL 比率。

Jul 16, 2026

Upload by Oliver