The “I Don’t Know” Filter: Enhancing Agentic Reliability in Function Calling

Jul 16, 2026

Upload by Hazel

Language models powering agents have advanced rapidly on function-calling benchmarks, yet training and evaluation metrics often reward positive answers even when uncertainty remains, causing hallucinations. For high-stakes settings that rely on function calls, such hallucinations can lead to severe failures. This work proposes an agent evaluation metric that penalizes incorrect function calls and introduces a lightweight, trainable filter to quantify uncertainty and suppress potentially harmful calls, enabling agents to safely respond with “I don’t know.”

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

Introduction

Background

Uncertainty quantification for LMs

Reliable function calling for agents

The “I Don’t Know” Filter: Enhancing Agentic Reliability in Function Calling Stefan Broecker1 , Mason del Rosario2 , Boris Selitser2 , Thomas Strohmer3 arXiv :2607 .04034v 1 [ cs . SE] 4 Jul 2026 Abstract—The language models that underpin agents have seen a rapid rise in performance on function calling benchmarks. However, the metrics used in the training and evaluation of these models often encourage models to make positive claims even when the answer is uncertain, leading to hallucinations. Such hallucinations can be disastrous when language models are trusted to use function calls to make decisions in high stakes applications. To that end, we propose an agent evaluation metric that takes into account the negative outcomes associated withincorrect function calls. Further, to catch hallucinations before they can cause harm, we propose a lightweight trainable filter that can quantify a language model’s uncertainty and remove potentially harmful function calls. By training that filter to detect and suppress uncertain function calls without modifying the underlying model, we demonstrate a practical path toward agents that know when to say “I don’t know,” a property we argue is essential to production reliability. I. INTRODUCTION Agents are language models (LMs) that combine tools, memory, and reasoning to perform tasks [1] . As agents have improved at tool use, developers in critical industries such as digital infrastructure [2], health care, and public services [3] have begun to trust agents to make decisions in production settings. As an example, errant tool calls in such production environments have resulted in airfares offered at disallowed discounts and eggs purchased at exorbitant prices [4] . A large scale study of agents in production showed a persistent gap in agent performance on benchmarks and in production systems [5] . In that study, reliability was most commonly cited as the bottleneck to agent deployment. A key part of that reliability gap is hallucinations, and given agents’ability to take autonomous action in real-world decision making, hallucinations can cause much more severe and material damage than standalone LMs. [6] makes the claim that hallucinations persist because of conventions in LM evaluations. In most evaluations, an incorrect answer is weighed the same as a non-answer, so guessing when uncertain improves accuracy. However, Bastounis et al. state that this tendency to provide a nonanswer when uncertain is a fundamental roadblock to “trustworthy” (i.e., reliable) AI, and they describe this tendency to hallucinate and the inability to state “I don’t know” as a critical component of the Consistent Reasoning Paradox [7] . In this work, we study “agent reliability” through this lens: the agent’s ability to express uncertainty, i.e., by explicitly 1University of California, Davis, Department of Computer Science, [sbroecker@ucdavis.edu](sbroecker@ucdavis.edu) 2 Okareo, Inc., {mason, [boris](boris}@okareo.com)[}](boris}@okareo.com)[@okareo.com](boris}@okareo.com) 3University of California, Davis, Department of Mathematics, [strohmer@math.ucdavis.edu](strohmer@math.ucdavis.edu) stating “I don’t know”(IDK) . Towards this end, we make the following contributions: • An accuracy metric that penalizes incorrect function calls, emphasizing the danger of an autonomous agent guessing at an answer instead of admitting uncertainty and encouraging IDK responses • A simple filter that uses “whitebox,” “graybox,” and“blackbox” features to identify likely-incorrect function calls (cases of IDK) to reduce the likelihood of hallucinations • A simple method for generating synthetic data that can be used to train that filter, enabling deployment even when labeled function calling data is limited • A study of the filter’s effectiveness in reducing uncertainty across different open source LMs and function calling benchmarks With these contributions, we demonstrate that explicitly measuring an age

FAQs

Why do hallucinations become a reliability risk in agent function calling?

Function-calling agents can take autonomous actions based on uncertain outputs, and evaluation conventions may encourage guessing instead of admitting uncertainty. In high-stakes applications, incorrect function calls can cause serious real-world damage.

What evaluation contribution does the paper propose?

It proposes an agent evaluation metric that accounts for negative outcomes tied to incorrect function calls, penalizing confident-but-wrong behavior and promoting “I don’t know” responses.

How does the proposed filter improve reliability without changing the base model?

The work introduces a lightweight trainable filter that estimates a language model’s uncertainty and removes potentially harmful function calls. The filter is trained to suppress uncertain calls without modifying the underlying language model.

Jul 16, 2026

Upload by Hazel