Reason Less, Verify More - Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents

Sep 21, 2026

Tool-using LLM agents can successfully complete tasks while violating the very policies they are meant to enforce, producing silent wrong-state outcomes when permissive tools execute well-formed calls that correspond to forbidden state transitions. The study analyzes the 2-bench airline setting, quantifying silent failures and evaluating deterministic read-only preexecution gates. A four-gate suite increases benchmark success from 29.6% to 42.0% on gpt-4o-mini and shows replicated gains in a disjoint seed set, with mechanism-bound improvements on firing tasks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

AccessFree to download

Abstract

1 Introduction

1.1 A trust problem, not only a capability problem

1.2 A reliability gap, not measurement noise

Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents Vikas Reddy Independent Researcher Sumanth Reddy Challaram Indian Institute of Technology Kharagpur Kharagpur, India Abhishek Basu Massachusetts Institute of Technology Cambridge, USA arXiv :2607 .07405v 1 [ cs .AI] 8 Jul 2026 Abstract Tool-using LLM agents can violate the very policies they are deployed to enforce while appearing to complete the task successfully. In policy-permissive environments, a tool may execute any well-formed call even when the corresponding state transition is forbidden by domain policy. The result is a silent wrong state: the booking is cancelled, the passenger count is changed, or a user claim is acted on without verification, and neither the tool nor the agent’s self-report exposes the violation. We study this failure mode in the 􀁧2-bench airline domain. On a budget agent, 78% of observed failures are silent wrong-state failures with no tool error, and the aggregate failure rate is reproducible across disjoint seeds rather than reflecting sampling noise. We then evaluate a lightweight intervention: deterministic, read-only preexecution gates that inspect the proposed tool call and current database state before allowing a write. A four-gate suite raises fullbenchmark success from 29.6% to 42.0% on gpt-4o-mini (+12.4pp; paired task-level bootstrap 􀀥 = 0. 0012), and the lift reproduces on a disjoint 15-seed replication set (+12.3pp; 􀀥 = 0. 0008) . The effect is concentrated where the gates actually fire: on the 26/50 firing tasks, success rises by +19.2pp, while movement on the 24 non-firing tasks does not exclude zero. Two negative controls, a self-enforcing retail domain and BFCL, bound the mechanism: gates help when tools are policy-permissive and add little where tools already enforce their own preconditions. Finally, we report suggestive evidence, not a central claim, that the same failure mode persists in a frontier-model harness: gpt-5.2 at default reasoning still attempts policy-violating writes, and the same gate suite improves success from 61.2% to 71.6%(+10.4pp; 􀀥 = 0. 020; n=5, no replication) . The contribution is a bounded evaluation and reliability result: deterministic gates do not guarantee task success, but they can deterministically prevent a known class of silent policy-violating writes at the action boundary. CCS Concepts • Computing methodologies → Artificial intelligence; • Software and its engineering → Software safety; • Security and privacy → Formal methods and theory of security. Keywords LLM agents; tool use; policy compliance; deterministic verification; agent evaluation; runtime enforcement; agent reliability 1 Introduction 1.1 A trust problem, not only a capability problem When an LLM agent operating a customer-service tool cancels anon-refundable reservation, modifies a passenger count that policy forbids modifying, or processes a request on the strength of an unverified user claim, two failures occur at once. The agent takes a prohibited action, and because the underlying tool accepts any syntactically valid call, the action succeeds. No exception is raised, no error is returned, and the agent may confidently report the task complete. This is a trust problem. The operator observing the run sees a clean transcript and a successful final response, but the system state is wrong. The issue is therefore not merely whether the model can reason about a policy in the abstract. It is whether the deployed agent system has any reliable signal when the model fails to apply that policy before issuing a mutating tool call. The root cause is a property of the agent environment. In realistic tool benchmarks such as 􀁧 -bench and 􀁧2-bench [1, 11], some tools are deliberately policy-permissive: a write tool executes when its arguments are well-formed, even if the domain policy forbids the state transition. The policy lives in a natu

FAQs

What is the silent policy-violation failure mode in tool-using LLM agents?

A tool may execute a syntactically valid call even when the corresponding state transition is forbidden by domain policy. The result is a silent wrong state, with no tool error and no reliable signal in the agent trace to expose the violation.

How prevalent are silent wrong-state failures in the studied airline domain?

In the 2-bench airline tasks, 78% of observed failures are silent wrong-state failures, where the final database is wrong but no tool raises an error.

How do deterministic read-only preexecution gates improve performance?

The approach inspects the proposed tool call and the current database state before allowing a write, blocking known forbidden transitions. The four-gate suite raises full-benchmark success from 29.6% to 42.0% on gpt-4o-mini and reproduces an uplift on a disjoint 15-seed replication set.

Sep 21, 2026

Uploaded by Valentina