Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use

Jul 16, 2026

Upload by Hazel

Large language model (LLM) agents can perform well on static benchmarks, yet real-world deployment is constrained by the non-stationary nature of user queries, available tools, and interaction dynamics. This work formalizes OpenAgent (Tool-Use Agent in Open-World), capturing distributional shifts across query, tool, observation, and task-domain dimensions. A controlled sandbox injects fine-grained perturbations across Perception, Interaction, Reasoning, and Internalization tiers, enabling systematic diagnosis. Experiments show performance degradation for both SFT and RL under open shifts, with distinct failure modes. Perturbation-Augmented Fine-Tuning (PAFT) improves SFT robustness via disturbance-based training.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages43

Reading Time1 h 48 min

Introduction

OpenAgent problem setting

Controlled sandbox and tiered perturbations

Evaluation results

Failure modes of SFT and RL

Perturbation-Augmented Fine-Tuning (PAFT)

Threefold contributions

Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use Song-Lin Lv * 1 2 Weiming Wu * 1 Rui Zhu 1 Zi-Jian Cheng 1 2 Lan-Zhe Guo 1 2 arXiv :2607 .0 1084v 1 [ cs .AI] 1 Jul 2026 Abstract While Large Language Model (LLM) agents demonstrate proficiency in static benchmarks, their deployment in real-world scenarios is hindered by the dynamic nature of user queries, tool sets, and interaction dynamics. To address this generalization gap, we formalize OpenAgent (Tool-Use Agent in Open-World), a problem setting characterized by distributional shifts across query, action, observation, and domain dimensions. To systematically diagnose its impact, we construct a controlled sandbox environment where we define fine-grained environmental shifts across a four-tier hierarchy, Perception, Interaction, Reasoning, and Internalization, and conduct a comprehensive series of experiments. Our analysis yields a series of key insights, demonstrating that agents trained via both Supervised Fine-Tuning(SFT) and Reinforcement Learning suffer from varying degrees of performance degradation when confronting open environmental shifts. Building on these insights, we propose Perturbation-Augmented Fine-Tuning, a disturbance-based intervention strategy for SFT that lays the foundation for enhancing agent robustness and utility in realistic environments. Our code will be released at: [https://github](https://github) . com/LAMDA-NeSy/OpenAgent. 1. Introduction The integration of Tool Learning and the Model Context Protocol (MCP) (Anthropic, 2024) has catalyzed a paradigm shift in Large Language Model (LLM) agents, enabling *Equal contribution 1 School of Intelligence Science and Technology, Nanjing University, Nanjing, China 2National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China. Correspondence to: Lan-Zhe Guo <[guolz@lamda.nju.edu.cn](guolz@lamda.nju.edu.cn) >. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) . them to navigate external environments to solve complex, multi-step tasks (Guo et al., 2024 ; Wang et al., 2024b ; Qu et al., 2025) . Optimized via Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), recent open-source models (Hsieh et al., 2023 ; Qu et al., 2024 ; Qwen et al., 2024 ; Bai et al., 2025) have achieved tool invocation proficiency that rivals proprietary frontiers like GPT-4 (Achiam et al., 2023) and Gemini (Team et al., 2023) . As illustrated in Figure 1, under the prevailing static world assumption, where the distribution of tools, schemas, and interaction logic remains consistent between training and inference (Shen et al., 2023 ; Yin et al., 2025 ; He et al., 2025), both SFT and RL paradigms demonstrate stable and continuous performance gains, eventually converging on near-perfect success rates. However, this stability is often an artifact of the closed-set nature of current benchmarks. Real-world deployment is fundamentally non-stationary: APIs are deprecated, tool schemas evolve, and user instructions are frequently ambiguous. When these environmental dynamics diverge from training priors, the apparent mastery observed in static benchmarks often proves fragile. This discrepancy raises a fundamental research question: Do current training paradigms enable agents to generalize to the open world? To rigorously address this, we formally define OpenAgent (Tool-Use Agent in Open-World), a problem setting characterizing shifts across four dimensions: User Queries (∆Q), Tool Sets (∆A), Interaction Dynamics (∆O) and Task Domain (∆D) . To isolate the impact of these shifts from the inherent instability and transient noise of real-world APIs, we establish a controlled sandbox environment. This setup allows for controlled probing, enabling us to maintain a pure closed-set baseline while systematically injecting openworld perturbations across a four-t

FAQs

What problem does the document address regarding LLM tool-use agents?

It addresses the gap between strong performance on static benchmarks and weaker generalization in real-world scenarios where queries, tools, and interaction dynamics change over time.

How is OpenAgent (Open-World Tool-Use Agent) defined in this work?

OpenAgent formalizes distributional shifts across four dimensions: User Queries, Tool Sets, Interaction Dynamics, and Task Domain, capturing the non-stationarity of real deployments.

What are the reported failure modes of SFT and RL in open-world settings?

SFT models are described as prone to trajectory overfitting and brittle symbolic anchoring, while RL models may exhibit boundary blindness driven by teleological bias in reward structures.

Jul 16, 2026

Upload by Hazel