Write, Execute, Refine - From Skill Followers to Skill Optimizers via Reinforcement Learning from Execution Feedback

Sep 15, 2026

Uploaded by Violet

Expert-written natural language skills can improve tool-using agents, yet agent-authored skills degrade performance by 8–11 points versus using no skill. This gap indicates that following procedural guidance and learning to improve it from execution evidence are distinct abilities. The work studies how to turn intermediate-skill execution experience into training states for a skill optimizer. It proposes WER, a multi-phase framework that trains an optimizer outside a frozen executor using verifier-based relative credit and paired success/failure refinement states. Experiments on BFCL v4 multi-turn and τ2-bench show consistent gains over no-skill baselines and over the same backbone without optimizer training.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Introduction

Motivation and capability gap

WER framework overview

Training phases and refinement states

Experimental results and comparisons

Write, Execute, Refine: From Skill Followers to Skill Optimizers via Reinforcement Learning from Execution Feedback Kang Peng1 * , Zhiwei Zhang2,3 * , Yichen Zhang4 , Zezhong Wang2 , Yiming Du2 , Geng Tu 1 , Baojun Wang5 , Bin Liang2,3 , Ruifeng Xu 1†, Kam-Fai Wong2,3 1Harbin Institute of Technology, Shenzhen, China 2The Chinese University of Hong Kong 3MoE Key Laboratory of High Confidence Software Technologies 4Harbin Institute of Technology, Harbin, China 4Huawei Technologies Co., Ltd. [26s165138@stu.hit.edu.cn](26s165138@stu.hit.edu.cn) , [zhangzhiwei1019@link.cuhk.edu.hk](zhangzhiwei1019@link.cuhk.edu.hk) arXiv : 2608 . 17587v 1 [cs .CL] 18 Aug 2026 Abstract Expert-written natural language skills can improve tool-using agents, yet agent-authored skills perform 8–11 points worse than using no skill. This gap suggests that following procedural guidance and improving it from execution evidence are distinct capabilities. Inferencetime loops can repair skills but do not improve the model that writes the next one. We study how to organize execution experience from intermediate skills into training states for an optimizer. We introduce WER (Write, Execute, and Refine), a multi-phase framework that trains a Skill Optimizer outside a frozen executor. The optimizer proposes skills, a frozen agent executes each repeatedly, anda programmatic verifier scores the outcomes. The scores provide relative credit and select mixed-outcome records. Matched successful and failed trajectories from these records form the next phase’s refinement states, so the optimizer learns from the consequences of its earlier outputs. On BFCL v4 multi-turn and τ 2-bench, WER improves average Pass@1 over the no-skill baseline by 7.80 and 3.85 points, respectively. Under an identical refinement workflow, it outperforms the same backbone without optimizer training by 9.35 and 10.29 points. The trained 4B optimizer reaches 76.63% on BFCL v4, outperforming all evaluated off-theshelf general-purpose models used as skill optimizers on average. Our code is available at [https://github.com/littlepkk/](https://github.com/littlepkk/)[ ](https://github.com/littlepkk/)[WER4skill-optimizer-training](WER4skill-optimizer-training.)[.](WER4skill-optimizer-training.) 1 Introduction Reliable tool use requires procedural knowledge: which tool to invoke, how to validate its arguments, and when to retry (Yao et al., 2023 ; Schick et al., 2023 ; Patil et al., 2025) . A common approach represents this knowledge as skills, compact *Equal contribution.†Corresponding author. Skill optimizer phase-wise GRPO refinement skill Execution Sandbox Agent Tool environment Figure 1: The Skill Optimizer remains outside the environment and observes execution. It writes a skill for a frozen agent and revises the skill from the resulting trajectory. natural-language instructions added to an agent’s context at inference time (Wang et al., 2023 ; Jiang et al., 2026) . On SkillsBench (Li et al., 2026), expert-curated skills raise the average pass rate from 33.9% to 50.5%, yet agent-authored skills fall 8–11 points below using no skill at all. This reveals a capability gap: agents can benefit substantially from procedural guidance without being able to write it reliably. A common response to this gap is an inferencetime loop in which an LLM drafts a skill, observesits execution, and revises it (Shinn et al., 2023 ; Liu et al., 2025 ; Alzubi et al., 2026 ; Yang et al., 2026b ; Liu et al., 2026) . More broadly, reflecting on trajectories and execution outcomes, then distilling the lessons into reusable experience, has become a common workflow recipe for improving agents. Yet this recipe presupposes a capability that language models may not reliably possess: converting behavioral evidence into the procedural instruction needed to prevent the observed failure. Although such loops can improve the current artifact, they typically leave the skill writer unchanged: each

FAQs

Why do agent-authored skills underperform expert-written procedural guidance?

Agent-authored skills trail expert guidance by 8–11 points, suggesting that following instructions and converting execution evidence into reliable procedural improvements are different capabilities.

What is WER and how does it train a Skill Optimizer?

WER (Write, Execute, and Refine) is a multi-phase framework that trains a Skill Optimizer outside a frozen executor. The optimizer proposes skills, a frozen agent executes them, and a verifier scores outcomes to guide refinement via relative credit.

How does WER construct refinement states across phases?

WER uses verifier results to select mixed-outcome records and forms the next phase’s refinement states by pairing successfully and unsuccessfully executed trajectories. This lets the optimizer learn to correct deficiencies exposed by its own previous outputs.

Sep 15, 2026

Uploaded by Violet