GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation (VA) Tasks

Jul 16, 2026

Upload by Aria

Robots deployed in commercial and industrial settings require reliability under open-world variability, yet agentic coding systems must reconcile interpretable robot programs with the adaptability of model-free policies. The paper studies “Variational Automation” (VA) tasks, where object geometry and pose vary beyond fixed automation, causing model-free approaches to miss the reliability gap for persistent execution. It introduces Graph-as-Policy (GaP), a multi-agent harness that builds directed computation graphs from a Modular Open Robot Skill Library (MORSL) and refines them via parallel simulation. Evaluation on 8 new VA benchmarks shows success rates outperforming baselines.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages52

Reading Time2 h 11 min

Abstract

Introduction

Variational Automation tasks and reliability gap

Agentic coding harness and steering coding agents

Code-as-Policy background and challenges

GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation (VA) Tasks Kaiyuan Chen 1 ,∗ Shuangyu Xie 1 ,∗ Letian Fu 1 Justin Yu 1 William Pacini 1 Sandeep Bajamahal 1 Hudson Kim 1 Jaimyn Drake 1 Daehwa Kim3 Haoru Xue 1 Jonathan Francis3 ,4 Christian Juette4 Peter Schaldenbrand3 ,4 Muhammet Yunus Seker3 ,4 Ruwan Wickramarachchi4 Uksang Yoo 1 ,3 Guanzhi Wang2 Adithyavairavan Murali2 Balakumar Sundaralingam2 S. Shankar Sastry 1 Spencer Huang2 Yuke Zhu2 Linxi “Jim” Fan2 Ken Goldberg 1 arXiv :2607 .05369v 1 [ cs .RO] 6 Jul 2026 1 University of California, Berkeley 2 NVIDIA 3 Carnegie Mellon University 4 Bosch ∗ equal contribution Project Website: [https://graph-robots.github.io/gap](https://graph-robots.github.io/gap) Figure 1: GaP system architecture. Given a “Variational Automation (VA)” task specification, GaP uses a multi-agent harness for coding tools such as Claude and Gemini to automatically generate computation graphs that include “skill” nodes from a Modular Open Robot Skill Library (MORSL), which includes model-based procedures (e.g. ROS [1]) and model-free policies (e.g. GraspGen [2]) . GaP then orchestrates self-learning using simulation (e.g. NVIDIA Isaac [3]) to iteratively refine the graph, which can then be tributinterpreted without the agents on an edge device for persistent execution over time. Bottom: 8 Variational Automation Benchmark Tasks (4 Sim and 4 Real) . Abstract: For robots to work reliably in commercial and industrial applications, can recent advances in agentic coding systems combine interpretable robot programming with the open-world adaptability of model-free policies? We focus on “Variational Automation”(VA), a class of tasks that have larger variations in object geometry and pose than fixed automation. Model-free policies often struggle to close the reliability gap for VA tasks, which must be executed persistently and reliably in commercial and industrial applications. Motivated by prior work on Task and Motion Planning (TAMP) and the Robot Operating System (ROS), we introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs with perception, planning, and control nodes from a Modular Open Robot Skill Library (MORSL) . GaP then generates an internal simulation environment to rehearse task instances with different graphs in parallel to iteratively refine the graph structure and parameters to improve success rates and throughput. Evaluation with 8 new open VA task benchmarks, 4 in simulation and 4 in real-world, suggests that GaP can achieve success rates that significantly outperform baselines. Details, code, and data will be posted online: [https://graph-robots.github.io/gap](https://graph-robots.github.io/gap) Keywords: Agentic Coding, Automation, Self-Learning 1 Introduction A majority of robot learning research focuses on generalist robotics, where a robot must perform abroad variety of tasks. Most of this research has focused on model-free end-to-end Vision-LanguageAction (VLA) models [4, 5, 6, 7, 8, 9] . However fully generalist robots struggle to achieve commercial / industrial levels of reliability [10] and there is increasing interest in how robot learning could be useful for more specialized task classes [11] . In this paper, we identify a class of “Variational Automation (VA)” tasks that differ from“fixed automation”(which blindly repeats the same motions e.g., spot welding or spray painting) . In VA tasks, a robot persistently performs varying instances of a task with non-trivial variation in the geometry and pose of objects (e.g., to sort packages, make coffee in a cafe, or build sandwiches in acommerical kitchen) . Today, fixed automation is set up and tuned by humans using rigorous, classical engineering methods in logistics, service, agriculture and manufacturing [12, 13, 14, 15] . This provides high reliability and throughput, and human effort can be justified and amortized over years

FAQs

What are “Variational Automation (VA)” tasks and why are they difficult for robots?

VA tasks require persistent execution across varying object geometry and pose rather than repeating fixed motions. This variability creates a reliability gap that model-free policies often struggle to close for commercial and industrial use.

What is Graph-as-Policy (GaP) and what does it generate?

GaP is a multi-agent coding harness that generates directed computation graphs containing perception, planning, and control nodes derived from a Modular Open Robot Skill Library (MORSL).

How does GaP improve the computation graph after generating it?

GaP builds an internal simulation environment to rehearse task instances with different graphs in parallel. It then iteratively refines graph structure and parameters to improve success rates and throughput.

Jul 16, 2026

Upload by Aria