FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space

Jul 16, 2026

Upload by Miles

Pretrained generative robot policies using flow matching and diffusion can solve many manipulation tasks, yet real deployments expose failure modes outside the pretraining distribution. FlowDAgger adapts frozen generative robot policies from human interventions without costly large-scale data collection or online reinforcement learning on hardware. The method inverts each expert action into a latent-space noise target via reverse-time integration and local refinement, then trains a lightweight latent policy to steer the base model during deployment. Evaluations in simulation and real bimanual and single-arm settings show superior performance over supervised fine-tuning and latent-space RL baselines, while preserving pretrained skills.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages14

Reading Time35 min

Introduction

Problem setting and limitations of existing adaptation methods

FlowDAgger approach via action inversion in latent space

arXiv :2607 .08877v 1 [ cs .RO] 9 Jul 2026 FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space Michael Murray 1 , Daphne Chen 1,4 , Simran Bagaria 1 , Dean Fortier 1 , Tess Hellebrekers 1 , Galen Mullins 1 , Harshavardhan Gajarla 1 , Oier Mees2,3 , Maya Cakmak4 , and Andrey Kolobov 1 1Microsoft Research 2Microsoft 3ETH Zurich 4University of Washington Abstract: Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real-world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large-scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample-and compute-efficient method for adapting frozen generative robot policies from human interventionsin latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse-time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation, adapting both action-head VLAs and worldaction models from a handful of interventions. FlowDAgger outperforms supervised fine-tuning and latent-space RL baselines and preserves pretrained skills on held-out tasks, offering a practical path for adapting robot foundation models in the real world. Website: [https://microsoft.github.io/FlowDAgger](https://microsoft.github.io/FlowDAgger) [Keywords:](Keywords: Generative Policies)[ Generative Policies](Keywords: Generative Policies), [Online Learning](Online Learning), [Human-in-the-Loop](Human-in-the-Loop), [Robotic](Robotic) Manipulation 1 Introduction Foundation models for robot manipulation have achieved impressive results on a wide range of tasks by leveraging large-scale demonstration datasets and expressive generative architectures. The dominant approach across modern systems is a learned generative process that maps random noise to actions conditioned on observations. Building on the visuomotor diffusion policy [1], this approach underlies recent vision-language-action (VLA) models [2, 3], world-action models (WAM) [4, 5, 6], and large multitask diffusion policies [7, 8] . Trained on large, diverse demonstration corpora, these policies encode broad behavioral priors that transfer surprisingly well across embodiments and tasks. However, in any specific deployment they routinely fail due to unfamiliar objects, novel scene dynamics, embodiment quirks, and long-tail edge cases not covered by the pretraining mixture. Existing approaches for closing these gaps are problematic. Collecting additional demonstrations to cover the state-action space for subsequent offline finetuning is tedious; finetuning large-scale models is computationally expensive; moreover, the resulting policy tends to erode broader skills already present in the base model. Interactive imitation in the action space [9, 10, 11] handles the covariateshift problem, but it, too, works by adapting the entire generative policy, an expensive and unstable update that can corrupt the learned prior; residual methods [12, 13] keep the base frozen and learn Figure 1: An overview of FLOWDAGGER. A pre-trained generative policy πgp producing actions ais deployed with a human operator in the loop. When the operator intervenes, the corrective action a∗ is inverted back to a vector w ∗ in the policy’s latent space. The resulting (s, w∗ ) pairs supervise a lightweight latent-space policy that adapts the policy without modifying the base model’s weights. an additive cor

FAQs

What problem does FlowDAgger address in robot manipulation deployments?

FlowDAgger targets failures that arise when real-world environments differ from the pretraining distribution, including unfamiliar objects, novel scene dynamics, and embodiment-specific edge cases.

How does FlowDAgger convert human corrections into supervision for latent adaptation?

FlowDAgger performs action inversion: each expert corrective action is mapped to the latent noise that would generate it under the frozen base policy using reverse-time integration followed by local refinement.

Why is FlowDAgger practical compared with online reinforcement learning or large-scale data collection?

FlowDAgger avoids extensive new data collection and avoids training on physical hardware; it adapts a lightweight latent policy using only the frozen base policy’s forward passes and the small number of human interventions.

Jul 16, 2026

Upload by Miles