WARP-RM A Warp-Augmented Relative Progress Reward Model for Data Curation

Jul 16, 2026

Scaling imitation learning depends on large datasets, but human teleoperation produces mixed-quality demonstrations with hesitations and recoveries. WARP-RM addresses progress-reward supervision noise and costly subtask labeling by learning dense, signed relative progress fully self-supervised from successful demonstrations. Time-warp augmentations (variable playback speeds and reversals) generate per-frame progress targets, while WARP-BC uses predicted scalar reward estimates to upweight high-advantage action chunks during behavior cloning. Experiments on robotic T-shirt folding and bottle-in-bin show robustness to suboptimal data, higher success and throughput, and released evaluation artifacts.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages24

Reading Time1 h 0 min

Introduction

Method

WARP-RM

WARP-BC

Experiments

Data curation via episode-quality proxies

Real-world and simulation evaluation

Contributions

arXiv :2606 .28320v3 [ cs .RO] 10 Jul 2026 WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation Justin Yu1 3 * † Andrew Goldberg1 * Kavish Kondap1 * Karim El-Refai1 * Ethan Ransing1 Qianzhong Chen2 Mac Schwager2 Fred Shentu3 Philipp Wu3 Ken Goldberg1 1University of California, Berkeley 2 Stanford University 3XDOF * Equal contribution. †Corresponding author: [yujustin@berkeley.edu](yujustin@berkeley.edu) Figure 1: WARP-RM signed progress measure vˆt on an unseen mixed-quality teleoperated T-shirt-folding demonstration. Large negative magnitudes occur when the right gripper drops the shirt in (b), and near-zero magnitude during stagnation between (g) and (h) . These values are used to filter and weight downstream policy training. Predictions on more examples in Appendix G. Abstract: Scaling imitation learning requires large datasets, yet human teleoperation inevitably produces mixed-quality demonstrations containing hesitationsand recoveries. Prior frame-level progress reward models supervise on absolute temporal progress proxies that suffer from label noise, or require costly human annotations to define subtask boundaries. We present WARP (Warp-Augmented Relative Progress), a novel fully self-supervised algorithm for learning dense, signed relative progress magnitudes directly from successful demonstrations. WARP generates per-frame progress targets via time-warp augmentations of demonstrations (variable playback speeds and reversals) and we train WARP-RM to predict the normalized elapsed time between input frames. Aggregating these predictions across overlapping windows yields a dense frame-level progress signal. We then introduce WARP-BC, which leverages these scalar reward estimates to upweight high-advantage action chunks during behavior cloning, where chunk-level advantage is obtained by aggregating per-frame rewards. We evaluate our approach on a physical bimanual robot system performing a long-horizon deformable object manipulation task: folding T-shirts from a random crumpled start. To evaluate policy robustness against suboptimal data, we construct training datasets of varying quality using episode length as a proxy for teleoperation sub-optimality. As the dataset is widened to admit more inefficiencies, WARP-BC maintains a 19/20 success rate compared to vanilla BC’s collapse to 2/20, improving throughput by up to ∼ 18× . Furthermore, we evaluate a bottle-in-bin placement task in the real-world, as well as in a reproducible simulation of the task, where gains in success, speed, and throughput hold under paired significance tests, and we release all simulation code and evaluation artifacts. Project page: [https://uynitsuj.github.io/warp-rm/](https://uynitsuj.github.io/warp-rm/) . Keywords: Robot Imitation Learning, Data Curation, Reward Models 1 Introduction Imitation learning has emerged as a powerful framework for long-horizon robot manipulation, enabling complex visuomotor behaviors from human teleoperated demonstrations. Recent advancesin policy modeling and large-scale pretraining [1, 2, 3, 4, 5, 6, 7, 8, 9] have significantly improved expressivity and generalization. However, these methods remain highly sensitive to demonstration quality, particularly in long-horizon manipulation settings, as policies learn to mimic suboptimal pauses and fumbles present in human teleoperation [10, 11, 12] . If trained on, these behaviors can derail policy performance. However, suboptimal sequences often contain valuable recovery behaviors, akin to DAgger data [13, 14, 15, 16, 17] . Some approaches to data curation operate at the trajectory level, discarding entire episodes that fall below a quality threshold [18, 19, 20] . However, this coarse filtering may suffer from two limitations: it discards valuable, high-advantage segments embedded within otherwise suboptimal executions, while simultaneously failing to prune localized hesitations or fumbles within retained demonstrations. To addr

FAQs

What problem does WARP-RM solve in imitation learning data curation?

WARP-RM learns progress rewards despite mixed-quality teleoperated demonstrations, avoiding label noise from absolute progress proxies and reducing reliance on expensive human annotations for subtask boundaries.

How does WARP generate training targets for relative progress?

WARP applies time-warp augmentations to successful demonstrations by replaying them with non-uniform velocities, including reversals, then trains a model to predict the normalized elapsed time between input frames.

How does WARP-BC use WARP-RM outputs during behavior cloning?

WARP-BC leverages scalar reward estimates to upweight high-advantage action chunks, where chunk-level advantage is computed by aggregating per-frame rewards during behavior cloning.

Jul 16, 2026

Upload by Ava Thompson