HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models

Sep 21, 2026

Uploaded by Ivy

World-action (WA) models generate long-horizon action chunks for general-purpose robotic manipulation, yet they often fail in real-world precision tasks due to calibration, perception, and contact-dynamics errors that accumulate in the final millimeters. HALO-WA proposes a hybrid-attention latent-guided online reinforcement learning framework that uses latent features and action priors from WA generation via a lightweight actor-critic adapter for fast adaptation. Experiments on four real-world tasks raise success from 26.4% to 87.1%.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Introduction

Online reinforcement learning for precision control

Need for WA-native adaptation

arXiv :2607 .04265v 1 [ cs .RO] 5 Jul 2026 HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models Angen Ye 1 ,2 ,∗ Weijie Ke3 ,∗ Xiaofeng Wang3 ,4 Xinze Chen3 Chaojun Ni3 Guosheng Zhao3 Boyuan Wang 1 ,2 Zheng Zhu3 ,† Junjie Xie 1 ,2 Dapeng Zhang 1 ,2 1Institute of Automation, Chinese Academy of Sciences, Beijing, China 2 School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China 3 GigaAI, Beijing, China 4Tsinghua University, Beijing, China ∗Equal contribution †Corresponding author: [zhengzhu@ieee.org](zhengzhu@ieee.org) Abstract: World-action (WA) models can generate long-horizon action chunks for general-purpose robotic manipulation, but they remain vulnerable to calibration, perception, and contact-dynamics errors in real-world precision tasks, often failing in the final few millimeters of alignment or insertion. We propose HALO-WA, a hybrid-attention latent-guided online reinforcement learning (RL) framework for WA models, which leverages latent features and action priors from the WA generation process through a lightweight actor-critic adapter to enable fast online adaptation to real deployment errors. HALO-WA introduces a hybridattention structure that preserves the temporal consistency of action chunks while reading task-relevant information from WA latents conditioned on visual context and end-stage correction requirements, thereby producing refined action chunks. We validate HALO-WA on four real-world precision manipulation tasks, where it improves the average success rate from 26.4% for WA-base to 87.1%, outperforming the strongest baseline by 19.2 percentage points while requiring only 45–75 minutes of online training per task. To facilitate reproducibility, we further conduct supplementary simulation experiments in RoboTwin and release the code at [https://github.com/YeanRoot/HALO-WA](https://github.com/YeanRoot/HALO-WA). Keywords: World-Action Models, Online Reinforcement Learning, Precision Manipulation 1 Introduction World-action (WA) models provide a promising route toward general-purpose robotic manipulation. They can generate long-horizon action chunks conditioned on visual observations, robot states, and task specifications[1][2][3][4][5] . However, deploying such models on real robots remains challenging. In precision manipulation, small errors in calibration, perception, control, or contact dynamics can be amplified during the final execution stage. As a result, even if a WA model generates a seemingly reasonable action sequence, it may still fail in the last few millimeters of insertion, alignment, or contact. This precision deployment gap is difficult to eliminate through offline training alone, because the most critical errors are often tied to a specific robot, a specific environment, and real physical interaction. Online reinforcement learning offers a potential solution to this problem[6][7][8] . By practicing in the target environment, a robot can learn from real successes, failures, and human intervention feedback, with a particular focus on improving the contact-rich stages where offline policies are most fragile. However, directly applying online reinforcement learning to a large-scale WA model is impractical: full-parameter updates are computationally expensive, slow to adapt, and prone to overfitting under limited real-robot interaction data. Training a small policy from scratch is also undesirable, because it discards the visual, temporal, and behavioral priors already encoded in the WA model. Therefore, the key question is: how can we enable fast online reinforcement learning for precision control while preserving the general capability of a pretrained WA model? Recent studies have shown that frozen robotic foundation models can be combined with lightweight online reinforcement learning to improve real-world deployment performance. GR-RL[9] shows that online RL can specialize a general visi

FAQs

Why do world-action (WA) models struggle with precision tasks in real robots?

Small calibration, perception, control, and contact-dynamics errors can be amplified during the last execution stage, causing failures in insertion, alignment, or contact despite reasonable action sequences.

How does HALO-WA enable fast online reinforcement learning for WA models?

HALO-WA freezes the pretrained WA backbone and trains a lightweight actor-critic adapter, guided by WA reference action chunks and latent features while using the current robot state as supplementary context.

What improvements does HALO-WA achieve in real-world experiments?

HALO-WA increases the average success rate from 26.4% (WA-base) to 87.1%, outperforming the strongest baseline by 19.2 percentage points with 45–75 minutes of online training per task.

Sep 21, 2026

Uploaded by Ivy