World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning

Jul 16, 2026

Upload by Rowan

Robust motion planning in dense traffic requires autonomous vehicles to handle rare, safety-critical interactions that are underrepresented in naturalistic driving logs. Existing adversarial training approaches often depend on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which limits integration with modern autoregressive planners. The work formulates adversarially robust planner learning as a constrained min-max game and introduces Adversarial World Modeling (AWM), a multi-agent self-play fine-tuning framework with a decoupled solver. Experiments on nuPlan and InterPlan show transferable adversarial interactions and competitive closed-loop performance across nominal and long-tail scenarios, supported by theoretical analysis.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages36

Reading Time1 h 31 min

Abstract

Introduction

From nominal training to rare safety-critical interactions

Limitations of existing adversarial training

Endogenous environment adversaries via world models

Optimization challenges in multi-agent min-max training

Method: Adversarial World Modeling (AWM)

arXiv :2607 . 10630v1 [ cs .RO] 12 Jul 2026 World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning Tong Nie,1,2,† Yuewen Mei,2,† Junlin He,1 Yihong Tang,3,4 Jian Sun,2,B Wei Ma1,B 1The Hong Kong Polytechnic University, 2Tongji University, 3McGill University, 4Mila-Quebec AI Institute [tong.nie@connect.polyu.hk](tong.nie@connect.polyu.hk) [wei.w.ma@polyu.edu.hk](wei.w.ma@polyu.edu.hk) † Equal contribution. B Corresponding authors. Abstract Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM) , a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner’s predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components. 1 Introduction Developing robust motion planners for closed-loop autonomous driving remains a central open problem [44, 23] . Recently, the leading paradigm has shifted from rule-based systems toward generative motion modeling [38, 33, 46, 53, 49] . Drawing inspiration from Large Language Models (LLMs), modern discrete architectures [46, 54, 43, 52, 48] treat map elements and motion primitivesas discrete tokens, exhibiting remarkable performance when trained on large-scale human driving logs. However, these autoregressive planners are primarily optimized against nominal, naturalistic data distributions. Consequently, they remain brittle on interactive long-tail cases that are rare but safety-critical, such as aggressive cut-ins, coordinated blocking, and collision-inducing maneuvers from surrounding vehicles [11, 25] . To expose and rectify these vulnerabilities, adversarial training offers a compelling solution by actively searching hard scenarios to robustify the planner [15] . Despite the promise of adversarial training, existing frameworks are not well aligned to modern autoregressive architecture. Many existing adversarial frameworks rely on external adversary agents, heuristic scenario perturbations, or simulator-intensive rollouts [51, 29, 26, 40, 30] . These attacks are often designed around low-dimensional control policies, fixed objects of interest, or handcrafted rules. As a result, they are infeasible to scale to the high-dimensional tokenized generation paradigm. Preprint. Rather than forcing incompatible external attackers into this paradigm, we argue that an endogenous solution lies in the environment simulator itself. We observe that behavioral World Models [46, 52], which inherently learn multi-agent traffic evolution as sequential token predictions, are naturally suited to bridge this gap. Operating on the same autoregressive architecture, motion vocabulary, and scene representation as the ego planner, they can be repur

FAQs

What problem does the paper address in motion planning?

The paper targets brittleness in autonomous driving when planners face rare but safety-critical interactive scenarios that are underrepresented in naturalistic data.

Why are existing adversarial training methods difficult to use with autoregressive planners?

They often rely on external adversary agents, heuristic perturbations, or simulator-intensive rollouts and are designed for low-dimensional control or handcrafted attack rules, making them hard to scale to tokenized autoregressive generation.

What is Adversarial World Modeling (AWM) and how does it work?

AWM casts robust planner learning as a constrained min-max game and uses a decoupled solver. It converts a predictive world model into a role-conditioned adversary for inner minimization and trains the ego planner with regret-aware robust best responses in the outer maximization.

Jul 16, 2026

Upload by Rowan