From World Models to World Action Models: A Concise Tutorial for Robotics

Jul 16, 2026

Upload by Anda

World models are widely used in embodied intelligence and generative simulation, but their scope varies across research communities. This tutorial reframes world models as action-conditioned predictive models that estimate future task-relevant observations or states. It organizes existing approaches into observation-space and state-space world models, contrasting visual fidelity, spatial structure, physical interpretability, and control usability. It then introduces world action models that link predicted futures to executable robot actions via four representative paradigms for embodied prediction and control.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages10

Reading Time25 min

Architectural View of World (Action) Models in Robotics

World

Embodied AI Task and Policy

World Models and World Action Models

A Design-Space View of World Models

Observation-space World Models

From World Models to World Action Models: A Concise Tutorial for Robotics Xiaoxiong Zhang 1 , Xiong Zeng 1 , and Wei Zhang 1 2 arXiv :2607 .00836v 3 [ cs .RO] 6 Jul 2026 Abstract World models are increasingly used in embodied intelligence and generative simulation, yet their scope remains ambiguous across communities. This tutorial presents a design-space view of world models as action-conditioned predictive models that estimate the future evolution of taskrelevant observations or states. We categorize existing methods into observation-space and statespace world models, comparing their trade-offsin visual fidelity, spatial structure, physical interpretability, and control usability. We further introduce world action models, which connect predicted futures with executable robot actions, and summarize four representative paradigms: imagine-then-execute, video-feature-conditioned action prediction, joint video-action modeling, and auxiliary video prediction for policy learning. The goal of this tutorial is to clarify the conceptual scope of world (action) models and provide a structured taxonomy for embodied prediction and control. Project page: World (Action) Models. 1. Architectural View of World (Action) Models in Robotics Predicting how the world evolves under actions is a central capability for embodied artificial intelligence (AI) and generative simulation. In these scenarios, the predictive models can support controller design, planning, decision making, and policy learning. Recently, the term world (action) model has been used to describe a broad range of methods, including latent dynamics models, video prediction models, physics-informed simulators, and action-conditioned generative models. However, these methods differ substantially in what they model, what they predict, and how their predictions are used. This motivates a unified definition and 1 School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China 2LimX Dynamics, Shenzhen, China. Correspondence to: Xiaoxiong Zhang <[12433017@mail.sustech.edu.cn](12433017@mail.sustech.edu.cn) >. taxonomy of world (action) models. Figure 1 . Illustration of the components of a world. 1.1. World We define a world as the set of task-relevant entities, including both the robot and its environment. The environment contains the objects of interest and the ambient environment, as shown in Fig. 1. 1.2. Embodied AI Task and Policy We define the world configuration as the configuration of both the robot and all objects inside the environment. An embodied AI task is to design a policy to transform the initial world configuration into a target configuration by controlling the robot. Different embodied AI tasks correspond to different instantiations of the world. Fig. 2 provides two examples of task-specific worlds. In humanoid locomotion on flat ground, the world consists of a humanoid robot and a ground surface, and the goal is to induce stable and feasible motion configurations. In robotic table-cleaning tasks after dinner, the world includes the robot and all relevant objects such as dishes, a table, anda household environment, and the objective is to rearrange objects into acceptable configurations (e.g., dishes placed ina sink) . Fig. 3 illustrates that, in an embodied AI task, the policy receives a language instruction l and the current observation Figure 2. Embodied AI tasks: Humanoid locomotion and robotic table-cleaning. Figure 3 . A language-conditioned closed-loop policy framework. ot from the world, and then outputs an action at to the robot. The policy might be a proportional-integral-derivative (PID) controller, a model predictive controller (MPC), a visionlanguage-action (VLA) model, or a world action model (WAM) . 1.3. World Models and World Action Models For a specified world, a world model is a model to predict how its future observation ot+1 or state xt+1 evolves under act

FAQs

What does the tutorial define as a “world model” in robotics?

A world model is an action-conditioned predictive model that estimates how future task-relevant observations or states evolve under a given action, typically conditioned on the current observation.

How are observation-space and state-space world models distinguished?

Observation-space world models predict future observations directly in the observation space (predicting ot+1), while state-space world models predict future task-relevant states (predicting xt+1).

What is a world action model, and how does it relate predicted futures to robot actions?

A world action model is a policy that extends world models by explicitly associating predicted future observations or states with actions, enabling executable control. The tutorial summarizes four paradigms such as imagine-then-execute and action prediction conditioned on video features.

Jul 16, 2026

Upload by Anda