[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83121-en":3,"doc-seo-83121-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},83121,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation","Open-world robotic manipulation requires predicting how a scene’s 3D structure evolves under interaction, not only recognizing visual appearance. RynnWorld-4D argues that synchronized RGB, depth, and optical flow (RGB-DF) yields a physically grounded representation capturing 4D scene dynamics closer to end-effector actions than 2D pixel videos. It introduces a generative diffusion model that jointly produces future RGB, depth, and optical flow from a single RGB-D image and a language instruction, using a tri-branch cross-modal architecture. A large Rynn4DDataset 1.0 with 254.4M frames supports training, and RynnWorld-4D-Policy enables closed-loop action via inverse dynamics, achieving state-of-the-art dexterous bimanual manipulation.","arXiv :2607 .06559v 1 [ cs .RO] 7 Jul 2026  \nRynnWorld-4D: 4D Embodied World Models for Robotic Manipulation  \nHaoyu Zhao∗1 ,2 ,3 , Xingyue Zhao∗1 , Siteng Huang†1 ,4 , Xin Li 1 ,4 , Deli Zhao†1 , Zhongyu Li†2 ,3  \n1 DAMO Academy, Alibaba Group, 2 Hong Kong Embodied AI Lab, 3 CUHK, 4 Hupan Lab  \n∗ Equal contribution, †Corresponding author  \nRobotic manipulation in the open world requires not only recognizing what a scene looks like, but also anticipating how its 3D structure moves under interaction. We argue that synchronized RGB, depth, and optical flow (RGB-DF) provide a physically grounded representation that captures the underlying 4D dynamics of a scene. Compared to 2D pixel videos, this multi-modal synergy aligns visual appearance with geometric structure and temporal motion, creating a representation space significantly closer to low-level end-effector actions demanded by robotic systems, narrowing the gap between world prediction and policy learning. Building on this insight, we introduce RynnWorld-4D, a generative model that co-produces future RGB frames, depth maps, and optical flow from a single RGB-D image and a language instruction within one unified diffusion process. This 4D world model features a tri-branch architecture that integrates cross-modal attention with frame-wise 3D RoPE, ensuring that appearance, geometry, and motion evolve consistently. To supply training data at scale, we curate Rynn4DDataset 1.0 , a massive dataset of over 254.4 million frames across egocentric human and robotic manipulation videos with high-quality pseudo-labels for depth and optical flow. We further propose RynnWorld-4D-Policy, an inverse dynamics head that consumes the internal 4D representations of RynnWorld-4D in a single forward pass, bypassing expensive multi-step denoising, to output robot actions in a closed-loop manner. Experiments show that RynnWorld-4D produces temporally and spatially coherent 4D predictions, and that RynnWorld-4D-Policy achieves state-of-the-art performance on real-world dexterous bimanual manipulation tasks, particularly excelling in tasks demanding spatial precision and temporal coordination.  \n [https://alibaba-damo-academy.github.io/RynnWorld-4D.github.io](https://alibaba-damo-academy.github.io/RynnWorld-4D.github.io)  \n [https://github.com/alibaba-damo-academy/RynnWorld-4D](https://github.com/alibaba-damo-academy/RynnWorld-4D)  \n [https://huggingface.co/Alibaba-DAMO-Academy/RynnWorld-4D](https://huggingface.co/Alibaba-DAMO-Academy/RynnWorld-4D)  \n [https://www.modelscope.cn/models/DAMO_Academy/RynnWorld-4D](https://www.modelscope.cn/models/DAMO_Academy/RynnWorld-4D)  \nDate: July 8, 2026   \n1 Introduction  \nRobotic manipulation in the open world could greatly benefit from visual world models that predict how the environment would evolve given an agent’s interactions (Zhao et al. , 2026b, 2025a ; Li et al. , 2026 ; Agarwalet al. , 2025 ; Ali et al. , 2025) . While recent generative video models (Ha and Schmidhuber, 2018a ; Xianget al. , 2024 ; Zheng et al. , 2024 ; Wang et al. , 2025a) have shown encouraging progress in policy synthesis (Duet al. , 2023b ; Liang et al. , 2024 ; Zhen et al. , 2025), data simulation and generation (Zhu et al. , 2024), and long-horizon planning (Du et al. , 2023a ; Li et al. , 2025a), they remain limited by the 2D projective nature of pixels. This inherent limitation leads to a loss of critical spatial relationships, preventing precise 6-DoF pose estimation and depth-aware interaction (Hu et al. , 2024 ; Agarwal et al. , 2025 ; Li et al. , 2026) . Furthermore, 2D models often lack geometric grounding, leading to temporal inconsistencies such as fluctuating object scales and unphysical shape morphing, which hinders their utility in robust policy learning. Consequently, transitioning generative world modeling from 2D videos to geometry-integrated 4D scene evolution is an essential step toward a solid foundation for embodied intelligence.  \nRGB Video  \n\"Move the apple f","cbCaivEIODN5JYqj","https://ap.wps.com/l/cbCaivEIODN5JYqj","pdf",10192290,6,1,26,"English","en",105,"# Introduction\n## 4D world modeling background\n## Limitations of 2D pixel-based video models\n## Projective RGB-DF 4D representation\n## Proposed approach: RynnWorld-4D","[{\"question\":\"为什么仅使用2D像素视频不足以支持开放世界中的机器人操作？\",\"answer\":\"2D像素的投影性质会丢失关键空间关系，难以实现精确的6-DoF位姿与基于深度的交互；同时也容易产生时间不一致，如物体尺度波动或不符合物理的形变，从而降低对鲁棒策略学习的支持。\"},{\"question\":\"RynnWorld-4D如何把场景从2D视频扩展到更具几何约束的4D演化表示？\",\"answer\":\"通过同步预测RGB、深度与光流（RGB-DF）。深度可将像素提升到3D位置，而深度与光流结合在针孔相机假设下可回投形成3D场景流，从而显式刻画外观、几何与运动随时间的一致演化。\"},{\"question\":\"RynnWorld-4D-Policy如何从4D世界模型内部表示生成机器人动作？\",\"answer\":\"RynnWorld-4D-Policy使用逆动力学头，在单次前向计算中读取RynnWorld-4D的内部4D表征，绕过昂贵的多步去噪过程，并以闭环方式输出机器人动作。\"}]",1784185413,66,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"rynnworld-4d-4d-embodied-world-models-for-robotic-manipulation","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/rynnworld-4d-4d-embodied-world-models-for-robotic-manipulation/83121/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-23","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"为什么仅使用2D像素视频不足以支持开放世界中的机器人操作？","Question",{"text":76,"@type":77},"2D像素的投影性质会丢失关键空间关系，难以实现精确的6-DoF位姿与基于深度的交互；同时也容易产生时间不一致，如物体尺度波动或不符合物理的形变，从而降低对鲁棒策略学习的支持。","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"RynnWorld-4D如何把场景从2D视频扩展到更具几何约束的4D演化表示？",{"text":81,"@type":77},"通过同步预测RGB、深度与光流（RGB-DF）。深度可将像素提升到3D位置，而深度与光流结合在针孔相机假设下可回投形成3D场景流，从而显式刻画外观、几何与运动随时间的一致演化。",{"name":83,"@type":74,"acceptedAnswer":84},"RynnWorld-4D-Policy如何从4D世界模型内部表示生成机器人动作？",{"text":85,"@type":77},"RynnWorld-4D-Policy使用逆动力学头，在单次前向计算中读取RynnWorld-4D的内部4D表征，绕过昂贵的多步去噪过程，并以闭环方式输出机器人动作。","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]