[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83906-en":3,"doc-seo-83906-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},83906,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","UNIVERSE Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation","World Action Models (WAMs) aim to improve action generalization in autonomous driving by using future video prediction as dense supervision for scene dynamics and temporal causality. However, existing cascaded or dual-DiT architectures separate video imagination from action prediction, weakening transfer of world-dynamics knowledge to trajectory generation. UNIVERSE proposes a unified single mask-modulated Diffusion Transformer with co-training over shared generative parameters. A Modality-Decoupling Visibility Mask preserves historical context while blocking mutual attention to prevent future leakage, enabling efficient trajectory-only inference.","arXiv :2607 .05 133v 1 [ cs .CV] 6 Jul 2026  \nUNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation  \nMengmeng Liu1 Diankun Zhang2 Jiuming Liu3 Jianfeng Cui2 Hongwei Xie2 Guang Chen2 Hangjun Ye2 Francesco Nex 1 Hao Cheng 1 Michael Ying Yang4,*  \n1University of Twente 2Xiaomi EV  \n3University of Cambridge 4University of Bath  \n* Corresponding author.  \nAbstract  \nWorld Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving by using future video prediction as dense supervision for scene dynamics and temporal causality. However, it remains unclear which architecture better transfers video-modeling benefits to trajectory generation.  \nExisting cascaded or dual-DiT designs separate video imagination from action prediction, weakening the transfer of video-learned world dynamics to the trajectory branch: the action model may still overfit dataset-specific driving priors, while the video model only indirectly regularizes planning. We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Diffusion Transformer.  \nBy co-training future video latents and ego-trajectory tokens within shared generative parameters, UNIVERSE allows dense video supervision to directly shape trajectory denoising, leading to stronger cross-domain action generalization. To ensure causal validity and efficient deployment, we introduce a Modality-Decoupling Visibility Mask, which shares historical context across modalities while blocking mutual attention between future video and trajectory tokens. This prevents futuretarget leakage and enables trajectory-only inference by removing future-video denoising at test time, achieving a 4.3 × speedup over joint video-action rollout while maintaining comparable planning accuracy. The same model also supports video-only and joint video-action rollouts. Experiments show that UNIVERSE achieves 91 .0 PDMS on NAVSIM (vs. 89.6 for the Two-DiT variant), and demonstrates strong zero-shot transfer to nuScenes and Bench2Drive without fine-tuning, while ablations confirm the importance of single-DiT unification, video co-training, and mask-based modality decoupling.  \n1 Introduction  \nLearning generalizable action policies is a central challenge in autonomous driving, where a planner trained on one dataset must safely transfer to unseen cities, traffic patterns, sensor distributions, and simulation environments. Standard end-to-end planners directly map historical observations to future ego trajectories, which is efficient but can overfit dataset-specific driving priors such as common trajectory patterns, road layouts, or benchmark-specific behaviors [1, 2, 3] . World Action Models (WAMs) offer a promising alternative by coupling future prediction with action generation [4] . Compared with sparse trajectory supervision, future world modeling provides dense signals about  \nPreprint.  \nscene geometry, object motion, traffic-agent interactions, and temporal causality, helping the model learn both how the world evolves and how the ego vehicle should act [5, 6, 7, 8] . Recent driving world models and embodied video-action models further show that future visual prediction can improve planning, simulation, transferability, and zero-shot policy generalization [9, 4, 10, 11, 12, 13] . However, it remains unclear which architecture best transfers video-learned world-dynamics priorsto trajectory generation. Fast-WAM [14] shows that future video prediction can be more useful as training-time world-dynamics supervision than as mandatory test-time imagination, since bypassing video rollout significantly accelerates action inference. This raises a complementary training-side question: if future videos and ego trajectories describe the same future driving state, should they bedenoised by separate modules, or by a shared generative backbone? As shown in Fig. 1, existing WAMs often use cascaded or dual-DiT designs that","cbCaij0XWyJzmMQW","https://ap.wps.com/l/cbCaij0XWyJzmMQW","pdf",3569867,3,1,18,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What problem does UNIVERSE address in video-based action modeling for autonomous driving?\",\"answer\":\"UNIVERSE targets the unclear transfer effectiveness between video-model architectures and trajectory generation, especially when prior designs separate video imagination from action prediction.\"},{\"question\":\"How does UNIVERSE achieve stronger action generalization compared with cascaded or dual-DiT designs?\",\"answer\":\"It uses a unified single mask-modulated Diffusion Transformer, co-training future video latents and ego-trajectory tokens within shared generative parameters so dense video supervision directly shapes trajectory denoising.\"},{\"question\":\"How does UNIVERSE enable efficient trajectory-only inference without future-video leakage?\",\"answer\":\"It introduces a Modality-Decoupling Visibility Mask that shares historical context across modalities while blocking mutual attention between future video and trajectory tokens, allowing removal of future-video denoising at test time.\"}]",1784191360,45,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"universe-unified-video-action-models-for-autonomous-driving-with-flexible-mask-modulated-modality-generation","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/universe-unified-video-action-models-for-autonomous-driving-with-flexible-mask-modulated-modality-generation/83906/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-26","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does UNIVERSE address in video-based action modeling for autonomous driving?","Question",{"text":75,"@type":76},"UNIVERSE targets the unclear transfer effectiveness between video-model architectures and trajectory generation, especially when prior designs separate video imagination from action prediction.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does UNIVERSE achieve stronger action generalization compared with cascaded or dual-DiT designs?",{"text":80,"@type":76},"It uses a unified single mask-modulated Diffusion Transformer, co-training future video latents and ego-trajectory tokens within shared generative parameters so dense video supervision directly shapes trajectory denoising.",{"name":82,"@type":73,"acceptedAnswer":83},"How does UNIVERSE enable efficient trajectory-only inference without future-video leakage?",{"text":84,"@type":76},"It introduces a Modality-Decoupling Visibility Mask that shares historical context across modalities while blocking mutual attention between future video and trajectory tokens, allowing removal of future-video denoising at test time.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]