[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83112-en":3,"doc-seo-83112-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83112,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation","Evaluating end-to-end autonomous driving (E2E-AD) remains difficult because existing simulators often force a trade-off between closed-loop interactivity and real-world visual fidelity. Point as Skeleton introduces a generative sensor simulation framework that produces driving video observations via a state-updated autoregressive generator using ego states, actor states, scene maps, and point-cloud skeleton conditions. Resetand-Roll adapts rolling diffusion for simulation rollouts, while point-cloud skeletons stabilize stepwise error accumulation. A nuPlan-based renderer-level closed-loop interface enables evaluation under ego deviations from logs, validated on nuScenes and nuPlan.","arXiv :2607 .065 16v 1 [ cs .CV] 7 Jul 2026  \nPoint as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation  \nSongbur Wong 1 , Xiaosong Jia3✉ , Junqi You 1 , Bo Zhang4 , Pei Xu2 Renqiu Xia 1 , Yuping Qiu2 , Shaofeng Zhang5 , Zelin Zhao6 , Xuechao Yan2 Yuchen Zhou3 , Yurui Chen2 , Wen Guo2 , Hang Xu2 , Junchi Yan 1✉  \n1 Shanghai Jiao Tong University 2Yinwang Intelligent Technology Co., Ltd.  \n3Fudan University 4 Shanghai Artificial Intelligence Laboratory  \n5University of Science and Technology of China 6 Georgia Institute of Technology  \n✉ Corresponding authors  \nAbstract  \nEvaluating end-to-end autonomous driving (E2E-AD) remains challenging, as existing driving simulation methods often trade off closed-loop interactivity (e.g., CARLA) and real-world visual fidelity (e.g., nuScenes) . We present Point as Skeleton, a generative sensor simulation framework for state-updated autoregressive driving video generation, in which an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and pointcloud skeleton conditions. To support closed-loop rollout, we introduce Resetand-Roll, which adapts rolling diffusion inference to simulation by preventing future-conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step-wise autoregressive rollout, we introduce point-cloud skeletons that decouple foreground and background assets and project them into camera-view painted-point and template-depth conditions, providing appearance and geometric cues. We further implement a nuPlan-based renderer-level closed-loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that Point as Skeleton improves autoregressive generation quality during closed-loop rollout, demonstrating its potential for visually faithful closed-loop driving simulation. The code is available at [https://github.com/krauwu/point-as-skeleton](https://github.com/krauwu/point-as-skeleton).  \n1 Introduction  \nEnd-to-end autonomous driving (E2E-AD) [1, 2, 3, 4, 5] has attracted increasing attention in recent years. A central challenge in evaluating E2E-AD systems is to provide sensor observations that respond to the actions selected by the driving policy. Open-loop benchmarks such as nuScenes [6] provide realistic sensor data, but the observations are fixed by the recorded trajectory and cannot react to policy-induced actions. In contrast, closed-loop simulators such as CARLA [7, 8, 9] provide interactive feedback, but their rendered sensor appearance can still differ from real-world camera data. This motivates generative sensor simulation, where a learned generative model synthesizes camera observations from simulator-provided ego states, actor states, and scene conditions. However, closed-loop simulation goes beyond standard driving-scene reconstruction [10] and generation [11, 12] . The ego vehicle may deviate from the logged trajectory, while the planner and traffic simulator update online. The generator therefore needs to render multi-view observations under off-log motions and repeatedly refreshed conditions. This introduces two additional requirements:(I) Spatiotemporal Extrapolation. Closed-loop policies may deviate from the logged motion in speed or heading, exposing vehicles and pedestrians that are not observed in the recorded images.  \nPreprint.  \n|  | Multi-view Image\u003Cbr>Point Cloud & 3D Bbox\u003Cbr>Agent Track ID | Paste\u003Cbr>Simulator: Transportation Flows Projection Matrices |\n| --- | --- | --- |\n| Log Trajectory Simulation Trajectory |  |  |\n\nGenerator  \nPoint Cloud Skeleton  \nControlling Transportation Simulator  \nCamera Control Generation  \nE2E-AD planner  \nFigure 1: Point as Skeleton Generative Simulator. We utilize offline logs to build a foregroundbackground decoupled Point Cloud Skeleton. At each time step, the ev","cbCaitonzLejZ28Y","https://ap.wps.com/l/cbCaitonzLejZ28Y","pdf",6873703,1,15,"English","en",105,"# Introduction\n## Spatiotemporal Extrapolation\n## Instantaneous Interactivity\n# Point as Skeleton Generative Simulator","[{\"question\":\"Why is closed-loop autonomous driving evaluation harder than open-loop benchmarks?\",\"answer\":\"Open-loop benchmarks use fixed recorded sensor observations tied to the logged trajectory, so they cannot react to policy actions. Closed-loop simulation is interactive, but its rendered sensor appearance may still differ from real-world camera data, motivating generative sensor simulation.\"},{\"question\":\"What is the core idea of Point as Skeleton?\",\"answer\":\"Point as Skeleton uses a generative sensor simulation framework where an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. Offline accumulated point-cloud assets provide stabilizing geometric cues for rollout.\"},{\"question\":\"How does Resetand-Roll support closed-loop rollout with rolling diffusion?\",\"answer\":\"Resetand-Roll adapts rolling diffusion inference to the simulation interface by preventing future-conditioned latent states from being carried across simulation steps. This helps align generation with step-wise condition updates during closed-loop execution.\"}]",1784185343,38,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"point-as-skeleton-accumulated-point-cloud-enhanced-autoregressive-generation-for-closed-loop-autonomous-driving-simulation","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/point-as-skeleton-accumulated-point-cloud-enhanced-autoregressive-generation-for-closed-loop-autonomous-driving-simulation/83112/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why is closed-loop autonomous driving evaluation harder than open-loop benchmarks?","Question",{"text":74,"@type":75},"Open-loop benchmarks use fixed recorded sensor observations tied to the logged trajectory, so they cannot react to policy actions. Closed-loop simulation is interactive, but its rendered sensor appearance may still differ from real-world camera data, motivating generative sensor simulation.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What is the core idea of Point as Skeleton?",{"text":79,"@type":75},"Point as Skeleton uses a generative sensor simulation framework where an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. Offline accumulated point-cloud assets provide stabilizing geometric cues for rollout.",{"name":81,"@type":72,"acceptedAnswer":82},"How does Resetand-Roll support closed-loop rollout with rolling diffusion?",{"text":83,"@type":75},"Resetand-Roll adapts rolling diffusion inference to the simulation interface by preventing future-conditioned latent states from being carried across simulation steps. This helps align generation with step-wise condition updates during closed-loop execution.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]