[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84671-en":3,"doc-seo-84671-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84671,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning","Dense video captioning generates temporally grounded descriptions for multiple events in untrimmed videos, enabling event-level understanding and video generation. Autoregressive video LLMs are widely used for their generative and cross-modal modeling, yet token-by-token decoding sharply limits inference efficiency and scalability as video length and event density grow. This work introduces a parallelized autoregressive framework that restructures causal dependencies using learned event-level planning and event-factorized decoding, enabling lossless parallel generation for tokens with weak cross-event links while preserving sequential decoding within events.","arXiv :2607 .02963v 1 [ cs .CV] 3 Jul 2026  \nParallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning  \nWenzheng Zeng, Siyi Jiao, Chen Gao, Hwee Tou Ng†, Mike Zheng Shou† National University of Singapore, Singapore  \nAbstract. Dense video captioning aims to generate temporally grounded descriptions of video events, benefiting both event-level video understanding and generation. In this domain, autoregressive video large language models have emerged as a prevalent paradigm due to their strong generative and cross-modal modeling capacity. However, generating dense captions under the token-by-token paradigm severely limits inference efficiency and hinders scalability as video length and event density increase.  \nIn this work, we propose a parallelized autoregressive framework that not only improves generation efficiency but also enhances temporally grounded captioning performance. Our key insight is to exploit the weak local dependencies across temporally distinct events to restructure the causal dependency graph, thereby enabling lossless parallel generation.  \nSpecifically, tokens with weak cross-event dependencies can be decoded in parallel, while tightly coupled tokens within each event retain sequential decoding to preserve local semantic coherence. To realize this insight, we introduce two key components for lossless parallel decoding: (1) a latent global planning mechanism that automatically learns the event-level structure and produces compact tokens encoding global inter-event causality while adaptively aggregating event-level audio-visual semantics, guiding subsequent dependency restructuring and parallel decoding; and (2) an event-factorized parallel decoding mechanism that effectively balances local focus with global inter-event awareness. Experiments on various benchmarks demonstrate the clear advantage of our approach in both efficiency and performance in omni-modal event grounding and captioning.  \nProject website: [https://github.com/showlab/PadCaptioner](https://github.com/showlab/PadCaptioner).  \nKeywords: Dense video captioning · Parallel decoding · Latent planning  \n1 Introduction  \nDense Video Captioning (DVC) addresses the problem of describing multiple temporally localized events in untrimmed long videos [13, 24 , 75] . Unlike conventional video captioning, which typically produces a single global description, DVC requires jointly localizing densely occurring, semantically meaningful events and generating a temporally grounded caption for each event. This makes DVC  \n†Corresponding authors.  \n2 W. Zeng et al.  \nPerformance  \n\n|  |  |\n| --- | --- |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n| +6.7% |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n\nEvent grounding (F1 ↑)  \n\n|  |  |\n| --- | --- |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n| +6.1% |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n\nEvent captioning (Sim ↑)  \nEfficiency  \n\n|  |  |\n| --- | --- |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  | 3.8× |\n|  |  |\n|  | speed up |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n|  |  |\n\n|  |\n| --- |\n|  |\n|  |\n|  |\n|  |\n|  |\n| 3.1× |\n|  |\n| speed up |\n|  |\n|  |\n|  |\n|  |\n|  |\n|  |\n|  |\n|  |\n\nDecoding time/video (ms ↓) Decoding time/token (ms ↓)  \n Prev. SOTA  Ours  \nFig. 1: Left: Comparison of token dependency modeling and decoding strategies between existing Video-LLM-based captioning models and ours. We achieve lossless parallel autoregressive decoding under a model-inferred restructured dependency graph. Right: Our method surpasses the previous SOTA [8] in both grounded captioning accuracy and decoding efficiency on the LongVALE benchmark [13] .  \nsubstantially more challenging, yet highly valuable for applications such as egocentric perception [5, 14 , 15], video narration [6, 7 , 63], embodied agents [4, 11], video search and indexing [64], and multi-event video generation [59, 60] .  \nTo tackle DVC, autoregressive","cbCailji9UNVnQul","https://ap.wps.com/l/cbCailji9UNVnQul","pdf",3895533,2,1,31,"English","en",105,"# Introduction\n# Performance","[{\"question\":\"What problem does dense video captioning address?\",\"answer\":\"Dense video captioning describes multiple temporally localized events in untrimmed long videos, generating a grounded caption for each event rather than a single global description.\"},{\"question\":\"Why is token-by-token autoregressive decoding inefficient for dense video captioning?\",\"answer\":\"Sequential decoding incurs substantial latency, and DVC requires multi-token descriptions for many events in long-form videos, so decoding steps grow rapidly with video length and event density.\"},{\"question\":\"How does PadCaptioner enable lossless parallel generation?\",\"answer\":\"PadCaptioner learns an event-level latent planning structure to capture global inter-event causality and uses event-factorized parallel decoding: tokens with weak cross-event dependencies are decoded in parallel, while tightly coupled tokens within each event remain sequential to preserve local semantic coherence.\"}]",1784197583,78,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"parallelized-autoregressive-decoding-for-omni-modal-dense-video-captioning","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/parallelized-autoregressive-decoding-for-omni-modal-dense-video-captioning/84671/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-20","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does dense video captioning address?","Question",{"text":75,"@type":76},"Dense video captioning describes multiple temporally localized events in untrimmed long videos, generating a grounded caption for each event rather than a single global description.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why is token-by-token autoregressive decoding inefficient for dense video captioning?",{"text":80,"@type":76},"Sequential decoding incurs substantial latency, and DVC requires multi-token descriptions for many events in long-form videos, so decoding steps grow rapidly with video length and event density.",{"name":82,"@type":73,"acceptedAnswer":83},"How does PadCaptioner enable lossless parallel generation?",{"text":84,"@type":76},"PadCaptioner learns an event-level latent planning structure to capture global inter-event causality and uses event-factorized parallel decoding: tokens with weak cross-event dependencies are decoded in parallel, while tightly coupled tokens within each event remain sequential to preserve local semantic coherence.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]