[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84591-en":3,"doc-seo-84591-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84591,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Towards Memory Efficient Autoregressive Video Generation via Instance Specific Parametric Absorption","Autoregressive (AR) streaming models enable long video generation, but their key-value (KV) cache grows linearly, causing GPU memory overload and reduced inference throughput. Dropping redundant KV tokens often harms long-range dependencies, producing temporal flickering and identity loss. This paper introduces Instance-Specific Parametric Absorption (ISPA), shifting compression from token discarding to distilling: selected layers absorb historical context into model weights via a warmup monitored transition and a closed-form least-squares weight modulation. Experiments (1.3B–14B) show up to 50% KV cache removal with near-lossless quality.","arXiv :2607 .007 12v 1 [ cs .CV] 1 Jul 2026  \nTowards Memory-Efficient Autoregressive Video Generation via Instance-Specific Parametric  \nAbsorption  \nXiaomeng Fu 1 ,3§ ,∗, Jia Li2§, Yiming Hu3‡, Yong Wang3†,‡, Hayden Kwok-Hay So2†, Jiao Dai 1†, Xiangxiang Chu3 , and Jizhong Han 1  \n1 Institute of Information Engineering, Chinese Academy of Sciences  \n2 The University of Hong Kong  \n3 AMAP, Alibaba Group  \nAbstract. Autoregressive (AR) streaming models have emerged as a powerful paradigm for long video generation. However, the linearly growing Key-Value (KV) cache poses a significant bottleneck, leading to memory overload and degraded inference throughput. A common compression method is to drop redundant KV tokens, which often breaks long-range dependencies, resulting in temporal flickering and identity loss. In this paper, we propose Instance-Specific Parametric Absorption (ISPA), a novel framework that shifts the KV cache compression from discarding to distilling. The core idea is to transit a subset of layers from FullAttention (F-Layers) to memory-efficient Local-Attention (L-Layers) by\"absorbing\" historical context into the model’s weights. Specifically, during a brief warmup phase, ISPA monitors the output discrepancy between global and local attention. At the transition point, we solve a closed-form least-squares problem to compute an instance-specific weight modulation that compensates for the missing history. Experiments across architectures (1.3B to 14B) demonstrate that ISPA can remove up to 50% of the KV cache with near-lossless visual quality. We hope this perspective encourages future work to explore parametric memory consolidation beyond external token-level cache management for streaming generative models.  \nKeywords: Autoregressive Video Generation · KV Cache Compression  \n1 Introduction  \nDiffusion models have become a central paradigm in visual generation, powering diverse tasks [8, 17] . Among these tasks, video generation [2, 27, 31, 38, 39, 45] is particularly challenging, as it requires both high visual fidelity and long-range temporal coherence. Recent works extend diffusion [11, 32] and flow-matching models [7, 21, 24, 26, 30] from image-level generation to video synthesis, typically  \n§ Equal contribution ‡ Project lead † Corresponding author.  \n* This work was done during an internship at AMAP, Alibaba Group.  \n2 X. Fu et al.  \nby generating a fixed-length clip holistically [2, 27, 31, 38, 39, 45] . While effective for short videos, this paradigm requires full spatiotemporal attention across all frames, whose quadratic memory and computation costs make long or infinite video generation prohibitively expensive.  \nTo address this limitation, the Autoregressive (AR) streaming paradigm [3, 13–15, 25, 29, 29, 37, 44, 50] has become a popular solution. By breaking a video into sequential segments, AR models generate frames one by one, using a KeyValue (KV) cache to store historical context. This method theoretically allows for videos of any length and supports real-time generation. However, the streaming process has a major drawback: to keep the video consistent over time, the KV cache must store continually accumulated history. As generation continues, this memory bank fills the GPU memory and significantly slows down the inference speed, losing the \"real-time\" advantage of the system.  \nA common way to reduce this memory pressure is to drop redundant tokens from the KV cache. While token dropping has been effective for Large Language Models (LLMs), directly applying this idea to dynamic video signals is often unstable. The key issue is that, in long video generation, the KV cache is not merely a storage buffer, but an external token memory that carries the model’s accumulated visual history. Existing pruning or eviction strategies manage this memory outside the model by deciding which tokens to keep or discard. Once important historical tokens are removed, long-range dependencies can be broken, causing t","cbCaiavYGpBZqPYF","https://ap.wps.com/l/cbCaiavYGpBZqPYF","pdf",14147344,2,1,19,"English","en",105,"# Introduction\n## Challenges in AR streaming video generation\n## KV cache compression and its limitations\n## ISPA overview: parametric memory consolidation\n# Instance-Specific Parametric Absorption (ISPA)","[{\"question\":\"为什么 AR 流式视频生成会出现显存和推理速度瓶颈？\",\"answer\":\"随着生成持续进行，KV cache 需要持续累积历史上下文，导致显存占用线性增长，从而显著减慢推理速度并削弱实时性优势。\"},{\"question\":\"直接丢弃 KV 缓存中的冗余 tokens 会带来什么问题？\",\"answer\":\"丢弃会破坏长距离依赖，使模型遗忘过去，进而引发背景闪烁、结构扭曲以及身份丢失等质量问题。\"},{\"question\":\"ISPA 如何在不丢弃历史 tokens 的情况下实现更高效的压缩？\",\"answer\":\"ISPA 将部分层从 FullAttention 转换为内存更高效的 Local-Attention，并通过“层吸收”把全局与局部注意力差异视为重建问题，使用闭式最小二乘求得实例特定的权重调制，从而让本地层能模拟长程历史的效果。\"}]",1784196979,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"towards-memory-efficient-autoregressive-video-generation-via-instance-specific-parametric-absorption","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,47,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":20},"https://docshare.wps.com/document/","Document",{"item":48,"name":12,"@type":43,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/towards-memory-efficient-autoregressive-video-generation-via-instance-specific-parametric-absorption/84591/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-21","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"为什么 AR 流式视频生成会出现显存和推理速度瓶颈？","Question",{"text":75,"@type":76},"随着生成持续进行，KV cache 需要持续累积历史上下文，导致显存占用线性增长，从而显著减慢推理速度并削弱实时性优势。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"直接丢弃 KV 缓存中的冗余 tokens 会带来什么问题？",{"text":80,"@type":76},"丢弃会破坏长距离依赖，使模型遗忘过去，进而引发背景闪烁、结构扭曲以及身份丢失等质量问题。",{"name":82,"@type":73,"acceptedAnswer":83},"ISPA 如何在不丢弃历史 tokens 的情况下实现更高效的压缩？",{"text":84,"@type":76},"ISPA 将部分层从 FullAttention 转换为内存更高效的 Local-Attention，并通过“层吸收”把全局与局部注意力差异视为重建问题，使用闭式最小二乘求得实例特定的权重调制，从而让本地层能模拟长程历史的效果。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]