[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83108-en":3,"doc-seo-83108-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},83108,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","DepthWeave-KV Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression","Long-context language model inference is constrained by memory bandwidth and capacity needed for key-value (KV) caches, while existing compression often uses uniform budgets across layers or tokens and hurts retrieval when lexical cues and semantic states require different preservation. DepthWeave-KV introduces token-adaptive cache compression that factorizes key/value states across neighboring transformer layers with shared low-rank channel bases plus lightweight token-specific residuals. A token-conditional depth router allocates higher reconstruction rank to instruction- and retrieval-critical tokens, guided by calibration-free online error tracking from attention-output probes. A fused CUDA kernel reduces decode-time traffic. Experiments on LongBench, Needle-in-a-Haystack, L-Eval and long-form QA/summarization achieve near-full task quality with 8.3x KV memory reduction and strong throughput at 64K context.","DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for  \nLong-Context KV Cache Compression  \nAnna Crdoba 1 Adam Puente Tercero 1 Nerea Angulo Hijo 1 Mar Linares Tercero 1 Julia Barrientos 1  \nAinhoa Miranda 1 Jess Olivera 1  \narXiv :2607 .06523v 1 [ cs .AI ] 7 Jul 2026  \nAbstract  \nLong-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different preservation. We introduce DepthWeave-KV, a token-adaptive cache compression method that factorizes key and value states across neighboring transformer layers using shared low-rank channel bases while retaining lightweight token-specific residuals where attention behavior is sensitive. DepthWeave-KV combines cross-depth residual factorization with a token-conditional depth router that allocates higher reconstruction rank to instruction-bearing and retrieval-critical tokens, and uses calibrationfree online error tracking from attention-output probes to adapt compression during generation without retraining the base model. A fused CUDA implementation jointly performs basis lookup, residual dequantization, and attention projection to reduce decode-time memory traffic. Across LongBench, Needle-in-a-Haystack, L-Eval, and long-form QA and summarization benchmarks, DepthWeave-KV achieves near-full-cache task quality with substantially lower memory use, improving average score and retrieval accuracy over prior compressed caches while reaching 8.3x KV memory reduction and 72.8 tokens per second at 64K context.  \n1Instituto de Investigacin en Visin Artificial. Correspondence to: Anna Crdoba \u003Canna.cordoba@iiva.tibeu> .  \nProceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 . Copyright 2024 by the author(s) .  \n1. Introduction  \nLong-context language models increasingly depend on efficient inference systems as prompts grow from short interactive turns to documents, conversations, codebases, and retrieval-augmented contexts. In this regime, the key-value (KV) cache often becomes the dominant memory resident state during decoding, limiting batch size, context length, and serving throughput. Prior work has reduced this burden through token eviction, token merging, quantization, low-rank attention, and layer-wise cache sharing (Li et al., 2024 ; Jiang et al., 2025 ; Saxena et al., 2024 ; Xiang et al., 2025 ; Yang et al., 2024a ; Yao et al., 2025) . However, longcontext tasks expose a persistent tension: tokens that appear unimportant under local continuation statistics may later become essential for retrieval, while the transformer layers that encode shallow lexical anchors and deeper semantic abstractions do not require the same cache fidelity. Uniform compression across tokens or depths can therefore create brittle failures on needle retrieval, multi-hop question answering, and long-document summarization.  \nDepth-wise KV compression is a particularly promising direction because adjacent transformer layers often contain correlated cache structure. MiniCache (Liu et al., 2024a) demonstrates that exploiting redundancy along the depth dimension can reduce KV memory while preserving much of the model’s behavior. Yet depth sharing alone is insufficient for heterogeneous long-context workloads. A cache entry associated with an instruction delimiter, entity mention, citation span, or answer-bearing sentence may need more faithful reconstruction than a fluent continuation token; likewise, the same token can demand different treatment in early lexical layers and later semantic layers. Recent studies have also highlighted that aggressive KV compression can degrade retrieval behavior in ways that average perplexity or short-context benchmarks may obscure (Chen et al., 2025 ; Haverbeck et al., 2026 ; Bui et al., 2026) .","cbCaiaIyAop1sjGN","https://ap.wps.com/l/cbCaiaIyAop1sjGN","pdf",2525881,3,1,9,"English","en",105,"# Abstract\n# Introduction\n## Motivation and Limitations of Existing Compression\n## Depth-wise KV Compression and Remaining Challenges\n## DepthWeave-KV Overview and Contributions","[{\"question\":\"为什么长上下文推理中 KV 缓存成为瓶颈？\",\"answer\":\"随着提示从短对话扩展到文档、对话、代码库与检索增强上下文，解码阶段需要的 KV 缓存往往主导常驻内存，限制批大小、上下文长度与服务吞吐。\"},{\"question\":\"DepthWeave-KV 如何在跨层之间实现更高效的压缩？\",\"answer\":\"它不为每一层存储独立 KV 张量，而是在每个注意力头上为相邻层共享一组低秩通道基，并用共享基加上稀疏的令牌特定残差来重建层特定的 key/value 状态。\"},{\"question\":\"模型如何在生成过程中自适应压缩强度而无需重训练？\",\"answer\":\"DepthWeave-KV 使用来自 attention-output probes 的校准无关在线误差跟踪，使压缩比例能够在生成期间随注意行为自适应调整，而不需要对基础 LLM 进行再训练。\"}]",1784185331,23,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"depthweave-kv-token-adaptive-cross-layer-residual-factorization-for-long-context-kv-cache-compression","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/depthweave-kv-token-adaptive-cross-layer-residual-factorization-for-long-context-kv-cache-compression/83108/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"为什么长上下文推理中 KV 缓存成为瓶颈？","Question",{"text":75,"@type":76},"随着提示从短对话扩展到文档、对话、代码库与检索增强上下文，解码阶段需要的 KV 缓存往往主导常驻内存，限制批大小、上下文长度与服务吞吐。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"DepthWeave-KV 如何在跨层之间实现更高效的压缩？",{"text":80,"@type":76},"它不为每一层存储独立 KV 张量，而是在每个注意力头上为相邻层共享一组低秩通道基，并用共享基加上稀疏的令牌特定残差来重建层特定的 key/value 状态。",{"name":82,"@type":73,"acceptedAnswer":83},"模型如何在生成过程中自适应压缩强度而无需重训练？",{"text":84,"@type":76},"DepthWeave-KV 使用来自 attention-output probes 的校准无关在线误差跟踪，使压缩比例能够在生成期间随注意行为自适应调整，而不需要对基础 LLM 进行再训练。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,127,130,134],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]