[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84922-en":3,"doc-seo-84922-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84922,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","FreqDepthKV Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference","Long-context LLM inference is constrained by KV cache memory and bandwidth, while aggressive compression can erase layer-specific evidence required for retrieval and multi-step reasoning. FreqDepthKV factorizes adjacent-layer KV states into broadly shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact modes based on reconstruction-sensitive logits, adapting to prompt structure without retraining. Benchmarks show near full-KV accuracy with substantially smaller cache budgets, alongside improved throughput, lower TTFT, and reduced peak KV memory.","arXiv :2607 .065 19v 1 [ cs .AI ] 7 Jul 2026  \nFreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM  \nInference  \nAnna Córdoba Adam Puente Tercero Nerea Angulo Hijo Mar Linares Tercero  \nJulia Barrientos Ainhoa Miranda Jesús Olivera  \nInstituto de Investigación en Visión Artificial  \ncontact@iiva .tibeu  \nAbstract  \nLong-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals.  \nA lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.  \n1 Introduction  \nLong-context inference has shifted the bottleneck of large language models from parameter storage to the key-value (KV) cache. During autoregressive decoding, every generated token attends over cached keys and values from all previous tokens, layers, and heads, making memory traffic and cache footprint grow linearly with context length. This cost is especially acute for long-context question answering, multi-document summarization, and code completion, where prompts may contain many irrelevant tokens but still require preserving a small amount of evidence that determines the final answer. Existing KV cache compression methods reduce this footprint through token eviction, heavyhitter retention, quantization, or structured sharing Li et al. [2024], Jiang et al. [2025], Javidnia et al.[2025] . However, these approaches often treat redundancy as either a token-level or precision-level phenomenon, while leaving underexplored the fact that adjacent transformer layers frequently encode similar depth-wise cache structure.  \nMiniCache demonstrates that KV caches can be compressed along the depth dimension by exploiting redundancy between neighboring layers Liu et al. [2024a] . This observation is powerful: many layers store correlated representations, so sharing or merging their caches can substantially reduce memory without modifying model weights. Yet uniform depth compression introduces a new failure mode. In retrieval-heavy and reasoning-heavy prompts, the decisive evidence is often localized to particular token-head-layer interactions. A layer that appears redundant on average may still contain highfrequency residual information needed to disambiguate a needle sentence, preserve a code dependency,  \n37th Conference on Neural Information Processing Systems (NeurIPS 2023) .  \nFigure 1: Overview of the core idea: FreqDepthKV removes redundant depth information shared by neighboring layers while preserving sparse layer-specific token evidence that controls retrievalsensitive attention.  \nor maintain a reasoning chain. Once these layer-specific residuals are erased, downstream attention logits can shift enough to change the generated answer, even when the reconstructed cache has low aggregate error. Recent work has similarly noted that cache compression can be workload-dependent and can fail under adversarial or evidence-sensitive prompts Chen et al. [2025], Haverbeck et al.  \n[2","cbCaicx7ZZjx1MJJ","https://ap.wps.com/l/cbCaicx7ZZjx1MJJ","pdf",2635147,1,11,"English","en",105,"# Introduction\n## Core idea and motivation\n## Contributions","[{\"question\":\"What problem does FreqDepthKV address in long-context LLM inference?\",\"answer\":\"It targets the growing memory and bandwidth costs of KV caches, where compression can remove layer-specific evidence needed for retrieval and multi-step reasoning.\"},{\"question\":\"How does FreqDepthKV compress KV caches across layers?\",\"answer\":\"It decomposes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals, preserving sparse token evidence that drives retrieval-sensitive attention.\"},{\"question\":\"How does FreqDepthKV choose which heads use shared, residual, or exact cache modes?\",\"answer\":\"During prefill, a lightweight online probe estimates each head’s contribution using reconstruction-sensitive attention logits and routes heads to shared-depth, residual-depth, or exact modes without retraining.\"}]",1784199362,28,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"freqdepthkv-frequency-guided-depth-sharing-for-robust-kv-cache-compression-in-long-context-llm-inference","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/freqdepthkv-frequency-guided-depth-sharing-for-robust-kv-cache-compression-in-long-context-llm-inference/84922/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does FreqDepthKV address in long-context LLM inference?","Question",{"text":75,"@type":76},"It targets the growing memory and bandwidth costs of KV caches, where compression can remove layer-specific evidence needed for retrieval and multi-step reasoning.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does FreqDepthKV compress KV caches across layers?",{"text":80,"@type":76},"It decomposes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals, preserving sparse token evidence that drives retrieval-sensitive attention.",{"name":82,"@type":73,"acceptedAnswer":83},"How does FreqDepthKV choose which heads use shared, residual, or exact cache modes?",{"text":84,"@type":76},"During prefill, a lightweight online probe estimates each head’s contribution using reconstruction-sensitive attention logits and routes heads to shared-depth, residual-depth, or exact modes without retraining.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]