[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81538-en":3,"doc-seo-81538-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81538,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782698725881665579",8,"Research & Report","REAL REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression","Large language models face memory and efficiency challenges as sequence length grows, making key-value (KV) caches increasingly costly for GPUs. Existing KV cache eviction methods focus mainly on successful retrieval-reasoning attention-head behavior and often ignore failure behaviors like bias and distraction, limiting heterogeneous head exploitation. REAL introduces an Attention Behavior Matrix to analyze success and failure behaviors, maximizing signal-to-noise ratio and enabling logic-constructed eviction. Evaluations show strong, comparable accuracy to top baselines with far lower cache space, enabling more failure-aware long-context modeling.","REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression  \nMengjie Li 1 , Yuan Feng2 , Xike Xie3 , and William J. Song†1  \n1 School of Electrical and Electronic Engineering, Yonsei University  \n2 School of Computer Science, University of Science and Technology of China  \n3 School of Biomedical Engineering, University of Science and Technology of China [lemoji@yonsei.ac.kr](lemoji@yonsei.ac.kr), [yfung@mail.ustc.edu.cn](yfung@mail.ustc.edu.cn), [xkxie@ustc.edu.cn](xkxie@ustc.edu.cn), [wjhsong@yonsei.ac.kr](wjhsong@yonsei.ac.kr)  \nAbstract  \nThe growing sequence length of large language models poses significant challenges for keyvalue (KV) caches. Existing state-of-the-art cache eviction methods primarily analyze the inference behavior of attention heads in successful retrieval-reasoning cases, often overlooking diverse behaviors in failure cases, such as bias and distraction. This oversight limits the potential to leverage heterogeneous head behaviors for improved eviction performance.  \nInspired by the confusion matrix, we introduce an Attention Behavior Matrix to comprehensively analyze attention head behaviors in both success and failure scenarios. By maximizing the signal-to-noise ratio—strengthening valid reasoning pathways in success cases while inhibiting noise from bias and distraction in failure cases—we propose REtrievalreAsoning and Logic-constructed (REAL) KV cache eviction, the first method to leverage multi-behavior analysis. Comprehensive evaluations show that REAL achieves remarkable performance across various models and benchmarks; notably, on LongBench v2, it achieves comparable accuracy to the strongest baseline, HeadKV-R2, while requiring 32x less space (Figure 1) . By offering a novel perspective on behavior analysis, we pave the way for a shift from success-only to comprehensive, failureaware methods in long-context modeling. Our code is available at [https://github.com/](https://github.com/)[ ](https://github.com/)yonseicasl/REAL.  \n1 Introduction  \nRetrieval-driven and logic-faithful Transformerbased (Vaswani et al., 2017) large language models (LLMs) (OpenAI, 2025 ; Anthropic, 2025) have shown remarkable performance on tasks such as question answering (QA) (Kamalloo et al., 2023) . To speed up inference, these models rely on keyvalue (KV) caches, which store key and value vec-  \n†Corresponding Author.  \nLB Avg  \n30  \n25  \n20  \n15  \n10  \n64 512 1024 2048 4096 8192 128  \n256 Token Budget  \nFigure 1: Results on question-aware LongBench v2 with Mistral-Large-Instruct-2411 . REAL matches SOTA HeadKV-R2 using 4,096 vs. 8,192 cache tokens (2x smaller), and matches SOTA at 4,096 tokens with 128 tokens (32x smaller) . See Section 4 for full results.  \nRetrieval-Reasoning  Bias  Distraction  Widespread  \nFigure 2: Heads dominated by different behaviors.  \ntors to avoid recalculations. However, as the size of the KV cache grows with sequence length, model dimensionality, and batch size, it quickly overwhelms the memory capacity of graphics processing units (GPUs) . For example, the KV cache for 64 x 4,096 tokens in GPT-3 (Brown et al., 2020) requires about 1,208 GB, whereas the device memory capacity of an NVIDIA H200 GPU is only 141 GB, underscoring the necessity of efficient KV cache compression.  \nTo address this problem, various cache eviction methods have been proposed (Zhang et al., 2023 ; Cai et al., 2024 ; Li et al., 2024 ; Feng et al., 2025a,c) . They enforce a fixed budget by retaining only a subset of KV entries within each attention head and discarding the rest. This reduces memory usage and speeds up decoding, enabling efficient longcontext inference. However, most approaches ignore functional differences across attention heads and therefore apply uniform compression budgets, which limits the effectiveness of cache eviction. AdaKV (Feng et al., 2025b) recognizes this issue  \nThis author’s version has been accepted for the Main Conference at the 64th Ann","cbCailPUt21w30yK","https://ap.wps.com/l/cbCailPUt21w30yK","pdf",1104674,3,1,18,"English","en",105,"# Introduction\n## KV cache challenges for long-context LLMs\n## Limits of success-only attention-head analysis\n## REAL method: attention behavior matrix and logic-constructed eviction\n# Experiments and results\n## LongBench v2 performance and space efficiency\n## Behavior analysis beyond success cases","[{\"question\":\"Why is KV cache compression necessary for long-context LLM inference?\",\"answer\":\"KV caches grow rapidly with sequence length, model dimensionality, and batch size, quickly exceeding GPU memory capacity. Compression reduces memory use and speeds decoding for long-context tasks.\"},{\"question\":\"What key limitation do existing KV cache eviction methods have?\",\"answer\":\"Most methods ignore functional differences across attention heads and primarily analyze successful retrieval-reasoning cases. This overlooks failure behaviors such as bias and distraction, reducing eviction effectiveness.\"},{\"question\":\"How does REAL improve KV cache eviction compared with prior approaches?\",\"answer\":\"REAL builds an Attention Behavior Matrix to analyze attention-head behaviors in both success and failure scenarios. By maximizing the signal-to-noise ratio, it strengthens valid reasoning pathways in success cases and suppresses noise from bias and distraction in failure cases.\"}]",1784174139,45,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"real-retrieval-reasoning-and-logic-constructed-attention-behaviors-for-long-context-kv-cache-compression","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/real-retrieval-reasoning-and-logic-constructed-attention-behaviors-for-long-context-kv-cache-compression/81538/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is KV cache compression necessary for long-context LLM inference?","Question",{"text":75,"@type":76},"KV caches grow rapidly with sequence length, model dimensionality, and batch size, quickly exceeding GPU memory capacity. Compression reduces memory use and speeds decoding for long-context tasks.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What key limitation do existing KV cache eviction methods have?",{"text":80,"@type":76},"Most methods ignore functional differences across attention heads and primarily analyze successful retrieval-reasoning cases. This overlooks failure behaviors such as bias and distraction, reducing eviction effectiveness.",{"name":82,"@type":73,"acceptedAnswer":83},"How does REAL improve KV cache eviction compared with prior approaches?",{"text":84,"@type":76},"REAL builds an Attention Behavior Matrix to analyze attention-head behaviors in both success and failure scenarios. By maximizing the signal-to-noise ratio, it strengthens valid reasoning pathways in success cases and suppresses noise from bias and distraction in failure cases.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]