[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83590-en":3,"doc-seo-83590-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},83590,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Hypic Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching","Retrieval augmented generation (RAG) and agentic LLM serving assemble long prompts from independent segments, causing prefill computation to dominate per-request cost and making cache-miss tail time-to-first-token (TTFT) reach tens of seconds. Position-independent caching (PIC) enables KV reuse for non-contiguous segments, while hybrid-attention models replace most full-attention layers with linear attention. Existing PIC cannot be applied to hybrid stacks because KV splice-and-correction primitives do not transfer to per-request recurrent states. Hypic addresses this gap with segmentcumulative transition caching, seam-window hidden-state repair, and segment parallelism.","Hypic: Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching  \nYifei Liu∗ Xiaohongshu Inc. [liuyifei5@xiaohongshu.com](liuyifei5@xiaohongshu.com)  \nJunhao Hu† Peking University [junhaohu@stu.pku.edu.cn](junhaohu@stu.pku.edu.cn)  \nJuntong Wu∗ Peking University [wujt@stu.pku.edu.cn](wujt@stu.pku.edu.cn)  \nMinghao Li  \nXiaohongshu Inc. [leimuchen@xiaohongshu.com](leimuchen@xiaohongshu.com)  \nYang Liu  \nShanghai Jiao Tong University [liuyang370@sjtu.edu.cn](liuyang370@sjtu.edu.cn)  \nXiaoxu Chen  \nXiaohongshu Inc. [chenxiaoxu@xiaohongshu.com](chenxiaoxu@xiaohongshu.com)  \nWeihang Chen  \nXiaohongshu Inc. [chenjinzhi@xiaohongshu.com](chenjinzhi@xiaohongshu.com)  \narXiv :2607 .01299v2 [ cs .DC] 12 Jul 2026  \nAbstract  \nIn retrieval augmented generation (RAG) and agentic LLM serving, prompts are assembled from independent segments into long contexts, making the prefill stage dominate the perrequest computation cost. To reduce this cost, two directions have emerged in parallel: position-independent caching (PIC) admits KV reuse for non-contiguous segments shared across different requests, while hybrid-attention models reduce computation complexity by replacing most full-attention layers with linear attention. However, they cannot coexist: applying existing PIC methods to hybrid-attention models breaks down because per-token KV-cache reuse primitives do not transfer to the per-request recurrent state.  \nIn this work, we present Hypic, the first system to accelerate hybrid-attention LLM serving with position-independent caching. For linear-attention layers, we identify the segmentcumulative transition operator as the missing algebraic primitive and cache it alongside each segment’s zero-start endstate, enabling near-exact and constant-time state composition of independently cached segments. For the remaining full-attention layers, existing PIC methods also fail because linear layers do not expose the per-token hidden states needed for selective recomputation. We show that the largest deviations concentrate at segment beginnings and construct a small seam window that propagates hidden states through the hybrid-attention stack to repair cross-segment attention. Finally, Hypic introduces segment parallelism, which exploits PIC’s segment-level self-containment to parallelize cache-miss prefill across instances, turning long cold requests—a major tail-latency contributor under both prefix caching and prior PIC—into an accelerable workload. Evaluated across four hybrid-attention models and five workloads, Hypic reduces time-to-first-token (TTFT) by 3. 25× on average and improves QPS by 1. 66× over Prefix Cache, while preserving task quality with a 1. 71-point gap from Full Recompute.  \n∗ Equal contribution.†Corresponding author.  \nFigure 1. Existing PIC methods reuse per-token KV cache in full-attention models via splice and correction (left); on hybrid stacks, both primitives fail because linear-attention layers expose only a per-request recurrent state, with no per-token handle (right) .  \n1 Introduction  \nLarge language model (LLM) serving is shifting from singleturn chat toward retrieval-augmented question answering [10, 14, 35, 48], multi-document summarization [2, 7, 9], and long-horizon agents [13] . These workloads pull independent text segments (skills, memory files, etc.) from local or remote sources and embed them into a fixed prompt template, assembling contexts oftens to hundreds of thousands of tokens [2, 52] . At these lengths, prefill dominates the per-request compute bill and becomes one of the most prominent serving expenses for providers [37, 38] . Worse, on a cache miss, tail time-to-first-token (TTFT) can reach tens of seconds, directly hurting interactive user experience [1, 26, 27, 55] .  \nTo reduce this cost, a growing body of work proposes position-independent caching (PIC) [11, 22–24, 38, 39, 41, 49, 50]. Unlike strict-prefix KV reuse, PIC caches each semantically independent segment once and allo","cbCaiu61hGtPVkAV","https://ap.wps.com/l/cbCaiu61hGtPVkAV","pdf",5408054,10,1,14,"English","en",105,"# Introduction\n## Problem: Prefill cost and cache-miss TTFT in RAG/agents\n## Prior work: Position-independent caching and its KV primitives\n## Background: Linear attention and hybrid attention stacks\n## Gap: Incompatibility of PIC with hybrid-attention models\n## Contribution overview: Hypic’s key ideas","[{\"question\":\"Why does prefill dominate cost in RAG and agentic LLM serving?\",\"answer\":\"Prompts are assembled from independent segments into very long contexts (often hundreds of thousands of tokens), so the prefill stage becomes the largest per-request computation expense.\"},{\"question\":\"What makes applying existing PIC methods to hybrid-attention models fail?\",\"answer\":\"PIC relies on token-level KV splice and correction, but linear-attention layers expose only a per-request recurrent state rather than per-token hidden states or KV handles, so the primitives do not transfer.\"},{\"question\":\"How does Hypic enable position-independent caching for linear-attention layers?\",\"answer\":\"Hypic caches the segment-cumulative transition operator and a segment’s zero-start endstate, allowing near-exact constant-time composition of independently cached segments.\"}]",1784189059,35,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"hypic-accelerating-hybrid-attention-llm-serving-with-position-independent-caching","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/hypic-accelerating-hybrid-attention-llm-serving-with-position-independent-caching/83590/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-27","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why does prefill dominate cost in RAG and agentic LLM serving?","Question",{"text":76,"@type":77},"Prompts are assembled from independent segments into very long contexts (often hundreds of thousands of tokens), so the prefill stage becomes the largest per-request computation expense.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What makes applying existing PIC methods to hybrid-attention models fail?",{"text":81,"@type":77},"PIC relies on token-level KV splice and correction, but linear-attention layers expose only a per-request recurrent state rather than per-token hidden states or KV handles, so the primitives do not transfer.",{"name":83,"@type":74,"acceptedAnswer":84},"How does Hypic enable position-independent caching for linear-attention layers?",{"text":85,"@type":77},"Hypic caches the segment-cumulative transition operator and a segment’s zero-start endstate, allowing near-exact constant-time composition of independently cached segments.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":20,"slug":134},"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]