[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84560-en":3,"doc-seo-84560-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84560,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","When Classic Cache Policies Fail: Learning-Augmented Replacement for Semantic Retrieval Buffers","LLM agents depend on retrieval buffers to reuse past experience, yet buffer cache management often follows ad-hoc rules. This work formalizes the task as an online semantic cache replacement problem with switching costs, using embedding-similarity matching and continuous hit quality. Experiments on two MemoryBench-Full datasets (LoCoMo, DialSim) with eight replacement policies show classic heuristics (LRU, LFU) underperform naive FIFO for semantic workloads. The paper introduces SOLAR, a learning-augmented method that learns modification timing from regret accumulation and content selection from Bayesian online learning, with proven competitive guarantees and capacity-constrained retrieval noise behavior.","When Classic Cache Policies Fail: Learning-Augmented Replacement for Semantic Retrieval Buffers  \nYushi Sun∗ LIGHTSPEED, Tencent Shenzhen, China [ysunbp@connect.ust.hk](ysunbp@connect.ust.hk)  \nBowen Cao∗ The Chinese University of Hong Kong Hong Kong, China [bwcao@link.cuhk.edu.hk](bwcao@link.cuhk.edu.hk)  \nWai Lam  \nThe Chinese University of Hong Kong Hong Kong, China [wlam@se.cuhk.edu.hk](wlam@se.cuhk.edu.hk)  \narXiv :2607 .00394v 1 [ cs .DB] 1 Jul 2026  \nABSTRACT  \nLLM agents increasingly rely on retrieval buffers to store and reuse past experience, yet the cache management policies governing these buffers remain largely ad-hoc. We formalize this as an online semantic cache replacement problem with switching costs, where items are matched by embedding similarity and hit quality is continuous rather than binary. Through experiments on two datasets from MemoryBench-Full (LoCoMo, DialSim) with 8 replacement policies, we reveal a surprising finding: classic heuristics (LRU, LFU) consistently underperform the naive FIFO baseline on semantic workloads, due to the absence of temporal locality and frequency concentration. We propose SOLAR, a learning-augmented framework that derives modification timing from regret accumulation (achieving ∼ 17% modification rate) and content selection from Bayesian online learning over implicit retrieval feedback. We prove SOLAR achievesa constant competitive ratio ≤ 3, independent of cache size and horizon (vs. Ω (􀀠) for FIFO), and eviction regret 􀀤( √︁􀀠􀀩 log􀀩 ) , matching the Ω( √􀀠􀀩) lower bound up to logarithmic factors. Experiments demonstrate 5–75% relative improvement over FIFO at tight cache sizes, with a clearly characterized phase transition atthe working set boundary. Synthetic experiments with 5000-item pools further reveal an inverted-U relationship between pool size and retrieval quality, justifying capacity constraints as a retrieval noise phenomenon rather than a storage limitation.  \nPVLDB Reference Format:  \nYushi Sun, Bowen Cao, and Wai Lam. When Classic Cache Policies Fail: Learning-Augmented Replacement for Semantic Retrieval Buffers. PVLDB, 14(1): XXX-XXX, 2027 .  \ndoi:XX.XX/XXX.XX  \n1 INTRODUCTION  \nLarge language model (LLM) agents are deployed in increasingly complex, long-running tasks: personal assistants maintaining months of conversation history [22], game-playing agents accumulating strategies over thousands of episodes [33], and research agents that iteratively refine their knowledge through tool use [23] . A common architectural pattern has emerged across these systems: the agent maintains a retrieval buffer of past experience that is queried at each  \n∗ Both authors contributed equally to this research. This work was done during Bowen’s internship at Tencent LIGHTSPEED.  \nThis work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit [https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of](https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of)[ ](https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of)[this license. For any use beyond those covered by this license](this license. For any use beyond those covered by this license), [obtain permission by](obtain permission by)[emailing info@vldb.org. Copyright](emailing info@vldb.org. Copyright) is held by the owner/author(s). Publication rights licensed to the VLDB Endowment.  \nProceedings of the VLDB Endowment, Vol. 14, No. 1 ISSN 2150-8097 .  \ndoi:XX.XX/XXX.XX  \nstep to inform its current response. This buffer serves as the agent’s long-term memory, supplementing the fixed context window of the underlying language model.  \nThe retrieval buffer operates as follows. At each interaction step, the agent generates a new experience item: in conversational deployments this is typically a user utterance or a stated preference, while in fully agentic deployments it is more often an environment observation, a tool-invocation result, or a completed subtask trac","cbCaidBfUj2rHbSg","https://ap.wps.com/l/cbCaidBfUj2rHbSg","pdf",753222,1,13,"English","en",105,"# Abstract\n# Introduction\n## Retrieval buffer architecture and cache analogy\n## Current practice and motivation for experiments\n## Experimental setup and surprising findings\n# Method: SOLAR (learning-augmented replacement)\n# Theoretical analysis and guarantees\n# Experiments and phase transition observations","[{\"question\":\"What problem does the paper study about retrieval buffers in LLM agents?\",\"answer\":\"It studies how to manage a semantic retrieval buffer as an online cache replacement problem, including switching costs and continuous hit quality rather than binary hits.\"},{\"question\":\"What surprising experimental result is reported for classic cache heuristics?\",\"answer\":\"Across semantic workloads, classic heuristics like LRU and LFU consistently underperform even the naive FIFO baseline, due to missing temporal locality and frequency concentration.\"},{\"question\":\"How does SOLAR improve semantic cache replacement compared with FIFO and classical policies?\",\"answer\":\"SOLAR learns modification timing from regret accumulation and performs content selection using Bayesian online learning over implicit retrieval feedback, achieving measurable improvement and providing competitive guarantees.\"}]",1784196739,33,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"when-classic-cache-policies-fail-learning-augmented-replacement-for-semantic-retrieval-buffers","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/when-classic-cache-policies-fail-learning-augmented-replacement-for-semantic-retrieval-buffers/84560/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the paper study about retrieval buffers in LLM agents?","Question",{"text":75,"@type":76},"It studies how to manage a semantic retrieval buffer as an online cache replacement problem, including switching costs and continuous hit quality rather than binary hits.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What surprising experimental result is reported for classic cache heuristics?",{"text":80,"@type":76},"Across semantic workloads, classic heuristics like LRU and LFU consistently underperform even the naive FIFO baseline, due to missing temporal locality and frequency concentration.",{"name":82,"@type":73,"acceptedAnswer":83},"How does SOLAR improve semantic cache replacement compared with FIFO and classical policies?",{"text":84,"@type":76},"SOLAR learns modification timing from regret accumulation and performs content selection using Bayesian online learning over implicit retrieval feedback, achieving measurable improvement and providing competitive guarantees.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]