[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84872-en":3,"doc-seo-84872-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84872,8796095461564,"Liam","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",6,"Technology","MemDefrag: Latent Memory Defragmentation for Large Language Models","Latent memory stores past knowledge fragments as per-layer hidden states, forming a long-term memory approach for large language models (LLMs). Updating latent memory can degrade performance because positional encoding becomes misaligned and there is no mechanism to trace which stored fragments are relevant versus irrelevant. By probing layer-wise attention density, a small set of middle transformer layers is found to consistently concentrate on target fragments. MemDefrag uses this tracing signal to rank, reorder, and filter memories, plus informativeness-guided proportional forgetting under capacity limits, improving knowledge retention and long-context benchmarks across model variants.","MemDefrag: Latent Memory Defragmentation for Large Language Models  \nRuiyi Yan*  \nKyoto University [ruiyi@nlp.ist.i.kyoto-u.ac.jp](ruiyi@nlp.ist.i.kyoto-u.ac.jp)  \nZhuoyuan Mao  \nLIGHTSPEED  \n[zhuoyuanmao@global.tencent.com](zhuoyuanmao@global.tencent.com)  \nYiwen Guo† Independent Researcher [guoyiwen89@gmail.com](guoyiwen89@gmail.com)  \narXiv :2607 .05969v 1 [ cs .CL] 7 Jul 2026  \nAbstract  \nLatent memory, which stores past knowledge fragments as per-layer hidden states, has emerged as a promising paradigm (e.g., MemoryLLM and M+) for long-term memory in large language models (LLMs) . However, the paradigm suffers from significant performance degradation during memory updates, due to positional encoding misalignment and the absence of any tracing mechanism to distinguish target memory fragments from irrelevant ones. To discover such a tracing mechanism, we probe the layer-wise attention density over stored memory fragments, and find that a small set of middle transformer layers consistently concentrates the highest density on the target fragment—exposing an inherent tracing signal. In light of this, we propose MemDefrag, a training-free and model-agnostic framework that (1) uses a middle-layer tracing signal to conduct memory defragmentation (rank, reorder, and filter memories), and (2) applies an informativenessguided proportional forgetting mechanism once capacity is exceeded. Experiments show that MemDefrag substantially outperforms MemoryLLM and M+ on knowledge retention (e.g., 43.0% vs. 17.4%/17.6% after 50 memory updates) and long-context benchmarks, and generalizes well across various LLMs and latentmemory variants.  \n1 Introduction  \nDespite the impressive capabilities of Large Language Models (LLMs), a long-standing challenge persists: how can a deployed LLM continually absorb new knowledge, while faithfully retaining what has already been stored? Previous solutions can be broadly grouped into three classes. (1) Tokenlevel memory (Packer et al., 2024 ; Chhikara et al., 2025) stores past information as explicit text tokens, while it is bottlenecked by the context window and is vulnerable to the lost-in-the-middle  \n*This work was done during internship at LIGHTSPEED.  \n†Corresponding author.  \nphenomenon (Liu et al., 2024) . (2) Parametric memory either encodes knowledge into the base model (Zhao et al., 2025) or into auxiliary adapters such as LoRA (Hu et al., 2022) . This form incurs no extra inference cost but requires costly tuning and is susceptible to catastrophic forgetting (Kirkpatrick et al., 2017) . (3) Latent memory carries information implicitly in the model’s internal representations such as KV-caches, activations, or perlayer hidden states. The long-term form (e.g., MemoryLLM (Wang et al., 2024), M+ (Wang et al., 2025), and NextMem (Zhang et al., 2026)) maintains a persistent pool of per-layer hidden states. By avoiding the repeated re-encoding of the full context and compressing information more densely in latent representations than discrete text tokens, this latent paradigm offers a compelling foundation for continually updatable LLMs.  \nHowever, the current long-term latent memory paradigm encounters the following limitations. (1) Positional encoding misalignment: When a new knowledge fragment is appended, the concatenated memory fragments are indexed with positions that no longer match those they were formed at, distorting the attention computation at inference time (Gao et al., 2024) . (2) The absence of any tracing mechanism: Existing methods treat all memory fragments as a flat, undifferentiated prefix and cannot dispel the noise information or single out fragment(s) relevant to the current query.  \nTo address these issues, we propose MemDefrag, a training-free and model-agnostic framework that reinvents the way long-term latent memory is used and evolved. MemDefrag is fully plugand-play: it requires neither additional training nor auxiliary modules to the underlying LLM. Specifically, the contri","cbCaiaGOvS639ppr","https://ap.wps.com/l/cbCaiaGOvS639ppr","pdf",1291089,1,20,"English","en",105,"# Abstract\n# 1 Introduction\n# 2 Investigation: Positional Distortion of Latent Memory\n## 2.1 Preliminaries: Vanilla Latent Memory","[{\"question\":\"What problem does MemDefrag address in long-term latent memory for LLMs?\",\"answer\":\"Latent memory updates can cause performance degradation due to positional encoding misalignment and the lack of a tracing mechanism to identify which stored fragments are relevant.\"},{\"question\":\"How does MemDefrag identify a tracing signal for relevant memory fragments?\",\"answer\":\"It probes layer-wise attention density over stored fragments for a given prompt, finding that a small set of middle transformer layers concentrates the highest density on the target fragment.\"},{\"question\":\"What memory update strategy does MemDefrag use when memory capacity is exceeded?\",\"answer\":\"It applies an informativeness-guided proportional forgetting mechanism: forgetting quotas are allocated proportionally across fragments, and within each fragment it evicts tokens with the lowest self-information.\"}]",1784198937,50,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"memdefrag-latent-memory-defragmentation-for-large-language-models","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/memdefrag-latent-memory-defragmentation-for-large-language-models/84872/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does MemDefrag address in long-term latent memory for LLMs?","Question",{"text":74,"@type":75},"Latent memory updates can cause performance degradation due to positional encoding misalignment and the lack of a tracing mechanism to identify which stored fragments are relevant.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does MemDefrag identify a tracing signal for relevant memory fragments?",{"text":79,"@type":75},"It probes layer-wise attention density over stored fragments for a given prompt, finding that a small set of middle transformer layers concentrates the highest density on the target fragment.",{"name":81,"@type":72,"acceptedAnswer":82},"What memory update strategy does MemDefrag use when memory capacity is exceeded?",{"text":83,"@type":75},"It applies an informativeness-guided proportional forgetting mechanism: forgetting quotas are allocated proportionally across fragments, and within each fragment it evicts tokens with the lowest self-information.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,111,116,121,125,128,132],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":28,"slug":110},"technology",{"id":112,"doc_module":4,"doc_module_name":45,"category_name":113,"show_sort_weight":114,"slug":115},7,"Healthcare",40,"healthcare",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":118,"show_sort_weight":119,"slug":120},8,"Research & Report",30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":21,"slug":124},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":126,"show_sort_weight":21,"slug":127},"World Cup","world-cup",{"id":129,"doc_module":4,"doc_module_name":45,"category_name":130,"show_sort_weight":129,"slug":131},10,"Lifestyle","lifestyle",{"id":133,"doc_module":4,"doc_module_name":45,"category_name":134,"show_sort_weight":105,"slug":135},19,"General","general"]