[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82535-en":3,"doc-seo-82535-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},82535,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","Auditing Forgetting in Limited Memory Language Models","Limited Memory Language Models (LMLMs) externalize factual knowledge to a database, enabling deletion-based unlearning without retraining. Existing evaluations score post-deletion correctness overall but cannot reveal whether a deleted fact survives via residual parametric memory, alternative retrieval routes, or near-neighbor retrieval artifacts. A causal auditing framework fixes the model and varies database state during inference (FULL, DEL-ON, DEL-OFF). Results show near-zero parametric leakage and that surviving correctness is predominantly reconstituted from retrieval-mediated near-neighbor access.","Auditing Forgetting in Limited Memory Language Models  \nArya Raeesi 1 Hanna Roed 1  \narXiv :2607 .00605v 1 [ cs .CL] 1 Jul 2026  \nAbstract  \nLimited Memory Language Models (LMLMs) externalize factual knowledge to a database to enable deletion-based unlearning without retraining. Existing evaluations measure post-deletion correctness in aggregate and cannot tell whether a deleted fact persists through residual parametric memory, alternative retrieval paths, or near-neighbor retrieval artifacts. We propose a causal auditing framework that holds the model fixed and varies the database state at inference time across three interventions: FULL, DEL-ON, and DEL-OFF. The framework decomposes postdeletion behavior into parametric leakage L (f), retrieval-mediated correctness R (f), and a retrieval artifact rate grounded in the inference-time retrieval trace. We apply it to 12 ,228 alias-closure deletions across thirteen databases, including four adversarial topologies (Base, Alias, Noise, Collision) we construct in three domains, and six prompt formulations. Parametric leakage is near zero in every variant and every prompt style: the model rarely returns the deleted answer in the absence of retrieval. The residual that does survive lives in the retrieval graph: retrieval-mediated correctness and the retrieval artifact rate match within rounding everywhere, so post-deletion correctness is, in our audit, predominantly reconstituted from near-neighbor retrieval. This residual ranges from 0.7% on the released LMLM database to 13.6% on the most adversarial variant, and prompt formulation does not independently control how much of a deleted fact survives. These results suggest that, for this class of LMLM and deletion procedure, the unlearning boundary is drawn primarily by the database administrator rather than by the model.  \n*Equal contribution 1University of California, Berkeley, Berkeley, California, United States of America. Correspondence to: Arya Raeesi \u003C[aryaraeesi@berkeley.edu](aryaraeesi@berkeley.edu) >, Hanna Roed \u003C[hanna.roed@berkeley.edu](hanna.roed@berkeley.edu) >.  \n1. Introduction  \nModern language models increasingly rely on hybrid architectures that combine parametric knowledge with external memory. Limited Memory Language Models (LMLMs) are a prominent example of this paradigm, explicitly separating linguistic competence encoded in model parameters from factual knowledge stored in an external database (Zhao et al., 2025) . As illustrated in Figure 1, an LMLM retains the linguistic competence of a standard language model but routes factual recall through an external database rather than holding it in parameters. This design enables deletion-based unlearning, where removing entries from the database is intended to eliminate access to specific facts without requiring retraining (Zhao et al., 2025) . Such capabilities are particularly important for applications involving data governance, privacy, and model editing.  \nFigure 1 . Comparison of a standard retrieval-augmented language model (LLM + RAG) and a LMLM. Both architectures pair a parametric model with an external database, but LMLMs are pretrained to limit the internal storage of factual knowledge, so factual recall is routed through the external store rather than reconstructed from parameters (Zhao et al., 2025) .  \nHowever, it remains unclear whether deletion in these systems truly removes knowledge. Existing evaluations of forgetting typically measure whether a model produces the correct answer before and after deletion, but do not distinguish the underlying mechanism of post-deletion correctness (Zhao et al., 2025) . A model may still answer correctly due to residual parametric memory, alternative retrieval paths, or semantically related matches in the external database. As a result, current metrics cannot determine whether knowledge has been successfully externalized or whether it persists internally in the model.  \nIn this work, we propose a causal auditing framew","cbCaipFLfmYlbcVZ","https://ap.wps.com/l/cbCaipFLfmYlbcVZ","pdf",998651,5,1,17,"English","en",105,"# Abstract\n# Introduction\n# Related Work","[{\"question\":\"What problem does the paper address in deletion-based unlearning for LMLMs?\",\"answer\":\"It addresses the gap in existing evaluations that only measure aggregate post-deletion correctness, without identifying whether deleted knowledge remains through parametric memory, different retrieval paths, or retrieval artifacts.\"},{\"question\":\"How does the proposed causal auditing framework isolate where deleted facts come from?\",\"answer\":\"It holds the model fixed and compares inference under three database interventions: FULL (retrieval on), DEL-ON (deletion with retrieval on), and DEL-OFF (deletion with retrieval disabled), then decomposes outcomes into parametric leakage, retrieval-mediated correctness, and retrieval artifacts.\"},{\"question\":\"What do the experiments conclude about the source of residual knowledge after deletion?\",\"answer\":\"Parametric leakage is near zero across variants and prompt styles, while the remaining post-deletion correctness aligns with retrieval-mediated correctness and retrieval artifact rate, indicating survival mainly through retrieval graph near-neighbor access.\"}]",1784181343,43,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"auditing-forgetting-in-limited-memory-language-models","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/auditing-forgetting-in-limited-memory-language-models/82535/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-23","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the paper address in deletion-based unlearning for LMLMs?","Question",{"text":76,"@type":77},"It addresses the gap in existing evaluations that only measure aggregate post-deletion correctness, without identifying whether deleted knowledge remains through parametric memory, different retrieval paths, or retrieval artifacts.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does the proposed causal auditing framework isolate where deleted facts come from?",{"text":81,"@type":77},"It holds the model fixed and compares inference under three database interventions: FULL (retrieval on), DEL-ON (deletion with retrieval on), and DEL-OFF (deletion with retrieval disabled), then decomposes outcomes into parametric leakage, retrieval-mediated correctness, and retrieval artifacts.",{"name":83,"@type":74,"acceptedAnswer":84},"What do the experiments conclude about the source of residual knowledge after deletion?",{"text":85,"@type":77},"Parametric leakage is near zero across variants and prompt styles, while the remaining post-deletion correctness aligns with retrieval-mediated correctness and retrieval artifact rate, indicating survival mainly through retrieval graph near-neighbor access.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":20,"slug":138},19,"General","general"]