[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82267-en":3,"doc-seo-82267-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82267,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","Forget Narrowly Retain Broadly Unlearning as an Asymmetric Generalization Problem","Machine unlearning in large language models targets the removal of specific knowledge while preserving all other capabilities, which is essential for privacy and safety. Existing evaluation benchmarks fail to reliably verify two complementary errors: under-forgetting, where forgotten facts reappear under paraphrased or indirect queries, and over-forgetting, where unrelated knowledge is degraded. Both issues stem from an asymmetric generalization problem between a finite forget set and an implicitly defined retain set. The work introduces SUITE, a protocol and training corpus that captures forget–retain structure for real-world factual domains, improving methods and enabling JensUn++ for better trade-offs.","arXiv :2607 .09236v 1 [ cs .LG] 10 Jul 2026  \nForget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem  \nAmit Peleg Naman Deep Singh Naama Pearl  \nBibhabasu Mohapatra Matthias Hein  \nTübingen AI Center, University of Tübingen  \nAbstract  \nMachine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety. Yet existing benchmarks measure it unreliably. They miss knowledge that resurfaces under paraphrased or indirect queries, a failure we call under-forgetting, and lack the semantic, syntactic, and lexical probes needed to verify that unrelated knowledge is preserved, a failure we call over-forgetting. Both failures reflect an asymmetric generalization problem. Forget evaluation must cover diverse query formulations of the same target facts, testing whether forgetting holds beyond exact training prompts. Retain evaluation must probe a far larger and implicitly defined set, namely every fact disjoint from the forget target. The retain set thus defines the effective forget set, yet current datasets provide no fine-grained annotation of this forget-retain boundary. We address this with SUITE, an evaluation protocol and training corpus that captures forget-retain structure for real-world factual domains. Methods trained on SUITE improve substantially, showing that training data is as important as algorithmic design. Building on the obtained insights, we introduce JensUn++, an unlearning algorithm that achieves the best forget-retain utility trade-off across three LLMs, in both sequential and joint unlearning settings. Code and datasets are available at [https://amitpeleg.github.io/forget-narrowly-retain-broadly](https://amitpeleg.github.io/forget-narrowly-retain-broadly).  \n1 Introduction  \nAlthough several benchmarks evaluate unlearning in LLMs [19, 42, 7, 14, 30, 21, 36], one fundamental question remains open: what does it mean to forget a fact while retaining knowledge of everything else? We argue that forgetting and retaining are governed by an asymmetric generalization problem. The forget set is finite, but successful forgetting must generalize intensively across all query formulations of the target facts: paraphrases [28, 43], indirect queries, and latent inferential dependencies on correlated facts the model still retains [48, 3, 20, 2, 13, 25] . When this fails, target information resurfaces under alternative phrasings [43, 44] or indirect queries-a failure mode we call under-forgetting. The retain scope, by contrast, cannot be enumerated: it spans “everything else” the model knows, and must generalize extensively across a much larger and only implicitly defined set-every fact disjoint from the forget set. Unlearning that ripples outward into semantically related concepts of the forget facts produces over-forgetting [37, 3, 44] . The retain set thus defines the effective forget set, and any benchmark that does not annotate this forget-retain boundary at a fine-grained level cannot distinguish genuine forgetting from suppression, nor genuine retention from collateral damage. Existing benchmarks [14, 19, 30, 45] largely lack this structure, which is a major obstacle for further progress in unlearning for LLMs.  \nTo address this, we propose SUITE: Selective Unlearning of Isolated Topics and Events, a finegrained evaluation protocol and training corpus for unlearning real-world factual knowledge that captures both sides of the asymmetry. On the forget side, we probe under-forgetting using indirect  \nPreprint.  \nquestions requiring multi-hop reasoning, which are unseen during training (where only direct and reverse questions are used) . Combined with paraphrases of every question type, this tests whether knowledge has been genuinely forgotten or merely suppressed. On the retain side, we probe overforgetting along several axes. We check over-fitting to query form (syntactic structure) and to query entities (lexical structure)","cbCaioKLgnQbUrca","https://ap.wps.com/l/cbCaioKLgnQbUrca","pdf",1416538,1,58,"English","en",105,"# Introduction\n## Asymmetric generalization: under-forgetting and over-forgetting\n## SUITE: selective unlearning evaluation and training corpus\n## JensUn++: unlearning algorithm and improvements","[{\"question\":\"What is the core meaning of “forgetting” and “retaining” in LLM unlearning?\",\"answer\":\"Forgetting removes targeted knowledge while retaining preserves all other capabilities. The challenge is ensuring deletion generalizes across all query formulations of the target facts, not just the exact prompts seen in training.\"},{\"question\":\"What are under-forgetting and over-forgetting?\",\"answer\":\"Under-forgetting is when supposedly forgotten information resurfaces under paraphrased or indirect queries. Over-forgetting is when the unlearning process harms unrelated knowledge, revealed by semantic, syntactic, and lexical probes.\"},{\"question\":\"How does SUITE address the evaluation gap in current unlearning benchmarks?\",\"answer\":\"SUITE evaluates both sides of the asymmetry by probing under-forgetting with indirect multi-hop questions and paraphrases, and probing over-forgetting across graded semantic proximity and syntactic/lexical sensitivity. It also constructs training and evaluation splits independently to test whether performance comes from data rather than only algorithm design.\"}]",1784179274,146,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"forget-narrowly-retain-broadly-unlearning-as-an-asymmetric-generalization-problem","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/forget-narrowly-retain-broadly-unlearning-as-an-asymmetric-generalization-problem/82267/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What is the core meaning of “forgetting” and “retaining” in LLM unlearning?","Question",{"text":74,"@type":75},"Forgetting removes targeted knowledge while retaining preserves all other capabilities. The challenge is ensuring deletion generalizes across all query formulations of the target facts, not just the exact prompts seen in training.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What are under-forgetting and over-forgetting?",{"text":79,"@type":75},"Under-forgetting is when supposedly forgotten information resurfaces under paraphrased or indirect queries. Over-forgetting is when the unlearning process harms unrelated knowledge, revealed by semantic, syntactic, and lexical probes.",{"name":81,"@type":72,"acceptedAnswer":82},"How does SUITE address the evaluation gap in current unlearning benchmarks?",{"text":83,"@type":75},"SUITE evaluates both sides of the asymmetry by probing under-forgetting with indirect multi-hop questions and paraphrases, and probing over-forgetting across graded semantic proximity and syntactic/lexical sensitivity. It also constructs training and evaluation splits independently to test whether performance comes from data rather than only algorithm design.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]