[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84460-en":3,"doc-seo-84460-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84460,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782698725881665579",8,"Research & Report","BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearning","Large Language Models increasingly shape online content, making it crucial to remove targeted information from deployed models via LLM unlearning. A core obstacle is sample-wise imbalance in the forget set: individual samples vary widely in unlearning difficulty, causing asynchronous forgetting where some knowledge stays insufficiently erased while other parts are over-forgotten. BalDRO models unlearning through an inner–outer minimax process, learning worst-case distributions that upweight hard samples and updating parameters accordingly. Two efficient variants—BalDRO-G and BalDRO-DV—improve both forgetting quality and utility on TOFU and MUSE.","BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearning  \nPengyang Shao  \nNational University of Singapore Singapore  \nYonghui Yang  \nNational University of Singapore Singapore  \nNaixin Zhai  \nUniversity of Science and Technology of China Hefei, China  \nFengbin Zhu∗ [zhfengbin@gmail.com](zhfengbin@gmail.com)[ ](zhfengbin@gmail.com)National University of Singapore Singapore  \nLei Chen  \nUniversity of Science and Technology of China Hefei, China  \nXun Yang∗ [xyang21@ustc.edu.cn](xyang21@ustc.edu.cn)[ ](xyang21@ustc.edu.cn)University of Science and Technology of China Hefei, China  \nMeng Wang  \nHefei University of Technology Hefei, China  \narXiv :2601 .09 172v 3 [ cs .LG] 12 Jul 2026  \nAbstract  \nAs Large Language Models (LLMs) increasingly shape online content, how to remove targeted information from well-trained LLMs (also known as LLM unlearning) has become increasingly critical for web governance. A key challenge in LLM unlearning lies in the sample-wise imbalance within the forget set: different samples exhibit widely varying unlearning difficulty, leading to asynchronous forgetting speeds where some knowledge remains insufficiently erased while others become over-forgotten. To address this challenge, we propose BalDRO, a novel and efficient framework for balanced LLM unlearning. BalDRO formulates unlearning as amin–sup process, where the inner process identifies a worst-case data distribution that adaptively emphasizes hard-to-unlearn samples, while the outer process updates model parameters based on the worst-case data distribution. We instantiate this formulation through two efficient variants: BalDRO-G, a discrete GroupDRObased approximation that focuses on high-loss subsets, and BalDRODV, a continuous Donsker–Varadhan dual method that enables smooth, adaptive weighting within standard LLM training pipelines. Extensive experiments on the TOFU and MUSE benchmarks demonstrate the effectiveness of our proposed BalDRO, yielding significant improvements in both forgetting quality and model utility over existing methods. For reproducibility, we have released the code for BalDRO1 .  \nCCS Concepts  \n• Security and privacy → Privacy protections; • Computing methodologies → Natural language processing; Machine learning.  \n∗ Corresponding authors.  \n1[https://github.com/nxZhai/BalDRO](https://github.com/nxZhai/BalDRO)  \nThis work is licensed under a Creative Commons Attribution 4 .0 International License. WWW’26, Dubai, United Arab Emirates  \n© 2026 Copyright held by the owner/author(s) .  \nACM ISBN 979-8-4007-2307-0/2026/04  \n[https://doi.org/10.1145/3774904.3792975](https://doi.org/10.1145/3774904.3792975)  \nKeywords  \nLarge Language Models, Machine Unlearning, Trustworthy AI  \nACM Reference Format:  \nPengyang Shao, Naixin Zhai, Lei Chen, Yonghui Yang, Fengbin Zhu, Xun Yang, and Meng Wang. 2026. BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearning. In Proceedings of the ACM Web Conference 2026 (WWW’26), April 13–17, 2026, Dubai, United Arab Emirates. ACM, New York, NY, USA, 11 pages. [https:](https:)//[doi.org/10.1145/3774904.3792975](doi.org/10.1145/3774904.3792975)  \n1 Introduction  \nAs Large Language Models (LLMs) become increasingly embedded in web platforms and services [1, 8, 18, 41, 43], ensuring that these models behave in a trustworthy and responsible manner has become essential for maintaining the reliability of web-based information ecosystems [27, 39, 58, 59]. A key aspect of achieving such reliability is the ability to remove outdated, incorrect, or privacy-sensitive knowledge from LLMs so that their behavior remains aligned with public values [7, 40, 47], and safety requirements [6, 21] . This need is further reinforced by legal frameworks such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) , which mandate the “right to be forgotten” and require machine learning systems to support","cbCainDqWTWbIu92","https://ap.wps.com/l/cbCainDqWTWbIu92","pdf",1393369,1,11,"English","en",105,"# Introduction\n## Sample-wise imbalance in LLM unlearning\n## Distributionally robust optimization formulation\n## Proposed BalDRO variants and experimental evaluation","[{\"question\":\"What problem does BalDRO address in LLM unlearning?\",\"answer\":\"BalDRO targets sample-wise imbalance in the forget set, where different samples have very different unlearning difficulty and thus forget at different rates.\"},{\"question\":\"How does BalDRO formulate the unlearning objective?\",\"answer\":\"BalDRO formulates unlearning as an inner–outer minimax (worst-case distribution) process that adaptively emphasizes hard-to-unlearn samples for parameter updates.\"},{\"question\":\"What are BalDRO-G and BalDRO-DV?\",\"answer\":\"BalDRO-G is a discrete group-DRO approximation focusing on high-loss subsets, while BalDRO-DV uses a continuous Donsker–Varadhan dual approach to enable smooth adaptive weighting within standard training pipelines.\"}]",1784195774,28,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"baldro-a-distributionally-robust-optimization-based-framework-for-large-language-model-unlearning","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/baldro-a-distributionally-robust-optimization-based-framework-for-large-language-model-unlearning/84460/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does BalDRO address in LLM unlearning?","Question",{"text":75,"@type":76},"BalDRO targets sample-wise imbalance in the forget set, where different samples have very different unlearning difficulty and thus forget at different rates.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does BalDRO formulate the unlearning objective?",{"text":80,"@type":76},"BalDRO formulates unlearning as an inner–outer minimax (worst-case distribution) process that adaptively emphasizes hard-to-unlearn samples for parameter updates.",{"name":82,"@type":73,"acceptedAnswer":83},"What are BalDRO-G and BalDRO-DV?",{"text":84,"@type":76},"BalDRO-G is a discrete group-DRO approximation focusing on high-loss subsets, while BalDRO-DV uses a continuous Donsker–Varadhan dual approach to enable smooth adaptive weighting within standard training pipelines.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]