[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117345-en":3,"doc-seo-117345-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117345,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Forgetting in Machine Learning and Beyond - A Survey","This survey investigates the multifaceted nature of forgetting in machine learning, drawing insights from neuroscientific findings that treat forgetting as an adaptive function rather than a flaw. It emphasizes how forgetting can enhance learning, improve generalization, and help prevent overfitting. The work surveys selective forgetting and its model-performance and data-privacy benefits, and it discusses challenges, future directions, and ethical considerations for integrating forgetting mechanisms into learning systems.","arXiv :2405 .20620v1 [ cs .LG] 31 May 2024  \n\"Forgetting\" in Machine Learning and Beyond: A Survey  \nALYSSA SHA, Australian National University, Australia  \nBERNARDO PEREIRA NUNES, Australian National University, Australia ARMIN HALLER, Australian National University, Australia  \nThis survey investigates the multifaceted nature of forgetting in machine learning, drawing insights from neuroscientific research that posits forgetting as an adaptive function rather than a defect, enhancing the learning process and preventing overfitting. This survey focuses on the benefits of forgetting and its applications across various machine learning sub-fields that can help improve model performance and enhance data privacy. Moreover, the paper discusses current challenges, future directions, and ethical considerations regarding the integration of forgetting mechanisms into machine learning models.  \nCCS Concepts: • Applied computing → Computers in other domains; • General and reference → Reliability.  \nAdditional Key Words and Phrases: Forgetting, Survey, Machine Learning, Selective forgetting, Reinforcement Learning, Learning enhancement, Regularisation  \nACM Reference Format:  \nAlyssa Sha, Bernardo Pereira Nunes, and Armin Haller. 2018. \"Forgetting\" in Machine Learning and Beyond: A Survey. 1, 1 (June 2018), 35 pages. [https://doi.org/XXXXXXX.XXXXXXX](https://doi.org/XXXXXXX.XXXXXXX)  \n1 INTRODUCTION  \nHuman brain is a complex system, where forgetting serves as a dynamic nature that allows us to avoid cognitive overload, update information to adapt to changing environments [77], and can potentially enhance our learning capabilities [25] . The advantages of forgetting have been investigated in various research fields, including education, philosophy, ecology and linguistics, where forgetting has been found to contribute significantly to the enhancement of humans’decision-making, creativity, and diversity from multiple perspectives.  \nForgetting, an intrinsic aspect of human memory, does not naturally occur in machines, highlighting a fundamental distinction between humans and artificial systems. In the context of the human brain, overfitting arises when we simply memorise specific examples rather than generalise patterns from them [96] . This narrow focus can cause inflexibility in our thinking and problem-solving abilities, as well as lead to erroneous predictions or assumptions when confronted with unfamiliar situations. Overfitting is also a challenge in machine learning (ML) [50] . By mimicking the human brain, incorporating a forget-and-relearn function into machines has been proposed to be a powerful paradigm for shaping the learning trajectories of artificial neural networks [269], as not all content in the past is equally important for models to remember [203] .  \nAuthors’ addresses: Alyssa Sha, [alyssa.sha@anu.edu.au](alyssa.sha@anu.edu.au), Australian National University, Canberra, ACT, Australia; Bernardo Pereira Nunes, Bernardo. [Nunes@anu.edu.au](Nunes@anu.edu.au), Australian National University, Canberra, ACT, Australia; Armin Haller, [Armin.Haller@anu.edu.au](Armin.Haller@anu.edu.au), Australian National University, Canberra, ACT, Australia.  \nPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, [requires prior specific permission and/or a fee. Request permissions from permissions@acm.org](requires prior specific permission and/or a fee. Request permissions from permissions@acm.org).  \n© 2018 Association for Computing Machinery.  \nManuscript submitted to ACM  \nManuscript submitted to ACM 1  \n2 Alyssa Sha, ","cbCaia7kLz9xozuB","https://ap.wps.com/l/cbCaia7kLz9xozuB","pdf",2096418,1,35,"English","en",105,"# Introduction\n## Forgetting as an adaptive function\n## Distinction between human memory and machines\n## Overfitting and forget-and-relearn paradigms\n# Types of forgetting\n## Selective forgetting\n## Detrimental forgetting and catastrophic forgetting\n# Scope of the survey\n## Applying selective forgetting across disciplines\n# Research questions\n## Manifestation of forgetting across knowledge areas","[{\"question\":\"What does the survey argue about forgetting in machine learning?\",\"answer\":\"Forgetting is presented as potentially adaptive and beneficial, not merely a defect, and can support better learning and generalization.\"},{\"question\":\"How does selective forgetting help machine learning models?\",\"answer\":\"Selective forgetting selectively ignores irrelevant or noisy data, improving memory utilization, generalization, adaptability, and supporting privacy compliance.\"},{\"question\":\"What is detrimental forgetting and why is it important to continuous learning?\",\"answer\":\"Detrimental forgetting, known as catastrophic forgetting, occurs when a model loses previously learned information after learning new information, harming performance on earlier tasks in continuous settings.\"}]","Forgetting in Machine Learning and Beyond - A Survey | PDF",1785675293,88,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"forgetting-in-machine-learning-and-beyond-a-survey","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/forgetting-in-machine-learning-and-beyond-a-survey/117345/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What does the survey argue about forgetting in machine learning?","Question",{"text":75,"@type":76},"Forgetting is presented as potentially adaptive and beneficial, not merely a defect, and can support better learning and generalization.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does selective forgetting help machine learning models?",{"text":80,"@type":76},"Selective forgetting selectively ignores irrelevant or noisy data, improving memory utilization, generalization, adaptability, and supporting privacy compliance.",{"name":82,"@type":73,"acceptedAnswer":83},"What is detrimental forgetting and why is it important to continuous learning?",{"text":84,"@type":76},"Detrimental forgetting, known as catastrophic forgetting, occurs when a model loses previously learned information after learning new information, harming performance on earlier tasks in continuous settings.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]