[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121897-en":3,"doc-seo-121897-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121897,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","Negotiated Representations to Prevent Forgetting in Machine Learning Applications","Catastrophic forgetting undermines continual learning in neural networks as training on a new task updates weights and representations, often overwriting knowledge needed for earlier tasks. The paper proposes a negotiated-representations method that preserves prior task knowledge while still enabling effective learning of incoming information. Experiments on benchmark splits—Split MNIST, Split CIFAR-10, Split Fashion MNIST, and Split CIFAR-100—simulate sequential non-overlapping tasks. Results demonstrate improved balance between retention and adaptation, addressing forgetting in continual learning settings.","NEGOTIATED REPRESENTATIONS TO PREVENT FORGETTING IN MACHINE LEARNING APPLICATIONS ∗  \narXiv :2312 .00237v1 [ cs .LG] 30 Nov 2023  \nNuri Korhan  \nIstanbul Technical University Maslak, Istanbul, Turkey [korhan@itu.edu.tr](korhan@itu.edu.tr)  \nCeren Öner  \nIstanbul Technical University Maslak, Istanbul, Turkey [csalkin@itu.edu.tr](csalkin@itu.edu.tr)  \n1 Abstract  \nCatastrophic forgetting is a significant challenge in the field of machine learning, particularly in neural networks. When a neural network learns to perform well on a new task, it often forgets its previously acquired knowledge or experiences. This phenomenon occurs because the network adjusts its weights and connections to minimize the loss on the new task, which can inadvertently overwrite or disrupt the representations that were crucial for the previous tasks. As a result, the network’s performance on earlier tasks deteriorates, limiting its ability to learn and adapt to a sequence of tasks. In this paper, we propose a novel method for preventing catastrophic forgetting in machine learning applications, specifically focusing on neural networks. Our approach aims to preserve the network’s knowledge across multiple tasks while still allowing it to learn new information effectively. We demonstrate the effectiveness of our method by conducting experiments on various benchmark datasets, including Split MNIST, Split CIFAR-10, Split Fashion MNIST, and Split CIFAR-100 . These datasets are created by dividing the original datasets into separate, non-overlapping tasks, simulating a continual learning scenario where the model needs to learn multiple tasks sequentially without forgetting the previous ones.  \nOur proposed method tackles the catastrophic forgetting problem by incorporating negotiated representations into the learning process, which allows the model to maintain a balance between retaining past experiences and adapting to new tasks. By evaluating our method on these challenging datasets, we aim to showcase its potential for addressing catastrophic forgetting and improving the performance of neural networks in continual learning settings.  \n2 Introduction  \nThe problem of catastrophic forgetting (the loss or disruption of previously learned information when new information is learned) in neural networks is commonly faced in deep learning tasks due to selected optimizers that are relied on Gradient Descent algorithm. [1, 2, 3] . Decreasing the number of epochs and exploring rehearsal mechanisms (theretraining of some of the previously learned information as the new information is added) is critical for extraction of a potential solution.This point comes from the idea of the “stability / plasticity dilemma” [4, 5], and emphasis on the representations developed by neural networks should be plastic enough to change to adapt to changing environmentsand learn new things, but stable enough so that important information is preserved over time. The dilemma is that while both are desirable properties, the requirements of stability and plasticity are in conflict. Stability depends on preserving the structure of representations, plasticity depends on altering it. An appropriate balance is difficult to achieve.  \nWhile stability / plasticity issues are very general, the term “catastrophic forgetting” has tended to be associated with a specific class of networks, namely static networks employing supervised learning. This broad class includes probably the majority of commonly used and applied networks (such as the very influential back-propagation family and Hopfield nets) . Other types of network – for example dynamic / “constructive” networks and unsupervised networks – are not necessarily prone to catastrophic forgetting as it is typically described in this context. Dynamic networks are those that use learning mechanisms where the number of units and connections in the network grows or shrinks in response to the  \n∗  Citation: GitHub Link [https://github.com/nurikorhan/Negotia","cbCaigWHKUtb6x2o","https://ap.wps.com/l/cbCaigWHKUtb6x2o","pdf",771988,1,19,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"什么是神经网络中的灾难性遗忘？\",\"answer\":\"灾难性遗忘指模型在学习新信息后，之前已学到的知识或表征会被覆盖或破坏，导致旧任务表现下降。\"},{\"question\":\"作者提出的核心方法是什么？\",\"answer\":\"作者提出在学习过程中引入“negotiated representations”，在保留过去经验与适应新任务之间实现平衡，从而缓解灾难性遗忘。\"},{\"question\":\"实验使用了哪些基准数据集来验证方法？\",\"answer\":\"实验在多个Split数据集上进行，包括 Split MNIST、Split CIFAR-10、Split Fashion MNIST 和 Split CIFAR-100，这些数据集通过将原始数据划分为互不重叠的连续任务来模拟持续学习场景。\"}]","Negotiated Representations to Prevent Forgetting in Machine Learning Applications | PDF",1785807635,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"negotiated-representations-to-prevent-forgetting-in-machine-learning-applications","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/negotiated-representations-to-prevent-forgetting-in-machine-learning-applications/121897/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"什么是神经网络中的灾难性遗忘？","Question",{"text":75,"@type":76},"灾难性遗忘指模型在学习新信息后，之前已学到的知识或表征会被覆盖或破坏，导致旧任务表现下降。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"作者提出的核心方法是什么？",{"text":80,"@type":76},"作者提出在学习过程中引入“negotiated representations”，在保留过去经验与适应新任务之间实现平衡，从而缓解灾难性遗忘。",{"name":82,"@type":73,"acceptedAnswer":83},"实验使用了哪些基准数据集来验证方法？",{"text":84,"@type":76},"实验在多个Split数据集上进行，包括 Split MNIST、Split CIFAR-10、Split Fashion MNIST 和 Split CIFAR-100，这些数据集通过将原始数据划分为互不重叠的连续任务来模拟持续学习场景。","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]