[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-1-en-105":3,"doc-seo-204912-105":53,"doc-detail-204912-en":126},{"code":4,"msg":5,"data":6},0,"success",[7,14,19,24,29,34,39,44,49],{"id":8,"doc_module":9,"doc_module_name":10,"category_name":11,"show_sort_weight":12,"slug":13},11,1,"Template","Presentations",90,"presentations",{"id":15,"doc_module":9,"doc_module_name":10,"category_name":16,"show_sort_weight":17,"slug":18},12,"Resumes",80,"resumes",{"id":20,"doc_module":9,"doc_module_name":10,"category_name":21,"show_sort_weight":22,"slug":23},14,"Invoices",70,"invoices",{"id":25,"doc_module":9,"doc_module_name":10,"category_name":26,"show_sort_weight":27,"slug":28},15,"Posters",60,"posters",{"id":30,"doc_module":9,"doc_module_name":10,"category_name":31,"show_sort_weight":32,"slug":33},16,"Social Media",50,"social-media",{"id":35,"doc_module":9,"doc_module_name":10,"category_name":36,"show_sort_weight":37,"slug":38},17,"Forms",40,"forms",{"id":40,"doc_module":9,"doc_module_name":10,"category_name":41,"show_sort_weight":42,"slug":43},18,"Letters",30,"letters",{"id":45,"doc_module":9,"doc_module_name":10,"category_name":46,"show_sort_weight":47,"slug":48},21,"Paper Templates",5,"papers-templates",{"id":50,"doc_module":9,"doc_module_name":10,"category_name":51,"show_sort_weight":4,"slug":52},158,"General","general-158",{"code":4,"msg":54,"data":55},"ok",{"site_id":56,"language":57,"slug":58,"title":59,"keywords":60,"description":61,"schema_data":62,"social_meta":119,"head_meta":121,"extra_data":123,"updated_unix":125},105,"en","a-stitch-in-time-autonomous-model-management-via-reinforcement-learning-concept-drift-model-retraining","A Stitch in Time - Autonomous Model Management via Reinforcement Learning - Concept Drift Model Retraining","","Concept drift poses a persistent threat to maintaining high-performing learned models in autonomous systems when data dynamics change over time. Instead of relying on outdated models, discarding history, or retraining with limited new data, the paper frames model management as a sequential decision-making problem. It formalizes the response as a Markov Decision Process and uses reinforcement learning to learn a robust policy for updating models. Results on one synthetic and two real-world datasets show promising performance and a trade-off between learning costs and long-term performance.",{"@graph":63,"@context":118},[64,80,101],{"@type":65,"itemListElement":66},"BreadcrumbList",[67,71,74,77],{"item":68,"name":69,"@type":70,"position":9},"https://docshare.wps.com","Home","ListItem",{"item":72,"name":10,"@type":70,"position":73},"https://docshare.wps.com/template/",2,{"item":75,"name":11,"@type":70,"position":76},"https://docshare.wps.com/template/presentations/",3,{"item":78,"name":59,"@type":70,"position":79},"https://docshare.wps.com/template/a-stitch-in-time-autonomous-model-management-via-reinforcement-learning-concept-drift-model-retraining/204912/",4,{"url":78,"name":59,"@type":81,"image":82,"author":87,"headline":59,"publisher":90,"fileFormat":93,"inLanguage":57,"description":61,"dateModified":94,"datePublished":95,"encodingFormat":93,"isAccessibleForFree":96,"interactionStatistic":97},"DigitalDocument",{"url":83,"@type":84,"width":85,"height":86},"https://docshare.wps.com/thumbnails/a-stitch-in-time-autonomous-model-management-via-reinforcement-learning-concept-drift-model-retraining/204912.png","ImageObject",442,249,{"name":88,"@type":89},"Lute","Person",{"url":68,"name":91,"@type":92},"DocShare","Organization","application/pdf","2026-10-28","2026-09-05",true,{"@type":98,"interactionType":99,"userInteractionCount":8},"InteractionCounter",{"@type":100},"ViewAction",{"@type":102,"mainEntity":103},"FAQPage",[104,110,114],{"name":105,"@type":106,"acceptedAnswer":107},"What challenge does the paper address in autonomous systems?","Question",{"text":108,"@type":109},"The paper addresses concept drift, meaning sudden or gradual changes in the properties of data that can degrade learned model performance over time.","Answer",{"name":111,"@type":106,"acceptedAnswer":112},"How does the paper model autonomous model management?",{"text":113,"@type":109},"It formulates the model retraining response to concept drift as a sequential decision-making problem and formally frames it as a Markov Decision Process.",{"name":115,"@type":106,"acceptedAnswer":116},"What are the reported results and datasets?",{"text":117,"@type":109},"The reinforcement learning approach shows promising results on one synthetic dataset and two real-world datasets, demonstrating effectiveness beyond baseline policies.","https://schema.org",{"og:url":78,"og:type":120,"og:title":59,"og:site_name":91,"og:description":61},"article",{"robots":122,"canonical":78},"index,follow",{"doc_id":124,"site_id":56},204912,1791518703,{"code":4,"msg":5,"data":127},{"doc_id":124,"user_id":128,"nickname":88,"user_avatar":129,"doc_module":9,"category_id":8,"category_name":11,"doc_title":59,"doc_description":61,"doc_content":130,"file_id":131,"file_url":132,"file_type":133,"file_size":134,"view_count":8,"is_deleted":4,"is_public":9,"is_downloadable":9,"audit_status":9,"page_count":135,"language":136,"language_code":57,"site_id":56,"html_lang":57,"table_of_contents":137,"faqs":138,"seo_title":139,"seo_description":61,"update_tm":140,"read_time":76},137454149569,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","A Stitch in Time-Autonomous Model Management via  \nReinforcement Learning  \nElad Liebman  \nThe University of Texas at Austin [eladlieb@cs.utexas.edu](eladlieb@cs.utexas.edu)  \nEric Zavesky  \nAT&T Research [ezavesky@research.att.com](ezavesky@research.att.com)  \nPeter Stone  \nThe University of Texas at Austin [pstone@cs.utexas.edu](pstone@cs.utexas.edu)  \nABSTRACT  \nConcept drift-a change, either sudden or gradual, in the underlying properties of data-is one of the most prevalent challenges to maintaining high-performing learned models over time in autonomous systems. In the face of concept drift, one can hope that the old model is sufficiently representative of the new data despite the concept drift, one can discard the old data and retrain a new model with (often limited) new data, or one can use transfer learning methods to combine the old data with the new to create an updated model. Which of these three options is chosen affects not only near-term decisions, but also future needs to transfer or retrain. In this paper, we thus model response to concept drift as a sequential decision making problem and formally frame it as a Markov Decision Process. Our reinforcement learning approach to the problem shows promising results on one synthetic and two real-world datasets.  \nKEYWORDS  \nReinforcement Learning; Model Retraining; Concept Drift  \nACM Reference Format:  \nElad Liebman, Eric Zavesky, and Peter Stone. 2018. A Stitch in Time Autonomous Model Management via Reinforcement Learning. In Proc. of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018), Stockholm, Sweden, July 10–15, 2018, IFAAMAS, 9 pages.  \n1 INTRODUCTION  \nAs automation grows, more and more industry control systems around us make decisions autonomously, ranging from image understanding [19] to movie recommendation systems [9] and network and service virtualization [3]. Underneath their hoods, many of these systems rely on models. These models can be descriptive (capturing the properties of data) or predictive (using known data to predict other, latent properties) . However, such systems are often susceptible to the nonstationary, ever-changing dynamics of data in the real world. In recommendation systems, tastes and fashion change. In climate prediction, the properties of the environment change constantly. These changes, either gradual or abrupt, are commonly referred to as concept drift. Such shifts in the feature distribution and underlying label correspondence constitute a significant challenge to learning systems. In the face of concept drift, we consider the problem of how to generically and adaptively adjust models to mitigate the risks of drift. We refer to this challenge as model retraining, or model management. In this paper we propose a novel reinforcement learning (RL) approach, framing the model  \nProc. of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018), M. Dastani, G. Sukthankar, E. André, S. Koenig (eds.), July 10–15, 2018, Stockholm, Sweden. © 2018 International Foundation for Autonomous Agents and Multiagent Systems ([www.ifaamas.org](www.ifaamas.org)). All rights reserved.  \nretraining problem as a sequential decision making task, and harnessing ideas from the RL literature to learn a robust policy for model update.  \nThis paper makes two main contributions. First, we formally frame the autonomous model retraining problem as a reinforcement learning task. To our knowledge, we are the first to consider the benefits of this approach. Second, we propose a robust and general framework for learning an autonomous model adaptation policy that balances learning costs with overall performance and is completely decoupled from the underlying models. We show empirically that this approach is superior to other baseline metalearning policies, including two competitive drift detection methods (DDM[7] and HDDM[6]) . We also show how our approach can be scaled to more complex, mult","cbCaitjKGp4Ofi6L","https://ap.wps.com/l/cbCaitjKGp4Ofi6L","pdf",2294347,9,"English","# Introduction\n## Concept drift and model retraining\n## Contributions and approach\n# Related Work\n## Concept drift methods\n## Online learning and lifelong learning\n## Transfer learning connections","[{\"question\":\"What challenge does the paper address in autonomous systems?\",\"answer\":\"The paper addresses concept drift, meaning sudden or gradual changes in the properties of data that can degrade learned model performance over time.\"},{\"question\":\"How does the paper model autonomous model management?\",\"answer\":\"It formulates the model retraining response to concept drift as a sequential decision-making problem and formally frames it as a Markov Decision Process.\"},{\"question\":\"What are the reported results and datasets?\",\"answer\":\"The reinforcement learning approach shows promising results on one synthetic dataset and two real-world datasets, demonstrating effectiveness beyond baseline policies.\"}]","A Stitch in Time - Autonomous Model Management via Reinforcement Learning - Concept Drift Model Retraining | PDF",1788576462]