[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124102-en":3,"doc-seo-124102-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124102,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems - Preprint","Widespread deployment of large-scale machine learning systems requires understanding long-term effects on their environment, including loss of trustworthiness, bias amplification, and violations of AI safety requirements. The paper introduces a repeated learning process capturing phenomena from unintended hidden feedback loops, where the environment state becomes causally dependent on the learner and breaks standard data-distribution assumptions. It presents a dynamical systems model and proves limiting probability distributions for positive and negative feedback-loop modes, validated by computational experiments on synthetic datasets.","A MATHEMATICAL MODEL OF THE HIDDEN FEEDBACK LOOP EFFECT IN MACHINE LEARNING SYSTEMS  \nA PREPRINT  \narXiv :2405 .02726v1 [ cs .LG] 4 May 2024  \nAndrey Veprikov  \nDepartment of Intelligent Systems MIPT Dolgoprudny, Russia [veprikov.as@phystech.edu](veprikov.as@phystech.edu)  \nAlexander Afanasyev  \nIITP Moscow, Russia [apa@isa.ru](apa@isa.ru)  \nAnton Khritankov  \nHSE University, MIPT Moscow, Russia [akhritankov@hse.ru](akhritankov@hse.ru)  \nABSTRACT  \nWidespread deployment of societal-scale machine learning systems necessitates a thorough understanding of the resulting long-term effects these systems have on their environment, including loss of trustworthiness, bias amplification, and violation of AI safety requirements. We introduce a repeated learning process to jointly describe several phenomena attributed to unintended hidden feedback loops, such as error amplification, induced concept drift, echo chambers and others. The process comprises the entire cycle of obtaining the data, training the predictive model, and delivering predictions to end-users within a single mathematical model. A distinctive feature of such repeated learning setting is that the state of the environment becomes causally dependent on the learner itself over time, thus violating the usual assumptions about the data distribution. We present a novel dynamical systems model of the repeated learning process and prove the limiting set of probability distributions for positive and negative feedback loop modes of the system operation. We conduct a series of computational experiments using an exemplary supervised learning problem on two synthetic data sets. The results of the experiments correspond to the theoretical predictions derived from the dynamical model. Our results demonstrate the feasibility of the proposed approach for studying the repeated learning processes in machine learning systems and open a range of opportunities for further research in the area.  \nKeywords machine learning · repeated learning · hidden feedback loop · dynamical systems · concept drift  \n1 Introduction  \nSocietal-scale machine learning and decision making systems are, by definition, intended to have a major impact on society as a whole. Recent analysis (CAIS, 2023) presents a wide range of potential problems and areas of concern associated with such systems (Suresh et al., 2020) . Addressing these challenges and various aspects of engineering trustworthy systems (Li et al., 2023) requires different methods for designing machine learning and artificial intelligence systems, combining formal mathematical modelling, data-driven engineering methods, long-term risk analysis (Sifakisand Harel, 2023; Pei et al., 2022; He et al., 2021) . One of the key quality attributes of trustworthy ML systems (Serban et al., 2021; Toreini et al., 2020; Siebert et al., 2020) and socially responsible AI (SRA) algorithms (Cheng et al., 2021) systems is their ability to behave in a way that users expect without any unintended side-effects.  \nA repeated machine learning process describes a situation in a machine learning system where the input data to a learning algorithm may depend in part on the previous predictions made by the system.  \nMachine learning methods usually take specific assumptions about the data, such as data has to be i.i.d., or stationary with white noise, or the environment the agent operates in remains the same, or there is a data drift independent from the learning agent. A distinctive feature of the repeated learning setting—the one that justifies introducing the name—is that the state of the environment in the process becomes causally dependent on the learning algorithm and the predictions given.  \nWhen there is a high automation bias, that is, when the use of predictions is high and adherence to them is tight, aso-called positive feedback loop occurs (Khritankov, 2021) . As a result of the loop, the learning algorithm is repeatedly applied to the data containing previous prediction","cbCainw9w3amQ8LT","https://ap.wps.com/l/cbCainw9w3amQ8LT","pdf",5523599,1,21,"English","en",105,"# Introduction\n## Repeated learning and hidden feedback loops\n## Trustworthiness and socially responsible AI\n## Dynamical systems model and mapping of distributions\n## Paper structure and related work","[{\"question\":\"What is the hidden feedback loop effect in machine learning systems?\",\"answer\":\"It arises when the learning process repeatedly trains on data that depends on earlier predictions, creating unintended shifts in input and prediction distributions over time.\"},{\"question\":\"How does the paper model repeated learning mathematically?\",\"answer\":\"It defines a set of probability density functions and a mapping that transforms the current data distribution into the next one through sampling, model updating, and user-facing prediction actions.\"},{\"question\":\"What results does the paper prove and how are they tested?\",\"answer\":\"It proves limiting sets of probability distributions for positive and negative feedback-loop modes, and validates these theoretical predictions using computational experiments on supervised learning with two synthetic datasets.\"}]","A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems - Preprint | PDF",1785820373,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"a-mathematical-model-of-the-hidden-feedback-loop-effect-in-machine-learning-systems-preprint","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-mathematical-model-of-the-hidden-feedback-loop-effect-in-machine-learning-systems-preprint/124102/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the hidden feedback loop effect in machine learning systems?","Question",{"text":75,"@type":76},"It arises when the learning process repeatedly trains on data that depends on earlier predictions, creating unintended shifts in input and prediction distributions over time.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper model repeated learning mathematically?",{"text":80,"@type":76},"It defines a set of probability density functions and a mapping that transforms the current data distribution into the next one through sampling, model updating, and user-facing prediction actions.",{"name":82,"@type":73,"acceptedAnswer":83},"What results does the paper prove and how are they tested?",{"text":84,"@type":76},"It proves limiting sets of probability distributions for positive and negative feedback-loop modes, and validates these theoretical predictions using computational experiments on supervised learning with two synthetic datasets.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]