[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121973-en":3,"doc-seo-121973-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121973,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","Human-in-the-loop machine learning - A state of the art - Review and definitions","Researchers define Human-in-the-loop machine learning (HITL-ML) as a set of interaction patterns between humans and machine-learning algorithms. Approaches are categorized by control of the learning process—active learning, interactive machine learning, and machine teaching—while other roles include curriculum learning and explainable AI, where humans need both performance and rationale. The work reviews state-of-the-art techniques, clarifies overlapping and sometimes conflicting terms, and maps boundaries and relationships among methods, including extensions such as Usable and Useful AI.","Human‑in‑the‑loop machine learning: a state ofthe art  \nEduardo Mosqueira‑Rey1 · Elena Hernández‑Pereira1 · David Alonso‑Ríos1 · José Bobes‑Bascarán1 · Ángel Fernández‑Leal1  \nPublished online: 17 August 2022 © The Author(s) 2022  \nAbstract  \nResearchers are defining new types of interactions between humans and machine learning algorithms generically called human-in-the-loop machine learning. Depending on whois in control of the learning process, we can identify: active learning, in which the system remains in control; interactive machine learning, in which there is a closer interaction between users and learning systems; and machine teaching, where human domain experts have control over the learning process. Aside from control, humans can also be involved in the learning process in other ways. In curriculum learning human domain experts try to impose some structure on the examples presented to improve the learning; in explainable AI the focus is on the ability of the model to explain to humans why a given solution was chosen. This collaboration between AI models and humans should not be limited only to the learning process; if we go further, we can see other terms that arise such as Usable and Useful AI. In this paper we review the state of the art of the techniques involved in the new forms of relationship between humans and ML algorithms. Our contribution is not merely listing the different approaches, but to provide definitions clarifying confusing, varied and sometimes contradictory terms; to elucidate and determine the boundaries between the different methods; and to correlate all the techniques searching for the connections and influences between them.  \nKeywords Human-in-the-loop machine learning · Active learning · Interactive machine learning · Machine teaching · Curriculum learning · Explainable AI  \n* Eduardo Mosqueira-Rey [eduardo@udc.es](eduardo@udc.es)  \nElena Hernández-Pereira  \n[elena.hernandez@udc.es](elena.hernandez@udc.es)  \n[David Alonso-R](David Alonso-R)í[os](os)  \n[david.alonso@udc.es](david.alonso@udc.es)  \n[Jos](Jos)é Bobes-Bascarán  \njose.bobes@udc.es  \nÁngel Fernández-Leal  \n[angel.fleal@udc.es](angel.fleal@udc.es)  \n1 Department of Computer Science and Information Technologies, Universidade da Coruña (CITIC), Campus de Elviña, 15071 A Coruña, Spain  \n1 Introduction  \nThere is currently a great demand for machine learning (ML) solutions. This is because the advances that have occurred in recent years around this technology have popularized it and have brought it closer to the general public. But building machine learning systems is a complex process that requires deep knowledge of machine learning techniques.  \nUsually, humans are required at various points in the loop of the machine learning process but following a kind of monolithic conception in which the machine learning algorithm is modeled, built, tested and then offered to the public without further changes.  \nModels that are developed under this scenario might run the risk of not scaling well, becoming static, being hard to evaluate, and degrading their performance due to changes in the context they are deployed into. Also, due to the limitations of the dominating connectionist approach, they usually lack logical reasoning and the possibility of identifying causal relations (Holmberg et al. 2020) .  \nResearchers are defining new types of interactions between humans and machine learning algorithms, which we can group under the umbrella term of Human-in-the-loop machine learning (HITL-ML) (Munro 2020) . The idea is not only to make machine learning more accurate or to obtain the desired accuracy faster, but also to make humans more effective and more efficient.  \nDepending on who is in control of the learning process, we can identify different approaches to HITL-ML (Holmberg et al. 2020):  \n• Active learning (AL) (Settles 2009), in which the system remains in control of the learning process and treats humans as oracles to annotate unlabeled da","cbCaiqQ3UqxFu3RQ","https://ap.wps.com/l/cbCaiqQ3UqxFu3RQ","pdf",2901799,1,50,"English","en",105,"# Introduction\n## Background and motivation\n## Approaches based on control: AL, IML, MT\n## Additional roles: curriculum learning and explainable AI\n## Beyond learning: usability-oriented collaboration","[{\"question\":\"What is Human-in-the-loop machine learning (HITL-ML)?\",\"answer\":\"HITL-ML refers to interaction types between humans and machine-learning algorithms where humans contribute to improving accuracy, efficiency, or understanding. The key difference lies in how humans participate in the learning process.\"},{\"question\":\"How do active learning, interactive machine learning, and machine teaching differ?\",\"answer\":\"Active learning keeps the system in control while humans act as oracles to label data. Interactive machine learning emphasizes closer, incremental user interaction during learning. Machine teaching gives human domain experts control by delimiting knowledge to transfer to the model.\"},{\"question\":\"What roles do curriculum learning and explainable AI play?\",\"answer\":\"Curriculum learning structures the training examples to accelerate and improve learning outcomes. Explainable AI focuses on enabling the model to explain why a solution was chosen, making results more understandable to humans.\"}]","Human-in-the-loop machine learning - A state of the art - Review and definitions | PDF",1785808094,126,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"human-in-the-loop-machine-learning-a-state-of-the-art-review-and-definitions","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/human-in-the-loop-machine-learning-a-state-of-the-art-review-and-definitions/121973/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is Human-in-the-loop machine learning (HITL-ML)?","Question",{"text":75,"@type":76},"HITL-ML refers to interaction types between humans and machine-learning algorithms where humans contribute to improving accuracy, efficiency, or understanding. The key difference lies in how humans participate in the learning process.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do active learning, interactive machine learning, and machine teaching differ?",{"text":80,"@type":76},"Active learning keeps the system in control while humans act as oracles to label data. Interactive machine learning emphasizes closer, incremental user interaction during learning. Machine teaching gives human domain experts control by delimiting knowledge to transfer to the model.",{"name":82,"@type":73,"acceptedAnswer":83},"What roles do curriculum learning and explainable AI play?",{"text":84,"@type":76},"Curriculum learning structures the training examples to accelerate and improve learning outcomes. Explainable AI focuses on enabling the model to explain why a solution was chosen, making results more understandable to humans.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":21,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]