[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119093-en":3,"doc-seo-119093-105":30,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119093,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","An Interactive Human-Machine Learning Interface for Collecting and Learning from Complex Annotations - Paper","Human-Computer Interaction can enhance machine learning by improving model performance, accelerating learning, and strengthening user confidence. This work removes the assumption that annotators must conform to rigid constraints of traditional labels by collecting more flexible supervision signals. The proposed human-machine learning interface supports binary classification and lets annotators use counterfactual examples alongside standard labels as dataset annotations. The paper also reviews challenges for extending the approach.","An Interactive Human-Machine Learning Interface for Collecting and Learning  \nfrom Complex Annotations  \nJonathan Erskine 1 , Matt Clifford 1 , Alexander Hepburn 1 , Ral Santos-Rodrguez 1  \n1University of Bristol  \n{jonathan.erskine, matt.clifford, alex.hepburn, [enrsr](enrsr}@brisol.ac.uk)[}](enrsr}@brisol.ac.uk)[@brisol.ac.uk](enrsr}@brisol.ac.uk)  \narXiv :2403 . 19339v1 [ cs .LG] 28 Mar 2024  \nAbstract  \nHuman-Computer Interaction has been shown to lead to improvements in machine learning systems by boosting model performance, accelerating learning and building user confidence. In this work, we aim to alleviate the expectation that human annotators adapt to the constraints imposed by traditional labels by allowing for extra flexibility in the form that supervision information is collected. For this, we propose a human-machine learning interface for binary classification tasks which enables human annotators to utilise counterfactual examples to complement standard binary labels as annotations for adataset. Finally we discuss the challenges in future extensions of this work.  \n1 Introduction  \nWhile the field of Artificial intelligence (AI) research has undoubtedly made remarkable progress in recent years, it is important to recognize that this expansion does not necessarily imply a proportionate step towards a form of AI that has ability to understand, learn, and apply knowledge across various domains. Instead, we are witnessing the proliferation of specialized tools; each designed to tackle specific tasks with remarkable proficiency, often with a high associated cost.  \nFor example, Large Language Models (LLMs) excel in their designated domains [Wei et al., 2022], but their exceptional performance is often reliant on the availability of large datasets [Devlin et al., 2019] and their capabilities are often limited when confronted with unfamiliar or unforeseen challenges [Collins et al., 2022] . In this paper we ask if the reliance on large amounts of data and poor model generalisation can be alleviated through human-machine interaction, and propose an interface that enables human annotators to evaluate model behaviour and increase the complexity of individual annotations where required.  \nMany recent approaches in human-in-the-loop (HITL) machine learning have focused on providing post-hoc explanations for the decisions generated by complex machine learning models [Wang et al., 2021; Kurzendorfer et al., 2017] . By integrating these explanations into the pipeline, researchers aim to empower humans to detect the presence of spurious correlations. Such correlations often imply biases present in  \nthe training data, which in turn presents a need for more intelligent data selection. Active learning typically engages humans in the annotation process to strategically choose samples that will most effectively improve the model’s performance [Ren et al., 2022], usually selected to reduce a form of uncertainty [Prince, 2004], mitigating the need for large datasets. However, solely relying on labels for supervision information may be too inflexible for some tasks and may under-utilise the skills of the annotators. The field of Machine Teaching provides flexibility by allowing humans to select groups of samples that effectively describe concepts and patterns [Zhu, 2015] . Recent work combines active learning and machine teaching to automatically generate the minimum viable dataset to learn a concept, and then apply an active learning approach to refine this initial prototype [MosqueiraRey et al., 2021] .  \nHuman involvement in the learning pipeline is not only restricted to data selection. Humans can provide feedback on the explanations produced by models e.g. by adjusting decision boundaries for a 3D segmentation task [Kurzendorfer et al., 2017] according to human input, or to augment training examples. Kaushik et al. create a new dataset of counterfactual pairs [Kaushik et al., 2020]; a counterfactual is usually defined as a point be","cbCaiaFuI1NXGtWR","https://ap.wps.com/l/cbCaiaFuI1NXGtWR","pdf",462162,1,4,"English","en",105,"# Abstract\n# 1 Introduction\n## Human-in-the-loop and related approaches\n## Active learning and machine teaching\n## Counterfactual annotation motivation\n# 2 Methodology\n## Learning task\n## Interface and learning process","[{\"question\":\"What problem does the proposed interface aim to solve for human annotators?\",\"answer\":\"It reduces the expectation that annotators must adapt to rigid constraints from traditional labels by allowing additional flexibility in how supervision information is collected.\"},{\"question\":\"How does the interface incorporate counterfactuals in binary classification?\",\"answer\":\"It enables human annotators to use counterfactual examples to complement standard binary labels as annotations for the dataset.\"},{\"question\":\"What kind of learning task is used to demonstrate the approach?\",\"answer\":\"The paper demonstrates the method on a simple synthetic, two-dimensional binary classification setup with separable clusters.\"}]","An Interactive Human-Machine Learning Interface for Collecting and Learning from Complex Annotations - Paper | PDF",1785722328,10,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":28},"an-interactive-human-machine-learning-interface-for-collecting-and-learning-from-complex-annotations-paper","",{"@graph":36,"@context":84},[37,53,67],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":21},"https://docshare.wps.com/document/an-interactive-human-machine-learning-interface-for-collecting-and-learning-from-complex-annotations-paper/119093/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the proposed interface aim to solve for human annotators?","Question",{"text":74,"@type":75},"It reduces the expectation that annotators must adapt to rigid constraints from traditional labels by allowing additional flexibility in how supervision information is collected.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does the interface incorporate counterfactuals in binary classification?",{"text":79,"@type":75},"It enables human annotators to use counterfactual examples to complement standard binary labels as annotations for the dataset.",{"name":81,"@type":72,"acceptedAnswer":82},"What kind of learning task is used to demonstrate the approach?",{"text":83,"@type":75},"The paper demonstrates the method on a simple synthetic, two-dimensional binary classification setup with separable clusters.","https://schema.org",{"og:url":52,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":46,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":29,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":29,"slug":132},"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]