[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120727-en":3,"doc-seo-120727-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120727,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","FeedbackLogs - Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines","Machine learning pipelines increasingly affect many stakeholders, yet current practice lacks a systematic way to record how stakeholder input is gathered and then incorporated into model development. FeedbackLogs are proposed as addenda to existing pipeline documentation, capturing what stakeholders said, how feedback collection occurred, and how each feedback item updates the pipeline over time. The paper formalizes a process for collecting FeedbackLogs and demonstrates use cases as evidence for algorithmic auditing and as a mechanism for documenting stakeholder-driven updates.","arXiv :2307 . 15475v1 [ cs .HC] 28 Jul 2023  \nFeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines  \nMATTHEW BARKER, University of Cambridge, United Kingdom  \nEMMA KALLINA, University of Cambridge, United Kingdom and Responsible AI Institute, United Kingdom DHANANJAY ASHOK, Carnegie Mellon University, USA  \nKATHERINE M. COLLINS, University of Cambridge, United Kingdom ASHLEY CASOVAN, Responsible AI Institute, United Kingdom  \nADRIAN WELLER, University of Cambridge, United Kingdom and The Alan Turing Institute, United Kingdom AMEET TALWALKAR, Carnegie Mellon University, USA  \nVALERIE CHEN∗ , Carnegie Mellon University, USA  \nUMANG BHATT∗ , University of Cambridge, United Kingdom and The Alan Turing Institute, United Kingdom  \nEven though machine learning (ML) pipelines affect an increasing array of stakeholders, there is little work on how input from stakeholders is recorded and incorporated. We propose FeedbackLogs, addenda to existing documentation of ML pipelines, to track the input of multiple stakeholders. Each log records important details about the feedback collection process, the feedback itself, and how the feedback is used to update the ML pipeline. In this paper, we introduce and formalise a process for collecting a FeedbackLog. We also provide concrete use cases where FeedbackLogs can be employed as evidence for algorithmic auditing and as a tool to record updates based on stakeholder feedback.  \nACM Reference Format:  \nMatthew Barker, Emma Kallina, Dhananjay Ashok, Katherine M. Collins, Ashley Casovan, Adrian Weller, Ameet Talwalkar, Valerie Chen, and Umang Bhatt. 2023. FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines. 1, 1 (July 2023), 22 pages. [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn)  \n1 INTRODUCTION  \nStakeholders, who interact with or are affected by machine learning (ML) models, should be involved in the model development process [2, 22, 27] . Their unique perspectives, however, may not be adequately accounted for by practitioners, who are responsible for developing and deploying models (e.g., ML engineers, data scientists, UX researchers) [16] . We notice a gap in the existing literature around documenting how stakeholder input was collected and incorporated in the ML pipeline, which we define as a model’s end-to-end lifecycle, from data collection to model development to system deployment and ongoing usage. A lack of documentation can create difficulties when practitioners attempt to justify why certain design decisions were made through the pipeline: this may be important for compiling defensible evidence of compliance to governance practices [6], anticipating stakeholder needs [90], or participating in the model  \n∗ [Both last authors contributed and advised equally. Order was decided by a coin flip. Correspondence to: valeriechen@cmu.edu and usb20@cam.ac.uk](Both last authors contributed and advised equally. Order was decided by a coin flip. Correspondence to: valeriechen@cmu.edu and usb20@cam.ac.uk)  \nAuthors’ addresses: Matthew Barker, [mrlb3@cam.ac.uk](mrlb3@cam.ac.uk), University of Cambridge, United Kingdom; Emma Kallina, University of Cambridge, United Kingdom and Responsible AI Institute, United Kingdom; Dhananjay Ashok, Carnegie Mellon University, USA; Katherine M. Collins, University of Cambridge, United Kingdom; Ashley Casovan, Responsible AI Institute, United Kingdom; Adrian Weller, University of Cambridge, United Kingdom and The Alan Turing Institute, United Kingdom; Ameet Talwalkar, Carnegie Mellon University, USA; Valerie Chen, [valeriechen@cmu.edu](valeriechen@cmu.edu), Carnegie Mellon University, USA; Umang Bhatt, [usb20@cam.ac.uk](usb20@cam.ac.uk), University of Cambridge, United Kingdom and The Alan Turing Institute, United Kingdom.  \n2023. Manuscript submitted to ACM  \nManuscript submitted to ACM 1  \n2 Barker, et al.  \nFig. 1. (Left) Existing documentation u","cbCaicGv5tviRnh8","https://ap.wps.com/l/cbCaicGv5tviRnh8","pdf",974653,1,22,"English","en",105,"# Introduction\n## Problem: Missing documentation of stakeholder input\n## FeedbackLogs as an addendum to pipeline documentation\n## FeedbackLog structure and records\n## Practical evaluation via practitioner interviews and examples","[{\"question\":\"What problem do FeedbackLogs address in machine learning development?\",\"answer\":\"FeedbackLogs address the lack of systematic documentation showing how stakeholder input is collected and incorporated throughout an ML pipeline lifecycle.\"},{\"question\":\"What information does a FeedbackLog record?\",\"answer\":\"Each log records details about the feedback collection process, the stakeholder feedback itself, and how the feedback is used to update the ML pipeline.\"},{\"question\":\"How can FeedbackLogs be used beyond development?\",\"answer\":\"FeedbackLogs can serve as evidence for algorithmic auditing and help document updates driven by stakeholder feedback.\"}]","FeedbackLogs - Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines | PDF",1785731732,55,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"feedbacklogs-recording-and-incorporating-stakeholder-feedback-into-machine-learning-pipelines","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/feedbacklogs-recording-and-incorporating-stakeholder-feedback-into-machine-learning-pipelines/120727/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem do FeedbackLogs address in machine learning development?","Question",{"text":75,"@type":76},"FeedbackLogs address the lack of systematic documentation showing how stakeholder input is collected and incorporated throughout an ML pipeline lifecycle.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What information does a FeedbackLog record?",{"text":80,"@type":76},"Each log records details about the feedback collection process, the stakeholder feedback itself, and how the feedback is used to update the ML pipeline.",{"name":82,"@type":73,"acceptedAnswer":83},"How can FeedbackLogs be used beyond development?",{"text":84,"@type":76},"FeedbackLogs can serve as evidence for algorithmic auditing and help document updates driven by stakeholder feedback.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]