[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119230-en":3,"doc-seo-119230-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119230,687197207639,"Asher","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","From Model Performance to Claim - How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap","The paper addresses two closely related yet often separated aims in machine learning ethics: improving replicability and strengthening accountability. It argues that the so-called responsibility gap—blaming ML scientists for harms when they are distant from deployment sites—can be narrowed by reframing replicability. Instead of focusing on model-performance replicability, the work proposes claim replicability so accountability follows non-replicable, harm-prone claims. It further treats claim replicability as a social project, highlighting competing epistemological principles and implications for research communication.","arXiv :2404 . 13131v1 [ cs .CY] 19 Apr 2024  \nFrom Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap  \nTIANQI KOU, Penn State University, USA  \nTwo goals – improving replicability and accountability of Machine Learning research respectively, have accrued much attention from the AI ethics and the Machine Learning community. Despite sharing the measures of improving transparency, the two goals are discussed in different registers-replicability registers with scientific reasoning whereas accountability registers with ethical reasoning. Given the existing challenge of the responsibility gap – holding Machine Learning scientists accountable for Machine Learning harms due to them being far from sites of application, this paper posits that reconceptualizing replicability can help bridge the gap. Through a shift from model performance replicability to claim replicability, Machine Learning scientists can be held accountable for producing non-replicable claims that are prone to eliciting harm due to misuse and misinterpretation. In this paper, I make the following contributions. First, I define and distinguish two forms of replicability for ML research that can aid constructive conversations around replicability. Second, I formulate an argument for claim-replicability’s advantage over model performance replicability in justifying assigning accountability to Machine Learning scientists for producing non-replicable claims and show how it enacts a sense of responsibility that is actionable. In addition, I characterize the implementation of claim replicability as more of a social project thana technical one by discussing its competing epistemological principles, practical implications on Circulating Reference, Interpretative Labor, and research communication.  \nCCS Concepts: • Social and professional topics → Socio-technical systems; Computing profession; Codes of ethics; • Comput  \ning methodologies → Machine learning; Artificial intelligence.  \nAdditional Key Words and Phrases: Replicability, Accountability, Transparency, Research Communication, Sociology of Science ACM Reference Format:  \nTianqi Kou. 2024. From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap. In ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT ’24), June 3–6, 2024, Rio de Janeiro, Brazil. ACM, New York, NY, USA, 20 pages. [https://doi.org/10.1145/3630106.3658951](https://doi.org/10.1145/3630106.3658951)  \n1 INTRODUCTION  \nIn recent years, the AI ethics community has produced much literature on improving Machine Learning (ML) transparency. On the one hand, transparency measures can serve the goal of improving accountability. For this goal, transparency measures focus on making artifacts, actors, and development processes open for external auditing and regulations to make prevention, identification, and mitigation of harms easier, and to hold relevant parties accountable. On the other hand, transparency measures have been called for to uphold replicability – maintaining the scientific rigor and integrity of ML research [5] by ensuring that the research process and artifacts are adequately shared to facilitate re-run of studies for verifying the study’s finding’s validity [33] .  \nAuthor’s address: Tianqi Kou, [tfk5237@psu.edu](tfk5237@psu.edu), Penn State University, University Park, PA, USA.  \nPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specifi[c permission and","cbCaiiLMH3NTeick","https://ap.wps.com/l/cbCaiiLMH3NTeick","pdf",323145,1,21,"English","en",105,"# Introduction\n## Transparency Measures and the Origins of Concern\n## Replication Crisis and Open Science Movement\n# Framing Replicability for Accountability\n## Model Performance Replicability vs Claim Replicability\n# Claim Replicability as a Social Project\n## Epistemological Principles and Practical Implications","[{\"question\":\"What problem does the paper target in machine learning research?\",\"answer\":\"It targets the responsibility gap, where ML scientists can be held accountable for harms they may not directly observe due to being far from real application sites.\"},{\"question\":\"How does the paper redefine replicability to support accountability?\",\"answer\":\"It shifts from model-performance replicability to claim replicability, arguing that accountability should be assigned for producing non-replicable claims likely to be misused or misunderstood.\"},{\"question\":\"What does the paper say about implementation of claim replicability?\",\"answer\":\"It characterizes claim replicability as more of a social project than a technical one, involving social communication and competing epistemological principles.\"}]","From Model Performance to Claim - How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap | PDF",1785723204,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"from-model-performance-to-claim-how-a-change-of-focus-in-machine-learning-replicability-can-help-bridge-the-responsibility-gap","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/from-model-performance-to-claim-how-a-change-of-focus-in-machine-learning-replicability-can-help-bridge-the-responsibility-gap/119230/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the paper target in machine learning research?","Question",{"text":75,"@type":76},"It targets the responsibility gap, where ML scientists can be held accountable for harms they may not directly observe due to being far from real application sites.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper redefine replicability to support accountability?",{"text":80,"@type":76},"It shifts from model-performance replicability to claim replicability, arguing that accountability should be assigned for producing non-replicable claims likely to be misused or misunderstood.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the paper say about implementation of claim replicability?",{"text":84,"@type":76},"It characterizes claim replicability as more of a social project than a technical one, involving social communication and competing epistemological principles.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]