[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125981-en":3,"doc-seo-125981-105":31,"detail-sidebar-cat-0-en-105":93},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125981,687207024478,"Liam","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Reproducibility in Machine Learning-based Research - Overview, Barriers and Drivers","Research across scientific domains faces mounting challenges in reproducing published results, a problem that is particularly acute in machine learning (ML). Causes include unpublished data and source code as well as high sensitivity to ML training conditions and sources of randomness. Although remedies have been proposed, reproducibility in ML-driven research remains inadequate. This article analyzes barriers, categorizes them across description, code, data, and experiment reproducibility, discusses technology, procedural, and awareness/education drivers, and maps drivers to barriers to support informed solution adoption.","arXiv :2406 . 14325v2 [ cs . SE] 2 Jul 2024  \nReproducibility in Machine Learning-based Research: Overview, Barriers and Drivers  \nHarald Semmelrock2 , Tony Ross-Hellauer 1 ,2 , Simone Kopeinik 1 , Dieter Theiler 1 , Armin Haberl3 , Stefan Thalmann3 , and Dominik Kowald (B) 1 ,2  \n1 Know-Center GmbH, Graz, Austria  \ntross,skopeinik,dtheiler,[dkowald@know-center.at](dkowald@know-center.at)  \n2 Graz University of Technology, Graz, Austria  \n[h.semmelrock@alumni.tugraz.at](h.semmelrock@alumni.tugraz.at)  \n3 University of Graz, Graz, Austria  \narmin.haberl,[stefan.thalmann@uni-graz.at](stefan.thalmann@uni-graz.at)  \nAbstract. Research in various fields is currently experiencing challenges regarding the reproducibility of results. This problem is also prevalent in machine learning (ML) research. The issue arises, for example, due to unpublished data and/or source code and the sensitivity of ML training conditions. Although different solutions have been proposed to address this issue, such as using ML platforms, the level of reproducibility in MLdriven research remains unsatisfactory. Therefore, in this article, we discuss the reproducibility of ML-driven research with three main aims: (i) identifying the barriers to reproducibility when applying ML in research as well as categorize the barriers to different types of reproducibility (description, code, data, and experiment reproducibility),(ii) discussing potential drivers such as tools, practices, and interventions that support ML reproducibility, as well as distinguish between technology-driven drivers, procedural drivers, and drivers related to awareness and education, and (iii) mapping the drivers to the barriers. With this work, we hope to provide insights and to contribute to the decision-making process regarding the adoption of different solutions to support ML reproducibility.  \nKeywords: Machine Learning · Artificial Intelligence · Reproducibility  \n· Irreproducibility  \n1 Introduction  \nAs in many scientific fields [9,68], e.g., medicine and behavior science, research in artificial intelligence (AI) in general, and machine learning (ML) in particular, faces crucial doubts over the reproducibility of research [36] . This has raised concerns about the reliability and validity of many scientific findings, risking diminishing confidence in the overall body of scientific knowledge. Misleading or irreproducible results can lead to wasted resources, hinder scientific progress, reduce trust in science, and impact decision-making in various fields [51,29] . The“reproducibility crisis” has been a topic of discussion since the mid-2010s, particularly in the medical and behavioral science fields [9,55] . Recently, this issue has  \n2 Semmelrock, Ross-Hellauer, Kopeinik, Theiler, Haberl, Thalmann, & Kowald  \nalso been associated with research involving ML methods [36] . Apart from the common challenges faced by many disciplines, such as limited sharing of data and source code, the use of ML introduces unique challenges for reproducibility, including sensitivity to ML training conditions, sources of randomness [65], and the increasing use of AutoML tools [42] .  \nWhile the literature on these issues is expanding [46,30,33,40,26,4,70,27], there is still no comprehensive overview of the associated barriers and drivers. For example, in [27], the authors identify and categorize sources of irreproducibility in ML and how these sources affect conclusions drawn from ML experiments. However, this study does not investigate drivers to address these sources of irreproducibility. Thus, our paper provides a contextual categorization of the barriers and drivers to the four types of ML reproducibility proposed by (description, code, data, and experiment) [26], with specific reference to research in both Computer Science and Biomedical fields. We also propose a Drivers-Barriers-Matrix to summarize and visualize the results of the discussion. Such an analysis stands to clarify the current state regardin","cbCaioWdxnhwWhoL","https://ap.wps.com/l/cbCaioWdxnhwWhoL","pdf",590901,9,1,23,"English","en",105,"# Introduction\n# Defining Reproducibility\n# Barriers to Reproducibility\n# Drivers Supporting Reproducibility\n# Drivers-Barriers Mapping","[{\"question\":\"What are the main barriers to reproducibility in ML-driven research discussed in the paper?\",\"answer\":\"The paper identifies barriers including limited sharing of data and source code, sensitivity to ML training conditions, and sources of randomness. It categorizes barriers across description, code, data, and experiment reproducibility.\"},{\"question\":\"What kinds of drivers for ML reproducibility does the paper discuss?\",\"answer\":\"The paper discusses drivers such as tools, practices, and interventions. It distinguishes technology-driven drivers, procedural drivers, and drivers related to awareness and education.\"},{\"question\":\"How does the paper connect drivers and barriers?\",\"answer\":\"The paper maps the identified barriers to the corresponding drivers, using a Drivers-Barriers-Matrix to summarize and visualize how solutions may address specific reproducibility gaps.\"}]","Reproducibility in Machine Learning-based Research - Overview, Barriers and Drivers | PDF",1785902367,58,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":88,"head_meta":90,"extra_data":92,"updated_unix":29},"reproducibility-in-machine-learning-based-research-overview-barriers-and-drivers","",{"@graph":37,"@context":87},[38,55,70],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,52],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":51},"https://docshare.wps.com/document/research-report/",3,{"item":53,"name":13,"@type":44,"position":54},"https://docshare.wps.com/document/reproducibility-in-machine-learning-based-research-overview-barriers-and-drivers/125981/",4,{"url":53,"name":13,"@type":56,"author":57,"headline":13,"publisher":59,"fileFormat":62,"inLanguage":24,"description":14,"dateModified":63,"datePublished":64,"encodingFormat":62,"isAccessibleForFree":65,"interactionStatistic":66},"DigitalDocument",{"name":9,"@type":58},"Person",{"url":42,"name":60,"@type":61},"DocShare","Organization","application/pdf","2026-08-24","2026-08-05",true,{"@type":67,"interactionType":68,"userInteractionCount":20},"InteractionCounter",{"@type":69},"ViewAction",{"@type":71,"mainEntity":72},"FAQPage",[73,79,83],{"name":74,"@type":75,"acceptedAnswer":76},"What are the main barriers to reproducibility in ML-driven research discussed in the paper?","Question",{"text":77,"@type":78},"The paper identifies barriers including limited sharing of data and source code, sensitivity to ML training conditions, and sources of randomness. It categorizes barriers across description, code, data, and experiment reproducibility.","Answer",{"name":80,"@type":75,"acceptedAnswer":81},"What kinds of drivers for ML reproducibility does the paper discuss?",{"text":82,"@type":78},"The paper discusses drivers such as tools, practices, and interventions. It distinguishes technology-driven drivers, procedural drivers, and drivers related to awareness and education.",{"name":84,"@type":75,"acceptedAnswer":85},"How does the paper connect drivers and barriers?",{"text":86,"@type":78},"The paper maps the identified barriers to the corresponding drivers, using a Drivers-Barriers-Matrix to summarize and visualize how solutions may address specific reproducibility gaps.","https://schema.org",{"og:url":53,"og:type":89,"og:title":13,"og:site_name":60,"og:description":14},"article",{"robots":91,"canonical":53},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":94},[95,99,103,107,112,117,122,125,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":96,"show_sort_weight":97,"slug":98},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":100,"show_sort_weight":101,"slug":102},"Literature",80,"literature",{"id":54,"doc_module":4,"doc_module_name":47,"category_name":104,"show_sort_weight":105,"slug":106},"Exam",70,"exam",{"id":108,"doc_module":4,"doc_module_name":47,"category_name":109,"show_sort_weight":110,"slug":111},5,"Comic",60,"comic",{"id":113,"doc_module":4,"doc_module_name":47,"category_name":114,"show_sort_weight":115,"slug":116},6,"Technology",50,"technology",{"id":118,"doc_module":4,"doc_module_name":47,"category_name":119,"show_sort_weight":120,"slug":121},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":123,"slug":124},30,"research-report",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":108,"slug":139},19,"General","general"]