[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122680-en":3,"doc-seo-122680-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122680,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","reforms - Reporting Standards for Machine Learning Based Science","Machine learning methods are increasingly used in scientific research, but their adoption often coincides with failures in validity, reproducibility, and generalizability. These problems slow progress, promote false consensus around invalid findings, and weaken the credibility of ML-based science. The paper introduces clear reporting standards—an ML-based science reforms checklist—derived from extensive literature review, offering 32 questions with paired guidelines to support researchers, reviewers, and journals in transparency and reproducibility.","arXiv :2308 .07832v1 [ cs .LG] 15 Aug 2023  \nreforms: Reporting Standards for Machine Learning Based Science  \nSayash Kapoor 1 Emily Cantrell Kenny Peng Thanh Hien Pham  \nChristopher A. Bail Odd Erik Gundersen Jake M. Hofman Jessica Hullman Michael A. Lones Momin M. Malik Priyanka Nanayakkara Russell A. Poldrack Inioluwa Deborah Raji Michael Roberts Matthew J. Salganik Marta Serra-Garcia Brandon M. Stewart Gilles Vandewiele Arvind Narayanan  \nAugust 15, 2023  \nAbstract  \nMachine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity, reproducibility, and generalizability. These failures can hinder scientific progress, lead to false consensus around invalid claims, and undermine the credibility of ML-based science. ML methods are often applied and fail in similar ways across disciplines. Motivated by this observation, our goal is to provide clear reporting standards for ML-based science. Drawing from an extensive review of past literature, we present the reforms checklist (Reporting Standards For Machine Learning Based Science) . It consists of 32 questions and a paired set of guidelines. reforms was developed basedon a consensus of 19 researchers across computer science, data science, mathematics, social sciences, and biomedical [sciences. re](sciences. re)forms can serve as a resource for researchers when designing and implementing a study, for referees when reviewing papers, and for journals when enforcing standards for transparency and reproducibility.  \nIntroduction  \nML methods are being widely adopted for scientific research [1–11] . Compared to older statistical methods, they offer increased predictive accuracy[1], the ability to process large amounts of data [12], and the ability to use different types of data for scientific research, such as text, images, and video [7] . However, the rapid uptake of ML methods has been accompanied by concerns of validity, reproducibility, and generalizability [13–19] . There are several reasons for concern. Performance evaluation is notoriously tricky in ML [20–23] . ML code tends to be complex and as yet lacks standardization [24, 25], leading to a lack of computational reproducibility [26] . Subtle pitfalls arise from the differences between explanatory and predictive modeling [27] . The hype and over-optimism about commercial AI may spillover into scientific research [28] . In addition, publication biases that have led to past reproducibility crises [29] are also present in ML research [30, 31] . If left unchecked, these  \n1 Corresponding author: [sayashk@princeton.edu. The](sayashk@princeton.edu. The) latest version of the paper and the appendices are available at [https://reforms.cs.princeton.edu](https://reforms.cs.princeton.edu)  \nflaws can lead to a feedback loop of overoptimism since nonreplicable findings are cited more than replicable ones [32] . There is an urgent need to systematically address errors in ML-based science rather than finding errors in individual studies after publication.  \nIn this paper, we focus on a specific subset of ML applications: ML-based science. We use this term to refer to research that makes a scientific claim using the performance of an ML model as evidence. For example, Salganik et al.  \n[33] use ML methods to investigate the predictability of life outcomes. This contrasts with ML methods research, which involves improving widely applicable ML methods instead of making scientific claims using ML methods. In the next section, we clarify the distinctions between ML methods research and ML-based science and outline the scope of the paper in greater detail. Box 1 summarizes this discussion.  \nOne promising way to detect and prevent errors in scientific research is by improving reporting standards [34–36] . Clear expectations for using ML methods can allow researchers and referees to spot errors early. Despite the use ofML methods across disciplines, there are no widely ","cbCaio3tJgi0F8V4","https://ap.wps.com/l/cbCaio3tJgi0F8V4","pdf",362856,1,20,"English","en",105,"# Abstract\n# Introduction\n# Reporting Standards and Checklist","[{\"question\":\"What issues does the reforms checklist address in ML-based science?\",\"answer\":\"It targets failures of validity, reproducibility, and generalizability that can mislead scientific conclusions and slow progress.\"},{\"question\":\"What is ML-based science in the context of this paper?\",\"answer\":\"It refers to research that makes a scientific claim using the performance of an ML model as evidence, as opposed to improving ML methods without making scientific claims.\"},{\"question\":\"How is the reforms checklist structured and how many items does it include?\",\"answer\":\"It consists of 32 questions across 8 modules, accompanied by paired guidelines for reporting each item.\"}]","reforms - Reporting Standards for Machine Learning Based Science | PDF",1785812148,50,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"reforms-reporting-standards-for-machine-learning-based-science","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/reforms-reporting-standards-for-machine-learning-based-science/122680/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What issues does the reforms checklist address in ML-based science?","Question",{"text":75,"@type":76},"It targets failures of validity, reproducibility, and generalizability that can mislead scientific conclusions and slow progress.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is ML-based science in the context of this paper?",{"text":80,"@type":76},"It refers to research that makes a scientific claim using the performance of an ML model as evidence, as opposed to improving ML methods without making scientific claims.",{"name":82,"@type":73,"acceptedAnswer":83},"How is the reforms checklist structured and how many items does it include?",{"text":84,"@type":76},"It consists of 32 questions across 8 modules, accompanied by paired guidelines for reporting each item.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,126,129,133],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":29,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":21,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":21,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":106,"slug":136},19,"General","general"]