[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119169-en":3,"doc-seo-119169-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119169,8796095461564,"Liam","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research","Machine learning increasingly supports diagnosis and planning by replacing some traditional first-principle approaches with data-driven models. Empirical research is central to this field: clear, testable hypotheses guide experiment design; careful execution enables reliable measurements; and statistical analysis either supports or refutes the initial claims. Yet research practice varies widely across the community, with no shared quality criteria. The work proposes a model for the empirical research process and guidelines to strengthen validity, improving consistency, reliability, and impact.","Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research  \nDaniel Vranješ \\#   \nHelmut Schmidt University, Hamburg, Germany Jonas Ehrhardt \\#   \nHelmut Schmidt University, Hamburg, Germany René Heesch \\#   \nHelmut Schmidt University, Hamburg, Germany Lukas Moddemann \\#   \nHelmut Schmidt University, Hamburg, Germany Henrik Sebastian Steude \\#  Helmut Schmidt University, Hamburg, Germany Oliver Niggemann \\#   \nHelmut Schmidt University, Hamburg, Germany  \n~~ Abstract ~~  \nMachine learning is becoming increasingly important in the diagnosis and planning fields, where data-driven models and algorithms are being employed as alternatives to traditional first-principle approaches. Empirical research plays a fundamental role in the machine learning domain. At the heart of impactful empirical research lies the development of clear research hypotheses, which then shape the design of experiments. The execution of experiments must be carried out with precision to ensure reliable results, followed by statistical analysis to interpret these outcomes. This process is key to either supporting or refuting initial hypotheses. Despite its importance, there is a high variability in research practices across the machine learning community and no uniform understanding of quality criteria for empirical research. To address this gap, we propose a model for the empirical research process, accompanied by guidelines to uphold the validity of empirical research. By embracing these recommendations, greater consistency, enhanced reliability and increased impact can be achieved.  \n2012 ACM Subject Classification Computing methodologies → Machine learning  \nKeywords and phrases machine learning, hypothesis design, research design, experimental research, statistical testing, diagnosis, planning  \nDigital Object Identifier 10.4230/OASIcs.DX.2024.7  \nRelated Version Previous Version: [https://arxiv.org/abs/2405.18077](https://arxiv.org/abs/2405.18077)  \n 1  Introduction  \nDeductive and abductive reasoning play crucial roles in both theoretical and empirical research. Deductive reasoning, often used in theoretical research, involves deriving specific predictions from general principles or axioms. This approach ensures internal consistency within a logical framework but does not require empirical testing. In contrast, abductive reasoning, which is foundational in empirical research, involves forming plausible hypotheses to explain observations. These hypotheses are then tested through experiments to gather empirical evidence. While deductive reasoning provides a solid foundation for developing theoretical models, abductive reasoning bridges the gap between theory and practice by enabling the formulation and empirical validation of testable hypotheses.  \n© Daniel Vranješ, Jonas Ehrhardt, René Heesch, Lukas Moddemann, Henrik Sebastian Steude, and Oliver Niggemann;  \nlicensed under Creative Commons License CC-BY 4.0  \n35th International Conference on Principles of Diagnosis and Resilient Systems (DX 2024) .  \nEditors: Ingo Pill, Avraham Natan, and Franz Wotawa; Article No. 7; pp. 7:1–7:13  \nOpenAccess Series in Informatics  \n Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany  \n7:2 Design Principles for Falsifiable, Replicable and Reproducible Empirical ML Research  \nTheoretical research in machine learning (ML) involves the development and analysis of models and algorithms through mathematical formalisms and proofs, offering insights into their properties, performance guarantees, and limitations. This foundational work is essential for understanding the principles that govern ML systems, guiding the design of new algorithms, and providing a basis for interpreting their behavior. The analytical and formal nature of this approach allows for certain and specific evidence, but is not applicable formost hypotheses.  \nEmpirical research, on the other hand, plays a crucial role in the ML domain through","cbCaik4RbHI7TJ6L","https://ap.wps.com/l/cbCaik4RbHI7TJ6L","pdf",577486,1,13,"English","en",105,"# Introduction\n## Deductive vs. abductive reasoning in ML research\n## Role of empirical research in validating ML models\n## Challenges: falsifiability, replicability, reproducibility, and generalizability","[{\"question\":\"Why are deductive and abductive reasoning important for empirical machine learning research?\",\"answer\":\"Deductive reasoning derives predictions from general principles and mainly supports theoretical consistency. Abductive reasoning forms plausible hypotheses from observations and enables those hypotheses to be tested through experiments for empirical evidence.\"},{\"question\":\"What are the main issues affecting scientific rigor in empirical ML research?\",\"answer\":\"The document highlights falsifiability, replicability, reproducibility, and generalizability as major concerns, especially when conditions for failure are unclear, documentation is insufficient, or proprietary tools and datasets limit independent verification.\"},{\"question\":\"How do the proposed guidelines improve empirical research quality?\",\"answer\":\"They provide a structured model for the empirical research process and guidelines intended to uphold validity. Adopting these recommendations aims to increase consistency, reliability of results, and overall research impact.\"}]","Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research | PDF",1785722884,33,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"design-principles-for-falsifiable-replicable-and-reproducible-empirical-machine-learning-research","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/design-principles-for-falsifiable-replicable-and-reproducible-empirical-machine-learning-research/119169/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are deductive and abductive reasoning important for empirical machine learning research?","Question",{"text":75,"@type":76},"Deductive reasoning derives predictions from general principles and mainly supports theoretical consistency. Abductive reasoning forms plausible hypotheses from observations and enables those hypotheses to be tested through experiments for empirical evidence.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the main issues affecting scientific rigor in empirical ML research?",{"text":80,"@type":76},"The document highlights falsifiability, replicability, reproducibility, and generalizability as major concerns, especially when conditions for failure are unclear, documentation is insufficient, or proprietary tools and datasets limit independent verification.",{"name":82,"@type":73,"acceptedAnswer":83},"How do the proposed guidelines improve empirical research quality?",{"text":84,"@type":76},"They provide a structured model for the empirical research process and guidelines intended to uphold validity. Adopting these recommendations aims to increase consistency, reliability of results, and overall research impact.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]