[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117641-en":3,"doc-seo-117641-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":11,"language":21,"language_code":22,"site_id":23,"html_lang":22,"table_of_contents":24,"faqs":25,"seo_title":26,"seo_description":14,"update_tm":27,"read_time":28},117641,13056703019662,"Evangeline","https://ap-avatar.wpscdn.com/avatar/be000253a8e92610077?_k=1778726343310543188",8,"Research & Report","Continuously Reproducing Toolchains in Pattern Recognition and Machine Learning Experiments","Pattern recognition and machine learning research relies on real-world experimental results that support hypotheses and enable comparison of ideas through tables and figures. Result reproducibility is often neglected because software for these reports is complex to install, maintain, and distribute, while experiments involve many steps and parameters that are hard to document. Growing research complexity makes replication even more difficult over time. This paper advocates reproducible work that is repeatable, shareable, extensible, and stable, and presents detailed lessons using a face recognition use case.","View metadata, citation and similar [papers at ](papers at core.ac.uk)[core.ac.uk](papers at core.ac.uk) brought to you by CORE  \nprovided by Infoscience- École polytechnique fédérale de Lausanne  \nContinuously Reproducing Toolchains in Pattern Recognition and Machine Learning Experiments  \nA. Anjos  \nIdiap Research Institute Martigny, Switzerland [andre.anjos@idiap.ch](andre.anjos@idiap.ch)  \nM. Gnther  \nVision and Security Technology University of Colorado Colorado Springs, USA[mgunther@vast.uccs.edu](mgunther@vast.uccs.edu)  \nT. Pereira, P. Korshunov, A. Mohammadi, S. Marcel  \nIdiap Research Institute Martigny, Switzerland [marcel@idiap.ch](marcel@idiap.ch)  \nAbstract  \nPattern recognition and machine learning research work often contains experimental results on real-world data, which corroborates hypotheses and provides a canvas for the development and comparison of new ideas. Results, in this context, are typically summarized as a set of tables and figures, allowing the comparison of various methods, highlighting the advantages of the proposed ideas. Unfortunately, result reproducibility is often an overlooked feature of original research publications, competitions, or benchmark evaluations. The main reason for such a gap is the complexity on the development of software associated with these reports. Software frameworks are difficult to install, maintain, and distribute, whilescientific experiments often consist of many steps and parameters that are difficult to report. The increasingly rising complexity of research challenges make it even more difficult to reproduce experiments and results. In this paper, we emphasize that a reproducible research work should be repeatable, shareable, extensible, and stable, and discuss important lessons we learned in creating, distributing, and maintaining software and data for reproducible research in pattern recognition and machine learning. We focus on a specific use-case of face recognition and describe in details how we can make the recognition experiments reproducible in practice.  \n1 Introduction  \nThe popularity of machine learning, especially neural networks, has been growing exponentially in the recent years. This growth manifested in an explosive number of available machine learning software, datasets, models, and techniques. Every respectable software company or a university is now providing and maintaining a machine learning software suite, ranging from open-source tools, such as TensorFlow [1] by Google, Caffe [2] by UC Berkeley, and Torch [3] by Facebook, to businessoriented solutions, including Azure [4] by Microsoft and Watson platform [5] by IBM. Datasets are also growing in size and numbers, with regularly organized competitions that provide challenging databases. Examples include the NIST speaker recognition evaluations [6], with more than a thousand hours recordings of more than two thousand identities, the MegaFace [7, 8] face recognition challenge, with four million images of more than six hundred thousand identities, and generic object recognition dataset COCO [9], with more than two million instances of eighty different object categories.  \nSuch abundance of tools and data has a positive impact on advances in research, allowing scientists to quickly test their hypotheses and setup experiments, however, it also increases the complexity of an experiment setup. A paper that was published eighty years ago, e.g., on Linear Discriminant Analysis (LDA) by Ronald Fisher [10], is a self-contained piece of knowledge, and even now, its results can be verified and reproduced with pen and paper. A machine learning paper today is no  \n31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.  \nmore bound by the 10-page text limit, and often corresponds just to the tip of an iceberg, since the experiments and the results depend on various software, data, configuration options, operating systems, and even hardware. Therefore, a researcher needs to make a","cbCaibL1sPIHaDgd","https://ap.wps.com/l/cbCaibL1sPIHaDgd","pdf",218191,1,"English","en",105,"# Abstract\n# Introduction\n## Reproducibility challenges\n## Towards systematic reproducibility\n# Reproducibility properties for published papers\n## Repeatable\n## Shareable\n## Extensible\n## Stable\n# Practical components of reusable toolchains","[{\"question\":\"Why is experiment and result reproducibility often overlooked in machine learning publications?\",\"answer\":\"Reproducibility is frequently missed because the associated software is difficult to install, maintain, and distribute, and because experiments contain many steps and parameters that are hard to report clearly.\"},{\"question\":\"What does the paper require for a published work to be reproducible?\",\"answer\":\"A reproducible paper should be repeatable, shareable, extensible, and stable, meaning experiments can be re-run reliably, materials can be distributed, the infrastructure can support new directions, and behavior remains consistent over time on a best-effort basis.\"},{\"question\":\"How does the paper plan to address reproducibility in practice?\",\"answer\":\"It shares approaches and lessons for organizing research software and data so that papers satisfy reproducibility properties, emphasizing reusable experimental toolchains and practices such as version control, packaging, continuous integration, unit testing, and documentation.\"}]","Continuously Reproducing Toolchains in Pattern Recognition and Machine Learning Experiments | PDF",1785677555,20,{"code":4,"msg":30,"data":31},"ok",{"site_id":23,"language":22,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"continuously-reproducing-toolchains-in-pattern-recognition-and-machine-learning-experiments","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/continuously-reproducing-toolchains-in-pattern-recognition-and-machine-learning-experiments/117641/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":22,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-08-05","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is experiment and result reproducibility often overlooked in machine learning publications?","Question",{"text":75,"@type":76},"Reproducibility is frequently missed because the associated software is difficult to install, maintain, and distribute, and because experiments contain many steps and parameters that are hard to report clearly.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the paper require for a published work to be reproducible?",{"text":80,"@type":76},"A reproducible paper should be repeatable, shareable, extensible, and stable, meaning experiments can be re-run reliably, materials can be distributed, the infrastructure can support new directions, and behavior remains consistent over time on a best-effort basis.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the paper plan to address reproducibility in practice?",{"text":84,"@type":76},"It shares approaches and lessons for organizing research software and data so that papers satisfy reproducibility properties, emphasizing reusable experimental toolchains and practices such as version control, packaging, continuous integration, unit testing, and documentation.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":23},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":28,"slug":126},9,"Religion & Spirituality","religion-spirituality",{"id":28,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":28,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]