[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117404-en":3,"doc-seo-117404-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117404,7971461740909,"Levi","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Reproducible machine learning research in mental workload classification using EEG","This review addresses reproducibility concerns in scientific research by focusing on electroencephalography (EEG) combined with machine learning for mental workload estimation. The study develops guidelines for reproducible EEG-based machine learning, then applies them to evaluate the current reproducibility state of mental workload modeling. Using summaries of reproducibility efforts, a systematic literature review across major databases identifies studies on EEG-driven prediction, structures resulting recommendations via the CRISP-DM framework, and re-evaluates reproducibility to highlight reporting and sharing gaps.","TYPE Review  \nPUBLISHED 10 April 2024  \nDOI 10. 3389/fnrgo.2024.1346794  \nOPEN ACCESS  \nEDITED BY  \nEdmund Wascher,  \nLeibniz Research Centre for Working Environment and Human Factors (IfADo), Germany  \nREVIEWED BY  \nFelix Putze,  \nUniversity of Bremen, Germany Emad Alyan,  \nLeibniz Research Centre for Working Environment and Human Factors (IfADo), Germany  \n*CORRESPONDENCE  \nGüliz Demirezen  \n [guliz.demirezen@metu.edu.tr](guliz.demirezen@metu.edu.tr)  \nRECEIVED 29 November 2023  \nACCEPTED 22 March 2024  \nPUBLISHED 10 April 2024  \nCITATION  \nDemirezen G, Ta¸skaya Temizel T and Brouwer A-M (2024) Reproducible machine learning research in mental workload classification using EEG.  \nFront. Neuroergon. 5:1346794 .  \ndoi: 10.3389/fnrgo.2024.1346794  \nCOPYRIGHT  \n© 2024 Demirezen, Ta¸skaya Temizel and Brouwer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.  \nReproducible machine learning research in mental workload classification using EEG  \nGüliz Demirezen1*, Tuˇgba Ta¸skaya Temizel2 and Anne-Marie Brouwer3,4  \n1 Department of Information Systems, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye, 2 Department of Data Informatics, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye, 3 Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands, 4 Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands  \nThis study addresses concerns about reproducibility in scientific research, focusing on the use of electroencephalography (EEG) and machine learning to estimate mental workload. We established guidelines for reproducible machine learning research using EEG and used these to assess the current state of reproducibility in mental workload modeling. We first started by summarizing the current state of reproducibility e􀀀orts in machine learning and in EEG. Next, we performed a systematic literature review on Scopus, Web of Science, ACM Digital Library, and Pubmed databases to find studies about reproducibility in mental workload prediction using EEG. All of this previous work was used to formulate guidelines, which we structured along the widely recognized Cross-Industry Standard Process for Data Mining (CRISP-DM) framework. By using these guidelines, researchers can ensure transparency and comprehensiveness of their methodologies, therewith enhancing collaboration and knowledge-sharing within the scientific community, and enhancing the reliability, usability and significance of EEG and machine learning techniques in general. A second systematic literature review extracted machine learning studies that used EEG to estimate mental workload. We evaluated the reproducibility status of these studies using our guidelines. We highlight areas studied and overlooked and identify current challenges for reproducibility. Our main findings include limitations on reporting performance on unseen test data, open sharing of data and code, and reporting of resources essential for training and inference processes.  \nKEYWORDS  \nneuroergonomics, reproducibility, EEG, physiological measurement, mental workload, machine learning, brain-computer interface, neuroscience  \n1 Introduction  \nReproducibility is fundamental for research advancement. Reproducing results, not only by the owners of the original study but also by other researchers, enables establishing a solid foundation that can be built upon for global research progress. The ability to repeat a study of others using the exact same methodology and produce the same res","cbCaiqn2Wkc5MHdP","https://ap.wps.com/l/cbCaiqn2Wkc5MHdP","pdf",737314,1,21,"English","en",105,"# Introduction\n# Reproducibility and research progress\n# Neuroergonomics and BCI context\n# EEG-based mental workload modeling\n# Guidelines and evaluation approach\n## CRISP-DM structured reproducibility framework\n## Systematic literature review and assessment","[{\"question\":\"What is the main focus of the study?\",\"answer\":\"The study focuses on improving reproducibility in research that uses EEG signals with machine learning to estimate mental workload, and on assessing the reproducibility of existing mental workload modeling work.\"},{\"question\":\"How do the authors develop and apply their reproducibility guidelines?\",\"answer\":\"The authors derive guidelines from a synthesis of reproducibility efforts and prior literature, then structure the guidelines using the CRISP-DM framework and evaluate existing studies against them.\"},{\"question\":\"Which reproducibility gaps are highlighted in the reviewed EEG-and-ML studies?\",\"answer\":\"Key gaps include limited reporting of performance on unseen test data, insufficient open sharing of data and code, and missing reporting of essential resources needed for training and inference.\"}]","Reproducible machine learning research in mental workload classification using EEG | PDF",1785675688,53,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"reproducible-machine-learning-research-in-mental-workload-classification-using-eeg","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/reproducible-machine-learning-research-in-mental-workload-classification-using-eeg/117404/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main focus of the study?","Question",{"text":75,"@type":76},"The study focuses on improving reproducibility in research that uses EEG signals with machine learning to estimate mental workload, and on assessing the reproducibility of existing mental workload modeling work.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the authors develop and apply their reproducibility guidelines?",{"text":80,"@type":76},"The authors derive guidelines from a synthesis of reproducibility efforts and prior literature, then structure the guidelines using the CRISP-DM framework and evaluate existing studies against them.",{"name":82,"@type":73,"acceptedAnswer":83},"Which reproducibility gaps are highlighted in the reviewed EEG-and-ML studies?",{"text":84,"@type":76},"Key gaps include limited reporting of performance on unseen test data, insufficient open sharing of data and code, and missing reporting of essential resources needed for training and inference.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]