Reproducible machine learning research in mental workload classiﬁcation using EEG

Aug 2, 2026

Upload by Levi

This review addresses reproducibility concerns in scientific research by focusing on electroencephalography (EEG) combined with machine learning for mental workload estimation. The study develops guidelines for reproducible EEG-based machine learning, then applies them to evaluate the current reproducibility state of mental workload modeling. Using summaries of reproducibility efforts, a systematic literature review across major databases identifies studies on EEG-driven prediction, structures resulting recommendations via the CRISP-DM framework, and re-evaluates reproducibility to highlight reporting and sharing gaps.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

AccessFree to download

Introduction

Reproducibility and research progress

Neuroergonomics and BCI context

EEG-based mental workload modeling

Guidelines and evaluation approach

CRISP-DM structured reproducibility framework

Systematic literature review and assessment

TYPE Review PUBLISHED 10 April 2024 DOI 10. 3389/fnrgo.2024.1346794 OPEN ACCESS EDITED BY Edmund Wascher, Leibniz Research Centre for Working Environment and Human Factors (IfADo), Germany REVIEWED BY Felix Putze, University of Bremen, Germany Emad Alyan, Leibniz Research Centre for Working Environment and Human Factors (IfADo), Germany *CORRESPONDENCE Güliz Demirezen [guliz.demirezen@metu.edu.tr](guliz.demirezen@metu.edu.tr) RECEIVED 29 November 2023 ACCEPTED 22 March 2024 PUBLISHED 10 April 2024 CITATION Demirezen G, Ta¸skaya Temizel T and Brouwer A-M (2024) Reproducible machine learning research in mental workload classiﬁcation using EEG. Front. Neuroergon. 5:1346794 . doi: 10.3389/fnrgo.2024.1346794 COPYRIGHT © 2024 Demirezen, Ta¸skaya Temizel and Brouwer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Reproducible machine learning research in mental workload classiﬁcation using EEG Güliz Demirezen1*, Tuˇgba Ta¸skaya Temizel2 and Anne-Marie Brouwer3,4 1 Department of Information Systems, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye, 2 Department of Data Informatics, Graduate School of Informatics, Middle East Technical University, Ankara, Türkiye, 3 Human Performance, Netherlands Organisation for Applied Scientiﬁc Research (TNO), Soesterberg, Netherlands, 4 Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands This study addresses concerns about reproducibility in scientiﬁc research, focusing on the use of electroencephalography (EEG) and machine learning to estimate mental workload. We established guidelines for reproducible machine learning research using EEG and used these to assess the current state of reproducibility in mental workload modeling. We ﬁrst started by summarizing the current state of reproducibility e􀀀orts in machine learning and in EEG. Next, we performed a systematic literature review on Scopus, Web of Science, ACM Digital Library, and Pubmed databases to ﬁnd studies about reproducibility in mental workload prediction using EEG. All of this previous work was used to formulate guidelines, which we structured along the widely recognized Cross-Industry Standard Process for Data Mining (CRISP-DM) framework. By using these guidelines, researchers can ensure transparency and comprehensiveness of their methodologies, therewith enhancing collaboration and knowledge-sharing within the scientiﬁc community, and enhancing the reliability, usability and signiﬁcance of EEG and machine learning techniques in general. A second systematic literature review extracted machine learning studies that used EEG to estimate mental workload. We evaluated the reproducibility status of these studies using our guidelines. We highlight areas studied and overlooked and identify current challenges for reproducibility. Our main ﬁndings include limitations on reporting performance on unseen test data, open sharing of data and code, and reporting of resources essential for training and inference processes. KEYWORDS neuroergonomics, reproducibility, EEG, physiological measurement, mental workload, machine learning, brain-computer interface, neuroscience 1 Introduction Reproducibility is fundamental for research advancement. Reproducing results, not only by the owners of the original study but also by other researchers, enables establishing a solid foundation that can be built upon for global research progress. The ability to repeat a study of others using the exact same methodology and produce the same res

FAQs

What is the main focus of the study?

The study focuses on improving reproducibility in research that uses EEG signals with machine learning to estimate mental workload, and on assessing the reproducibility of existing mental workload modeling work.

How do the authors develop and apply their reproducibility guidelines?

The authors derive guidelines from a synthesis of reproducibility efforts and prior literature, then structure the guidelines using the CRISP-DM framework and evaluate existing studies against them.

Which reproducibility gaps are highlighted in the reviewed EEG-and-ML studies?

Key gaps include limited reporting of performance on unseen test data, insufficient open sharing of data and code, and missing reporting of essential resources needed for training and inference.

Aug 2, 2026

Upload by Levi