System Usability Prediction Based on Eye-Tracking Data Using Machine Learning - A Case of Text-Entry in Virtual Reality

Aug 5, 2026

Extended Reality (XR) adoption drives the need for improved user experience through testing and evaluation. This thesis investigates when users may encounter usability problems by classifying above- and below-average usability based on System Usability Scale (SUS) scores, using eye-tracking data and machine learning from a Virtual Reality text-entry experiment. Two text-entry methods were evaluated: speech-to-text and a virtual keyboard. Eye-tracking metrics were extracted with a sliding-window approach, used as features for Random Forest models (with KNN as baseline). The best-performing models were analyzed with SHAP to interpret feature impact and relate gaze behavior, cognitive processes, and system usability. The symbiosis model reached 71.46% average accuracy, while method-specific models achieved 78.00% for speech-to-text and 74.05% for the virtual keyboard.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages61

Reading Time2 h 34 min

AccessFree to download

Introduction

Background and motivation

Research goal and approach

Methodology

Text-entry methods in VR

Eye-tracking data extraction (sliding window)

Feature engineering for ML models

Model training and baselines (Random Forest, KNN)

Model interpretation and analysis

Feature impact explanation with SHAP

Links between gaze, cognition, and usability

Results

Classification accuracy across models and methods

Similarities and differences in gaze behavior

Aditya Iqbal Bagaskara SYSTEM USABILITY PREDICTION BASED ON EYE-TRACKING DATA USING MACHINE LEARNING: A CASE OF TEXT-ENTRY IN VIRTUAL REALITY Faculty of Information Technology and Communication Sciences (ITC) Master’s thesis October 2023 Abstract Aditya Iqbal Bagaskara: System Usability Prediction Based on Eye-Tracking Data Using Machine Learning: A Case of Text-Entry in Virtual Reality Master’s thesis Tampere University Master’s Degree Programme in Data Science October 2023 The vast development and adoption of Extended Reality (XR) technology requires improving user experience through testing and evaluation. The ability to separate cases where users potentially encounter issues would be a beneficial solution to complement efforts in enhancing user experience. Therefore, this thesis explores the potential use cases of classifying above and below-average usability based on System Usability Scale (SUS) scores using eye-tracking data and machine learning from a text-entry experiment in Virtual Reality (VR) . Two text-entry methods were tested which were speech-to-text and virtual keyboard. The recorded eye-tracking data from the experiment were extracted using a sliding window approach. The extracted metrics served as features for machine learning models, with Random Forest as the chosen algorithm and K-Nearest Neighbors (KNN) as the baseline algorithm. The best-performing models were further analyzed using SHapley Additive exPlanations (SHAP) to explain features’ impact on the models’ output and the potential links between eye-tracking data, cognitive processes, and system usability. The best accuracy of the symbiosis model (i.e., the model trained using data from the two text-entry methods) achieved an average accuracy of 71.46% . Meanwhile, the method-specific models (i.e., the models trained using data from each specific text-entry method) achieved an average accuracy of 78.00% for the speechto-text method and 74 .05% for the virtual keyboard method. The SHAP analysis revealed the variations of features’ impact on the classification output highlighting the similarity and distinctiveness of participants’ gaze behavior between the two text-entry methods. Furthermore, some eye-tracking metrics indicated a correlation between previous research on eye-tracking data as a psychophysiological parameter of cognitive processes such as cognitive load and mental effort which might influence users’ opinion on system usability. Keywords: Machine Learning, Eye-Tracking Data, Virtual Reality. The originality of this thesis has been checked using the Turnitin Originality Check service. Preface On the orientation day, I remembered a presentation that highlighted the importance of understanding domain-specific knowledge in Data Science. Thus, I would like to express my gratitude to my supervisors, Martti Juhola and Markku Turunen for their guidance in developing this Master’s thesis. Their help and advice have helped me to combine and apply pieces of knowledge from Data Science and Human-Computer Interaction. I hope that this thesis will be a good beginning for my endeavors as I believe that continuous learning and curiosity will make me a better data scientist and a person in general. I am equally thankful to the members of the Pervasive Interaction Research Group (PIRG) and the TAUCHI Research Center. This community has been very supportive and encouraging critical thinking at my work and during the development of this thesis. Their contributions, particularly in providing data, knowledge, and support have significantly enriched this work. I would like to extend my special appreciation to John Mäkelä, Alisa Burova, and Jari Kangas, whose insights and assistance have played important roles in refining this work. In addition, I would like to express my gratitude to my family and friends who have supported me during my studies even though it has been more than 4 years since we have been together. Their encouragement

FAQs

What usability metric does the thesis use for classification?

The thesis classifies usability using System Usability Scale (SUS) scores, separating above- and below-average usability cases.

Which text-entry methods were tested in the Virtual Reality experiment?

Two methods were tested: speech-to-text and a virtual keyboard.

How were eye-tracking features extracted for machine learning models?

Eye-tracking data were extracted using a sliding window approach, and the resulting metrics were used as features for the models.

Aug 5, 2026

Upload by Lucas Martin