Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments - PhD thesis

Aug 4, 2026

Upload by Anda

This thesis investigates real-time touchless hand interaction and the use of machine learning agents to enhance immersive learning environments. It surveys immersive technology in education, touchless interaction technologies, and intelligent agents, then designs and implements AGILEST, an agent-driven approach for self-guided learning in XR. Prototypes target both handheld devices and HMDs, including learning, practice, training, and quiz modules. Evaluation examines AGILEST performance through user-based assessment.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages261

Reading Time10 h 58 min

AccessFree to download

Introduction

Research Motivation

Research Questions

Objectives

Contributions

Thesis Overview

Immersive Technology in Education

XR Learning Applications

Immersive Learning in Different Domains

Analysis

Conclusion

Touchless Interaction Technology

What is Touchless Technology?

Types of Touchless (Contactless) Interfaces

Touchless Technology in Education

Conclusion

Intelligent Agents

Defining an Intelligent Agent

Why Agents are Useful?

Rules and Types of Agents

Training Intelligent Agents

Unity ML-Agents in Immersive Learning

Conclusion

AGILEST Approach

Machine Learning Agents for self-guided learning

Unity ML-Agents and Remote Learning in XR

Learning PC Assembling

Conclusions

AGILEST Prototypes Implementation

AGILEST Approach-Handheld

AGILEST Approach-HMDs

HMD vs Handheld Prototype

Conclusion

Evaluation

Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments by Muhammad Zahid Iqbal This thesis is submitted to University College Dublin in fulfilment of the requirements for the degree of Doctor of Philosophy in School of Computer Science and Informatics Head of School: Prof. Chris Bleakley Principal Supervisor: Dr. Abraham G. Campbell CONTENTS Abstract x Acknowledgements xiii Publications xiv 1 Introduction 1 1.1 Research Motivation ............................... 4 1.2 Research Questions ............................... 5 1.3 Objectives .................................... 6 1.4 Contributions ................................... 6 1.5 Thesis Overview ................................. 7 2 Immersive Technology in Education 10 2.1 XR Learning Applications ............................ 13 2.1.1 XR Learning in Formal Classrooms .................. 15 2.1.2 XR Learning in Special Education ................... 15 2.1.3 XR Learning Outside the Classroom .................. 15 2.1.4 XR for Collaborative Learning ..................... 15 2.2 Immersive Learning in Different Domains .................... 16 2.2.1 Interactive Learning for Early classes ................. 16 2.2.2 Interactive Learning for Higher Classes ................ 18 2.2.3 STEM(Science, Technology, Engineering, and Mathematics) Education 20 2.2.4 Language Learning ........................... 24 2.2.5 Collaborative Learning ......................... 25 2.2.6 Environment & History Learning ................... 25 2.2.7 Special Education ............................ 27 2.2.8 MOOC (Massive Open Online Courses) ................ 27 2.2.9 Technical Training ........................... 28 2.2.10 XR Authoring Tools .......................... 29 2.2.11 Intelligent Agents in XR ........................ 31 2.3 Analysis ..................................... 40 2.3.1 Education Level ............................. 40 2.3.2 Educational Domain .......................... 41 2.3.3 Experiments Conducted to Evaluate XR Education .......... 41 2.3.4 Libraries Used ............................. 42 2.3.5 Devices ................................. 43 2.3.6 Tracking ................................. 44 2.3.7 User Interaction ............................ 45 2.3.8 Collaboration .............................. 45 2.3.9 Agents .................................. 45 2.3.10 Potential Future Benefits of XR in Remote Education ......... 46 2.3.11 Limitations ............................... 49 2.4 Conclusion .................................... 49 3 Touchless Interaction Technology 51 3.1 What is Touchless Technology? ......................... 52 3.2 Types of Touchless (Contactless) Interfaces ................... 53 3.2.1 Personal Devices for Touchless Interaction ............... 56 3.2.2 Gesture Recognition & Hand Interaction ................ 56 3.2.3 Motion Sensing ............................. 56 3.2.4 Eye/Gaze Tracking ........................... 57 3.2.5 Touchless Typing ............................ 57 3.2.6 Contactless Payments .......................... 58 3.2.7 Voice Control / Virtual Assistant .................... 58 3.3 Touchless Technology in Education ....................... 59 3.3.1 Kinesthetic Learning .......................... 62 3.4 Conclusion .................................... 67 4 Intelligent Agents 68 4.1 Defining an Intelligent Agent .......................... 69 4.2 Why Agents are Useful? ............................. 70 4.3 Rules of Agents ................................. 72 4.4 Types of Agents ................................. 72 4.4.1 Reflex Agents .............................. 72 4.4.2 Model-based Agents .......................... 72 4.4.3 Goal-based Agents ........................... 73 4.4.4 Utility-based Agents .......................... 73 4.4.5 Learning Agents .........................

FAQs

What core problems does the thesis address in immersive learning environments?

The thesis focuses on real-time touchless hand interaction and on how machine learning agents can support adaptive, self-guided learning in immersive (XR) environments.

What is AGILEST and how does it support learning?

AGILEST is an agent-based approach that combines learning and practice modules with training using Unity ML-Agents to enable self-guided learning and interaction.

How are the prototypes evaluated and what devices are involved?

Evaluation targets AGILEST on handheld devices and HMDs, using participant recruitment and an evaluation procedure to compare and assess learning-related outcomes.

Aug 4, 2026

Upload by Anda