Real-time hand interaction and self-directed machine learning agents in immersive learning environments

Aug 4, 2026

Upload by Anda

Integration of extended reality (XR) in education is increasingly used to reshape traditional classrooms through immersive learning environments. Growing interest in XR interaction techniques and artificial intelligence enables interactive, self-directed learning, yet research remains limited on combining these tools with kinesthetic, hands-on pedagogy using real-time hand interaction and agent-guided instruction. This paper proposes machine learning agent support for interactive kinesthetic learning in science and engineering, demonstrated via a chemistry case study and evaluated for usability, workload, and perceived acceptance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Introduction

Extended Reality for immersive learning

Self-directed learning and immersive learning environments

Hand interaction for kinesthetic learning in STEM

Intelligent agents and learning support

Computers & Education: X Reality 3 (2023) 100038 Contents lists available at ScienceDirect Computers & Education: X Reality journal [homepage: www.journals.elsevier.com/computers-and-education-x-reality](homepage: www.journals.elsevier.com/computers-and-education-x-reality) | Real-time hand interaction and self-directed machine learning agents inimmersive learning environments
*
Muhammad Zahid Iqbal , Abraham G. Campbell
School of Computer Science, University College Dublin, Ireland | | | | | --- | --- | --- | --- | | A R T I C L E I N F O | | A B S T R A C T | | | Keywords:
Mixed reality
Virtual reality
Extended reality
Technology enhanced learning Immersive learning
XR
Learning technology | | Integration of extended reality (XR) in education is becoming popular to transform the traditional classroom with immersive learning environments. The adoption of immersive learning is accelerating as an innovative approach for science and engineering subjects. With new powerful interaction techniques in XR and the latest developments in artificial intelligence, interactive and self-directed learning are becoming important. However, there is a lack of research exploring these emerging technologies research with kinesthetic learning or “handsone learning" as a pedagogical approach using real-time hand interaction and agent-guided learning inimmersive environments. This paper proposes a novel approach that uses machine learning agents to facilitate interactive kinesthetic learning in science and engineering education through real-time hand interaction in the virtual world. To implement the following approach, this paper uses a chemistry-related case study and presentsa usability evaluation conducted with 15 expert reviewers and 2 subject experts. NASA task load index is used for cognitive workload measurement, and the technology acceptance model is used for measuring perceived ease of use and perceived usefulness in the evaluations. The evaluation with expert reviewers proposed self-directed learning using trained agents can help in the end-user training in learning technical topics and controller-freehand interaction for kinesthetic tasks can improve hands-on learning motivation in virtual laboratories. This success points to a novel research area where agents embodied in an immersive environment using machine learning techniques can forge a new pedagogical approach where they can act as both teacher and assessor. | | 1. Introduction Extended Reality (XR) is transforming learning technology by providing the ability to create more interactive learning content (Skult & Smed, 2020) and simulating immersive user experiences to understand complex technical concepts more practically (Dawley & Dede, 2014). The use of XR as learning technology, explained by Dengel (2022) asimmersive learning, has increased quickly in the last couple of years due to the higher immersion capabilities. The adoption of Head-Mounted Displays (HMDs) and smart glasses is increasing with time due to improved interaction techniques, more accessibility, decreasing cost, and increasing portability. Following these developments in HMDs and computer graphics, immersive learning technology has gained enough potential to take technology-enhanced learning in resource-constrained environments closer to real-world settings (Gao et al., 2021). Among all other novel approaches in learning technologies, Immersive Learning Environments (ILE) are the most revolutionary interactive platforms (Freina & Ott, 2015), that can become more productive with self-guided learning and hands-on learning capabilities. Self-guided learning which also known as self-directed learning (Abdullah, 2001) helps learners to take initiative and responsibility for planning, organizing, and executing their learning plans (Sandars & Walsh, 2016). In this type of learning, learners are not depending on direct supervision or guidance, rather they have control over wh

FAQs

What problem does the paper address about immersive learning?

It addresses the limited research exploring how real-time hand interaction and agent-guided learning can support kinesthetic, hands-on pedagogy in immersive environments.

How does the proposed approach support kinesthetic learning?

It uses machine learning agents to facilitate interactive kinesthetic learning in science and engineering education through real-time hand interaction within a virtual world.

How was the approach evaluated in the study?

A chemistry-related case study was used, with usability evaluation by 15 expert reviewers and 2 subject experts, including NASA task load index for cognitive workload and a technology acceptance model for perceived usefulness and ease of use.

Aug 4, 2026

Upload by Anda