AGILEST approach - Using machine learning agents to facilitate kinesthetic learning in STEM education through touchless interaction

Aug 3, 2026

Upload by Himbo

Interactive learning applications are increasingly sought in STEM, driven by innovative interaction technologies. Advances in machine learning enable near-perfect touchless hand tracking for augmented reality (AR) interaction, yet kinesthetic learning pedagogy combined with agents and AR remains underexplored. In a post-COVID learning context, the AGILEST approach proposes machine learning agents as both trainer and assessor for touchless, real-time hand-interaction. A chemistry AR case study is evaluated remotely with usability experts using NASA-TLX, PUEU, and PU.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Motivation and background

Remote learning and STEM hands-on challenges

XR/AR for education and engagement

Kinesthetic learning and hand-tracking in virtual environments

Proposed AGILEST approach

Touchless interaction in AR learning

Machine learning agents as trainer and assessor

Case study and evaluation

Chemistry AR learning application

Remote usability study and expert review

Metrics: NASA-TLX, PUEU, PU

Telematics and Informatics Reports 9 (2023) 100034 Contents lists available at ScienceDirect Telematics and Informatics Reports journal [homepage: www.elsevier.com/locate/teler](homepage: www.elsevier.com/locate/teler) | AGILEST approach: Using machine learning agents to facilitate kinesthetic learning in STEM education through real-time touchless hand interaction Muhammad Zahid Iqbal∗, Abraham G. Campbell
School of Computer Science, University College Dublin, Ireland | | | | | --- | --- | --- | --- | | a r t i c l e i n f o | | a b s t r a c t | | | Keywords:
Augmented reality Technology enhanced learning Touchless hand interaction Machine learning
Learning technology | | There is an increasing interest in creating interactive learning applications using innovative interaction technologies, especially in STEM (Science, technology, engineering, and mathematics) subjects. Recent developments in machine learning have allowed for nearly perfect hand-tracking recognition, introducing a touchless modality for interaction within Augmented Reality (AR) environments. However, the research community has not explored the pedagogical approach of Kinesthetic Learning or “Learning by Doing”, hand tracking, and machine learning agents combined with Augmented Reality technology. Fundamentally, this exploration of touchless interaction technologies has taken on new importance in the new post-COVID world. Meanwhile, machine learning has gained attention for its ability to enhance personalized learning and play a vital new role as a virtual instructor. This paper proposes a novel approach called the AGILEST approach, which uses machine learning Agents to facilitate interactive kinesthetic learning in STEM education through touchless interaction. The ﬁrst case study for this approach will be an AR learning application for chemistry. This application uses real-time touchless hand interaction for kinesthetic learning and uses a machine learning agent to act as both trainer and assessor of the user. The evaluation of this research has been conducted remotely through a usability study with expert reviewers, which includes 15 young researchers with peer-reviewed work in Human-Computer Interaction & AR and 2 subject experts STEM teachers at the secondary school level. The usability evaluation through NASA Task Load Index (NASA-TLX), Perceived Ease of Use (PUEU), and Perceived Usefulness (PU) with expert reviewers provide positive feedback about this approach for productive learning gain, engagement and interactiveness in learning STEM subjects. | | Learning Technology is continuously evolving, and its importance is now being recognized worldwide due to the COVID-19 pandemic. As a result, most countries have adopted some form of remote learning during the outbreak’s peak. The ﬁrst country to do this was China which adopted the Suspending Classes Without Stopping Learning Policy [46], which examined alternative learning solutions that facilitate the use of technology integration in remote learning. One area where remote learning suﬀers is naturally in hands-on experiments for STEM (Science, technology, engineering, and mathematics) subjects. However, with the development of new interaction techniques and intelligence, there are possibilities to support STEM subjects learning virtually. One approach is to use eXtended Reality (XR), which is an umbrella term for Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality(VR) as explained in Fig. 1 and especially suitable for presenting information during experimentation, as it can be used to integrate both physical and virtual lab work [2]. By augmenting the real world with virtual objects, AR provides new possibilities for education in diﬀerent educational contexts [23,45]. For example, in the learning process, student engagement is a critical component, and AR has been proven to be highly motivational for students to create more involvement in the learning process [38]

FAQs

What problem does the AGILEST approach address in STEM education?

It addresses the difficulty of delivering effective hands-on, kinesthetic learning for STEM topics in virtual or remote settings, especially when traditional physical experiments are hard to conduct.

How does the approach use touchless hand interaction in AR?

It uses real-time touchless hand interaction inside an AR learning environment to support learning by doing without requiring users to touch devices.

How was the proposed system evaluated and what metrics were used?

The work was evaluated remotely through a usability study with expert reviewers, using NASA-TLX for workload and PUEU/PU for perceived ease of use and perceived usefulness.

Aug 3, 2026

Upload by Himbo