ImmersiveIML - Immersive interactive machine learning for 3D point cloud classification - the neural network at your fingertips

Aug 4, 2026

Upload by Finn

ImmersiveIML proposes an immersive, interactive virtual reality (VR) dialogue for interactive machine learning (IML) aimed at classifying points in 3D point clouds. The system supports an iterative human-in-the-loop trial-and-error process that converges to classifications aligned with human intent. Interactions are designed to be expressive yet minimal, combining 6-DOF controllers for direct interaction via minimal brushing, a fast trainable machine learning model, and immediate visual feedback of classification results. Results are validated through scenarios including 3D edge detection and city LiDAR tree classification.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Abstract

1 Introduction

3D point clouds and classification challenges

Human-in-the-loop and interactive machine learning motivation

Human classification intent and iterative workflow

ImmersiveIML – Immersive interactive machine learning for 3D point cloud classification: the neural network at your fingertips Maxime Cordeil, Thomas Billy, Nicolas Mellado, Loïc Barthe, Nadine Couture, Patrick Reuter To cite this version: Maxime Cordeil, Thomas Billy, Nicolas Mellado, Loïc Barthe, Nadine Couture, et al.. ImmersiveIML – Immersive interactive machine learning for 3D point cloud classification: the neural network at your fingertips. IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMARAdjunct 2023), Oct 2023, Sydney, Australia. pp.81-85, 10 . 1109/ISMAR-Adjunct60411 .2023.00025 . hal-04246601 HAL Id: hal-04246601 [https://hal.science/hal-04246601](https://hal.science/hal-04246601) Submitted on 17 Oct 2023 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. ImmersiveIML – Immersive interactive machine learning for 3D point cloud classification: the neural network at your fingertips Maxime Cordeil* University of Queensland Lo¨ıc Barthe‡ CNRS, IRIT, Universit de Toulouse Thomas Billy University of Queensland Nicolas Mellado† CNRS, IRIT, Universit de Toulouse Nadine Couture§ Univ. Bordeaux, ESTIA Institute of Technology Patrick Reuter¶ Univ. Bordeaux, Inria, LaBRI, ESTIA Human classification intent (1) Direct input: labelling of two subsets P1 and P2 (2) Training and inference: classification of all points in P (3) Output: visualisation of the result Figure 1: Starting from a human intent, the iterative human-in-the-loop process is composed of (1) 6-DOF controllers to allow direct interaction in large 3D point clouds via a minimal brushing technique,(2) a fast trainable machine learning model, and (3) the direct visual feedback of classification results. In the shown iteration, the classification result does not correspond to the user’s intent, so they refine the input to eventually converge to an acceptable classification of the data, iteratively. ABSTRACT We propose an initial exploration of an interactive machine-learning (IML) dialogue in immersive, interactive Virtual Reality (VR) for the classification of points in 3D point clouds. We contribute ImmersiveIML, an Immersive Analytics tool which builds on humanmachine learning trial-and-error dialogue to support an iterative classification process of points in the 3D point cloud. The interactions in ImmersiveIML are designed to be both expressive and minimal; we designed the iterative process to be supported by (1) 6-DOF controllers to allow direct interaction in large 3D point clouds via a minimal brushing technique,(2) a fast trainable machine learning model, and (3) the direct visual feedback of classification results. *[e-mail: m.cordeil@uq.edu.au](e-mail: m.cordeil@uq.edu.au)[ ](e-mail: m.cordeil@uq.edu.au)†e-mail: [nicolas.mellado@irit.fr](nicolas.mellado@irit.fr)[ ](nicolas.mellado@irit.fr)‡e-mail: [loic.barthe@irit.fr](loic.barthe@irit.fr) § e-mail: [n.couture@estia.fr](n.couture@estia.fr)[ ](n.couture@estia.fr)¶ e-mail: [preuter@labri.fr](preuter@labri.fr) This constitutes an iterative human-in-the-loop process that eventually converges to a classification according to human intent. We argue that this approach is a novel contribution that supports a constant improvement of the classification model and fast tracks classification tasks with this type of data, in an immersive scenario. We report on the design and implementation of ImmersiveIML and demonstrate its capabilities with two emblemat

FAQs

What problem does ImmersiveIML address in 3D point cloud classification?

It targets the challenge of obtaining classifications that match specific user needs, which is difficult when humans are not involved during learning. ImmersiveIML integrates human input to refine and verify classification outputs.

How does the system enable interaction in large 3D point clouds?

It uses 6-DOF controllers to support direct interaction in large point clouds via a minimal brushing technique. Users provide input subsets (P1 and P2) and the model learns from them for training and inference.

What interaction loop leads to convergence toward the intended classification?

The process iteratively compares the classification result with the user’s intent and lets the user refine the input. Through repeated interaction cycles, the workflow converges to an acceptable classification.

Aug 4, 2026

Upload by Finn