SPHERE - Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL

Oct 12, 2026

While Large Language Models (LLMs) advance 3D indoor scene synthesis, current pipelines fail to retain user-specific preferences across sessions, making immersive authoring repetitive and physically fatiguing. SPHERE is an adaptive VR generation framework that turns isolated synthesis into continuous human-AI co-creation by extracting persistent spatial preferences from natural multimodal interactions, including speech and controller edits. It models edits as hierarchical constraints capturing local functional and global topological contexts, then uses human-in-the-loop reinforcement learning to update retrieval policies from final user-edited scenes. A mixed-design user study (N=42) and offline ablations show reduced corrective edits and physical demand, while improving geometric resilience and aligning layouts with demonstrated spatial logic without costly fine-tuning.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages11

Reading Time4 min

AccessFree to download

Introduction

Problem: preference loss in session-to-session editing

Approach: persistent preference memory with hierarchical constraints

Learning: human-in-the-loop reinforcement learning for policy updates

SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced arXiv : 26 10 .02023v 1 [cs .AI] 1 Oct 2026 Spatial Preference Learning and Human-in-the-Loop RL Hyeonmin Lee , Zheng Wei , Kyungmin Kwon , Jumin Seo , Jiwon Park , Hayoung Oh * , Figure 1: SPHERE incrementally adapts subsequent VR indoor scenes by inferring user preferences and spatial tendencies from an initial user-edited scene (left) . Across the generated environments (1-3), SPHERE reflects user-specific layout patterns, stylistic and design preferences, as well as object-level tendencies. Abstract—While Large Language Models (LLMs) advance 3D indoor scene synthesis, current pipelines fail to retain user-specific preferences across sessions, making immersive authoring a repetitive and physically fatiguing process. We present SPHERE, an adaptive VR generation framework that transforms isolated synthesis into continuous human-AI co-creation. SPHERE extracts persistent spatial preferences from natural multimodal interactions (speech and controller edits) . To ensure geometric resilience against spatial distortions, it abstracts these raw edits into hierarchical constraints modeling both local functional and global topological contexts. Furthermore, a human-in-the-loop reinforcement learning mechanism dynamically updates retrieval policies based on the user’s final edited scenes. A mixed-design user study (N = 42) and an offline ablation demonstrate that SPHERE significantly reduces corrective edits and physical demand, preventing bias toward shallow object-level traits to yield geometrically resilient, profile-aligned layouts. Ultimately, SPHERE demonstrates how capturing demonstrated spatial logic enables controlled spatial adaptation, establishing a reliable, governed human-AI collaboration framework for immersive authoring. Project page and source code will be available at: [https://github.com/hyeonmin11/SPHERE](https://github.com/hyeonmin11/SPHERE) Index Terms—Virtual Reality, 3D Scene Generation, Preference Learning 1 INTRODUCTION Recent advancements in 3D indoor scene synthesis (3DISS) have increasingly leveraged powerful generative models to construct complex, object-centric environments. Contemporary systems equipped with Large Language Models (LLMs) and multimodal architectures can now dynamically encode user-provided text to guide zero-shot objectselection and spatial placement [7, 27, 30, 38, 58] . However, immersive spatial design is not merely a one-off generation task; it is an ongoing process of centering humans to shape intelligence. In this process, establishing a governed AI collaboration framework is essential to reliably bridge human intent with predictable AI outcomes. Despite algorithmic leaps in text-to-3D generation, end-to-end LLMdriven pipelines exhibit a systemic shortcoming: they systematically fail to explicitly model the latent evolution of user preferences across repeated, temporally disjoint interactions. Existing sequential editing frameworks [28, 56, 60] predominantly treat user modifications as isolated, context-free error corrections. Consequently, users are burdened with repeatedly executing identical corrective operations across discrete authoring tasks, which significantly increases interaction friction and • Hyeonmin Lee, Kyungmin Kwon, Jumin Seo, Jiwon Park, and Hayoung Oh are with Sungkyunkwan University. E-mail: [llhyeonmin12@gmail.com](llhyeonmin12@gmail.com), [km0228kr@skku.edu](km0228kr@skku.edu), [zox2mm@g.skku.edu](zox2mm@g.skku.edu), [orilove1@g.skku.edu](orilove1@g.skku.edu), and [hyoh79@skku.edu](hyoh79@skku.edu). • Zheng Wei is with The Hong Kong University of Science and Technology. E-mail: [zwei302@connect.ust.hk](zwei302@connect.ust.hk). physical fatigue in VR environments. Furthermore, prevailing spatial reasoning mechanisms often anchor spatial relationships to rigid absolute Cartesian coordinates [23, 38] or flat, unstructured semantic graphs [27, 43] . These formulations inheren

FAQs

What limitation do existing LLM-based 3D indoor scene systems have for immersive authoring?

They fail to explicitly model and retain user preferences across temporally disjoint sessions, forcing users to repeat similar corrective edits and increasing interaction friction and physical fatigue.

How does SPHERE capture user preferences from VR interactions?

SPHERE extracts persistent spatial preferences from natural multimodal behaviors, combining speech-driven instructions with controller-grounded spatial edits as continuous implicit signals.

How does SPHERE ensure geometric resilience when applying user-informed constraints?

It abstracts raw edits into hierarchical constraints that model both local functional relationships and global topological context, helping layouts remain robust under geometric scaling or distortion.

Oct 12, 2026

Upload by Valentina