CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models

Jul 16, 2026

Upload by Asher

Existing 3D scene-grounded large language models concentrate on question answering in simplified single-room 3D scenes, without robust reasoning across real household settings with multiple interconnected rooms and varied object categories. CAIRN introduces a topology-aware 3D-LLM for multi-room scene understanding, aligning transformer attention with scene hierarchy to expose object relations and room connectivity. It enriches object tokens using graph neural networks, adds learned room tokens, and applies hierarchical attention with geometric bias. CAIRN is built on CAIRN-MR, a benchmark on HM3D with grounding, captioning, and four QA tasks to evaluate intra- to cross-room reasoning, achieving strong gains over prior 3D-LLMs while staying competitive on single-room benchmarks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

Introduction

Background and motivation

Proposed approach and benchmark

arXiv :2607 .06534v2 [ cs .CV] 13 Jul 2026 CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models He Liang1 ∗ Chenyang Ma1 Yiming Zhang3 Sangyun Shin1 Andrew Markham 1 Niki Trigoni 1 Yuhang He2† 1University of Oxford 2Microsoft Research 3 Simon Fraser University {he.liang, [chenyang.ma](chenyang.ma) , sangyun.shin, andrew.markham, [niki.trigoni}@cs.ox.ac.uk](niki.trigoni}@cs.ox.ac.uk) [yuhanghe@microsoft.com](yuhanghe@microsoft.com) [yza440@sfu.ca](yza440@sfu.ca) Abstract Existing 3D scene-grounded Large Language Models (3D-LLMs) focus on answer ing questions grounded in simplified single-room 3D scenes, lacking the ability to reason over real-world household environments containing multiple interconnected rooms and diverse object categories. We introduce CAIRN, a topology-aware 3D LLM for multi-room 3D scene understanding. CAIRN aligns transformer attention with scene hierarchy, giving the model explicit awareness of object-level relations and room-level connectivity. It enriches object tokens with room-local relational context via a graph neural network, introduces learned room tokens for room-level abstraction, and applies a hierarchical attention mask with geometric bias to route information according to scene topology. CAIRN is developed on CAIRN-MR, a benchmark we introduce on HM3D for multi-room 3D scene understanding, cover ing grounding, captioning, and four question-answering tasks that progressively evaluate from intra-room perception to cross-room reasoning. Experiments show that CAIRN outperforms prior 3D-LLMs by a large margin across all CAIRN-MR tasks while remaining competitive on five single-room benchmarks. 1 Introduction A capable home assistant agent should recall where an object was last seen in the kitchen and retrieve it from another room, or answer queries that relate objects across spaces (e.g., comparing furniture arrangements across bedrooms) . This requires understanding the full 3D scene structure of a household and performing consistent reasoning over the entire multi-room environment. There has been a rich history of prior work on developing agents that answer questions grounded in 3D scenes, using large-language models (LLMs) augmented with structured 3D scene representations (i.e., 3D-LLMs) as the reasoning backbone [28, 29, 31, 32, 57, 63] . However, these approaches largely restrict 3D scene understanding and spatial reasoning to a single room [8, 10, 20–22, 43, 60], representing scenes as flat sequences of object tokens without modeling higher-level room structure. In this work, we aim to move beyond the single-room paradigm and develop 3D-LLMs that can understand full household settings [42], where multiple interconnected rooms contain hundreds of objects. This unlocks more challenging tasks such as grounding across spatial contexts, tracing relationships between rooms, and reasoning over a scene’s topological structure. To enable the study of multi-room 3D scene understanding, we first introduce CAIRN-MR, a benchmark for structured multi-room 3D scene understanding built on HM3D [44], as existing 3D ∗ Part of this work was done during an internship at Microsoft Research.†Corresponding author Project Page: [https://oceansdepp.github.io/cairn_web/](https://oceansdepp.github.io/cairn_web/) Preprint. Figure 1: 3D-LLMs for multi-room scene understanding. We introduce CAIRN, a topology-aware 3D-LLM for multi-room scene understanding, along with CAIRN-MR, a multi-room 3D scene understanding benchmark with diverse tasks. By representing each scene as a hierarchical scene graph and aligning attention with scene topology through structured masking and geometric bias, CAIRN achieves substantial gains over prior 3D-LLMs. scene-language datasets [1, 5, 11, 37, 61] are mostly limited to single-room scans. Unlike prior benchmarks where reasoning is confined to a single room with a limited candidate set, CAIRN-MR shi

FAQs

What limitation do existing 3D scene-grounded 3D-LLMs have in household environments?

They focus on simplified single-room scenes and lack the ability to reason over multiple interconnected rooms with diverse object categories in real households.

How does CAIRN handle multi-room 3D scene understanding differently from prior models?

CAIRN models a hierarchical 3D scene graph and aligns transformer attention with scene topology using structured masking and geometric bias, enabling explicit room connectivity and object-level relations.

What is CAIRN-MR and how does it evaluate the model?

CAIRN-MR is a benchmark for structured multi-room 3D scene understanding on HM3D, providing grounding, captioning, and four progressively harder question-answering tasks that span from room-grounded perception to cross-room reasoning.

Jul 16, 2026

Upload by Asher