Hydra++ Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation

Jul 16, 2026

3D scene graphs encode objects, places, and their spatial relationships, yet existing systems often approximate geometry using centroids, bounding boxes, partial point clouds, or class-level CAD priors, limiting instance-specific detail. Hydra++ integrates learning-based, category-agnostic object shape estimation into a hierarchical 3D scene graph pipeline. The framework adds a reprojection-mask consistency check (RMCC) to filter degenerate outputs, evaluates slower modular estimators to study latency and generalization tradeoffs, and supports hybrid LiDAR-camera sensing for robust outdoor reconstruction. Experiments in simulation and real campus scenes improve object- and scene-level fidelity.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Introduction

Motivation and Problem

Hydra++ Approach and System Design

Contributions

arXiv :2607 .09455v1 [ cs .CV] 10 Jul 2026 Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation Hyungtae Lim 1 , Nathan Hughes 1 , Xihang Yu 1 , Ruihan Xu 1 , Yun Chang 1 , Jingnan Shi 1 , Rajat Talak2 , and Luca Carlone 1 Indoor (RGB-D Camera) 5 m Outdoor (3D LiDAR + RGB/RGB-D Camera) (b) 10 m Fig. 1. 3D scene graphs with object-level shape reconstruction generated by Hydra++ . (a) Indoor environment using the uHumans2 dataset [1] . Zoom-in views show full object shapes reconstructed from partial observations. Each color represents a semantic class (e.g., couches in pink, plants in green, trash bins in purple, and chairs in red) . The object shape estimation network in our framework captures intra-class variation, as illustrated by the reconstructed trash bins and chairs with different geometries. (b) Outdoor campus scene using the hybrid LiDAR-camera configuration. Our method reconstructs objects of various sizes, from a small fire hydrant to a large car, under realistic sensing conditions. Abstract—3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities (e.g., objects, places) and their relationships. However, existing scene graph systems model object geometry coarsely, relying on partial point clouds or class-level CAD templates, which limits instancespecific shape detail. This paper presents Hydra++, a systemlevel investigation into how learning-based object shape estimators can be integrated into a hierarchical 3D scene graph pipeline. Hydra++ incorporates category-agnostic shape estimation and a reprojection-mask consistency check (RMCC) to reject degenerate predictions from partial observations or imprecise segmentation. In its default CRISP-based configuration, Hydra++ performs online scene graph construction; slower estimators such as SAM3D are evaluated as modular alternatives to demonstrate generalization-latency tradeoffs. Furthermore, to address the challenges of sparse and noisy depth measurements in outdoor environments, Hydra++ supports a hybrid LiDAR-camera configuration for largescale operation, improving scene-level reconstruction quality. Experiments in both simulation and real-world outdoor campus scenarios demonstrate that Hydra++ improves object- and scene-level reconstruction quality. Project page is available at [https://hydra-plusplus.github.io/](https://hydra-plusplus.github.io/). 1H. Lim, N. Hughes, X. Yu, R. Xu, Y. Chang, J. Shi, and L. Carlone are with the Laboratory for Information & Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USA, {shapelim, na26933, jimmyyu, multyxu, yunchang, jnshi, [lcarlone}@mit.edu](lcarlone}@mit.edu) [2](2 R. Talak is with the Department of Electrical and Computer Engineering)[ R. Talak is with the Department of Electrical and Computer Engineering](2 R. Talak is with the Department of Electrical and Computer Engineering)[ ](2 R. Talak is with the Department of Electrical and Computer Engineering)at the National University of Singapore, Singapore, {[talak@nus.edu.sg}](talak@nus.edu.sg}) This work was partially funded by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. RS- 2024-00461409) and by the Department of the Air Force Artificial Intelligence Accelerator, accomplished under Cooperative Agreement Number FA8750-19-2-1000 . I. INTRODUCTION Modeling 3D scenes is crucial for many robotics tasks, from navigation to manipulation [2] . With the rise of deep learning and large language models (LLMs), many studies have examined semantic mapping, which captures semantic information about the environment to support reasoning and interaction [3]–[6] . In particular, recent 3D scene graph approaches encode semantics by treating entities (e.g., objects, rooms, and agents) as nodes, and relationships as edges (e.g., adjacency, inclusion, and support), yielding a lightweight and efficient abstract

FAQs

What sensing setup does Hydra++ support for large-scale outdoor scenes?

Hydra++ supports a hybrid LiDAR-camera configuration, combining metric-scale depth from a 3D LiDAR with image-based shape estimation to improve both object-level and scene-level reconstruction quality.

Jul 16, 2026

Upload by Theodora