EO-VGGT: Orbital Ray-Conditioned 3D Foundation Models for Satellite Multi-View Reconstruction

Q: What problem does EO-VGGT address in satellite multi-view reconstruction?

EO-VGGT addresses the structural mismatch between implicit perspective assumptions in feed-forward 3D foundation models and the explicit orbital pushbroom geometry, amplified by heterogeneous viewsets in satellite imagery.

Q: How does EO-VGGT condition a frozen model on orbital observations?

It embeds explicit physical geometry by using GCCS for optimized input sequence selection, a Sensor-Ray Encoder that tokenizes pushbroom lines of sight derived from the Rational Function Model (RFM), and a lightweight Ray-Pointing-Aware Adapter that integrates these tokens into the transformer backbone.

Q: What improvements does EO-VGGT achieve on the US3D benchmark?

Evaluations show EO-VGGT outperforms traditional photogrammetric pipelines, learning-based approaches, and neural rendering methods, reducing mean absolute error by 18.0% and the 95th percentile absolute residual (P95Abs) by 18.6% while maintaining dense spatial coverage under heterogeneous radiometric conditions.

Jul 16, 2026

Upload by Lucas Martin

In satellite constellations, multi-view optical imagery is central to Earth Observation and accurate Digital Surface Model (DSM) reconstruction. Feed-forward 3D foundation models are limited for remote sensing due to a mismatch between implicit perspective assumptions and the explicit orbital pushbroom geometry, further worsened by heterogeneous viewsets. EO-VGGT adapts a frozen perspective-driven model using explicit physical geometry embedding, combining GCCS view selection, RFM-derived Sensor-Ray Encoding, and a lightweight adapter with <0.1% overhead for unified dense 3D inference.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

Abstract

Introduction

Related Work

EO-VGGT: Orbital Ray-Conditioned 3D Foundation Models for Satellite Multi-View Reconstruction Qiyan Luoa , Yingdong Pia,∗, Lekang Wena , Jie Yanga , Xiaoyu Wanga , Haiming Zhanga,b , Mi Wanga,b a State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, 430079, China b Hubei Luojia Laboratory, Wuhan, 430079, China Abstract In the era of satellite constellations, multi-view optical satellite imagery is pivotal for Earth Observation (EO) and high-quality Digital Surface Model (DSM) reconstruction. Although feed-forward 3D foundation models have transformed computer vision, their deployment in satellite remote sensing is inherently constrained by the structural discrepancy between implicit perspective assumptions and explicit orbital pushbroom geometry. This geometric incongruity is further compounded by pronounced viewset heterogeneity. We present EO-VGGT, a framework that adapts a frozen perspective-driven model to orbital observations via explicit physical geometry embedding.First, the Geometry-Correlation Constrained Selection (GCCS) strategy prunes sub-optimal observations by balancing geometric diversity and radiometric consistency to optimize the input sequence. Second, a Sensor-Ray Encoder (SRE) parameterizes pixel-level pushbroom lines of sight derived from the Rational Function Model (RFM) into highdimensional space-geometric tokens, reconciling the mathematical discrepancy between central projection and orbital kinematics. Third, a lightweight Ray-Pointing-Aware Adapter (RPAA) employs gated residual blocks to integrate these tokens directly into the frozen transformer backbone. With under 0.1 % parameter overhead, this joint conditioning paradigm aligns cross-domain latent representations and facilitates unified forward inference to reconstruct accurate 3D structures.Evaluations on the US3D benchmark demonstrate that EO-VGGT substantially outperforms traditional photogrammetric pipelines, learning-based approaches, and recent neural rendering methods. By overcoming the limited radiometric adaptability and poor cross-domain generalization inherent in these conventional approaches, our framework achieves an 18.0 % reduction in mean absolute error and an 18.6 % reduction in the 95th percentile absolute residual P95Abs compared to the most competitive alternative method while maintaining full dense spatial coverage across heterogeneous radiometric conditions. Our findings underscore that integrating explicit physical geometry with optimized view selection is essential for robust feed-forward satellite 3D reconstruction. Keywords: Satellite multi-view reconstruction, Rational Function Model (RFM), 3D foundation models, Digital Surface Model (DSM) 1. Introduction The rapid proliferation of satellite networks and the deployment of dense orbital constellations have initiated a transformative era for modern Earth Observation (EO) (Li et al., 2017; Wang et al., 2024a) . Driven by the continuous expansion of these mega-constellations, the unprecedented global coverage and high revisit frequencies provide massive multi-view observation streams. Within this data-rich landscape, leveraging the cross-view synergy of multi-view optical imagery has emerged as a powerful paradigm for high-quality Digital Surface Model (DSM) reconstruction. This capability provides a critical data foundation with substantial potential for large-scale geographic mapping (Wang et al., 2025b; Pi et al., 2019; Li et al., 2024), city-scale digital twins (Qian et al., 2023, 2026), and dynamic change monitoring (Peng et al., 2020; Qin et al., 2016) . Traditionally, satellite multi-view reconstruction relies on a complex, decoupled pipeline involving Structure from Motion (SfM), Bundle Adjustment (BA), and dense matching (Pi et al., ∗ Corresponding author. Email address: [pyd_imars@whu.edu.cn](pyd_imars@whu.edu.cn) 2019; Yang et al., 2018; Gong et al., 2022; Huang et al., 2020) . Al

FAQs

What problem does EO-VGGT address in satellite multi-view reconstruction?

EO-VGGT addresses the structural mismatch between implicit perspective assumptions in feed-forward 3D foundation models and the explicit orbital pushbroom geometry, amplified by heterogeneous viewsets in satellite imagery.

How does EO-VGGT condition a frozen model on orbital observations?

It embeds explicit physical geometry by using GCCS for optimized input sequence selection, a Sensor-Ray Encoder that tokenizes pushbroom lines of sight derived from the Rational Function Model (RFM), and a lightweight Ray-Pointing-Aware Adapter that integrates these tokens into the transformer backbone.

What improvements does EO-VGGT achieve on the US3D benchmark?

Evaluations show EO-VGGT outperforms traditional photogrammetric pipelines, learning-based approaches, and neural rendering methods, reducing mean absolute error by 18.0% and the 95th percentile absolute residual (P95Abs) by 18.6% while maintaining dense spatial coverage under heterogeneous radiometric conditions.

Jul 16, 2026

Upload by Lucas Martin