Consistency-Augmented Geometric Gaussian Splatting for Ego-centric 3D Scene Generation

Jul 16, 2026

Upload by Hazel

Ego-centric 3D scene generation remains limited by small view overlap and the strong bias of individual perspectives, which obstruct viewpoint-consistent, semantically aligned visual synthesis and the recovery of accurate geometric structures. CGGS proposes a text-to-3D framework that improves 3D content awareness and reduces geometric distortions. It fine-tunes a Multi-View Latent Diffusion Model with consistency-augmented loss for consistent 2D priors, derives dense point clouds using optical flow and correspondence, and refines 3D Gaussians via entropy-based mutual information depth loss with hierarchical optimization, achieving superior results over prior methods.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Introduction

Text-to-3D and multi-view priors

Limitations of ego-centric generation

CGGS overview and motivation

© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This article has been accepted for publication in IEEE Transactions on Image Processing. CGGS: Consistency–Augmented Geometric Gaussian Splatting for Ego-centric 3D Scene Generation 1 Zhenyu Sun, Xiaohan Zhang, Qi Liu†, Senior Member, IEEE, and Huan Wang†, Senior Member, IEEE arXiv :2607 .038 19v2 [ cs .GR] 7 Jul 2026 Abstract— Challenges remain in ego-centric 3D scene generation due to limited view overlap and the dominant influence of individual perspectives on scene interpretation. These factors hinder the creation of viewpoint-consistent and semantically aligned visual content, as well as the construction of accurate geometric structures. In this paper, we propose CGGS, a textto-3D framework aiming to enhance 3D-content-awareness and address geometric distortions in ego-centric scene generation. Firstly, the Ego-centric Generator is proposed by fine-tuning a Multi-View Latent Diffusion Model with consistency-augmented loss to generate consistent, high-fidelity 2D content aligned with textual descriptions. Then, Layout Decorator leverages optical flow and point-track correspondence to estimate depth, therefore producing dense point clouds as coarse layouts from the egocentric 2D priors. Building on this initialization, Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy-based Mutual Information Depth Loss (MID) combined with a hierarchical optimization scheme for improving visual quality and geometric structure. Comprehensive experiments demonstrate that CGGS outperforms previous methods in generating coherent and accurate text-driven 3D scenes. Project page: [https://cggs-26.github.io/cggs26/](https://cggs-26.github.io/cggs26/). Index Terms—3D gaussian splatting, ego-centric generation, semantic alignment, global coherence. I. INTRODUCTION 3D scene generation has recently gained significant attention, fueled by advances in generative models and strong image priors. In particular, generating 3D scenes from textual descriptions holds great promise for a wide range of real-world applications in AR/VR, robotics, and autonomous driving. With the rapid development of text-to-image generation [2],[3], [4], progress has been made toward text-to-3D generation. Latent Diffusion Models (LDMs) [4], [5] have been leveraged to optimize Neural Radiance Fields (NeRF) [6] via CLIP This paper is supported by Young Scientists Fund of the National Natural Science Foundation of China (NSFC) (No. 62506305), Zhejiang Leading Innovative and Entrepreneur Team Introduction Program (No. 2024R01007), Key Research and Development Program of Zhejiang Province (No. 2025C01026), Scientific Research Project of Westlake University (No. WU2025WF003), Chinese Association for Artificial Intelligence (CAAI) & Ant Group Research Fund - AGI Track (No. 2025CAAI-ANT-13) . It is also supported by the research funds of the National Talent Program and Hangzhou Municipal Talent Program. It is also supported in part by the GJYC program of Guangzhou under Grant 2024D01J0081, in part by the ZJ program of Guangdong under Grant 2023QN10X455, and in part by the Fundamental Research Funds for the Central Universities under Grant 2025ZYGXZR053 . Zhenyu Sun, Xiaohan Zhang and Qi Liu are with the School of Future Technology, South China University of Technology, Guangzhou 511442, China (email: [ftsunzhenyu@mail.scut.edu.cn](ftsunzhenyu@mail.scut.edu.cn); [ftxiaohanzhang@mail.scut.edu.cn](ftxiaohanzhang@mail.scut.edu.cn); dr [liuqi@scut.edu.cn](liuqi@scut.edu.cn)). Huan Wang is with the School of Engineering, Westlake University, Hangzhou 310030,

FAQs

What key problems does CGGS address in ego-centric 3D scene generation?

Limited view overlap and the dominant influence of individual perspectives reduce viewpoint consistency and semantic alignment, and hinder accurate geometric structure construction.

How does CGGS generate consistent 2D priors from text?

It fine-tunes a Multi-View Latent Diffusion Model using a consistency-augmented loss in the proposed Ego-centric Generator.

How does CGGS turn ego-centric 2D priors into 3D geometry and refine it?

Layout Decorator estimates depth using optical flow and point-track correspondences to form dense point clouds as coarse layouts, then Geometric Refiner reconstructs 3D Gaussians using an entropy-based Mutual Information Depth Loss with hierarchical optimization.

Jul 16, 2026

Upload by Hazel