Reducing Overfitting in 3D Gaussian Splatting using Depth Supervision - Bachelor Thesis

Aug 25, 2026

Uploaded by Sage

3D Gaussian Splatting (3DGS) is effective for representing 3D scenes but tends to overfit when training data has limited viewpoint diversity, producing depth-related artifacts such as floating Gaussians. The study proposes 3D Gaussian Splatting with Depth by injecting depth supervision from RGB-D cameras into training. Depth-guided Gaussian placement reduces incorrect-depth artifacts. Quantitative and qualitative results show a threefold reduction in depth loss on challenging outdoor settings, without substantially harming regular-view performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

AccessFree to download

Introduction

Motivation: overfitting from limited viewpoints

Proposed solution: 3D Gaussian Splatting with Depth

Related Work

Pipeline of 3DGS with COLMAP and gradient descent

Reducing Overfitting in 3D Gaussian Splatting using Depth Supervision Tygo Hendrik Bertus Spanhoff1 Supervisor: Xucong Zhang1 1EEMCS, Delft University of Technology, The Netherlands A Thesis Submitted to EEMCS Faculty Delft University of Technology, In Partial Fulfilment of the Requirements For the Bachelor of Computer Science and Engineering June 23, 2024 Name of the student: Tygo Hendrik Bertus Spanhoff Final project course: CSE3000 Research Project Thesis committee: Xucong Zhang, Michael Weinmann An electronic version of this thesis is available at [http://repository.tudelft.nl/](http://repository.tudelft.nl/) . Abstract 3D Gaussian Splatting (3DGS) is a method for representing 3D scenes, but is prone to overfitting when trained with limited viewpoint diversity, often resulting in artifacts like floating Gaussians at incorrect depths. This paper addresses this issue by introducing 3D Gaussian Splatting with Depth, which incorporates depth supervision from RGB Depth (RGB-D) cameras into the training process. By using depth data to guide the placement of Gaussians, the proposed method aims to reduce artifacts. Through quantitative and qualitative analysis, this paper demonstrates that depth-supervised Gaussian splatting mitigates overfitting artifacts, particularly in outdoor scenes with a mediocre camera point diversity. The depth-supervised model is able to reduce the depth loss by a factor of three times without substantially increasing the loss on regular views. 1 Introduction The representation of 3-dimensional scenes is a fundamental challenge in the fields of computer graphics and computer vision. The ability to turn real-world objects into 3D representations on computers to generate novel views can be applied in media such as films and video games [10], for robots and navigation systems [10], and for medical applications such as CT scans [8] . First introduced in 2020, Neural Radiance Field (NeRF) [7] is a novel view synthesis method that represents 3-dimensional scenes using a Deep Neural Network. NeRF’s ability to render photorealistic views in a reasonable time sparked significant interest in the field of radiance field rendering. Although revolutionary, NeRF came with some significant disadvantages, one of the main ones being that the representation as a Multi-Layer Perceptron requires an entire model to be retrained in order to make changes to a scene. In July of 2023, Kerbl et al. introduced a novel way of representing 3D scenes using 3D Gaussians, which allowed for realtime rendering of scenes captured with multiple photos [5] . This technique was generally faster in training and rendering than previous techniques such as NeRF, and the representation also allowed for changes to the scene without having to retrain the model. 3D Gaussian Splatting (3DGS) is particularly prone to overfitting due to its sensitivity to limited viewpoint diversity in the training data. When training a Gaussian splat, it is possible for some parts of the scene to be visible in only one or a select amount of training frames. As such, the model will place a Gaussian of the corresponding color in the scene such that the rendered splat will look similar to the training frame when rendered from the same camera position, as can be seen in figure 1 . However, since the part of the scene is only shot from one angle, the model could place the Gaussian at any distance from the camera. This could cause Gaussians to appear at the wrong depth, floating in space when rendered from a novel view, as can be seen in figure 1 . Figure 1: The white Gaussians representing the sky look decent on one view, but can be seen floating in mid-air when rendering the scene from a novel view. This paper investigates a potential solution for this problem by training a Gaussian splat using images from RGB-D cameras. RGB-D cameras have corresponding depth data for every frame, and these cameras are becoming increasingly more

FAQs

Why does 3D Gaussian Splatting overfit in practice?

Overfitting arises when training viewpoints are limited, so some scene parts are only visible from one or a few frames. The model can then place Gaussians at incorrect depths while still matching the limited training views.

What is the core idea of 3D Gaussian Splatting with Depth?

The method renders depth maps from the Gaussian splat and supervises them using depth ground truth from RGB-D cameras, guiding Gaussian placement with additional depth constraints.

How is the proposed approach evaluated?

Evaluation combines training loss analysis and qualitative comparison of renders produced by models trained with both 3DGSw/Depth and the original 3DGS, focusing on floating-Gaussian behavior in novel views.

Aug 25, 2026

Uploaded by Sage

Reducing Overfitting in 3D Gaussian Splatting using Depth Supervision - Bachelor Thesis_Page 1

Open in WPS to view all 9 pages