RealityBridge Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos

Jul 16, 2026

Upload by Aurora

Long-tail safety-critical scenarios are crucial for safety-oriented autonomous driving, yet they are hard to collect and reproduce at scale. Editable 3D Gaussian Splatting (3DGS) simulation enables controllable scene editing by reconstructing real driving scenes, but edited videos still show a large Sim-to-Real gap: rendering artifacts, degraded foreground assets, illumination inconsistency, and temporal flickering. RealityBridge is proposed as a structure-preserving, asset-aware Sim-to-Real framework using multimodal controls and GateNet plus targeted training and reward-guided autoregressive long-video training.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages12

Reading Time30 min

Introduction

Safety-critical editable 3DGS simulation

The Sim-to-Real gap in edited driving videos

Related work and remaining challenges

RealityBridge method (proposed framework)

arXiv :2606 . 16278v2 [ cs .CV] 13 Jul 2026 RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videos Zhenhua Wu 1 ,2∗ Yun Pang 1∗ Mingkun Chang 1∗ Yuwei Ning 1 Liangzhi Wang 1 Yi Xiao 1 Guanbin Li 1 ,3§ 1 Sun Yat-sen University 2 Shanghai Innovation Institute 3 Shenzhen Loop Area Institute [wuzhh56@mail2.sysu.edu.cn](wuzhh56@mail2.sysu.edu.cn) , [pangy9@mail2.sysu.edu.cn](pangy9@mail2.sysu.edu.cn) , [mingkun502@gmail.com](mingkun502@gmail.com) , [ningyw@mail2.sysu.edu.cn](ningyw@mail2.sysu.edu.cn) , [wanglzh26@mail2.sysu.edu.cn](wanglzh26@mail2.sysu.edu.cn) , [xiaoy2622935705@gmail.com](xiaoy2622935705@gmail.com) , [liguanbin@mail.sysu.edu.cn](liguanbin@mail.sysu.edu.cn) Poor 3DGS Vehicle Reconstruction Pedestrian Harmonization Lighting Harmonization Small-Object Harmonization Figure 1: Restoration and harmonization results. Our method improves realism, foreground harmonization, and temporal consistency in 3DGS simulation degradation and editing scenarios. Top: four representative cases, including vehicle reconstruction restoration, vehicle, pedestrian, and small-object lighting harmonization. Bottom: three frames sampled at different timestamps from daytime and nighttime scenes, demonstrating stable and consistent results over time. Abstract Long-tail safety-critical scenarios are essential for safetyoriented autonomous driving, yet they are difficult to collect and reproduce at scale. Editable 3D Gaussian Splatting (3DGS) simulation offers a promising alternative by reconstructing real driving scenes and supporting controllable scene editing. However, edited 3DGS-rendered videos still suffer from a significant Sim-to-Real gap, including rendering artifacts, degraded foreground assets, inconsistent illumination, and temporal flickering. Existing restoration and ∗Equal contribution. §Corresponding author video generation methods are insufficient for this task, as they often fail to jointly repair 3DGS-specific artifacts, improve visual realism, and ensure temporal consistency. To fill this gap, we propose RealityBridge, a structure-preserving and asset-aware Sim-to-Real framework for edited 3DGS driving videos. RealityBridge uses multimodal controls, including rendered videos, category-level masks, edge maps, and semantic masks, together with a lightweight GateNet for adaptive condition allocation across backbone layers. We further construct targeted training data and introduce autoregressive long-video training with reward-guided post-training to improve restoration quality, temporal stability, and hallucination suppression. Extensive experiments on internal and public driving datasets show that RealityBridge outperforms exist- ing methods in artifact removal, illumination harmonization, and long-sequence temporal consistency. Introduction Safety-oriented training and evaluation of autonomous driving systems require diverse, controllable, and reproducible long-tail hazardous scenarios, which are rare, costly to collect, and difficult to reproduce in the real world. Recent advances in 3D Gaussian Splatting (3DGS) (Kerbl et al. 2023) provide a promising foundation for editable driving simulation (Zhao et al. 2024; Kaur et al. 2021; Yang et al. 2023; Wu et al. 2023; Turki et al. 2023; Tonderski et al. 2024; Zhou et al. 2024; Yan et al. 2024; Yang et al. 2024) . By reconstructing real-world driving scenes and supporting controllable edits such as object insertion, removal, and trajectory modification, 3DGS makes it possible to build editable simulators where safety-critical objects, trajectories, and interactions can be explicitly controlled for scalable autonomousvehicle training, evaluation, and safety validation. However, being editable does not necessarily make the simulator realistic. Edited 3DGS-rendered videos still suffer from a clear Sim-to-Real gap, including blurry textures, rendering artifacts, foreground illumination mismatch, missing

FAQs

What problem does RealityBridge address?

RealityBridge addresses the Sim-to-Real gap in edited 3D Gaussian Splatting (3DGS) driving videos, including rendering artifacts, degraded foreground assets, illumination mismatch, and temporal flickering.

How does RealityBridge preserve edited simulator layout and assets?

It uses a structure-preserving and asset-aware Sim-to-Real framework with multimodal controls (rendered videos, category-level masks, edge maps, and semantic masks) to keep simulator-defined scene layout and edited assets intact.

What training strategy improves restoration quality and temporal stability?

RealityBridge constructs targeted training data and introduces autoregressive long-video training with reward-guided post-training to enhance restoration quality, improve temporal stability, and suppress hallucinations.

Jul 16, 2026

Upload by Aurora