MiDSummer - ICCV 2025 - Multi-Guidance Diffusion for Controllable Zero-Shot Immersive Gaussian Splatting Scene Generation

Oct 28, 2026

Upload by （）

MiDSummer提出一种两阶段框架，用多种扩散引导信号生成沉浸式高斯溅射场景，通过结构化布局控制、物理真实性增强与视觉质量提升来应对现有方法的不足。本文同时解决三类挑战：可靠布局控制与开域泛化、语义推理与可直接控制的几何约束的平衡，以及利用布局知识进行视觉精修。规划阶段以双重布局扩散引导融合LLM语义推理与图扩散模型的几何精度，并用场景图结构与布局边界框的多层自一致性评分实现细粒度构图控制。装配阶段提出布局引导的优化以将规划得到的先验融入基于稳定扩散的精修过程，联合优化相机轨迹与场景细分，实现结构一致且可控的沉浸式结果。

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Abstract

Introduction

This ICCV paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore. MiDSummer: Multi-Guidance Diffusion for Controllable Zero-Shot Immersive Gaussian Splatting Scene Generation Anjun Hu 1,2 Richard Tomsett 1 Valentin Gourmet 1 Massimo Camplani 1 Jas Kandola 1 Hanting Xie 1 1Amazon Prime Video 2University of Oxford {anjunhu,[hanting](hanting}@amazon.com)[}](hanting}@amazon.com)[@amazon.com](hanting}@amazon.com) , [anjun.hu@eng.ox.ac.uk](anjun.hu@eng.ox.ac.uk) | a bedroom with a queen-sized bed, two night stands, a large wooden wardrobe... | | --- | | moonlight streams through glass windows, ... an ancient gothic cathedral. | | a chocolate factory ﬁlled with various sweet making stations including... | | | | --- | | | | | Abstract We present MiDSummer, a two-stage framework for generating immersive Gaussian Splatting scenes that leverages multiple diffusion guidance signals to enable structured layout control, enhanced physical realism, and improved visual quality. While 3D scene generation has seen signiﬁcant recent advances, current approaches could beneﬁt from: (1) achieving precise, reliable layout control while preserving open-world generalization and physical plausibility,(2) balancing high-level semantic reasoning with low-level, directly controllable geometric constraints, and (3) effectively utilizing layout knowledge for visual reﬁnement. Our work addresses these challenges through a structured two-stage planning-assembly framework. For planning, we introduce a dual layout diffusion guidance approach to bridge semantic reasoning and geometric controllability. Our approach uniquely integrates LLMs’ open-vocabulary reasoning with Graph Diffusion Models’(GDM) geometric precision by incorporating multi-level self-consistency scores over scene graph structures and layout bounding box parameters. This fusion enables ﬁne-grained control over scene composition while ensuring physical plausibility and faithful prompt interpretation. For assembly, we propose a layout-guided optimization technique for scene reﬁnement. We effectively incorporate layout priors obtained during the planning stage into a Stable Diffusion (SD)-based reﬁnement process that jointly optimizes camera trajectories and scene splats. This layout-aware joint optimization, constrained by multi-view consistency, produces visually compelling immersive scenes that are structurally coherent and controllable. 1. Introduction Scene design and prototyping are used in many artistic and engineering applications, including ﬁlm set design, game environment drafting, and interior visualization. Designers often need to enforce strict placement constraints on key story-relevant objects, while maintaining ﬂexibility in other elements to promote visual diversity and realism. This makes the ability to control spatial relationships and gauge their uncertainty fundamental to practical scene synthesis. Recent advances in LLMs [1, 2] and Graph Diffusion Models (GDMs) [3, 4] have shown promise in compositional scene synthesis, yet they exhibit distinct limitations: foundation models like LLMs excel in open-world semantic reasoning and are aware of object functionalities but struggle with precise spatial control, often failing to adhere to relationship constraints or introducing hallucinated elements. Conversely, specialist layout models like GDMs trained on domain-speciﬁc scenes can faithfully enforce geometric relationships but are constrained by their dataset priors, limiting their generalization to out-of-domain scenarios. While signiﬁcant advances in LLM-driven 3D scene generation have been made [5], we notice two potential opportunities for improvement. Firstly, relying only on LLM proposals can lead to high output variance (Figure 6) and diminished control over ﬁne-grained object rela

FAQs

MiDSummer的核心框架是什么？

MiDSummer采用两阶段规划-装配流程：规划阶段进行布局优化，装配阶段完成构图与视觉精修，从而提升沉浸式高斯溅射场景的可控性与一致性。

MiDSummer如何实现布局规划阶段的可控性？

规划阶段使用双重布局扩散引导，将LLM的开域语义推理与图扩散模型的几何精度结合，并通过多层自一致性评分来平衡语义与几何约束。

装配阶段如何进行场景精修与全局优化？

装配阶段提出布局感知的多视角精修：将规划阶段得到的布局先验输入基于稳定扩散的精修流程，联合优化相机轨迹与场景高斯点（splats），并以多视角一致性约束提升结构连贯与视觉质量。

Oct 28, 2026

Upload by （）