Structure-Detail Decoupled Autoregressive Generation for Fast and High-Fidelity Virtual Try-On

Jul 16, 2026

Virtual try-on (VTON) synthesizes a photorealistic person wearing a target garment, requiring faithful person preservation plus accurate garment deformation and detail synthesis. Diffusion-based VTON models combine these factors in compressed latent space but suffer from high inference latency and high-frequency detail loss from latent compression. Visual autoregressive generation offers faster inference yet lacks effective bi-conditional mechanisms for VTON. The document introduces VAR-VTON and STAR-VTON, decoupling structural synthesis in latent space from fine pixel-space detail recovery via a matching-informed refiner.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Abstract

Introduction

arXiv :2607 . 11233v1 [ cs .CV] 13 Jul 2026 STRUCTURE-DETAIL DECOUPLED AUTOREGRESSIVE GENERATION FOR FAST AND HIGH-FIDELITY VIRTUAL TRY-ON Lu Yang1 , Xiaonan Hu1 , Yanan Li2 , Daqi Liu3 , Xiang Bai1 & Hao Lu1, ∗ 1Huazhong University of Science and Technology, China 2Wuhan Institute of Technology, China 3Xiaomi EV, China {lu yang1, [hlu](hlu}@hust.edu.cn)[}](hlu}@hust.edu.cn)[@hust.edu.cn](hlu}@hust.edu.cn) ABSTRACT Virtual try-on (VTON) is a bi-conditional image generation problem that requires not only accurate person preservation but also faithful garment deformation and detail synthesis. Diffusion-based VTON methods can jointly model these factors in a compressed latent space, but suffer from high-frequency detail loss due to inherent latent compression, even with costly multi-step denoising. Recent visual autoregressive (VAR) models offer a promising alternative for high-quality generation with faster inference, yet remain unexplored for VTON due to the lack of effective bi-conditioning mechanisms. To bridge this gap, we first introduce VAR-VTON, a VAR-based VTON model that incorporates garment conditioning and structural guidance for efficient latent-space VTON. Despite its efficacy, latent-space generation still struggles to preserve fine-grained garment details. We argue that different VTON sub-tasks should be addressed in different representation spaces: structural synthesis such as garment warping and person layout is suited to the latent space, whereas fine-grained detail recovery should be tackled in the pixel space. Motivated by this insight, we further propose STARVTON, a Two-Stage AutoRegressive framework that builds upon VAR-VTON by decoupling latent-space structural synthesis from pixel-space detail recovery. Our idea is to resort to a matching-informed refiner to establish dense correspondences between the stage-one generation and the source garment to directly map fine-grained pixel-space details. Extensive experiments show that STAR-VTON achieves an impressive efficiency–fidelity trade-off: VAR-VTON runs at least 4× faster than diffusion-based counterparts without degrading quality, and the pixelspace refiner effectively restores fine details and acts as a plug-and-play module that can benefit existing VTON approaches. 1 INTRODUCTION Given a person image and a garment image, virtual try-on (VTON) aims to synthesize aphotorealistic image in which the person is naturally and accurately dressed in the target garment (Han et al., 2018) . It is a typical bi-conditional generative problem, because a faithful VTON generation requires not only preserving the identity and pose of the person but also the fidelity of the garment. Achieving high-fidelity VTON, however, is non-trivial. It needs to i) correctly deform the garment to align with the target body pose and ∗ Corresponding author. Figure 1: Comparison with the state of the art in terms of detail fidelity and efficiency. Bubble size denotes computational cost (TFLOPs) . We evaluate detail preservation using CLIP-I, with higher values indicating better fidelity. Our solutions achieve better fidelity-efficiency trade-off. Figure 2: Comparison between latent-space VTON and our structure-detail decoupled generation framework. (a) Existing VTON methods rely on compressed latent representations, where high-frequency garment details are inevitably lost during VAE encoding. Consequently,(b) latentspace generation struggles to faithfully preserve fine-grained garment textures. (c) Our framework leverages the VAR model for efficient holistic structure synthesis and dense image matching for pixel-space garment detail recovery, enabling both fast inference and faithful detail preservation. shape, ii) well preserve person attributes including appearance and pose, and iii) faithfully retain garment details such as logos and text. Early GAN-based methods (Han et al., 2018; Wang et al., 2018; Choi et al., 2021) typically follow a two-stage pipeli

FAQs

What problem does the document address in virtual try-on (VTON)?

It targets efficient and high-fidelity VTON that preserves the person while accurately deforming the garment and synthesizing fine garment details.

Why do diffusion-based VTON methods struggle with high detail fidelity?

They incur high inference latency and lose high-frequency garment details due to inherent compression in the VAE latent space.

What is the key idea behind STAR-VTON compared with earlier approaches?

STAR-VTON decouples latent-space structural synthesis from pixel-space fine-detail recovery, using a matching-informed refiner to map dense garment details from the source.

Jul 16, 2026

Upload by Evangeline