Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge

Jul 16, 2026

Upload by Patrick

Diffusion language models enable parallel decoding and can greatly reduce generation latency, yet their token independence limits output quality versus autoregressive models. Recent bridging through importance sampling suffers from expensive large-particle requirements due to distribution mismatch. PoE-Bridge introduces an intermediate Product-of-Experts distribution built from the DLM proposal and AR target. It drafts candidates in parallel, uses rejection sampling to move them toward the PoE, then importance-samples to correct toward the AR target. Mixed-temperature sampling and elastic rejection windows improve diversity and reduce wasted verification, yielding up to 5× speedup while recovering at least 95% of AR performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

Introduction

PoE-Bridge Method

Intermediate Product-of-Experts Distribution

Parallel Drafting and Rejection Sampling

Importance Sampling Toward Autoregressive Target

Experimental Results

Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge Juntong Shi 1 Brian L. Trippe 1 Jure Leskovec 1 Stefano Ermon 1 Minkai Xu 1 arXiv :2606 .08048v 1 [ cs .CL] 6 Jun 2026 Abstract Diffusion language models (DLMs) offer substantial speed advantages through parallel decoding, but the lack of token dependencies limits generation quality compared to autoregressive (AR) models. Recent progress attempts to bridge the gap via importance sampling, with DLM being the proposal and AR being the target. However, due to the huge gap between their distributions, the sampling requires a large number of particles and is thus expensive to compute. In this paper, we introduce PoE-Bridge, a novel decoding framework that drastically improves generation speed and accuracy by introducing an intermediate distribution to bridge the gap. The distribution is constructed as a Product-of-Experts (PoE) of the DLM proposal and the AR target. With the intermediate distribution, we first use the DLM to draft multiple continuations in parallel, then apply rejection sampling to verify the drafted tokens and move the resulting candidates toward the PoE. We then use importance sampling to further correct the PoE-aligned candidates toward the AR target. We further propose several improved techniques, including mixed-temperature sampling for enhanced diversity and elastic rejection windows for reducing wasted verification. Empirically, PoE-Bridge achieves significantly improved accuracy with 5 × speedup over the standard DLM decoding approach, and recovers at least 95% of the target AR model’s performance, efficiently advancing most of the quality gap on challenging mathematical reasoning and coding tasks. Our code is available at [https://github.com/](https://github.com/)[ ](https://github.com/)juntongshi48/poe-bridge. BT, JL, and SE co-supervised the project, secured resources, and contributed to the conceptualization and critical revision of the manuscript. 1 Stanford University. Correspondence to: Juntong Shi <[juntong@stanford.edu](juntong@stanford.edu)>, Minkai Xu <[minkai@cs.stanford.edu](minkai@cs.stanford.edu)>. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) . 1. Introduction Autoregressive (AR) language models remain the dominant approach to text generation (Vaswani et al., 2017 ; Wolf et al., 2020 ; Touvron et al., 2023), achieving strong performance on challenging tasks such as mathematical reasoning (Wei et al., 2022 ; Trinh et al., 2024 ; Yang et al., 2024) and code synthesis (Roziere et al., 2023 ; Hui et al., 2024) . However, AR decoding is inherently sequential—generating tokens one by one in a strict left-to-right order. This sequential dependency inherently limits inference-time parallelism and makes latency and throughput a major bottleneck. Diffusion language models (DLMs) (Sahoo et al., 2024 ; Shi et al., 2024 ; Ye et al., 2025 ; Nie et al., 2025), derived from the broader discrete diffusion framework (Austin et al., 2021b ; Campbell et al., 2022 ; Lou et al., 2024), provide an appealing alternative by enabling parallel decoding. By generating and refining multiple tokens simultaneously, DLM can, in principle, produce entire sequences in a fraction of the sequential steps required by AR models. In practice, however, DLMs still lag behind strong AR models in generation quality. This gap largely stems from the conditional independence assumptions used to enable parallel decoding, under which tokens generated within the same step are modeled independently rather than jointly. As a result, the anticipated speedups from parallel generation are difficult to realize without incurring substantial degradation in output quality. This gap suggests that effective parallel decoding requires a mechanism to inject inter-token dependencies into DLM drafts while preserving their parallel sampling efficiency. Notably, while sa

FAQs

Which enhancements does PoE-Bridge add to improve diversity and efficiency?

The paper proposes mixed-temperature sampling to enhance diversity and elastic rejection windows to reduce wasted verification, improving both generation quality and computational efficiency.

Jul 16, 2026

Upload by Patrick