BARD Bridging AutoRegressive and Diffusion Vision Language Models Via Highly Efficient Progressive Block Merging and Stage Wise Distillation

Jul 16, 2026

Autoregressive vision-language models deliver strong multimodal capability, but token-by-token decoding creates a hard inference bottleneck. Diffusion vision-language models enable more parallel refinement, yet naive conversion of pretrained autoregressive VLMs into large-block dVLMs causes significant quality loss. BARD bridges this gap by progressively merging blocks, then recovering performance with stage-wise intra-dVLM distillation from a small-block diffusion anchor, plus mixed noise scheduling and memory-friendly training for long multimodal sequences.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

Abstract

Keywords

Introduction

Autoregressive VLM bottleneck and diffusion decoding

Two diffusion modeling paradigms

The BARD bridging framework

arXiv :2604 . 16514v5 [ cs .CV] 12 Jul 2026 BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation Baoyou Chen 1,3,★ HanChen Xia 1,★ Peng Tu 1,★ Haojun Shi 1 Liwei Zhang 1 Yuxuan Yao2,3 Weihao Yuan4 Siyu Zhu 1,2,3,†∗ 1 Shanghai Academy of AI for Science, 2 Shanghai Innovation Institute, 3Fudan University 4Nanjing University Figure 1: Quality–efficiency comparison of BARD-VL and representative open dVLMs. Left: radar chart on seven multimodal benchmarks, where BARD-VL at 2B/4B/8B is compared with prior open diffusion VLMs. BARD-VL 4B and 8B exhibit the strongest overall performance among the compared dVLMs, showing that the proposed bridge substantially narrows the capability gap of existing diffusion VLMs. Right: OCRBench accuracy versus decoding throughput. Even though it is smaller than the 7B/8B baselines, BARD-VL 4B traces a clearly better accuracy–throughput trade-off, retaining higher accuracy across abroad range of decoding speeds. Abstract Autoregressive vision-language models (VLMs) deliver strong multimodal capability, but their token-by-token decoding imposes a fundamental inference bottleneck. Diffusion VLMs offer a more parallel decoding paradigm, yet directly converting a pretrained autoregressive VLM into a large-block diffusion VLM (dVLM) often leads to substantial quality degradation. In this work, we present BARD, a simple and effective bridging framework that converts apretrained autoregressive VLM into a same-architecture, decodingefficient dVLM. Our approach combines progressive supervised block merging, which gradually enlarges the decoding block size, with stage-wise intra-dVLM distillation from a fixed small-block diffusion anchor to recover performance lost at larger blocks. We further incorporate a mixed noise scheduler to improve robustness and token revision during denoising, and memory-friendly training ∗★ Equal contribution.† Corresponding author. to enable efficient training on long multimodal sequences. A key empirical finding is that direct autoregressive-to-diffusion distillation is poorly aligned and can even hurt performance, whereas distillation within the diffusion regime is consistently effective. Experimental results show that, with ≤ 4.4􀀢 data, BARD-VL transfers strong multimodal capability from Qwen3-VL to a large-block dVLM. Remarkably, BARD-VL establishes a new SOTA among comparable-scale open dVLMs on our evaluation suite at both 4Band 8B scales. At the same time, BARD-VL achieves up to 3× decoding throughput speedup compared to the source model. Code is available at: [https://github.com/fudan-generative-vision/Bard-VL](https://github.com/fudan-generative-vision/Bard-VL). CCS Concepts • Computing methodologies → Natural language generation; Neural networks; Computer vision. Conference acronym’34, November 10–14, 2026, Rio de Janeiro, Brazil Keywords vision-language models, multimodal understanding, discrete diffusion, parallel decoding, multimodal reasoning 1 Introduction Autoregressive vision-language models (VLMs) have become the dominant foundation for multimodal understanding and agentic interaction. Their success spans visual reasoning, document understanding, grounded question answering, and emerging multimodal agents. Yet their token-by-token causal decoding remains inherently sequential, imposing a hard inference bottleneck for practical deployment. Diffusion-based decoding offers a compelling alternative: by refining a partially generated response block by block, a diffusion VLM (dVLM) can update multiple tokens in parallel and therefore expose a fundamentally different quality–efficiency trade-off [2, 11, 29] . Recent progress has established diffusion models as a viable direction for both language and vision-language modeling. Broadly speaking, existing methods follow two main paradigms. The first builds diffusion-native models from scratch or from

FAQs

What problem does BARD address when converting autoregressive VLMs to diffusion VLMs?

Directly converting a pretrained autoregressive VLM into a large-block diffusion VLM often causes substantial quality degradation due to misalignment in the distillation process.

How does BARD enable efficient large-block diffusion decoding?

BARD first converts an autoregressive checkpoint into a stable small-block diffusion anchor, then gradually increases decoding parallelism via progressive supervised block merging rather than an abrupt transition.

What training improvements does BARD add beyond the bridging procedure?

It introduces a mixed noise scheduler to improve robustness and token revision during denoising, and memory-friendly training that packs clean and noisy responses with shared multimodal context to reduce overhead.

Jul 16, 2026

Upload by Ava Thompson