BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; November 10–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2026/06 4906 CCS: Computing methodologies Natural language generation CCS: Computing methodologies Neural networks CCS: Computing methodologies Computer vision
Baoyou Chen 1,3,⋆ HanChen Xia 1,⋆ Peng Tu 1,⋆ Haojun Shi 1 Liwei Zhang 1 Weihao Yuan 4 Siyu Zhu 1,2,3,† · 2026
Read BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; November 10–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2026/06 4906 CCS: Computing methodologies Natural language generation CCS: Computing methodologies Neural networks CCS: Computing methodologies Computer vision by Baoyou Chen 1,3,⋆ HanChen Xia 1,⋆ Peng Tu 1,⋆ Haojun Shi 1 Liwei Zhang 1 Weihao Yuan 4 Siyu Zhu 1,2,3,† (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16514.txt.
Introduction
Chen and colleagues bridge autoregressive and diffusion models. BARD converts a pretrained model into a large-block diffusion model of the same design, though a direct switch hurts quality. BARD slowly grows block size through guided merging, then wins back lost quality with distillation inside that regime from a small anchor. A mixed noise plan adds strength and fixes during cleaning. The final model reached up to three times the output speed of its source, while direct cross-regime training could even hurt.
Catalog record
| CREATOR | Baoyou Chen 1,3,⋆ HanChen Xia 1,⋆ Peng Tu 1,⋆ Haojun Shi 1 Liwei Zhang 1 Weihao Yuan 4 Siyu Zhu 1,2,3,† |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38564 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16514
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; November 10–14, 2026; Rio de Janeiro, Brazil ISBN: 978-1-4503-XXXX-X/2026/06 4906 CCS: Computing methodologies Natural language generation CCS: Computing methodologies Neural networks CCS: Computing methodologies Computer vision Baoyou Chen 1,3,⋆ HanChen Xia 1,⋆ Peng Tu 1,⋆ Haojun Shi 1 Liwei Zhang 1 Weihao Yuan 4 Siyu Zhu 1,2,3,† Abstract . Autoregressive vision-language models (VLMs) deliver strong multimodal capability, but their token-by-token decoding imposes a fundamental inference bottleneck. Diffusion VLMs offer a more parallel decoding paradigm, yet directly converting a pretrained autoregressive VLM into a large-block diffusion VLM (dVLM) often leads to substantial quality degradation. In this work, we present BARD , a simple and effective bridging framework that converts a pretrained autoregressive VLM into a same-architecture, decoding-efficient dVLM. Our approach combines progressive supervised block merging, which gradually enlarges the decoding block size, with stage-wise intra-dVLM distillation from a fixed small-block diffusion anchor to recover performance lost at larger blocks. We further incorporate a mixed noise scheduler to improve robustness and token revision during denoising, and memory-friendly training to enable efficient training on long multimodal sequences. A key empirical finding is that direct autoregressive-to-diffusion distillation is poorly aligned and can even hurt performance, whereas distillation within the diffusion regime is consistently effective. Experimental results show that, with ≤ 4.4 M \leq 4.4M data, BARD-VL transfers strong multimodal capability from Qwen3-VL to a large-block dVLM. Remarkably, BARD-VL establishes a new SOTA among comparable-scale open dVLMs on our evaluation suite at both 4B and 8B scales. At the same time, BARD-VL achieves up to 3 × \times decoding throughput speedup compared to the source model. Code is available at: https://github.com/fudan-generative-vision/Bard-VL . Figure 1. Quality–efficiency comparison of BARD-VL and representative open dVLMs. Left: radar chart on seven multimodal benchmarks, where BARD-VL at 2B/4B/8B is compared with prior open diffusion VLMs. BARD-VL 4B and 8B exhibit the strongest overall performance among the compared dVLMs, showing that the proposed bridge substantially narrows the capability gap of existing diffusion VLMs. Right: OCRBench accuracy versus decoding throughput. Even though it is smaller than the 7B/8B baselines, BARD-VL 4B traces a clearly better accuracy–throughput trade-off, retaining higher accuracy across a broad range of decoding speeds. 1. Introduction Autoregressive vision-language models (VLMs) have become the dominant foundation for multimodal understanding and agentic interaction. Their success spans visual reasoning, document understanding, grounded question answering, and emerging multimodal agents. Yet their token-by-token causal decoding remains inherently sequential, imposing a hard inference bottleneck for practical deployment. Diffusion-based decoding offers a compelling alternative: by refining a partially generated response block by block, a diffusion VLM (dVLM) can update multiple tokens in parallel and therefore expose a fundamentally different quality–efficiency trade-off ( 2 ; 29 ; 11 ) . Recent progress has established diffusion models as a viable direction for both language and vision-language modeling. Broadly speaking, existing methods follow two main paradigms. The first builds diffusion-native models from scratch or from dedicated diffusion backbones, represented by MDLM and LLaDA on the language side, and Dimple, LaViDa, and LLaDA-V on the multimodal side ( 15 ; 14 ; 27 ; 11 ; 26 ) . The second starts from strong autoregressive checkpoints and bridges them into diffusion models through post-training or continuous pretraining, as exemplified by Dream, NBDiff, Fast-dLLM v2, Efficient-DLM, and DiffusionVL ( 25 ; 16 ; 21 ; 6 ; 29 ) . Recent work ( 22 ) studies the text-only bridging setting and does not address teacher-forcing post-training from a frontier autoregressive VLM to a strong large-block dVLM. Despite this rapid progress, current dVLMs cannot match comparable scale VLMs in overall multimodal capability. In this work, we propose BARD , an efficient framework for bridging a strong autoregressive VLM to a same-architecture large-block diffusion VLM. Our approach begins with the pretrained autoregressive checkpoint and first converts it into a stable small-block diffusion anchor. We then progressively increase decoding parallelism through progressive block merging, which smoothly expands the block size instead of forcing an abrupt transition from causal decoding to large-block diffusion. To recover the capability drop that arises at higher block sizes, we further introduce stage-wise dVLM distillation, where each larger-block model is distilled from the fixed small-block diffusion anchor. Beyond the bridging procedure itself, we incorporate two practical improvements that make diffusion post-training more effective for multimodal models: a mixed noise scheduler, which trains the model to refine both masked and visibly corrupted tokens and thus improves iterative revision, and memory-friendly training, which packs clean and noisy responses with a shared multimodal context to significantly reduce training overhead. Together, these components form a simple and scalable recipe for teacher-forcing post-training of large-block dVLMs. Using fewer than 4.4M training samples, BARD transfers the capability of a frontier autoregressive VLM to a highly parallel diffusion VLM with essentially no performance loss. Using the proposed framework, we train a family of dVLMs, BARD-VL , at multiple model scales. Experiments on multimodal reasoning, visual question answering, and document understanding benchmarks show that models trained with BARD-VL effectively eliminate the performance gap between diffusion and autoregressive VLMs. At both 4B and 8B scales, BARD-VL consistently outperforms prior open dVLM baselines and achieves the strongest results among comparable-scale dVLMs in our evaluation suite, while retaining, matching, or slightly exceeding the performance of its Qwen3-VL source model on most reported benchmarks. At the same time, BARD-VL delivers up to 3 \times decoding throughput speedup compared to the source model. Figure 1 summarizes the resulting quality–efficiency profile. 2. Related Work Discrete Diffusion Language Models. Recent work on discrete diffusion models replaces causal next-token prediction with iterative denoising over discrete tokens. In language modeling, MDLM, LLaDA, and Dream show that simple mask-based training can already scale to strong performance ( 15 ; 14 ; 25 ) . Subsequent studies mainly examine two issues that are also central to our setting: the choice of corruption process and the mismatch between random corruption during training and progressive generation at inference. GIDD broadens the noising process beyond pure masking ( 17 ) , while MDPO studies how training can be better matched to the generation process used at inference time ( 8 ) . Discrete Vision-Language Models. This framework has also been extended to multimodal settings. MMaDA studies multimodal diffusion foundation models, while Dimple, LaViDa, LLaDA-V, and DiffusionVL develop diffusion-based vision-language models for multimodal understanding ( 23 ; 27 ; 11 ; 26 ; 29 ) . RIV and ReDiff place more emphasis on iterative correction, and FUDOKI explores a flow-based alternative ( 12 ; 10 ; 18 ) . Across these works, a recurring difficulty is revision: once a wrong token appears early, later denoising steps often fail to correct it. Our mixed-noise training is designed to improve this part of the generation process. Autoregressive-to-Diffusion Adaptation. A separate line of work asks whether strong pretrained autoregressive models can be converted into diffusion models through continued pretraining or post-training, rather than training diffusion models from scratch. In language modeling, several recent studies show that autoregressive pretraining provides a strong starting point for diffusion adaptation ( 7 ; 25 ; 21 ;