Qwen-Image-VAE-2.0 Technical Report
Qwen Team · 2026
Read Qwen-Image-VAE-2.0 Technical Report by Qwen Team (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13565.txt.
Introduction
Zekai Zhang and twenty-nine colleagues offer Qwen-Image-VAE-2.0, a family of high-compression variational autoencoders with strong reconstruction and spread. To ease training in high-dimensional latent space, they use a strong semantic matching plan making latent space ready for diffusion work. Tests on public sets, plus a new OmniDoc-TokenBench for text-rich texts, show state-of-the-art rebuilds in general and text-rich domains at high compression. Downstream transformer tests show better diffusability and faster training.
Catalog record
| CREATOR | Qwen Team |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41250 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13565
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Qwen-Image-VAE-2.0 Technical Report Qwen Team Abstract We present Qwen-Image-VAE-2.0, a suite of high-compression Variational Autoencoders (VAEs) that achieve significant advances in both reconstruction fidelity and diffusability 1 1 1 Diffusability describes how easily a distribution can be modeled by diffusion. . To address the reconstruction bottlenecks of high compression, we adopt an improved architecture featuring Global Skip Connections (GSC) and expanded latent channels. Moreover, we scale training to billions of images and incorporate a synthetic rendering engine to improve performance in text-rich scenarios. To tackle the convergence challenges of high-dimensional latent space, we implement an enhanced semantic alignment strategy to make the latent space highly amenable to diffusion modeling. To optimize computational efficiency, we leverage an asymmetric and attention-free encoder-decoder backbone to minimize encoding overhead. We present a comprehensive evaluation of Qwen-Image-VAE-2.0 on public reconstruction benchmarks. To evaluate performance in text-rich scenarios, we propose OmniDoc-TokenBench, a new benchmark comprising a diverse collection of real-world documents coupled with specialized OCR-based evaluation metrics. Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction performance, demonstrating exceptional capabilities in both general domains and text-rich scenarios at high compression ratio. Furthermore, downstream DiT experiments reveal our models possess superior diffusability, significantly accelerating convergence compared to existing high-compression baselines. These establish Qwen-Image-VAE-2.0 as a leading model with high compression, superior reconstruction, and exceptional diffusability. Introduction Latent Diffusion Models (LDMs) have become the dominant paradigm in image synthesis ( 30 ; 19 ; 39 ; 8 ; 9 ; 44 ) . These models typically employ a Variational Autoencoder (VAE) to project images into a compressed latent space for efficient diffusion modeling, where a widely adopted spatial compression ratio is 8 (denoted as f8 ). However, as the industry shifts toward native high-resolution synthesis, this standard ratio faces significant computational bottlenecks. Increasing the spatial compression ratio has thus become essential for computational efficiency, as the complexity of modern Diffusion Transformers (DiTs) ( 28 ) scales quadratically with the number of latent tokens. Over the past few years, several advances have been achieved in this field, demonstrating the significant potential of high-compression VAEs ( 11 ; 4 ; 1 ; 5 ) . Despite these advances, a critical challenge exists: the inevitable trade-off between high compression ratio, reconstruction fidelity, and diffusability ( 33 ) . Specifically, higher compression ratios often lead to severe degradation in reconstruction quality, particularly in text-rich scenarios where fine-grained detail is lost. While increasing the latent channel dimension can mitigate this information bottleneck, it frequently results in an over-complex and unstructured latent distribution, which significantly hinders the convergence and generative performance of downstream diffusion models ( 42 ; 29 ) . In this work, we introduce Qwen-Image-VAE-2.0, a series of high-compression image VAEs ( f16 & f32 ), designed to overcome these challenges through improved architecture, comprehensive data engineering, and enhanced training strategy. To address the challenge of reconstruction fidelity in high-compression regimes, we adopt an improved VAE architecture with Global Skip Connection (GSC), which establishes a global shortcut from pixels to latents, preserving fine-grained detail. Moreover, our design incorporates a higher latent dimensionality to alleviate the information bottleneck inherent in high-compression scenarios. On the data front, we scale our training corpus to billions of images and curate a specialized document collection (including academic papers, posters, slides, web pages, etc.) to enhance the reconstruction of text-rich images. Furthermore, we develop a synthetic pipeline that renders documents to provide dense supervisory signals for character-level reconstruction. Through these advancements, Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction performance, especially in text-rich scenarios, despite its high compression ratio. To address the challenge of diffusability brought by high compression ratio and expanded latent dimension, we demonstrate that semantic alignment with intermediate features of DINOv2 ( 26 ) can effectively accelerate DiT convergence. Furthermore, we adopt a staged semantic alignment paradigm that transitions from strict semantic alignment to a balanced optimization of reconstruction and generation. Leveraging these techniques, Qwen-Image-VAE-2.0 achieves superior diffusability compared to existing high-compression VAEs, despite its large channel dimension. To ensure computational efficiency, we leverage an asymmetric architecture that features a lightweight encoder to minimize encoding overhead during diffusion training. Additionally, we utilize an attention-free backbone to maintain high throughput even with ultra-high-resolution inputs. We conduct a comprehensive evaluation to validate the performance of Qwen-Image-VAE-2.0, focusing on two key aspects: reconstruction fidelity and latent space diffusability. For reconstruction fidelity, we evaluate it across general reconstruction tasks and introduce OmniDoc-TokenBench, a benchmark specifically targeting challenging scenarios like real-world text-rich document reconstruction. For latent space diffusability, we also perform extensive downstream DiT experiments to empirically verify it. The results demonstrate that Qwen-Image-VAE-2.0 not only achieves superior reconstruction fidelity, especially in text-rich scenarios despite high compression ratios, but also exhibits excellent latent space compatibility, facilitating rapid DiT convergence even with large latent dimension. The key contributions of Qwen-Image-VAE-2.0 can be summarized as follows: — High-Compression VAE : We introduce a suite of f16 and f32 image VAEs, providing a robust solution for efficient and native high-resolution image generation. — Superior Reconstruction Performance: Qwen-Image-VAE-2.0 achieves state-of-the-art reconstruction performance across multiple benchmarks. It maintains exceptional legibility in text-rich scenarios where traditional high-compression models typically fail. — Enhanced Latent Diffusability: Through a refined semantic alignment strategy, we demonstrate that large-channel VAEs can achieve excellent diffusability. This provides a promising solution to the tripartite trade-off between compression ratio, reconstruction fidelity, and diffusability. Model In this section, we present the detailed design and architectural innovations of Qwen-Image-VAE-2.0. 2.1 Design Principle: High Compression VAE with Large Channel To optimize the training efficiency of downstream DiTs, we prioritize a higher spatial compression ratio. Given an input image I\in\mathbb{R}^{H\times W\times 3} , the VAE maps it to a latent representation z\in\mathbb{R}^{\frac{H}{f}\times\frac{W}{f}\times C} , where f denotes the spatial compression ratio and C represents the channel dimension. This results in a sequence length of L=HW/f^{2} for the DiT. Since DiT’s computational complexity scales quadratically with sequence length ( 35 ) , the computation overhead of \mathcal{O}(L^{2})=\mathcal{O}(H^{2}W^{2}/{f^{4}}) presents a significant bottleneck in high-resolution image generation. To alleviate this, we move beyond the conventional f8 paradigm ( 30 ; 19 ; 36 ) , adopting higher compression ratios of f16 and f32 to significantly reduce DiT training costs. While high spatial compression ratio f promises training efficiency, it inevitably constrains the information capacity of the latent space, often resulting in the loss of fine-grained structural detail. To mitigate this, we rely on the principle that reconstruction fidelity is largely governed by the total information bottleneck N(z)={CHW}/{f^{2}} ( 4 ) . By increasing the channel dimension C , we compensate for the spatial information loss incurred by high compression ratio f . Notably, channel expansion does not compromise DiT training efficiency: during training, the DiT first projects latents into a fixed hidden dimension via a linear layer, ensuring that the computational complexity remains nearly invariant to channel dimension. 2.2 Model Architecture Figure 1: Comparison of No Skip Connection (NSC), Local Skip Connection (LSC), and Global Skip Connection (GSC) on model architecture, reconstruction loss and PSNR performance. S2C is the abbreviation of Space to Channel module. The experiments are conducted on f16c64 model training from scratch. Global Skip Connection (GSC). A primary challenge in high-compression VAEs is the preservation of fine-grained detail during the aggressive downsampling process. The encoder, particularly its