Tuna : T aming U nified Visual Representations for Na tive Unified Multimodal Models
Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, Shoufa Chen, Haonan Qiu, Xiaoke Huang, Zhaochong An, Fanny Yang, Aditya Patel, Viktar Atliha, Tony Ng, Xiao Han, Chuyan Zhu, Chenyang Zhang, Ding Liu, Juan-Manuel Perez-Rua, Sen He, Jürgen Schmidhuber, Wenhu Chen, Ping Luo, Wei Liu, Tao Xiang, Jonas · 2025
Read Tuna : T aming U nified Visual Representations for Na tive Unified Multimodal Models by Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, Shoufa Chen, Haonan Qiu, Xiaoke Huang, Zhaochong An, Fanny Yang, Aditya Patel, Viktar Atliha, Tony Ng, Xiao Han, Chuyan Zhu, Chenyang Zhang, Ding Liu, Juan-Manuel Perez-Rua, Sen He, Jürgen Schmidhuber, Wenhu Chen, Ping Luo, Wei Liu, Tao Xiang, Jonas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02014.txt.
Introduction
Most AI systems either understand images or generate them, and unified models try to do both in one framework. Zhiheng Liu and a large team present Tuna in 2025, a native unified multimodal model built around a single continuous representation of visual content.
Tuna cascades two encoders: a variational autoencoder that compresses images, and a representation encoder that adds meaning. The shared representation space then serves both understanding and generation, for images and video alike, processed end to end. A common visual language inside the model could make its two talents reinforce rather than compete with each other.
Catalog record
| CREATOR | Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, Shoufa Chen, Haonan Qiu, Xiaoke Huang, Zhaochong An, Fanny Yang, Aditya Patel, Viktar Atliha, Tony Ng, Xiao Han, Chuyan Zhu, Chenyang Zhang, Ding Liu, Juan-Manuel Perez-Rua, Sen He, Jürgen Schmidhuber, Wenhu Chen, Ping Luo, Wei Liu, Tao Xiang, Jonas |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-02014
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Tuna : T aming U nified Visual Representations for Na tive Unified Multimodal Models Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, Shoufa Chen, Haonan Qiu, Xiaoke Huang, Zhaochong An, Fanny Yang, Aditya Patel, Viktar Atliha, Tony Ng, Xiao Han, Chuyan Zhu, Chenyang Zhang, Ding Liu, Juan-Manuel Perez-Rua, Sen He, Jürgen Schmidhuber, Wenhu Chen, Ping Luo, Wei Liu, Tao Xiang, Jonas Schult, Yuren Cong Abstract Unified multimodal models (UMMs) aim to jointly perform multimodal understanding and generation within a single framework. We present Tuna , a native UMM that builds a unified continuous visual representation by cascading a VAE encoder with a representation encoder. This unified representation space allows end-to-end processing of images and videos for both understanding and generation tasks. Compared to prior UMMs with decoupled representations, Tuna ’s unified visual space avoids representation format mismatches introduced by separate encoders, outperforming decoupled alternatives in both understanding and generation. Moreover, we observe that stronger pretrained representation encoders consistently yield better performance across all multimodal tasks, highlighting the importance of the representation encoder. Finally, in this unified setting, jointly training on both understanding and generation data allows the two tasks to benefit from each other rather than interfere. Our extensive experiments on multimodal understanding and generation benchmarks show that Tuna achieves state-of-the-art results in image and video understanding, image and video generation, and image editing, demonstrating the effectiveness and scalability of its unified representation design. † † affiliation: Meta BizAI † † affiliation: HKU † † affiliation: University of Waterloo † † affiliation: KAUST † † contribution: Joint first authors, listed alphabetically by last name † † contribution: Core contributors † † contribution: Joint project lead † † correspondence: [email protected] , [email protected] , [email protected] , [email protected] † † Project page: https://tuna-ai.org 1 Introduction A long-term aspiration of multimodal AI is natively 1 1 1 We define native UMMs as models that are pretrained jointly on both understanding and generation objectives, instead of assembling understanding-only and generation-only models with learnable connectors. We refer to the latter as composite UMMs. unified multimodal generation, in which a single model can seamlessly understand and generate diverse modalities such as text, images, and videos. Recent advances in unified multimodal models ( 92 ; 13 ; 117 ) have shown promising results towards this vision, suggesting that truly integrated multimodal intelligence is increasingly within reach. Figure 1: We present Tuna , a native unified multimodal model built on a unified visual representation, enabling diverse multimodal understanding and generation capabilities such as image and video understanding, image and video generation, and image editing. A central challenge in developing native UMMs lies in how visual inputs are encoded into representations. Current UMMs adopt one of two approaches: (1) decoupled visual representations for understanding and generation tasks, or (2) a unified visual representation shared across both tasks. Intuitively, learning a unified visual representation for both tasks offers compelling advantages for UMMs. First, UMMs with decoupled representations, such as BAGEL ( 13 ) and Mogao ( 47 ) , often adopt MoE-style architectures to handle different visual encoders, introducing additional parameters that increase training and inference costs. In contrast, a unified representation allows the model to operate within a single representation space, simplifying training and improving efficiency. Second, different vision encoders typically produce representations with incompatible formats. For the same input, features from a representation encoder ( e.g . SigLIP ( 126 ) ) and a causal VAE encoder ( e.g . Wan 2.1 VAE ( 98 ) ) differ in (1) spatial compression (16 \times vs. 8 \times ), (2) temporal compression (none vs. 4 \times ), and (3) channel dimension (1152 vs. 16). These discrepancies can cause representation conflicts in decoupled models, whereas unified representations inherently avoid such inconsistencies. Finally, unified visual representations provide a clear pathway for achieving mutual enhancement between understanding and generation. While recent studies like Ross ( 100 ) and REPA ( 123 ) show task-specific improvements in understanding-only and generation-only models, this synergy remains underexplored in existing UMMs. Nevertheless, current UMMs with unified visual representations often underperform their decoupled counterparts. Most existing approaches adopt a single type of vision encoder for both understanding and generation. For example, Chameleon ( 92 ) and Transfusion ( 130 ) use VQ-VAE ( 17 ) , while Harmon ( 111 ) utilizes the MAR encoder ( 43 ) . This unified design tends to favor one task at the expense of the other. Show-o2 ( 117 ) attempts to mitigate this issue by fusing SigLIP ( 126 ) and VAE ( 98 ) features through a late-fusion strategy. However, our analysis in Section 3.4 reveals that its learned representation remains biased toward semantic features, resulting in limited generation quality. To systematically address these limitations, we propose Tuna , a native UMM that employs unified visual representations across understanding and generation. Our design is simple yet highly effective: by directly connecting a VAE encoder to a representation encoder, we obtain representations that are sufficiently expressive for diverse multimodal tasks. These unified visual features are fused with text tokens and processed by an LLM decoder, which subsequently generates new text tokens and denoised images through autoregressive next-token prediction and flow matching. As illustrated in Figure 1 , our unified visual representation enables Tuna to handle image and video understanding, image and video generation, and image editing within a single framework. By conducting a three-stage training, our model achieves state-of-the-art performance on multimodal understanding and generation benchmarks ( e.g . 61.2% on MMStar ( 10 ) and 0.90 on GenEval ( 23 ) ). Our main contributions can be summarized as follows: 1. We propose Tuna , a native unified multimodal model with unified visual representations, enabling image/video understanding, image/video generation, and image editing within a single framework. 2. Our extensive experiments show that Tuna ’s unified visual representation is highly effective, achieving state-of-the-art performance across multiple multimodal understanding and generation tasks. 3. We further perform a comprehensive ablation study, demonstrating the superiority of our unified visual representation design over existing methods such as Show-o2 and other models employing decoupled representations. 2 Our Method: Tuna In this section, we introduce Tuna , a native unified multimodal model that employs unified visual representations across all multimodal understanding and generation tasks. We begin by outlining the key motivations behind our model design in Section 2.1 , followed by a detailed description of Tuna ’s architecture and training pipeline in Sections 2.2 and 2.3 , respectively. An overview of our overall framework is shown in Figure 2 . Figure 2: Overview of the Tuna architecture. Our model employs a VAE encoder and a representation encoder to construct unified visual representations, which are then combined with text tokens and processed by an LLM decoder. The decoder performs autoregressive text generation for understanding tasks and flow-matching-based visual generation for generation tasks. ∗ During visual generation, noise is added to the visual tokens to enable diffusion-based generation. 2.1 Motivation and Design Principles We discuss the following observations, which motivate the design of Tuna and its unified visual representations: Autoregressive vs. diffusion. Both text and image/video generation can be achieved using autoregressive ( 24 ; 120 ; 90 ) or diffusion models ( 69 ; 6 ; 98