T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, Hongsheng Li · 2025
Read T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT by Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, Hongsheng Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00703.txt.
Introduction
Dongzhi Jiang and co-authors add step-by-step reasoning to text-to-image models. Their 2025 paper splits the reasoning into two levels: a semantic plan that reads the prompt as a whole, and a token path that fills the image patch by patch. Reward training tunes both levels in one step.
A bundle of reward scores guides the joint training, so plan and pixels improve together. Built on a known base model, the system gained thirteen percent on a layout test and nineteen percent on a knowledge test. It even passed a top rival model on the second test. Code is public. The work shows that image models, like text models, gain from taught reasoning.
Catalog record
| CREATOR | Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, Hongsheng Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00703
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, Hongsheng Li Abstract Recent advancements in large language models have demonstrated how chain-of-thought (CoT) and reinforcement learning (RL) can improve performance. However, applying such reasoning strategies to the visual generation domain remains largely unexplored. In this paper, we present T2I-R1 , a novel reasoning-enhanced text-to-image generation model, powered by RL with a bi-level CoT reasoning process. Specifically, we identify two levels of CoT that can be utilized to enhance different stages of generation: (1) the semantic-level CoT for high-level planning of the prompt and (2) the token-level CoT for low-level pixel processing during patch-by-patch generation. To better coordinate these two levels of CoT, we introduce BiCoT-GRPO with an ensemble of generation rewards, which seamlessly optimizes both generated CoTs within the same training step. By applying our reasoning strategies to the baseline model, Janus-Pro, we achieve superior performance with 13% improvement on T2I-CompBench and 19% improvement on the WISE benchmark, even surpassing the state-of-the-art model FLUX.1. Code is available at: https://github.com/CaraJ7/T2I-R1 . 1 Introduction The emergence of advanced Large Language Models (LLMs) [ 50 , 52 , 69 , 81 ] , such as OpenAI o1 [ 53 ] and DeepSeek-R1 [ 17 ] , has demonstrated considerable reasoning capabilities across domains including mathematics [ 1 , 21 , 46 ] and coding [ 6 , 2 , 24 ] . Through reinforcement learning (RL) [ 60 , 61 ] , these models analyze problems progressively with a comprehensive Chain-of-Thought (CoT) [ 72 , 27 , 19 , 26 , 86 , 18 ] before providing answers, significantly enhancing output accuracy. The CoT reasoning strategies have also been extended to the visual domain. Recent Large Multi-modal Models (LMMs) [ 5 , 47 , 84 , 87 ] have adapted the paradigm to accommodate the visual understanding task [ 43 , 86 , 26 ] . These advanced LMMs can jointly process images and their associated textual queries, performing step-by-step analyses of visual details and integrating them with reasoning steps to derive final answers. Concurrently, CoT-like reasoning has been initially investigated in the visual generation task, particularly in autoregressive text-to-image generation. The pioneering work, ‘Image Generation with CoT’ [ 19 ] , regards the progressive generation of the image tokens as a kind of CoT analogous to that of the text tokens, and proposes to optimize this intermediate process to enhance the image quality. Despite these advances, the exploration of CoT for image generation remains preliminary. Unlike image understanding, image generation requires the complex interpretation of cross-modal alignment and the synthesis of fine-grained visual details. To address these challenges, we identify two distinct levels of CoT reasoning that can be leveraged to enhance image generation, as illustrated in Fig. 1 : • Semantic-level CoT is the textual reasoning about the image to generate, which is introduced prior to the image generation. The semantic-level CoT designs the global structure of the image, e.g., the appearance and location of objects. In case the prompt requires reasoning shown in Fig. 2 , the semantic-level CoT also helps to deduce the objects to generate. Optimizing the semantic-level CoT could explicitly decouple the planning and reasoning of the prompt from the subsequent image tokens generation, making the generation easier. • Token-level CoT is the intermediate patch-by-patch generation process of the image, as originally introduced in [ 19 ] . This process could be viewed as a form of CoT as it outputs each subsequent token conditioned on all previous tokens within a discrete space, similar to the textual CoT. Unlike semantic-level CoT, token-level CoT focuses on low-level details like pixel generation and maintaining visual coherence between adjacent patches. Optimizing the token-level CoT can enhance both the generation quality and the alignment between the prompt and the resulting images. Figure 1: The Illustration of CoT in Image Understand and Generation Tasks. In the image understanding task, the CoT is the textual reasoning process. In the autoregressive visual generation task, we identify two levels of CoT: the semantic-level and token-level CoT. The semantic-level CoT is the high-level planning prior to the image generation, in the form of text. The token-level CoT is the intermediate patch-by-patch generation process, focusing on the local pixel details within a patch, in the form of image tokens. Despite recognizing these two levels of CoT, a critical question remains unaddressed: How can we enhance and coordinate them for text-to-image generation? Current mainstream generative models [ 63 , 67 , 58 , 28 ] are trained exclusively on generation targets, lacking the explicit textual understanding required for semantic-level CoT reasoning. Although introducing a separate model (e.g., an LLM) specifically for prompt interpretation [ 9 ] is technically feasible, this approach would significantly increase computational costs, complexity, and deployment challenges. Recently, a trend has arisen to merge visual understanding and generation within a single model. Building upon LMMs, these unified LMMs (ULMs) [ 74 , 79 , 88 , 7 ] could not only understand the visual inputs but also generate images from text prompts. However, their two capabilities are still decoupled, typically pre-trained in two independent stages, with no clear evidence that the understanding capabilities can benefit generation. Given these potentials and issues, we start from a ULM and enhance it to unite both the semantic-level and token-level CoT into one framework for text-to-image generation. To fulfill our target, we introduce BiCoT-GRPO , an RL method to jointly optimize the two levels of CoT for ULM. We opt for RL instead of supervised fine-tuning (SFT) for two reasons: First, the ULM has possessed the fundamental ability needed for the semantic-level and token-level CoT; our goal is only to elicit the fusion of these two abilities by guiding the model’s self-exploration. Second, RL methods have proven highly effective for enhancing reasoning capabilities, which are essential for both levels of CoT. Specifically, we first instruct the ULM to imagine and plan the image based on the prompt to obtain the semantic-level CoT. Then, we feed it into the ULM as the condition for the subsequent image generation for token-level CoT. We simultaneously generate multiple images from each prompt and then compute group-relative reward to optimize both levels of CoT within the same iteration. Unlike understanding tasks, where clearly defined rules for rewards exist, image generation lacks such standardized rules. Therefore, we propose to utilize an ensemble of diverse vision experts [ 77 , 70 , 41 , 19 ] as reward models. This reward design serves two critical purposes: it evaluates generated images from multiple dimensions to ensure reliable quality assessment, while also functioning as a regularization method to prevent the ULM from hacking a single reward model. Through the proposed reasoning strategies, we obtain T2I-R1 , the first reasoning-enhanced text-to-image model combining the semantic-level and token-level CoT. Empirical results show that our approach outperforms baseline models by 13% and 19% improvements on the T2I-CompBench and WISE benchmark, and even surpasses the previous state-of-the-art model FLUX.1. Qualitative analysis reveals that our method empowers the model to generate more human-aligned results by reasoning about the true intentions behind the prompt and demonstrates enhanced