T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation
Zi-Ao Ma, Tian Lan, Rong-Cheng Tu, Shu-Hang Liu, Heyan Huang, Zhijing Wu, Chen Xu, Xian-Ling Mao · 2025
Read T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation by Zi-Ao Ma, Tian Lan, Rong-Cheng Tu, Shu-Hang Liu, Heyan Huang, Zhijing Wu, Chen Xu, Xian-Ling Mao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17897.txt.
Introduction
Zi-Ao Ma and seven colleagues wrote a 2025 paper on judging AI-made images in a way people can follow. Text-to-image models now draw well, and someone must grade their output at scale. Human grading is slow, and using pricey closed models adds heavy cost.
Open models trained on critique sets are one fix, but good critique data is scarce. Their system, T2I-Eval-R1, uses reinforcement learning to teach an open model to reason about image quality. The grader explains its scores, not just spits out a number. Clear reasons help users trust or challenge each grade. The team aims for cheap grading that stays open to inspection.
Catalog record
| CREATOR | Zi-Ao Ma, Tian Lan, Rong-Cheng Tu, Shu-Hang Liu, Heyan Huang, Zhijing Wu, Chen Xu, Xian-Ling Mao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17897
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation Zi-Ao Ma, Tian Lan, Rong-Cheng Tu, Shu-Hang Liu, Heyan Huang, Zhijing Wu, Chen Xu, Xian-Ling Mao Abstract The rapid progress in diffusion-based text-to-image (T2I) generation has created an urgent need for interpretable automatic evaluation methods that can assess the quality of generated images, therefore reducing the human annotation burden. To reduce the prohibitive cost of relying on commercial models for large-scale evaluation, and to improve the reasoning capabilities of open-source models, recent research has explored supervised fine-tuning (SFT) of multimodal large language models (MLLMs) as dedicated T2I evaluators. However, SFT approaches typically rely on high-quality critique datasets, which are either generated by proprietary LLMs—with potential issues of bias and inconsistency—or annotated by humans at high cost, limiting their scalability and generalization. To address these limitations, we propose T2I-Eval-R1, a novel reinforcement learning framework that trains open-source MLLMs using only coarse-grained quality scores, thereby avoiding the need for annotating high-quality interpretable evaluation rationale. Our approach integrates Group Relative Policy Optimization (GRPO) into the instruction-tuning process, enabling models to generate both scalar scores and interpretable reasoning chains with only easy acessible annotated judgment scores or preferences. Furthermore, we introduce a continuous reward formulation that encourages score diversity and provides stable optimization signals, leading to more robust and discriminative evaluation behavior. Experimental results on three established T2I meta-evaluation benchmarks demonstrate that T2I-Eval-R1 achieves significantly higher alignment with human assessments and offers more accurate interpretable score rationales compared to strong baseline methods. 1 Introduction Recent advancements in diffusion models have significantly improved the quality of text-to-image (T2I) generation, enabling the synthesis of high-resolution, photorealistic images from natural language prompts. Foundational models such as SD1.5 [ 28 ] and SDXL [ 25 ] have pushed the frontier of generative fidelity and efficiency, while recent transformer-based architectures [ 24 ] further enhance visual realism. Despite these advances, T2I systems still suffer from semantic inconsistencies—such as object hallucinations, attribute entanglement, and misalignment with complex or compositional instructions [ 11 , 8 ] —posing challenges for both quality control and model development. These limitations underscore the pressing need for robust, fine-grained evaluation methods that can assess the semantic faithfulness of generated images to input prompts. However, standard evaluation metrics remain inadequate. Inception Score [ 29 ] and FID [ 14 ] capture only global fidelity and distributional similarity, while CLIPScore [ 13 ] offers a coarse proxy for image-text alignment using pretrained encoders but lacks interpretability and correlates poorly with human judgments. To overcome these shortcomings, recent studies have explored leveraging large language models (LLMs) and multimodal LLMs (MLLMs) [ 23 , 16 , 22 , 20 , 31 ] as evaluators, owing to their strong semantic understanding and reasoning abilities [ 20 , 23 ] . Commercial MLLMs like GPT-4o have shown remarkable capability in scoring T2I outputs with chain-of-thought explanations, as demonstrated in frameworks such as VIEScore [ 20 ] . Nevertheless, relying on closed-source APIs introduces high cost, limited accessibility, and scalability concerns for real-world deployment. Therefore, recent work has turned to training open-source MLLMs through supervised fine-tuning (SFT), aiming to replicate the evaluation ability of commercial models while avoiding their cost and constraints. For instance, EATG [ 31 ] formulates the evaluation task into sub-skills such as object presence, attribute correctness, and spatial alignment, and uses fine-grained supervision to train MLLMs to act as interpretable evaluators. However, this paradigm still relies on high-quality supervision dataset, which typically comes from either human annotations or synthetic critiques generated by proprietary models, such as GPT-4o [ 1 ] and Gemini [ 2 ] . Human annotations are costly and time-consuming, requiring fine-grained judgments about object presence, attributes, and semantic alignment—making them difficult to scale across diverse prompts and domains. On the other hand, synthetic critiques often suffer from inconsistency, bias, and limited generalization, especially when transferred across models or applied to complex compositional prompts [ 31 ] . These limitations significantly hinder the scalability and reliability of supervised fine-tuning for training open-source MLLMs as robust T2I evaluators. To address these limitations, we propose T2I-Eval-R1, a novel framework that applies Rule-based Reinforcement Fine-Tuning (RFT) to train open-source MLLMs using only coarse-grained judgment scores or preferences. Unlike SFT approaches that depend on high-quality interpretable evaluation rationales, our method optimizes a reward-driven policy that encourages interpretable reasoning and semantic discrimination. Specifically, we integrate Group Relative Policy Optimization (GRPO) into the instruction-tuning pipeline to induce relative ranking behavior and structured rationales under weak supervision. Furthermore, we introduce a continuous reward formulation that increases reward diversity within training groups, improving learning stability and evaluator sharpness. We validate T2I-Eval-R1 on the T2I-Eval [ 31 ] , TIFA v1.0 [ 16 ] , and the ImageReward human preference dataset [ 36 ] and demonstrate new state-of-the-art Spearman and Kendall correlations with human judgments. Our method not only surpasses strong open-source and GPT-4o-based baselines, but also provides interpretable rationales for each prediction, facilitating transparency and human trust. 2 Related Work 2.1 Evaluation Methods for Synthesized Images Early evaluation of text-to-image (T2I) generation focused on global, distributional metrics that trade interpretability for efficiency. Inception Score (IS) quantifies image quality and diversity via class-prediction confidence and variety from a pretrained Inception-v3 network [ 29 ] , while Fréchet Inception Distance (FID) measures the difference in mean and covariance between real and generated image embeddings [ 14 ] . Although these metrics are computationally cheap and widely used, they do not assess semantic fidelity to the input text and provide no insight into specific errors. To incorporate semantic alignment, joint vision–language embeddings were introduced. CLIPScore computes the cosine similarity between CLIP-encoded image and text representations to approximate text–image alignment without references [ 13 ] , and BLIPv2Score refines this approach with richer pretrained multimodal representations [ 21 ] . Despite improved alignment, these methods remain opaque, yielding a single score without explanatory reasoning. Recognizing the need for interpretability, subsequent work leveraged large language models (LLMs) and visual question answering (VQA). LLMScore decomposes prompts into subquestions answered through text-only reasoning, enabling compositional fidelity checks [ 23 ] , while TIFA frames evaluation as VQA over prompt-derived questions to deliver fine-grained faithfulness via QA accuracy [ 16 ] . These task-decomposed strategies provide more transparency but incur high computational overhead and rely on handcrafted pipelines. More recently, multimodal LLMs (MLLMs) have been applied directly as evaluators, generating natural-language rationales alongside scalar judgments. For example, VIEScore prompts GPT-4o to produce chain-of-thought critiques that align closely with human preferences [ 20 ] . Although effective, these proprietary-API–based systems face challenges in cost, scalability, and reproducibility. Open-source alternatives seek to overcome these barriers via supervised fine-tuning (SFT). T2I-Eval trains MLLMs on human-annotated or LLM-generated critiques, decomposing evaluation into sub-tasks such as object presence, attribute correctness, and spatial alignment [ 31 ] . While this yields interpretable outputs, the dependence on large-scale, fine-grained annotations limits scalability and may introduce synthetic biases. Overall, evaluation has progressed from coarse statistical metrics to semantically rich, interpretable methods, yet existing approaches struggle to reconcile annotation cost, model accessibility, and explanation quality. This motivates methods that leverage weak or coarse-grained supervision to achieve both interpretability and scalability. 2.2 Reinforcement Learning for Evaluation The prohibitive cost and limited scalability of supervised fine-tuning