RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning
Mingrui Wu, Lu Wang, Pu Zhao, Fangkai Yang, Jianjin Zhang, Jianfeng Liu, Yuefeng Zhan, Weihao Han, Hao Sun, Jiayi Ji, Xiaoshuai Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang, Rongrong Ji 1 1 Key Laboratory of Multimedia Trusted Perception and Efficient Computing,Ministry of Educat · 2025
Read RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning by Mingrui Wu, Lu Wang, Pu Zhao, Fangkai Yang, Jianjin Zhang, Jianfeng Liu, Yuefeng Zhan, Weihao Han, Hao Sun, Jiayi Ji, Xiaoshuai Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang, Rongrong Ji 1 1 Key Laboratory of Multimedia Trusted Perception and Efficient Computing,Ministry of Educat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17540.txt.
Introduction
Text to image models often miss what a user meant when the prompt is short and vague. Mingrui Wu and co-authors present RePrompt, a framework that teaches a language model to rewrite prompts with explicit reasoning. Earlier prompt enhancers leaned on fixed rules or style tricks and drifted into unrealistic content. RePrompt instead trains the rewriter with reinforcement learning. Reward models grade the images that result on human preference, meaning match, and composition, so the rewriter learns from visual outcomes.
No hand labeled data are needed for the training. On two standard benchmarks, the method improved how faithfully images follow spatial layout and how well they generalize to new compositions. It set new best results across several image model backbones. The authors argue that reasoning, trained end to end against image quality, grounds prompt rewriting in what pictures actually need.
Catalog record
| CREATOR | Mingrui Wu, Lu Wang, Pu Zhao, Fangkai Yang, Jianjin Zhang, Jianfeng Liu, Yuefeng Zhan, Weihao Han, Hao Sun, Jiayi Ji, Xiaoshuai Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang, Rongrong Ji 1 1 Key Laboratory of Multimedia Trusted Perception and Efficient Computing,Ministry of Educat |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17540
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning Mingrui Wu, Lu Wang, Pu Zhao, Fangkai Yang, Jianjin Zhang, Jianfeng Liu, Yuefeng Zhan, Weihao Han, Hao Sun, Jiayi Ji, Xiaoshuai Sun, Qingwei Lin, Weiwei Deng, Dongmei Zhang, Feng Sun, Qi Zhang, Rongrong Ji 1 1 Key Laboratory of Multimedia Trusted Perception and Efficient Computing,Ministry of Education of China, Xiamen University, 361005, P.R. China 2 2 Microsoft Abstract Despite recent progress in text-to-image (T2I) generation, existing models often struggle to faithfully capture user intentions from short and under-specified prompts. While prior work has attempted to enhance prompts using large language models (LLMs), these methods frequently generate stylistic or unrealistic content due to insufficient grounding in visual semantics and real-world composition. Inspired by recent advances in reasoning for language model, we propose RePrompt, a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning. Instead of relying on handcrafted rules or stylistic rewrites, our method trains a language model to generate structured, self-reflective prompts by optimizing for image-level outcomes. The tailored reward models assesse the generated images in terms of human preference, semantic alignment, and visual composition, providing indirect supervision to refine prompt generation. Our approach enables end-to-end training without human-annotated data. Experiments on GenEval and T2I-Compbench show that RePrompt significantly boosts spatial layout fidelity and compositional generalization across diverse T2I backbones, establishing new state-of-the-art results. Code is available at: https://github.com/microsoft/DKI_LLM/tree/main/RePrompt . 1 Introduction Text-to-image (T2I) generation has made rapid progress with the rise of large-scale generative models flux2024 ; esser2024scaling ; podell2023sdxl ; chen2024pixart , yet a persistent challenge remains: users typically provide concise and under-specified prompts, which often result in images that fail to reflect the intended semantics or visually coherent compositions. Generated outputs may misrepresent object counts, overlook spatial relations, or violate real-world plausibility. This misalignment arises from the gap between natural language descriptions and the structured visual logic required for faithful image generation yang2024mastering . Previous work on prompt enhancement in T2I can be divided into two main approaches. The first approach focuses on iterative refinement: an image is generated from an initial prompt, and subsequent feedback, derived either from human preference models or from automated scoring systems, is used to improve the prompt or intermediate representations over multiple rounds yang2024idea2img ; wu2024self ; wang2024genartist ; guo2025can . Although this approach can progressively improve image quality, it suffers from high latency and computational overhead due to repeated image generation, and it rarely incorporates explicit scene semantics or compositional reasoning. The second approach enriches prompts in a single pass by leveraging large language models (LLMs) to inject additional detail and context betker2023improving ; hao2023optimizing . While these methods produce linguistically fluent and expressive descriptions, they frequently generate prompts that produce images with semantically inconsistent or visually implausible content, such as conflicting object placements or unrealistic interactions, because the underlying LLMs lack grounding in physical reality and do not incorporate feedback from downstream visual tasks. As a result, they frequently hallucinate content or miss critical spatial and attribute-level relationships (see Figure 1 ). Figure 1: Given the user prompt " a photo of a couch below a vase ", existing models like DELL-E3 generate rich language descriptions but often produce unrealistic or physically implausible compositions. In contrast, our RePrompt performs explicit chain-of-thought reasoning to resolve spatial relations, resulting in enhanced prompts that guide text-to-image models towards realistic and semantically aligned generations. In contrast, we propose RePrompt, a reasoning-augmented prompt refinement framework trained via reinforcement learning. Rather than relying on stylistic rewriting or black-box completions, RePrompt trains a language model to generate structured and semantically grounded prompts through self-reflection and step-by-step decomposition. Motivated by recent advances in reasoning-augmented language models guo2025deepseek ; trung2024reft ; team2025kimi ; jaech2024openai , RePrompt enables the model to internally simulate the visual implications of a prompt—much like how humans mentally visualize a scene before drawing. This structured, logic-driven process anticipates potential errors (e.g., conflicting object positions, missing entities, or spatial incoherence) during prompt construction, thereby reducing the need for multiple rounds of image generation. A core component of RePrompt is a T2I RePrompt Reward Model tailored for text-to-image generation. Instead of relying on pre-labeled reasoning traces or handcrafted prompt templates, RePrompt learns from downstream visual feedback by optimizing prompt generation through reinforcement learning. To capture the multifaceted nature of image quality, we design a ensemble reward that evaluates generated images along three dimentions: human preference, visual realism, and semantic alignment with the input. By learning from diverse and grounded feedback signals, the model develops a more robust reasoning strategy that transfers across prompt types, scene structures, and T2I backbones, enabling stronger performance on unseen inputs without overfitting to specific linguistic patterns. Experiments on GenEval ghosh2023geneval and T2I-Compbench huang2023t2i demonstrate that RePrompt significantly improves compositional accuracy, semantic fidelity, and spatial coherence. Notably, on the GenEval benchmark, RePrompt surpasses Qwen2.5 3B-enhanced baselines by +77.1% (FLUX flux2024 ), +78.8% (SD3 esser2024scaling ) and +122.2% (Pixart- \Sigma chen2024pixart ) in the position category, highlighting its superior capability in grounding spatial relations. Furthermore, RePrompt achieves the best overall accuracy (0.76) among all evaluated methods while maintaining an order-of-magnitude lower latency than optimization-heavy baselines like Idea2Img (30s vs. 140s per image), offering a scalable and inference-efficient solution. These findings validate the effectiveness of explicit reasoning in prompt construction for closing the semantic-visual alignment gap in text-to-image generation, without relying on larger language models or expensive optimization at inference time. Figure 2: Overview of the proposed RePrompt. For each input prompt, RePrompt generates multiple reasoning trace and enhanced prompt pairs. The reasoning trace guides the model to produce more detailed, image-grounded prompts. These are used to synthesize candidate images via a T2I model, which are then scored by a reward model. Feedback is used to update RePrompt via GRPO. 2 Related Work Text to Image Generation. Recently, large-scale diffusion models flux2024 ; esser2024scaling ; podell2023sdxl ; chen2024pixart ; xie2025sana ; liu2024playground ; betker2023improving ; ma2025inference ; rombach2022high ; saharia2022photorealistic have achieved impressive progress in generating high-resolution, photorealistic images from complex textual prompts. To enhance alignment between text and visuals, prior work has explored prompt engineering hao2023optimizing ; mo2024dynamic ; yeh2024tipo ; manas2024improving ; yun2025learning ; cao2023beautifulprompt ; qin2024diffusiongpt ; yang2024idea2img ; wu2024self ; wang2024genartist , often relying on manual or heuristic strategies with limited generalization. We propose a reinforcement learning-based framework that automatically refines prompts through iterative reasoning, achieving better semantic alignment than static or rule-based methods. Reinforcement Learning. Reinforcement learning (RL) has proven effective in scenarios where iterative feedback is essential for task optimization. In the realm of generative models, RL-based approaches guo2025deepseek ; wallace2024diffusion ; yang2024dense ; guo2024versat2i ; gupta2025simple ; lee2024parrot ; zhao2024scores ; nabati2024personalized ; black2023training ; liang2024rich ; kirstain2023pick ; lee2023aligning ; shen2025vlm ; zhang2024itercomp ; wang2025simplear ; jiang2025t2i ; xue2025dancegrpo have been employed to fine-tune outputs by maximizing a reward function that encapsulates desired attributes such as image realism, diversity, and semantic fidelity. In our framework, we adopt RL techniques to drive the automatic optimization of text prompts. By defining a multi-faceted reward that not only evaluates the visual quality of the generated image but also the interpretability and relevance of the prompt, our approach enables the model to learn an optimal