From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuning

Le Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed Elhoseiny, Hongsheng Li · 2025

Read From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuning by Le Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed Elhoseiny, Hongsheng Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16080.txt.

Introduction

Le Zhuo and co-authors teach image generators to look at their own work, admit the flaws, and repaint. Their 2025 paper introduces ReflectionFlow, which scales effort at inference time along three axes. It searches for better starting noise and sharper prompts. Most notably, it adds explicit reflection, where the model assesses its last image and states what to fix.

Reflection needs training data, so the team built GenRef: a million triplets of a critique, a flawed image, and its improved version. A leading diffusion model was then reflection-tuned on it. The full system clearly beat naive noise scaling on hard generation tasks. Language models gained reasoning power by critiquing themselves at runtime; this paper ports the trick to pixels. Compute spent reflecting, it turns out, buys quality that compute spent sampling does not.

Catalog record

CREATORLe Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed Elhoseiny, Hongsheng Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16080

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuning Le Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed Elhoseiny, Hongsheng Li Abstract Recent text-to-image diffusion models achieve impressive visual quality through extensive scaling of training data and model parameters, yet they often struggle with complex scenes and fine-grained details. Inspired by the self-reflection capabilities emergent in large language models, we propose ReflectionFlow, an inference-time framework enabling diffusion models to iteratively reflect upon and refine their outputs. ReflectionFlow introduces three complementary inference-time scaling axes: (1) noise-level scaling to optimize latent initialization; (2) prompt-level scaling for precise semantic guidance; and most notably, (3) reflection-level scaling, which explicitly provides actionable reflections to iteratively assess and correct previous generations. To facilitate reflection-level scaling, we construct GenRef, a large-scale dataset comprising 1 million triplets, each containing a reflection, a flawed image, and an enhanced image. Leveraging this dataset, we efficiently perform reflection tuning on state-of-the-art diffusion transformer, FLUX.1-dev, by jointly modeling multimodal inputs within a unified framework. Experimental results show that ReflectionFlow significantly outperforms naive noise-level scaling methods, offering a scalable and compute-efficient solution toward higher-quality image synthesis on challenging tasks. All code, checkpoints, and datasets are available at https://diffusion-cot.github.io/reflection2perfection . Figure 1 : Overall pipeline of the ReflectionFlow framework with qualitative and quantitative results of scaling compute at inference time. † † footnotetext: ∗ Equal Contribution † † footnotetext: \dagger Corresponding Author 1 Introduction Recent advances in text-to-image (T2I) diffusion models [ 12 , 26 , 67 , 59 , 16 , 1 , 53 ] have led to remarkable progress in image synthesis. With the scaling of training resources, their capability to create high-resolution and photorealistic images steadily improves. Despite their success, their performance across various domains remains inconsistent, especially when tasked with generating complex human poses, multiple-object compositions, or scenes with complicated lighting and shadows. To alleviate these limitations, it is necessary to exponentially scale the training compute, model parameters, and data size, according to established scaling laws at training time [ 22 ] . This motivates us to rethink the prevailing paradigm: instead of continuously scaling pretrained models, how to effectively exploit the full capabilities of existing diffusion models during inference? We hypothesize that while T2I models may struggle to generate desirable images within a fixed compute budget, their performance can be significantly improved through an iterative refinement process. This idea of leveraging additional computational resources to enhance performance during inference has been validated in large language models (LLMs) [ 48 ] . Modern LLMs have demonstrated the ability to improve their outputs by reflecting on intermediate outputs and subsequently refining their responses [ 35 , 25 , 46 ] , leading to superior performance on complex tasks such as math problem solving and code generation. However, current inference-time optimizations for diffusion models [ 34 , 47 , 60 ] focus on parallelized noise-space search strategies, leaving sequential self-refinement paradigm that could enable diffusion models to reflect upon and correct their own mistakes largely unexplored. In this paper, we introduce ReflectionFlow , a novel inference-time self-refinement framework for diffusion models leveraging iterative reflection. Our approach explores three dimensions for scaling inference-time computation: (1) noise-level scaling, which searches for better noise initialization; (2) prompt-level scaling, which optimizes input prompt for precise semantic guidance; and (3) reflection-level scaling, which constructs explicit reflections to iteratively assess and correct previously generated outputs. Integrating these dimensions enables diffusion models to flexibly exploit additional computational resources at inference-time, continuously improving the image quality through an iterative process as illustrated in Fig. 1 . A fundamental challenge underlying our approach is: how to empower T2I diffusion models with self-refinement capabilities? Currently, no existing diffusion model is able to accurately interpret reflection prompts and leverage previously generated images for iterative refinement. Reflecting on the intrinsic similarity between self-refinement and image editing tasks, we observe that both involve generating a new output, jointly conditioned on textual and visual contexts. Inspired by the training paradigm of efficiently adapting diffusion priors to image editing [ 7 , 55 ] , we hypothesize that adopting similar methods could enable diffusion models to achieve self-refinement. However, another critical challenge arises: the absence of dedicated datasets specifically designed for reflection-guided refinement. Motivated by this insight, we propose GenRef , the first large-scale image reflection dataset comprising 1 million reflection triplets in multiple domains. To ensure the validity of these triplets ( e.g. , flawed images accurately capture potential errors, the reflections offer effective and actionable editing suggestions, and the good images exhibit sufficiently higher quality), we design a scalable, fully automatic data construction pipeline with four distinct data sources, leveraging verifiable objectives, ensemble reward models, and diverse rollout strategies. We further create a chain-of-thought (CoT) reflection subset, named GenRef-CoT , providing 227K high-quality progressive annotations from GPT-4o [ 21 ] and Gemini 2.0 [ 52 ] . We then employ an efficient reflection tuning tailored for T2I diffusion transformers, such as FLUX.1-dev [ 26 ] . In our approach, the original prompt, reflection prompt, flawed image, and high-quality image are concatenated into a single unified sequence to perform joint multimodal attention, thereby eliminating the need for additional modules [ 65 , 63 ] . Note that beyond laying the foundation for reflection tuning, our carefully curated dataset can also serve broader applications, such as general preference tuning [ 54 ] and reward modeling [ 32 ] . Through experiments, ReflectionFlow demonstrates the potential for enhancing image generation quality without requiring additional large-scale training. Compared to naive noise-level scaling, ReflectionFlow achieves significantly better performance under identical inference budgets, with performance consistently improved using better verifiers. Additionally, we explore the flexible trade-off between search width and reflection depth, as well as overall inference budgets. Further evaluation shows that our approach achieves particularly substantial gains on challenging prompts. Qualitative analyses illustrate how our model iteratively reflects on and corrects its outputs, progressively converging to superior solutions. In summary, (1) we propose ReflectionFlow, a reflection-based inference-time scaling framework for diffusion models, integrating three scaling axes, i.e. , noise-, prompt-, and reflection-level; (2) we propose a scalable pipeline for collecting high-quality image reflection data and introduce GenRef, the first large-scale dataset containing 1 million triplets, along with an additional 227K CoT reflection annotations; (3) we employ an efficient reflection tuning method for diffusion transformers with a tailored training strategy; (4) extensive experiments demonstrate that ReflectionFlow substantially boosts performance via scaling inference-time compute, unlocking reasoning capability. 2 Related Work Text-to-Image Diffusion Models. Text-to-image diffusion models have rapidly advanced in terms of model architecture and training strategies. In terms of architecture evolution, the community has transitioned from the prevalent U-Net diffusion models [ 44 ] towards diffusion transformers (DiT) [ 37 ] . Regarding training strategies, earlier complex hand-designed diffusion schedules [ 19 , 50 ] have given way to simpler flow-based formulations [ 33 , 30 ] , significantly enhancing training efficiency through techniques such as multi-resolution progressive training [ 9 ] . Recently, scaling up both the training datasets and model parameters has led to the emergence of various large-scale, flow-based diffusion transformer models [ 12 , 26 , 14 , 67 , 59 , 41 ] . Diffusion

Read the complete text