Self-Evaluation Unlocks Any-Step Text-to-Image Generation

Xiaojuan Qi ​, Zhengqi Li, Kai Zhang, Richard Zhang, Zhe Lin, Eli Shechtman, Tianyu Wang, Yotam Nitzan · 2025

Read Self-Evaluation Unlocks Any-Step Text-to-Image Generation by Xiaojuan Qi ​, Zhengqi Li, Kai Zhang, Richard Zhang, Zhe Lin, Eli Shechtman, Tianyu Wang, Yotam Nitzan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22374.txt.

Introduction

Xin Yu and eight co-authors introduce Self-E, a text-to-image model that grades its own homework while it learns. Flow and diffusion models learn only from local signals and so need many steps at generation time, while distilled fast models require a pretrained teacher. Self-E instead evaluates its own generated samples using its current score estimates, acting as a dynamic self-teacher alongside usual flow matching training.

Trained from scratch with no teacher, the model produces strong images at very low step counts yet also matches leading flow matching systems at fifty steps. Quality improves always as more steps are allowed, so one model covers both ultra-fast drafts and careful long runs. The authors describe Self-E as the first from-scratch, any-step text-to-image model.

Catalog record

CREATORXiaojuan Qi ​, Zhengqi Li, Kai Zhang, Richard Zhang, Zhe Lin, Eli Shechtman, Tianyu Wang, Yotam Nitzan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-22374

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Self-Evaluation Unlocks Any-Step Text-to-Image Generation Xiaojuan Qi ​, Zhengqi Li, Kai Zhang, Richard Zhang, Zhe Lin, Eli Shechtman, Tianyu Wang, Yotam Nitzan Abstract We introduce the Self-Evaluating Model (Self-E), a novel, from-scratch training approach for text-to-image generation that supports any-step inference. Self-E learns from data similarly to a Flow Matching model, while simultaneously employing a novel self-evaluation mechanism: it evaluates its own generated samples using its current score estimates, effectively serving as a dynamic self-teacher. Unlike traditional diffusion or flow models, it does not rely solely on local supervision, which typically necessitates many inference steps. Unlike distillation-based approaches, it does not require a pretrained teacher. This combination of instantaneous local learning and self-driven global matching bridges the gap between the two paradigms, enabling the training of a high-quality text-to-image model from scratch that excels even at very low step counts. Extensive experiments on large-scale text-to-image benchmarks show that Self-E not only excels in few-step generation, but is also competitive with state-of-the-art Flow Matching models at 50 steps. We further find that its performance improves monotonically as inference steps increase, enabling both ultra-fast few-step generation and high-quality long-trajectory sampling within a single unified model. To our knowledge, Self-E is the first from-scratch, any-step text-to-image model, offering a unified framework for efficient and scalable generation. 1 Introduction Figure 1 : Qualitative Any-Step Generation. We showcase diverse text-to-image results from our model at different inference step counts, demonstrating coherent semantics, strong text alignment. Text prompts are provided in the supplementary material. Diffusion models 18 ; 54 ; 55 and flow matching models 26 ; 29 currently dominate text-to-image generation due to their stability, scalability, and strong visual fidelity 2 ; 60 ; 61 ; 40 . These models are trained to approximate local supervision from data – either the score function or the instantaneous velocity field – which specifies how a noisy sample should infinitesimally move toward the data manifold at each timestep. Because this supervision is inherently local, it provides only short-range guidance: each update corrects small deviations but lacks a holistic global view of the target distribution. Consequently, diffusion and flow-based models typically require dozens of sequential steps to reliably traverse the curved reverse trajectory from noise to data, making inference computationally expensive and limiting their use in time-sensitive applications. A dominant strategy for reducing inference steps is distillation, where a pretrained teacher supervises a student model 43 ; 34 ; 63 ; 64 ; 44 ; 47 ; 25 ; 45 . Although these methods differ technically, they share a core principle: the student is optimized with global objectives that match the teacher’s distributions or trajectories, rather than data-derived local velocities, so that it can perform few-step inference. A key limitation, however, is the reliance on a strong pretrained teacher . This has recently motivated growing interest in self-contained, from-scratch training frameworks that natively yield few-step models. A prominent line of work is consistency-based methods 51 ; 22 ; 14 ; 30 ; 42 ; 4 ; 59 , which essentially learn the underlying flow maps 4 – or, equivalently, the average velocity 14 between two points along the reverse trajectory – so that, in principle, the model can follow a one-step shortcut instead of integrating many instantaneous velocities at test time. However, these objectives are typically unstable to optimize from scratch 72 ; 59 or suffer from quality degradation 70 , and have so far scaled reliably only on simpler benchmarks such as ImageNet 8 , while large text-to-image systems that do succeed in this regime still rely heavily on distillation 70 ; 42 , undermining the original teacher-free motivation. In this paper, we present the Self-Evaluating Model ( Self-E , pronounced like selfie) – a novel, self-contained, from-scratch training framework enabling any-step text-to-image inference. The model learns simultaneously from data, which provides local velocity supervision, and through a novel self-evaluation mechanism supervising the global distribution. The core idea is conceptually simple yet powerful: the model evaluates its own generated samples using its current local score estimate, effectively serving as a dynamic self-teacher . This self-evaluation becomes an increasingly accurate guidance signal – allowing the model to improve itself. By combining instantaneous local learning with self-driven global matching, Self-E naturally bridges the gap between flow-based and distillation-based paradigms. As a result, it can be trained entirely from scratch while supporting any-step text-to-image inference, generating high-quality images even at very low step counts (see Fig. 1 ). To the best of our knowledge, Self-E is the first native any-step text-to-image model, concurrent with TiM 59 . We conduct extensive experiments on large-scale text-to-image generation and show that Self-E achieves both strong few-step quality and graceful scaling across inference budgets. In the few-step ( <8 ) setting, Self-E surpasses the performance of diffusion and flow-based models including FLUX-1-dev 2 , SDXL 37 , SANA 61 , the distillation-based LCM 33 , and concurrent any-step model TiM 59 . Remarkably, although targeting this few-step generation, Self-E is also competitive or even surpasses some state-of-the-art flow-based methods at the 50 step setting. We also note that Self-E ’s performance improves monotonically as inference steps increase, enabling both ultra-fast few-step generation and high-quality long-trajectory sampling within a single unified model. Figure 2 : Self-Evaluating Model. (a) Overview. The model is trained with two complementary objectives: learning from data (b) and self-evaluation (c). (b) Learning from data. Given a real sample \mathbf{x}_{0} , we add noise to obtain \mathbf{x}_{t} and train G_{\theta}^{t\rightarrow s} with an x_{0} -prediction loss, providing local trajectory supervision. (c) Self-evaluation with classifier score. When s<t , we re-noise the generated \hat{\mathbf{x}}_{0} to \hat{\mathbf{x}}_{s} and run the same network in evaluation mode (stop-gradient) twice: once with condition \mathbf{c} and once with the null prompt \phi . The difference between these outputs yields a self-evaluation score, which is treated as a feedback gradient on \hat{\mathbf{x}}_{0} and back-propagated through the denoising path, enforcing global distribution matching in a teacher-free manner. 2 Background: Flow Matching Generative models aim to learn a parameterized distribution p_{\theta}(\mathbf{x}_{0}|\mathbf{c}) that approximates the real data distribution q(\mathbf{x}_{0}|\mathbf{c}) where \mathbf{c} is a condition such as text prompt. Flow matching and diffusion models achieve this by learning the instantaneous velocity field, or equivalently the score function, induced by a continuous-time forward diffusion process. Specifically, given real data samples \mathbf{x}_{0}\sim q(\mathbf{x}_{0}|\mathbf{c}) , these models define a trajectory indexed by t\in[0,1] : \mathbf{x}_{t}=\alpha_{t}\mathbf{x}_{0}+\sigma_{t}\boldsymbol{\epsilon},\quad\boldsymbol{\epsilon}\sim\mathcal{N}(0,\mathbf{I}), (1) where coefficients (\alpha_{t},\sigma_{t}) form a noise scheduler and defines the noisy distribution q(\mathbf{x}_{t}|\mathbf{c}) . Flow matching specifically refers to a particular parameterization that explicitly matches the velocity field: v_{t}(\mathbf{x}_{t}):=\frac{d\mathbf{x}_{t}}{dt}=\frac{d\alpha_{t}}{dt}\mathbf{x}_{0}+\frac{d\sigma_{t}}{dt}\boldsymbol{\epsilon}, (2) where coefficients (\alpha_{t},\sigma_{t})=(1-t,t). Conditional Flow Matching (CFM) trains a neural network V_{\theta}(\mathbf{x}_{t},t,\mathbf{c}) to predict the marginal velocity field (i.e., the expectation of instantaneous velocity) 26 by minimizing the mean squared error between predicted and conditional velocities: \mathcal{L}_{\text{CFM}}(\theta)=\mathbb{E}_{t,\mathbf{x}_{0},\boldsymbol{\epsilon}}\left[\|V_{\theta}(\mathbf{x}_{t},t,\mathbf{c})-v_{t}(\mathbf{x}_{t})\|^{2}\right]. (3) At inference time, we use the predicted velocity to follow the trajectory and generate samples. However, because the velocity is a local quantity, a single-step estimate of the original sample \mathbf{x}_{0} often falls short. Intuitively, a single step only captures the immediate direction and cannot account for the curvature of the trajectory, so it typically recovers just the average of the possible original samples. Formally, a naive one-step estimate is \hat{\mathbf{x}}_{0}=\mathbf{x}_{t}-t\,V_{\theta^{*}}(\mathbf{x}_{t},t,\mathbf{c})\approx\mathbb{E}[\mathbf{x}_{0}|\mathbf{x}_{t},\mathbf{c}], (4) where

Read the complete text