ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedule

Yilie Huang, Wenpin Tang, Xunyu Zhou · 2026

Read ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedule by Yilie Huang, Wenpin Tang, Xunyu Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18681.txt.

Introduction

Yilie Huang, Wenpin Tang, and Xunyu Zhou tune the clock of tools that build data by a slow walk back from noise. Those tools start from pure noise and step, time by time, to a clean case. The steps can be set at even gaps or by a hand-made plan. With a fixed count of steps, a poor plan wastes steps where they count least.

Their plan, named in the title, lets a step rule learn to speed or slow the clock as the walk goes. Steps grow thick where the walk needs care and thin where it runs free. The total time of the walk stays the same. A learned clock could give the same grade of case for fewer steps. That would cut the cost of each new shot or text such tools make.

Catalog record

CREATORYilie Huang, Wenpin Tang, Xunyu Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18681

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedule Yilie Huang, Wenpin Tang, Xunyu Zhou Abstract We consider time discretization for score-based diffusion models to generate samples from a learned reverse-time dynamic on a finite grid. Uniform and hand-crafted grids can be suboptimal given a budget on the number of time steps. We introduce Adaptive Reparameterized Time (ART) that controls the clock speed of a reparameterized time variable, leading to a time change and uneven timesteps along the sampling trajectory while preserving the terminal time. The objective is to minimize the aggregate error arising from the discretized Euler scheme. We derive a randomized control companion, ART-RL, and formulate time change as a continuous-time reinforcement learning (RL) problem with Gaussian policies. We then prove that solving ART-RL recovers the optimal ART schedule, which in turn enables practical actor–critic updates to learn the latter in a data-driven way. Empirically, based on the official EDM pipeline, ART-RL improves Fréchet Inception Distance on CIFAR-10 over a wide range of budgets and transfers to AFHQv2, FFHQ, and ImageNet without the need of retraining. 1 Introduction Diffusion models ( 7 ; 30 ; 31 ) are a family of generative models that create samples from unknown target distributions based on given samples. They have played a key role in the success in text-to-image creators such as DALL·E 2 ( 25 ) and Stable Diffusion ( 26 ) , in text-to-video generators such as Sora ( 24 ) , Make-A-Video ( 28 ) and Veo ( 6 ) and, more recently, in diffusion large language models such as Mercury ( 18 ) and LLaDA ( 23 ) . A diffusion model consists of two steps: learn the target score function by training on given samples (pretraining), and generate new samples from the learned model (inference). The focus of this paper is on diffusion sampling/inference, where it involves choosing timesteps or time discretization of the learned model. Existing work uses either uniform or hand-crafted timesteps ( 29 ; 31 ; 16 ; 2 ) , lacking a principled framework to schedule timesteps. Here our purpose is to provide a control-theoretic framework, which allows strategic and systematic time discretization in diffusion sampling in order to minimize the aggregate error when applying the Euler scheme. The main contribution of this paper is as follows: • Methodology : We propose an optimal control framework – Adaptive Reparameterized Time (ART), to schedule timesteps for diffusion sampling. The idea is to treat the speed of the diffusion sampler as control to reparameterize time and adaptively redistribute computation along the sampling trajectory. To solve the ART problem, we propose a continuous-time reinforcement learning (CTRL) approach – ART-RL, premised upon the recent development in 35 ; 13 ; 14 . • Theory : We provide a rigorous theoretical analysis of the CTRL formulation in the ART setting. We show that the optimal ART control coincides with the mean of the optimal randomized Gaussian policy in ART-RL, establishing a precise link between ART and ART-RL. Based on this connection, we derive actor–-critic update rules for learning adaptive time schedules. • Experiments : Our ART-RL schedule consistently outperforms uniform and EDM schedules ( 16 ) within the official EDM pipeline. It improves Fréchet Inception Distance (FID) across a broad range of sampling time budgets on CIFAR-10, with particularly strong gains at low budgets. Moreover, under the scheduled timesteps for CIFAR-10, the improvement transfers without retraining to other datasets such as AFHQv2, FFHQ, and ImageNet. To our best knowledge, this is the first paper that develops a principled approach to scheduling timesteps for generative diffusion sampling. The proposed ART-RL method is purely data-driven, and shows robust performance and generalizations empirically. Relevant literature : Diffusion models as generative tools were first proposed by 7 (DDPM) and 29 (DDIM) in the discrete setting. The pioneering work of 31 introduces a continuous-time formulation of diffusion models, providing a unified treatment that encompasses earlier discrete-time models. Various sampling methods have been proposed for diffusion inference, including predictor-corrector sampler ( 31 ) , exponential integrator ( 37 ) , and higher-order solvers ( 38 ; 36 ) . See also 20 ; 2 ; 3 ; 21 ; 1 ; 22 ; 8 for convergence analysis of diffusion models (with either uniform or hand-crafted timesteps). Continuous-time reinforcement learning (CTRL) was first formulated by 35 , where exploration is modeled via stochastic relaxed control to capture the trial-and-error nature of reinforcement learning. Subsequent work developed a model-free theoretical foundation via martingale methods ( 13 ; 14 ; 15 ; 34 ) , established performance guarantees ( 9 ; 10 ) , and studied policy optimization ( 41 ) . The CTRL framework has also been applied to training and fine-tuning diffusion models in generative AI ( 5 ; 39 ; 40 ) . Organization : Section 2 reviews diffusion models and introduces the ART control formulation. Section 3 describes how CTRL is used to solve ART, leading to the ART-RL approach. Section 4 presents the ART-RL algorithm, followed by empirical results in Section 5 . Section 6 concludes. The proofs and additional numerical results are placed in the appendix. 2 Diffusion Models and ART 2.1 Continuous-Time Score-Based Diffusion Models We briefly review continuous-time score-based diffusion models; see 33 for a recent survey. Given samples from an unknown target distribution, a forward process corrupts data over time \tau\in[0,T] toward a tractable reference distribution, and sampling proceeds by integrating a learned reverse-time process. Diffusion sampling. A forward process obeys the Itô SDE \mathrm{d}\,\bar{x}(\tau)=-\,f\bigl(\tau\bigr)\bar{x}(\tau)\,\mathrm{d}\tau+g(\tau)\,\mathrm{d}w(\tau), (1) where \tau\in[0,T] , \bar{x}(0)\sim p_{0} is the target data distribution, w is a standard Wiener process in \mathbb{R}^{d} , and f and g are coefficient functions. Let p_{\tau} be the law of \bar{x}(\tau) and S(\tau,x)=\nabla_{x}\log p_{\tau}(x) be the score. For sampling, we use the probability flow ODE associated with ( 1 ), which shares the same marginals as the reverse-time SDE; see e.g. 33 . Denote the backward state by \tilde{x}(\tau)\coloneqq\bar{x}(T-\tau) with initialization \tilde{x}(0)\sim p_{T} . Replacing S with a trained score model \hat{S} yields \frac{\mathrm{d}\tilde{x}(\tau)}{\mathrm{d}\tau}=f(T-\tau)\tilde{x}(\tau)+\tfrac{1}{2}g(T-\tau)^{2}\hat{S}(T-\tau,\tilde{x}(\tau)), (2) where \tau\in[0,T] . Euler discretization. We integrate ( 2 ) on a grid 0=\tau_{0}<\tau_{1}<\cdots<\tau_{K}=T with step sizes h_{i}=\tau_{i+1}-\tau_{i} , and write \tilde{x}_{i}\coloneqq\tilde{x}(\tau_{i}) . The explicit Euler update is \tilde{x}_{i+1}=\tilde{x}_{i}+h_{i}\left[f(T-\tau_{i})\tilde{x}_{i}+\tfrac{1}{2}g(T-\tau_{i})^{2}\hat{S}(T-\tau_{i},\tilde{x}_{i})\right]. (3) A uniform grid \tau_{i}=iT/K is simple but allocates evaluations uniformly even when the reverse dynamics vary along the trajectory. A simple intuition is that early stages, where samples resemble noise, may tolerate coarser steps, while later stages will benefit from finer resolutions. Such considerations motivate adaptive, data-driven schedules that redistribute steps under a given, fixed total time budget T . 2.2 ART: Time Reparameterization via Control Motivated by the drawback of uniform timesteps in backward sampling discussed above, we introduce a reparameterized sampling clock and formalize adaptive timestep selection in the reverse process as a control problem. The key idea is to replace the uniform progression of original time by an adaptive, controlled progression that can accelerate in some segments and decelerate in others, thereby redistributing computational effort along the trajectory. To this end, let \psi:[0,T]\to\mathbb{R} be a continuous time mapping from the new clock t to the original diffusion time \tau (i.e., \tau=\psi(t) ), with \psi(0)=0 and \psi(T)=T . On this new

Read the complete text