Antithetic Noise in Diffusion Models
Jing Jia, Sifan Liu, Bowen Song, Wei Yuan, Liyue Shen, Guanyang Wang · 2025
Read Antithetic Noise in Diffusion Models by Jing Jia, Sifan Liu, Bowen Song, Wei Yuan, Liyue Shen, Guanyang Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06185.txt.
Introduction
Jing Jia and co-authors find a free accuracy boost for image generators in a 2025 paper. Diffusion models start from random noise, and the team shows that pairing each noise sample with its exact opposite produces outputs that are strongly negatively linked. The effect held across data sets, model designs, guided and unguided sampling, and even other generator families.
To explain it, they propose that the model's learned score function is nearly antisymmetric, a mirror symmetry with a shift, and experiments back the guess. The practical payoff is trust: averaging such pairs shrinks confidence intervals by up to ninety percent, sharpening pixel estimates and inverse problem solvers. Extra designs improve editing and diversity, all without retraining or extra runtime.
Catalog record
| CREATOR | Jing Jia, Sifan Liu, Bowen Song, Wei Yuan, Liyue Shen, Guanyang Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06185
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Antithetic Noise in Diffusion Models Jing Jia, Sifan Liu, Bowen Song, Wei Yuan, Liyue Shen, Guanyang Wang Abstract We initiate a systematic study of antithetic initial noise in diffusion models. Across unconditional models trained on diverse datasets, text-conditioned latent-diffusion models, and diffusion posterior samplers, we find that pairing each initial noise with its negation consistently yields strongly negatively correlated samples. To explain this phenomenon, we combine experiments and theoretical analysis, leading to a symmetry conjecture that the learned score function is approximately affine antisymmetric (odd symmetry up to a constant shift), and provide evidence supporting it. Leveraging this negative correlation, we enable two applications: (1) enhancing image diversity in models like Stable Diffusion without quality loss, and (2) sharpening uncertainty quantification (e.g., up to 90% narrower confidence intervals) when estimating downstream statistics. Building on these gains, we extend the two-point pairing to a randomized quasi-Monte Carlo estimator, which further improves estimation accuracy. Our framework is training-free, model-agnostic, and adds no runtime overhead. * * footnotetext: Co-corresponding authors 1 Introduction Diffusion models have set the state of the art in photorealistic image synthesis, high - fidelity audio, and video generation sohl2015deep ; ho2020denoising ; song2021scorebased ; kong2021diffwave ; they also power applications such as text-to-image generation ldm2022high , image editing and restoration meng2022sdedit , and inverse problem solving song2022solving . For many pretrained diffusion models, sampling relies on three elements: the network weights, the denoising schedule, and the initial Gaussian noise. Once these are fixed, sampling is often deterministic: the sampler transforms the initial noise into an image via successive denoising passes. Much of the literature improves the first two ingredients and clusters into two strands: (i) architectural and training developments, which improve sample quality and scalability through backbone or objective redesign (e.g., EDM ( karras2022elucidating ) , Latent Diffusion Models ( ldm2022high ) , DiT ( peebles2023scalable ) ), and (ii) accelerated sampling, which reduces the number of denoising steps while retaining high-quality generation (e.g., DDIM ( songj2021denoising ) , Consistency Models ( song2023consistency ) , DPM - Solver++ ( lu2023dpmsolver ) , and progressive distillation ( salimans2022progressive ) ). However, the third ingredient—the initial Gaussian noise —has received comparatively little attention. Previous work examines it mostly through task - specific lenses: InitNO guo2024initno separates Gaussian noise seeds into “valid” and “invalid” groups according to their attention-score profiles. Qi et al. qi2024not show that inversion-stable noises reliably boost the quality of generated images. Zhou et al. zhou2024golden propose a learning framework to select high quality noises. The Crystal Ball detector ban2025the pinpoints “trigger patches” as crucial elements in object synthesis. FIND chen2024find optimizes the initial noise distribution so that outputs better match user prompts. CCS song2025ccs exploits an almost linear noise-to-image mapping to achieve controllable generation. Nevertheless, a systematic analysis of how the initial noise itself shapes the final outputs is still missing. Our perspective is orthogonal to prior work. Our first contribution is a simple yet model - agnostic finding : across unconditional, latent text - conditioned, and posterior - sampling diffusion models, pairing every Gaussian noise z with its negation -z —known as antithetic sampling mcbook —consistently produces samples that are strongly negatively correlated . This phenomenon is universal as it holds regardless of architecture, dataset, sampling schedule, and both conditional and unconditional sampling. To illuminate this phenomenon, we conduct explanatory experiments and theoretical analysis, conjecture that the learned score function exhibits approximate affine symmetry, and present initial supporting evidence. (a) Stable Diffusion (b) CelebA-HQ Figure 1: Antithetic noise produces visually “opposite” images. Left: a Stable Diffusion sketch with the prompt “cover, hyperdetailed painting, luminism, Bar lighting, complex, head and shoulders portrait.” Right: images from independent random-noise pairs (top) vs. an antithetic pair (bottom). Leveraging this universal property, we present two concrete applications: (i) Enhanced image diversity. As shown in Figure 1 , negative correlation implies diversity. Because each antithetic noise pair drives the reverse - diffusion trajectories toward distant regions of the data manifold, the paired - sampling scheme markedly increases diversity “for free” compared with independent noise inputs, while still maintaining high image quality. The procedure is fully plug - and - play : one simply feeds the model with antithetic noise vectors. (ii) Sharper uncertainty quantification. Because each antithetic pair generates strongly negatively correlated samples, these outputs form an effective control - variate pair. We apply this idea in an antithetic estimator for downstream statistics, reducing the 95% confidence-interval width by up to 90% and cutting computational effort by about 100 times compared to standard Monte Carlo sampling with independent noise. Building on the variance gains from antithetic pairs, we generalize the idea to apply quasi-Monte Carlo (QMC) and randomized QMC (RQMC). The resulting RQMC estimator often delivers further variance reduction. Although QMC has been widely used in computer graphics keller1995quasi ; waechter2011quasi , quantitative finance joy1996quasi ; l2009quasi , and Bayesian inference buchholz2018quasi ; liu2021quasi , this is, to our knowledge, its first application to diffusion models. Together, these results suggest that principled manipulations in the initial-noise space can be a promising avenue for further gains in diffusion model’s efficiency, output diversity, and reliability. The remainder of the paper is organized as follows. Section 2 defines the problem and outlines our motivation. Section 3 presents our central finding that antithetic noise pairs produce strongly negatively correlated outputs. We offer both theoretical and empirical explanations for this phenomenon, and present the symmetry conjecture in Section 3.2 . Section 4 details the construction of point estimates and confidence intervals using antithetic sampling for estimating downstream statistics, illustrates how antithetic sampling tightens confidence bounds, and discusses the generalization to QMC. Section 5 reports experiments on the aforementioned applications. Finally, Section 6 discusses our method’s potentials, limitations, and future directions. Appendix A , B , and C include proofs, additional experiments and detailed setups, and supplemental visualizations, respectively. 2 Setup and motivation Unconditional diffusion model: A diffusion model aims to generate samples from an unknown data distribution p_{0} . It first noises data towards a standard Gaussian by applying progressively stronger Gaussian perturbations, then learns to reverse the process so that pure white noise can be denoised step-by-step back into target samples. The forward process simulates a stochastic differential equation (SDE): \mathrm{d}\mathbf{x}_{t}=\mu(\mathbf{x}_{t},t)\mathrm{d}t+\sigma_{t}\mathrm{d}\mathbf{w}_{t}, where \{\mathbf{w}_{t}\}_{t=0}^{T} denotes the standard Brownion motion, and \mu(\mathbf{x},t),\sigma_{t} are chosen by the users. Let p_{t} denote the distribution of \mathbf{x}_{t} , where \mathbf{x}_{0}\sim p_{0} . A key result in song2021scorebased states that if we sample \mathbf{y}_{T}\sim p_{T} and simulate the probability-flow ordinary differential equation (PF-ODE) backward from time T to 0 as \displaystyle\mathrm{d}\mathbf{y}_{t}\;=\;\left(-\mu(\mathbf{y}_{t},t)-\frac{1}{2}\sigma_{t}^{2}\nabla\log p(\mathbf{y}_{t},t)\right)\mathrm{d}t, (1) then for every time t the marginal distribution of \mathbf{y}_{t} coincides with p_{t} . Thus in an idealized world, one can perfectly sample from p_{0} by simulating the PF-ODE ( 1 ). In practice, the score function \nabla\log p_{t}(\mathbf{x},t) is unavailable, and a neural network \epsilon_{\theta}^{(t)}(\mathbf{x}) is trained to approximate it, where \theta represents its weights. The drift term \mu and diffusion term \sigma are designed such that p_{t} converges to a standard Gaussian as t\rightarrow\infty . Therefore, one can generate new samples from p_{0} by first sampling a Gaussian noise and simulating the PF-ODE ( 1 ) through some numerical integrator from T to 0 . For example, DDIM