When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory
Ahmad Aghapour, Erhan Bayraktar · 2026
Read When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory by Ahmad Aghapour, Erhan Bayraktar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07969.txt.
Description
Ahmad Aghapour and Erhan Bayraktar explain why diffusion samplers stay efficient in high dimensions when theory said they should not. Many convergence guarantees charge the discretization error to the ambient dimension, which would make image models need far more steps than they use. Improved results assumed intrinsic geometric structure; this paper offers an information-theoretic alternative.
For Gaussian mixture targets, they prove the discretization error is controlled by the Shannon entropy of the latent mixture component, not by the ambient dimension. Step counts therefore scale with that entropy and only logarithmically with the data's second moment. The analysis extends to discrete targets, where the target's own entropy is the governing quantity. Compact latent structure, widely believed to underlie natural images, is what makes diffusion cheap.
Catalog record
| CREATOR | Ahmad Aghapour, Erhan Bayraktar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07969