A theory of learning data statistics in diffusion models, from easy to hard
Lorenzo Bardone, Claudia Merger, Sebastian Goldt · 2026
Read A theory of learning data statistics in diffusion models, from easy to hard by Lorenzo Bardone, Claudia Merger, Sebastian Goldt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12901.txt.
Description
Lorenzo Bardone and co-authors explain the order in which diffusion models learn data statistics, from easy to hard. Tests on natural images show a simplicity bias: pair-wise statistics come first. Higher-order correlations are learned only later. A minimal data model with controlled correlations reproduces that pattern. The authors define a scalar invariant that sets sample complexity for each level. Pair-wise traits need linear sample counts, while a fourth cumulant needs at least cubic. Shared latent structure can cut that cost back to linear.
Catalog record
| CREATOR | Lorenzo Bardone, Claudia Merger, Sebastian Goldt |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69522 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.12901