A theory of learning data statistics in diffusion models, from easy to hard

Lorenzo Bardone, Claudia Merger, Sebastian Goldt · 2026

Read A theory of learning data statistics in diffusion models, from easy to hard by Lorenzo Bardone, Claudia Merger, Sebastian Goldt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12901.txt.

Description

Lorenzo Bardone and co-authors explain the order in which diffusion models learn data statistics, from easy to hard. Tests on natural images show a simplicity bias: pair-wise statistics come first. Higher-order correlations are learned only later. A minimal data model with controlled correlations reproduces that pattern. The authors define a scalar invariant that sets sample complexity for each level. Pair-wise traits need linear sample counts, while a fourth cumulant needs at least cubic. Shared latent structure can cut that cost back to linear.

Catalog record

CREATORLorenzo Bardone, Claudia Merger, Sebastian Goldt
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69522
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.12901

Browse the full library · About the Tower of Babel