A solvable model of learning generative diffusion: theory and insights

Hugo Cui, Cengiz Pehlevan, Yue M Lu · 2025

Read A solvable model of learning generative diffusion: theory and insights by Hugo Cui, Cengiz Pehlevan, Yue M Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03937.txt.

Description

Hugo Cui, Cengiz Pehlevan, and Yue Lu build a model of image-generating AI that can be solved exactly, so its learning can be understood rather than guessed. The model is a small two-layer system trained step by step on data that secretly lies on a low-dimensional surface. The authors derive tight formulas for what the trained model produces.

Their formulas show how the quality of the samples depends on how much training data was used. They also explain how mode collapse arises, when the model drops whole kinds of images. Collapse can deepen when a new model is trained on the old model's own output. The work gives theory a firm case to test claims about generative models.

Catalog record

CREATORHugo Cui, Cengiz Pehlevan, Yue M Lu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE133 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-03937

Browse the full library · About the Tower of Babel