A solvable model of learning generative diffusion: theory and insights
Hugo Cui, Cengiz Pehlevan, Yue M Lu · 2025
Read A solvable model of learning generative diffusion: theory and insights by Hugo Cui, Cengiz Pehlevan, Yue M Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03937.txt.
Description
Hugo Cui, Cengiz Pehlevan, and Yue Lu build a model of image-generating AI that can be solved exactly, so its learning can be understood rather than guessed. The model is a small two-layer system trained step by step on data that secretly lies on a low-dimensional surface. The authors derive tight formulas for what the trained model produces.
Their formulas show how the quality of the samples depends on how much training data was used. They also explain how mode collapse arises, when the model drops whole kinds of images. Collapse can deepen when a new model is trained on the old model's own output. The work gives theory a firm case to test claims about generative models.
Catalog record
| CREATOR | Hugo Cui, Cengiz Pehlevan, Yue M Lu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 133 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-03937