Error Propagation and Model Collapse in Diffusion Models: A Theoretical Study
Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan · 2026
Read Error Propagation and Model Collapse in Diffusion Models: A Theoretical Study by Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16601.txt.
Description
Nail Khelifa, Richard Turner, and Ramji Venkataramanan analyze model collapse in score-based diffusion models trained repeatedly on synthetic data. Such recursive training is known to drift away from the target distribution. The authors study a realistic pipeline where each round mixes synthetic data with fresh samples from the target. They obtain upper and lower bounds on the accumulated divergence between generated and target distributions.
Notably, the lower bound is claimed as the first of its kind even for standard diffusion models. Different drift regimes emerge from score estimation error and the share of fresh data. In one regime, total divergence becomes a discounted sum of per-round score errors. Synthetic and image experiments illustrate the theory, giving practitioners a quantitative guide to how much fresh data retraining needs.
Catalog record
| CREATOR | Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 135 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-16601