Error Propagation and Model Collapse in Diffusion Models: A Theoretical Study

Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan · 2026

Read Error Propagation and Model Collapse in Diffusion Models: A Theoretical Study by Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16601.txt.

Description

Nail Khelifa, Richard Turner, and Ramji Venkataramanan analyze model collapse in score-based diffusion models trained repeatedly on synthetic data. Such recursive training is known to drift away from the target distribution. The authors study a realistic pipeline where each round mixes synthetic data with fresh samples from the target. They obtain upper and lower bounds on the accumulated divergence between generated and target distributions.

Notably, the lower bound is claimed as the first of its kind even for standard diffusion models. Different drift regimes emerge from score estimation error and the share of fresh data. In one regime, total divergence becomes a discounted sum of per-round score errors. Synthetic and image experiments illustrate the theory, giving practitioners a quantitative guide to how much fresh data retraining needs.

Catalog record

CREATORNaïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE135 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-16601

Browse the full library · About the Tower of Babel