Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence
Bingji Yi, Qiyuan Liu, Yuwei Cheng, Haifeng Xu · 2025
Read Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence by Bingji Yi, Qiyuan Liu, Yuwei Cheng, Haifeng Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16657.txt.
Description
Bingji Yi and three co-authors ask whether checking synthetic data can stop model collapse. Collapse is the steady decline seen when a generative model is retrained again and again on its own output. Their proposal adds an outside verifier, which may be a person or a stronger model, to judge the synthetic data before retraining.
The analysis uses linear regression as a test setting. It shows that guided retraining can improve the model for a while, but the estimate drifts toward what the verifier already knows. Unless the verifier is perfectly reliable, gains level off and may reverse. Experiments in several settings matched that prediction, giving the claim both theory and practice behind it.
Catalog record
| CREATOR | Bingji Yi, Qiyuan Liu, Yuwei Cheng, Haifeng Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-16657