Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
Anvit Garg, Sohom Bhattacharya, Pragya Sur · 2025
Read Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression by Anvit Garg, Sohom Bhattacharya, Pragya Sur (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22341.txt.
Description
Anvit Garg, Sohom Bhattacharya, and Pragya Sur study when training on fake data makes a model worse. This decay, called model collapse, grows as each new model learns from the last one's output. The team works with overparameterized linear regression, where the model has more knobs than data points. Each round mixes fresh real labels with fake labels from the prior fit.
They derive exact error formulas for two fitting methods. The best mix weight can stop collapse for good. For one method, that weight tends to a value tied to the golden ratio. For ridge fits, the best share of real data is at least one half.
Catalog record
| CREATOR | Anvit Garg, Sohom Bhattacharya, Pragya Sur |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22341