On Understanding of the Dynamics of Model Capacity in Continual Learning
Supriyo Chakraborty, Krishnan Raghavan · 2025
Read On Understanding of the Dynamics of Model Capacity in Continual Learning by Supriyo Chakraborty, Krishnan Raghavan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08052.txt.
Description
Supriyo Chakraborty and Krishnan Raghavan model how a learning system's usable capacity drifts as it takes on task after task, in work from twenty twenty-five. Continual learning forces one network to absorb new jobs without erasing old ones. The balance point between keeping and adapting is the core tension.
Their measure, effective model capacity, tracks that balance as training proceeds. A difference equation describes how network, data, and optimizer move it over time. The balance point turns out to move no matter the design or training method. When new task data differ from old, room for new tasks shrinks. Tests ran from small plain networks up to transformer language models, backing the theory.
Catalog record
| CREATOR | Supriyo Chakraborty, Krishnan Raghavan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08052