Optimal Rates in Continual Linear Regression via Increasing Regularization
Ran Levinstein, Amit Attia, Matan Schliserman, Uri Sherman, Tomer Koren, Daniel Soudry, Itay Evron · 2025
Read Optimal Rates in Continual Linear Regression via Increasing Regularization by Ran Levinstein, Amit Attia, Matan Schliserman, Uri Sherman, Tomer Koren, Daniel Soudry, Itay Evron (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06501.txt.
Description
Ran Levinstein and co-authors close a gap in the theory of learning that never stops, in a 2025 paper. Their setting is continual linear regression: tasks arrive in random order, and the learner must keep old skills while gaining new ones. The best possible error after many rounds falls like one over the round count, yet plain methods had only reached a far weaker rate.
Two common anti-forgetting tools do the job, they show: a steady pull toward zero, and a fixed budget of steps per task. Read the right way, both act like gradient descent on a decoy loss. A strength that grows over time reaches the optimal rate exactly. The moral: tighten the leash as tasks pile up.
Catalog record
| CREATOR | Ran Levinstein, Amit Attia, Matan Schliserman, Uri Sherman, Tomer Koren, Daniel Soudry, Itay Evron |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06501