Stochastic Gradient Descent with Momentum is Algorithmically Stable
Yunwen Lei, Zimeng Wang, Xiaoming Yuan · 2026
Read Stochastic Gradient Descent with Momentum is Algorithmically Stable by Yunwen Lei, Zimeng Wang, Xiaoming Yuan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28517.txt.
Description
Yunwen Lei, Zimeng Wang, and Xiaoming Yuan analyze whether stochastic gradient descent with momentum generalizes to unseen data. Momentum speeds training, but it was conjectured to harm generalization. They build a generalized framework covering Polyak and Nesterov momentum and prove tight on-average stability bounds for smooth convex problems. The bounds exploit small optimization errors along the path, cover any momentum below one, and need no Lipschitz assumption, yielding optimal excess population risk bounds for both schemes.
Catalog record
| CREATOR | Yunwen Lei, Zimeng Wang, Xiaoming Yuan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 99751 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.28517