Stochastic Gradient Descent with Momentum is Algorithmically Stable

Yunwen Lei, Zimeng Wang, Xiaoming Yuan · 2026

Read Stochastic Gradient Descent with Momentum is Algorithmically Stable by Yunwen Lei, Zimeng Wang, Xiaoming Yuan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28517.txt.

Description

Yunwen Lei, Zimeng Wang, and Xiaoming Yuan analyze whether stochastic gradient descent with momentum generalizes to unseen data. Momentum speeds training, but it was conjectured to harm generalization. They build a generalized framework covering Polyak and Nesterov momentum and prove tight on-average stability bounds for smooth convex problems. The bounds exploit small optimization errors along the path, cover any momentum below one, and need no Lipschitz assumption, yielding optimal excess population risk bounds for both schemes.

Catalog record

CREATORYunwen Lei, Zimeng Wang, Xiaoming Yuan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE99751
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28517

Browse the full library · About the Tower of Babel