Any-stepsize Gradient Descent for Separable Data under Fenchel–Young Losses
Han Bao, Shinsaku Sakaue, Yuki Takezawa · 2025
Read Any-stepsize Gradient Descent for Separable Data under Fenchel–Young Losses by Han Bao, Shinsaku Sakaue, Yuki Takezawa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04889.txt.
Description
Han Bao, Shinsaku Sakaue, and Yuki Takezawa prove that gradient descent can converge with any step size for a broad family of losses. Their 2025 paper studies training on data that a linear model can separate cleanly. Standard theory demands small steps, yet practice shows training often rides at the edge of stability and still works.
Earlier work proved any-step convergence for logistic loss, leaning on a special property that loss has. This team generalize through a wider loss family built from entropy-like functions. Borrowing the classic perceptron argument, they derive convergence rates for most losses in the family. Two named entropies give especially fast rates. The result widens the theory of why big steps do not break training.
Catalog record
| CREATOR | Han Bao, Shinsaku Sakaue, Yuki Takezawa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 115 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-04889