Step-Size Stability in Stochastic Optimization: A Theoretical Perspective
Fabian Schaipp, Robert M. Gower, Adrien Taylor · 2026
Read Step-Size Stability in Stochastic Optimization: A Theoretical Perspective by Fabian Schaipp, Robert M. Gower, Adrien Taylor (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09842.txt.
Description
Fabian Schaipp and two co-authors present a theory analysis of random tuning methods in terms of step-size feel in their 2026 paper. A key number is found that, for each method, tells how scores fall as the step size grows too large. For convex problems, this number straight impacts the gap bound of the method. Most key, the analysis gives straight theory evidence that tuned step-size methods, such as SPS or NGN, are more robust than random gradient descent.
The edge of tuned methods is thus sized beyond observed tests. Tests show the theory bound mirrors actual scores in shape as a function of step size, even for non-convex problems. Step-size staying power gains a theory footing: a single number captures fall, favors tuned methods on convex problems, and tracks real ways beyond them.
Catalog record
| CREATOR | Fabian Schaipp, Robert M. Gower, Adrien Taylor |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09842