Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise
Gaku Omiya, Pierre-Louis Poirion, Akiko Takeda · 2026
Read Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise by Gaku Omiya, Pierre-Louis Poirion, Akiko Takeda (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20399.txt.
Description
Gaku Omiya, Pierre-Louis Poirion, and Akiko Takeda analyze a randomized subspace method for stochastic optimization under heavy-tailed noise. Subspace methods cut the cost of each step by working in a lower-dimensional slice of the problem. Most theory for the nonconvex case gives bounds in expectation, and high-probability bounds are scarce even under mild noise.
The authors first prove a high-probability convergence bound under sub-Gaussian noise, matching the known order of cost. They then extend the study toward heavy tails. Guarantees that hold with high probability, not just on average, matter when one bad run is costly.
Catalog record
| CREATOR | Gaku Omiya, Pierre-Louis Poirion, Akiko Takeda |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 107 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20399