Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise

Gaku Omiya, Pierre-Louis Poirion, Akiko Takeda · 2026

Read Convergence Analysis of Randomized Subspace Normalized SGD under Heavy-Tailed Noise by Gaku Omiya, Pierre-Louis Poirion, Akiko Takeda (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20399.txt.

Description

Gaku Omiya, Pierre-Louis Poirion, and Akiko Takeda analyze a randomized subspace method for stochastic optimization under heavy-tailed noise. Subspace methods cut the cost of each step by working in a lower-dimensional slice of the problem. Most theory for the nonconvex case gives bounds in expectation, and high-probability bounds are scarce even under mild noise.

The authors first prove a high-probability convergence bound under sub-Gaussian noise, matching the known order of cost. They then extend the study toward heavy tails. Guarantees that hold with high probability, not just on average, matter when one bad run is costly.

Catalog record

CREATORGaku Omiya, Pierre-Louis Poirion, Akiko Takeda
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE107 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20399

Browse the full library · About the Tower of Babel