Non-Euclidean Gradient Descent Operates at the Edge of Stability

Rustem Islamov, Michael Crawshaw, Jeremy Cohen, Robert Gower · 2026

Read Non-Euclidean Gradient Descent Operates at the Edge of Stability by Rustem Islamov, Michael Crawshaw, Jeremy Cohen, Robert Gower (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05002.txt.

Description

Rustem Islamov and team interpret the Edge of Balance through Directional Smoothness, stretching it to non-Euclidean norms. Sharpness, the Hessian's largest eigenvalue, ways and hovers near the balance threshold during gradient descent, apparently violating classical smoothness assumptions. Their carried over sharpness measure covers vanilla and preconditioned gradient descent as special cases, plus before unstudied methods such as l-infinity descent, Block CD, and Spectral GD. Tests on neural networks show non-Euclidean gradient descent also exhibits progressive sharpening followed by swing around or above the threshold.

Catalog record

CREATORRustem Islamov, Michael Crawshaw, Jeremy Cohen, Robert Gower
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87862
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05002

Browse the full library · About the Tower of Babel