Dimension-Free Saddle-Point Escape in Muon

Yanlin Long, Yufei Gu, Zeke Xie · 2026

Read Dimension-Free Saddle-Point Escape in Muon by Yanlin Long, Yufei Gu, Zeke Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09331.txt.

Description

Long and two colleagues study how the Muon optimiser leaves saddle points in the extreme high-dimensional terrain of large language model training. There, flat saddle points, not local minima, are the main block. Step-wise adaptive optimisers like AdamW are trapped by an O(D) dimensional curse, the authors argue, while Muon resists it through non-linear spectral shaping.

Extending matrix perturbation theory, they build a framework for Muon's non-equilibrium optimisation paths. Resolvent calculus and large-scale Cauchy contour math sidestep isotropic noise assumptions and Tracy-Widom edge spikes. Structural incoherence shields the path from side drift, giving a dimension-free saddle-point escape with a set O(1) discrete ballistic ejection under a wide enough spectral gap. The paper closes with a dimension-free escape bound for Muon, stated in algebra, spelling out the mechanics of its non-convex optimisation dynamics.

Catalog record

CREATORYanlin Long, Yufei Gu, Zeke Xie
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE125 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09331

Browse the full library · About the Tower of Babel