Dimension-Free Saddle-Point Escape in Muon
Yanlin Long, Yufei Gu, Zeke Xie · 2026
Read Dimension-Free Saddle-Point Escape in Muon by Yanlin Long, Yufei Gu, Zeke Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09331.txt.
Description
Long and two colleagues study how the Muon optimiser leaves saddle points in the extreme high-dimensional terrain of large language model training. There, flat saddle points, not local minima, are the main block. Step-wise adaptive optimisers like AdamW are trapped by an O(D) dimensional curse, the authors argue, while Muon resists it through non-linear spectral shaping.
Extending matrix perturbation theory, they build a framework for Muon's non-equilibrium optimisation paths. Resolvent calculus and large-scale Cauchy contour math sidestep isotropic noise assumptions and Tracy-Widom edge spikes. Structural incoherence shields the path from side drift, giving a dimension-free saddle-point escape with a set O(1) discrete ballistic ejection under a wide enough spectral gap. The paper closes with a dimension-free escape bound for Muon, stated in algebra, spelling out the mechanics of its non-convex optimisation dynamics.
Catalog record
| CREATOR | Yanlin Long, Yufei Gu, Zeke Xie |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 125 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.09331