Delving into Muon and Beyond: Deep Analysis and Extensions

Xianbiao Qi, Marco Chen, Jiaquan Ye, Yelin He, Rong Xiao · 2026

Read Delving into Muon and Beyond: Deep Analysis and Extensions by Xianbiao Qi, Marco Chen, Jiaquan Ye, Yelin He, Rong Xiao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04669.txt.

Description

Xianbiao Qi and 4 co-authors analyze the Muon optimizer in a 2026 paper on training neural networks. Muon updates matrix-shaped parameters using orthogonalized steps and has won attention for strong results, yet its relation to adaptive methods like Adam was unclear.

The team views Muon as one endpoint of a family of spectral transformations, which reshape an update by its singular values. Variants apply these transformations to momentum updates and to normalized gradient updates. A coupled Newton iteration computes them efficiently without full decomposition. Experiments showed normalized updates were more stable. Muon did not consistently beat Adam. The authors conclude Muon is best seen as effective spectral normalization, not a universal winner.

Catalog record

CREATORXianbiao Qi, Marco Chen, Jiaquan Ye, Yelin He, Rong Xiao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-04669

Browse the full library · About the Tower of Babel