Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning

Binghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, Xiaomin Li · 2026

Read Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning by Binghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, Xiaomin Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08949.txt.

Description

Binghang Lu and eight coauthors fight the loss of old skills as models learn task on task. Guard plans steer new steps clear of old task paths. They tend to gauge all with the plain Frobenius gauge. The Muon plan's wins hint at a next gauge. Muon uses square, set right steps with a spectral air. That air may fit matrix weights more.

Muon-OGD casts each step as a task with a spectral cap and line guards. Dual loops and matrix sign near forms solve it fast. On stock suites, and on code, math, and care sets, it beats plain tuned runs and strong guard base lines. It stays lean at scale.

Catalog record

CREATORBinghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, Xiaomin Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08949

Browse the full library · About the Tower of Babel