Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
Binghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, Xiaomin Li · 2026
Read Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning by Binghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, Xiaomin Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08949.txt.
Description
Binghang Lu and eight coauthors fight the loss of old skills as models learn task on task. Guard plans steer new steps clear of old task paths. They tend to gauge all with the plain Frobenius gauge. The Muon plan's wins hint at a next gauge. Muon uses square, set right steps with a spectral air. That air may fit matrix weights more.
Muon-OGD casts each step as a task with a spectral cap and line guards. Dual loops and matrix sign near forms solve it fast. On stock suites, and on code, math, and care sets, it beats plain tuned runs and strong guard base lines. It stays lean at scale.
Catalog record
| CREATOR | Binghang Lu, Zheyuan Deng, Runyu Zhang, Bing Hu, Yunhan Zhao, Yuan Tian, Changhong Mou, Guang Lin, Xiaomin Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08949