FedMuon: Accelerating Federated Learning with Matrix Orthogonalization
Junkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, Jin Liu · 2025
Read FedMuon: Accelerating Federated Learning with Matrix Orthogonalization by Junkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, Jin Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27403.txt.
Description
Junkang Liu, Fanhua Shang, and four other co-authors speed up learning spread across many devices in this 2025 paper. In federated learning, phones or sites train a shared model locally and swap updates, not raw data. Fewer swap rounds mean faster learning. Most local training still uses simple per-number optimizers that ignore matrix shape.
Their method puts the Muon optimizer on each device. Muon reshapes updates using matrix orthogonalization, a step that keeps update directions clean and balanced. With alike data, it cut rounds sharply versus standard local methods. With unlike data, though, each device's reshaping pulled the model apart. Their fix, FedMuon, shares momentum, the running push from past updates, so devices start each round aligned.
Catalog record
| CREATOR | Junkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, Jin Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-27403