FedMuon: Accelerating Federated Learning with Matrix Orthogonalization

Junkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, Jin Liu · 2025

Read FedMuon: Accelerating Federated Learning with Matrix Orthogonalization by Junkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, Jin Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27403.txt.

Description

Junkang Liu, Fanhua Shang, and four other co-authors speed up learning spread across many devices in this 2025 paper. In federated learning, phones or sites train a shared model locally and swap updates, not raw data. Fewer swap rounds mean faster learning. Most local training still uses simple per-number optimizers that ignore matrix shape.

Their method puts the Muon optimizer on each device. Muon reshapes updates using matrix orthogonalization, a step that keeps update directions clean and balanced. With alike data, it cut rounds sharply versus standard local methods. With unlike data, though, each device's reshaping pulled the model apart. Their fix, FedMuon, shares momentum, the running push from past updates, so devices start each round aligned.

Catalog record

CREATORJunkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, Jin Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27403

Browse the full library · About the Tower of Babel