Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
Chen Fan, Mark Schmidt, Christos Thrampoulidis · 2025
Read Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data by Chen Fan, Mark Schmidt, Christos Thrampoulidis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04664.txt.
Introduction
Chen Fan, Mark Schmidt, and Christos Thrampoulidis study the hidden habits of training methods in a 2025 paper. When a model has more knobs than needed, many training methods reach zero error, yet each lands on a different answer with different skill on new data. The difference comes from each method's implicit bias.
The authors give the first full account of that bias for a family of steepest-descent methods used on multi-class problems. They prove the methods converge to the answer with the largest safety margin, measured in the matching norm, and they give convergence speeds. Two popular methods, Spectral Descent and Muon, fall out as special cases.
Catalog record
| CREATOR | Chen Fan, Mark Schmidt, Christos Thrampoulidis |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 177 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-04664