Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data

Chen Fan, Mark Schmidt, Christos Thrampoulidis · 2025

Read Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data by Chen Fan, Mark Schmidt, Christos Thrampoulidis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04664.txt.

Introduction

Chen Fan, Mark Schmidt, and Christos Thrampoulidis study the hidden habits of training methods in a 2025 paper. When a model has more knobs than needed, many training methods reach zero error, yet each lands on a different answer with different skill on new data. The difference comes from each method's implicit bias.

The authors give the first full account of that bias for a family of steepest-descent methods used on multi-class problems. They prove the methods converge to the answer with the largest safety margin, measured in the matching norm, and they give convergence speeds. Two popular methods, Spectral Descent and Muon, fall out as special cases.

Catalog record

CREATORChen Fan, Mark Schmidt, Christos Thrampoulidis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE177 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-04664

Browse the full library · About the Tower of Babel