The Role of Symmetry in Optimizing Overparameterized Networks
Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba, Mehran Shakerinava, Siamak Ravanbakhsh · 2026
Read The Role of Symmetry in Optimizing Overparameterized Networks by Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba, Mehran Shakerinava, Siamak Ravanbakhsh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25150.txt.
Description
Sareen and four colleagues study how symmetry helps optimize networks with far more parameters than needed. Extra parameters add weight-space symmetries that help in two ways. First, they precondition the Hessian diagonally, so each class of functionally identical solutions holds a better-conditioned minimum. Second, they raise the share of global minima near typical starting points, so good solutions are easier to reach. Tests confirm wider networks converge faster with smaller condition numbers.
Catalog record
| CREATOR | Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba, Mehran Shakerinava, Siamak Ravanbakhsh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85347 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25150