The Role of Symmetry in Optimizing Overparameterized Networks

Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba, Mehran Shakerinava, Siamak Ravanbakhsh · 2026

Read The Role of Symmetry in Optimizing Overparameterized Networks by Kusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba, Mehran Shakerinava, Siamak Ravanbakhsh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25150.txt.

Description

Sareen and four colleagues study how symmetry helps optimize networks with far more parameters than needed. Extra parameters add weight-space symmetries that help in two ways. First, they precondition the Hessian diagonally, so each class of functionally identical solutions holds a better-conditioned minimum. Second, they raise the share of global minima near typical starting points, so good solutions are easier to reach. Tests confirm wider networks converge faster with smaller condition numbers.

Catalog record

CREATORKusha Sareen, Mohammad Pedramfar, Sékou-Oumar Kaba, Mehran Shakerinava, Siamak Ravanbakhsh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85347
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.25150

Browse the full library · About the Tower of Babel