The Symmetries of Three-Layer ReLU Networks

Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar · 2026

Read The Symmetries of Three-Layer ReLU Networks by Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18319.txt.

Description

Johanna Marie Gegenfurtner, Moritz Grillo, and Guido Montúfar sort out which different weight settings of a deep ReLU network actually do the same thing. For three layer bottleneck nets, they pin down every typical family of equivalent settings in full. Their account writes each family out in explicit semi algebraic form and turns the sameness question into a test that runs in polynomial time. Some symmetries create local conserved quantities along gradient flow, while the rest leave no such law behind.

Catalog record

CREATORJohanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE138564
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18319

Browse the full library · About the Tower of Babel