Most ReLU Networks Admit Identifiable Parameters
Moritz Grillo, Guido Montúfar · 2026
Read Most ReLU Networks Admit Identifiable Parameters by Moritz Grillo, Guido Montúfar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03601.txt.
Introduction
Moritz Grillo and Guido Montúfar ask when the parameters of a deep ReLU neural network are determined by the function the network computes. Two networks can represent the same function with different weights, beyond the obvious rescalings and permutations of hidden units. The authors study the realization map that sends parameters to functions.
Using weighted polyhedral complexes, they prove that for every architecture whose input and hidden layers have width at least two, there is an open set of identifiable parameters. Hence the functional dimension equals the parameter count minus the hidden neuron count. They also show minimal representations can keep nontrivial redundancies, and they establish a generic depth hierarchy against shallower networks.
Catalog record
| CREATOR | Moritz Grillo, Guido Montúfar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 168 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03601