Non-identifiability distinguishes Neural Networks among Parametric Models

Sourav Chatterjee, Timothy Sudijono · 2025

Read Non-identifiability distinguishes Neural Networks among Parametric Models by Sourav Chatterjee, Timothy Sudijono (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18017.txt.

Description

Sourav Chatterjee and Timothy Sudijono prove, in a 2025 paper, two results that set neural networks apart from smooth classical models in regression. The dividing line is identifiability: whether different parameter settings can produce the same predictions. Networks famously fail it, with many weight sets giving one function.

Their first result says networks always learn a real relationship between input and outcome when one exists, never collapsing to the plain average. The second says any reasonable smooth model that is identifiable, locally and globally, can be forced into exactly that collapse by some honest data pattern. Read together, the theorems flip a common complaint: the networks' non-identifiability is not a bug but the source of their learning guarantee.

Catalog record

CREATORSourav Chatterjee, Timothy Sudijono
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-18017

Browse the full library · About the Tower of Babel