Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks
Ali Hussaini UmarSISSATrieste, Italy [email protected], Alessandro LaioSISSA & \& ICTPTrieste, Italy [email protected] · 2026
Read Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks by Ali Hussaini UmarSISSATrieste, Italy [email protected], Alessandro LaioSISSA & \& ICTPTrieste, Italy [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26973.txt.
Description
Umar and Laio study why neural networks trained in different ways end up with similar inner representations. Training nets on regression and classification tasks with noisy datasets, they find signal-to-noise ratio and sample size shape alignment in similar ways. The pattern holds for real nets and for a simple one-layer linear net. Alignment rises steadily with signal-to-noise ratio but dips near the interpolation threshold as sample size grows. Stronger alignment does not always mean better generalization.
Catalog record
| CREATOR | Ali Hussaini UmarSISSATrieste, Italy [email protected], Alessandro LaioSISSA & \& ICTPTrieste, Italy [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57658 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.26973