Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks

Ali Hussaini UmarSISSATrieste, Italy [email protected], Alessandro LaioSISSA & \& ICTPTrieste, Italy [email protected] · 2026

Read Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks by Ali Hussaini UmarSISSATrieste, Italy [email protected], Alessandro LaioSISSA & \& ICTPTrieste, Italy [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26973.txt.

Description

Umar and Laio study why neural networks trained in different ways end up with similar inner representations. Training nets on regression and classification tasks with noisy datasets, they find signal-to-noise ratio and sample size shape alignment in similar ways. The pattern holds for real nets and for a simple one-layer linear net. Alignment rises steadily with signal-to-noise ratio but dips near the interpolation threshold as sample size grows. Stronger alignment does not always mean better generalization.

Catalog record

CREATORAli Hussaini UmarSISSATrieste, Italy [email protected], Alessandro LaioSISSA & \& ICTPTrieste, Italy [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57658
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26973

Browse the full library · About the Tower of Babel