Shortcut Features as Top Eigenfunctions of NTK: A Linear Neural Network Case and More

Jinwoo LimSeoul National University [email protected] Suhyun KimKyung Hee University [email protected], Soo-Mook MoonSeoul National University [email protected] · 2026

Read Shortcut Features as Top Eigenfunctions of NTK: A Linear Neural Network Case and More by Jinwoo LimSeoul National University [email protected] Suhyun KimKyung Hee University [email protected], Soo-Mook MoonSeoul National University [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03066.txt.

Description

Why neural networks latch onto easy shortcuts is analyzed by Jinwoo Lim and 2 co-authors in a 2026 paper. When most training data is dominated by one feature, networks learn that feature even if it will not generalize. The authors study this shortcut learning through the Neural Tangent Kernel, a framework describing network training in function space.

They define a feature as an eigenfunction of that kernel. Shortcut features then correspond to eigenfunctions with larger eigenvalues, when shortcuts come from imbalanced sample counts across clusters. Those features keep influencing outputs even after training, due to variance within clusters. Controlling the output margin does not remove the preference. Experiments extend the linear-network findings to larger networks.

Catalog record

CREATORJinwoo LimSeoul National University [email protected] Suhyun KimKyung Hee University [email protected], Soo-Mook MoonSeoul National University [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03066

Browse the full library · About the Tower of Babel