Revisiting Generalization Measures Beyond IID: An Empirical Study under Distributional Shift

Sora Nakai, Youssef Fadhloun, Kacem Mathlouthi, Kotaro Yoshida, Ganesh Talluri, Ioannis Mitliagkas, Hiroki Naganuma · 2026

Read Revisiting Generalization Measures Beyond IID: An Empirical Study under Distributional Shift by Sora Nakai, Youssef Fadhloun, Kacem Mathlouthi, Kotaro Yoshida, Ganesh Talluri, Ioannis Mitliagkas, Hiroki Naganuma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01718.txt.

Description

Sora Nakai and colleagues revisit generalization measures for models under corruptions and perturbations beyond the identically distributed settings of benchmarks. On CIFAR-10-C/P, where labels stay fixed while inputs degrade, they test whether measures rank models well. Value proves regime tied: sharpness and input gradient measures lead among signals, while family results stay close and architecture tied. Tuning measures, info criteria and sharpness measures add regime tied signals. Model choice should not rest on IID favored measures, but should treat measures as signals for the corruption or shift setting.

Catalog record

CREATORSora Nakai, Youssef Fadhloun, Kacem Mathlouthi, Kotaro Yoshida, Ganesh Talluri, Ioannis Mitliagkas, Hiroki Naganuma
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69114
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01718

Browse the full library · About the Tower of Babel