Takeuchi’s Information Criteria as a Generalization Measure for Deep Neural Networks Close to the NTK Regime

Hiroki Naganuma, Taiji Suzuki, Rio Yokota, Masahiro Nomura, Kohta Ishikawa, Ikuro Sato, [1em] Department of Computer Science, Operations Research, Université de Montréal, Canada · 2026

Read Takeuchi’s Information Criteria as a Generalization Measure for Deep Neural Networks Close to the NTK Regime by Hiroki Naganuma, Taiji Suzuki, Rio Yokota, Masahiro Nomura, Kohta Ishikawa, Ikuro Sato, [1em] Department of Computer Science, Operations Research, Université de Montréal, Canada (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23219.txt.

Description

Hiroki Naganuma and five co-authors test whether a classical statistics tool, Takeuchi's information criterion, can measure how well deep neural networks generalize. Reliable generalization measures are hard to build for singular models such as deep networks, whose complex structure resists standard theory. The authors develop theory showing the criterion applies near the neural tangent kernel regime, where wide networks behave almost linearly.

The experiments are large: over 5,000 trained networks across 12 architectures, including VGG-16, on four data sets, with several feasible approximation methods for the criterion. Estimated criterion values track the generalization gap well near the neural tangent kernel regime, but the link vanishes outside it, as theory predicts. The criterion also beats existing methods at pruning trials in hyperparameter search.

Catalog record

CREATORHiroki Naganuma, Taiji Suzuki, Rio Yokota, Masahiro Nomura, Kohta Ishikawa, Ikuro Sato, [1em] Department of Computer Science, Operations Research, Université de Montréal, Canada
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-23219

Browse the full library · About the Tower of Babel