Takeuchi’s Information Criteria as a Generalization Measure for Deep Neural Networks Close to the NTK Regime
Hiroki Naganuma, Taiji Suzuki, Rio Yokota, Masahiro Nomura, Kohta Ishikawa, Ikuro Sato, [1em] Department of Computer Science, Operations Research, Université de Montréal, Canada · 2026
Read Takeuchi’s Information Criteria as a Generalization Measure for Deep Neural Networks Close to the NTK Regime by Hiroki Naganuma, Taiji Suzuki, Rio Yokota, Masahiro Nomura, Kohta Ishikawa, Ikuro Sato, [1em] Department of Computer Science, Operations Research, Université de Montréal, Canada (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23219.txt.
Description
Hiroki Naganuma and five co-authors test whether a classical statistics tool, Takeuchi's information criterion, can measure how well deep neural networks generalize. Reliable generalization measures are hard to build for singular models such as deep networks, whose complex structure resists standard theory. The authors develop theory showing the criterion applies near the neural tangent kernel regime, where wide networks behave almost linearly.
The experiments are large: over 5,000 trained networks across 12 architectures, including VGG-16, on four data sets, with several feasible approximation methods for the criterion. Estimated criterion values track the generalization gap well near the neural tangent kernel regime, but the link vanishes outside it, as theory predicts. The criterion also beats existing methods at pruning trials in hyperparameter search.
Catalog record
| CREATOR | Hiroki Naganuma, Taiji Suzuki, Rio Yokota, Masahiro Nomura, Kohta Ishikawa, Ikuro Sato, [1em] Department of Computer Science, Operations Research, Université de Montréal, Canada |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-23219