Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function

Maria-Florina Balcan, Anh Tuan Nguyen, Dravyansh Sharma · 2025

Read Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function by Maria-Florina Balcan, Anh Tuan Nguyen, Dravyansh Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13734.txt.

Description

Maria-Florina Balcan and two co-authors open in 2025 the formal study of how much data hyperparameter tuning for neural networks really needs. Tuning is the slow, costly ritual behind most deep learning wins, yet its learning-theoretic cost was uncharted. Their setting assumes a stream of related tasks and asks for settings that do well on average across them.

The catch is that performance as a function of the settings jumps and wiggles wildly, since it hides an inner training problem. New analysis tools from geometry tame those discontinuities and bound the needed sample size. First bounds of this kind give AutoML a firmer footing and hint at which task families are cheap to tune for.

Catalog record

CREATORMaria-Florina Balcan, Anh Tuan Nguyen, Dravyansh Sharma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE194 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13734

Browse the full library · About the Tower of Babel