Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function
Maria-Florina Balcan, Anh Tuan Nguyen, Dravyansh Sharma · 2025
Read Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function by Maria-Florina Balcan, Anh Tuan Nguyen, Dravyansh Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13734.txt.
Description
Maria-Florina Balcan and two co-authors open in 2025 the formal study of how much data hyperparameter tuning for neural networks really needs. Tuning is the slow, costly ritual behind most deep learning wins, yet its learning-theoretic cost was uncharted. Their setting assumes a stream of related tasks and asks for settings that do well on average across them.
The catch is that performance as a function of the settings jumps and wiggles wildly, since it hides an inner training problem. New analysis tools from geometry tame those discontinuities and bound the needed sample size. First bounds of this kind give AutoML a firmer footing and hint at which task families are cheap to tune for.
Catalog record
| CREATOR | Maria-Florina Balcan, Anh Tuan Nguyen, Dravyansh Sharma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 194 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-13734