From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learning

Hossein Zakerinia, Dorsa Ghobadi, Christoph H. Lampert · 2025

Read From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learning by Hossein Zakerinia, Dorsa Ghobadi, Christoph H. Lampert (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19067.txt.

Description

Hossein Zakerinia, Dorsa Ghobadi, and Christoph H Lampert present a 2025 paper on why deep networks that learn several tasks at once still generalize, despite having far more knobs than data points. One leading account says the real learning happens in a small hidden subspace of the full weight space. The authors test that account in multi-task learning.

They build the network directly inside the small subspace, using random expansion tricks to do so. High-accuracy shared solutions then appear with far fewer free parameters than single-task learning needs. From that low intrinsic size they derive generalization bounds that are not vacuous, meaning the promised error ceiling sits below blind guessing.

Catalog record

CREATORHossein Zakerinia, Dorsa Ghobadi, Christoph H. Lampert
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-19067

Browse the full library · About the Tower of Babel