Variational Geometric Information Bottleneck: Learning the Shape of Understanding
Ronald Katende · 2025
Read Variational Geometric Information Bottleneck: Learning the Shape of Understanding by Ronald Katende (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-02496.txt.
Description
Ronald Katende, a researcher in machine learning theory, argues that the shape of a model's representation deserves to be graded, not just its accuracy. In data-scarce learning, where labels are few, an encoder can pack its latent space wastefully. Katende defines a quality score that adds two geometric penalties, for curvature and for intrinsic dimension, to the usual information bottleneck trade-off between compression and task relevance.
Under smoothness and concentration assumptions, he derives generalisation bounds for the low-label regime in which the intrinsic dimension and covering complexity appear. He characterises the frontier between keeping information and keeping geometry simple. The practical version, V-GIB, adds those penalties to variational bottleneck training. On real benchmarks with label fractions from one to twenty percent, it lifted performance and reduced geometric complexity in several regimes, notably on two image datasets, though no regulariser won everywhere.
Catalog record
| CREATOR | Ronald Katende |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-02496