Variational Geometric Information Bottleneck: Learning the Shape of Understanding

Ronald Katende · 2025

Read Variational Geometric Information Bottleneck: Learning the Shape of Understanding by Ronald Katende (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-02496.txt.

Description

Ronald Katende, a researcher in machine learning theory, argues that the shape of a model's representation deserves to be graded, not just its accuracy. In data-scarce learning, where labels are few, an encoder can pack its latent space wastefully. Katende defines a quality score that adds two geometric penalties, for curvature and for intrinsic dimension, to the usual information bottleneck trade-off between compression and task relevance.

Under smoothness and concentration assumptions, he derives generalisation bounds for the low-label regime in which the intrinsic dimension and covering complexity appear. He characterises the frontier between keeping information and keeping geometry simple. The practical version, V-GIB, adds those penalties to variational bottleneck training. On real benchmarks with label fractions from one to twenty percent, it lifted performance and reduced geometric complexity in several regimes, notably on two image datasets, though no regulariser won everywhere.

Catalog record

CREATORRonald Katende
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-02496

Browse the full library · About the Tower of Babel