Contrastive Entropy Bounds for Density and Conditional Density Decomposition
Bo Hu, José C. Príncipe · 2025
Read Contrastive Entropy Bounds for Density and Conditional Density Decomposition by Bo Hu, José C. Príncipe (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12903.txt.
Introduction
Bo Hu and Jose C. Principe reinterpret two classic neural network tools through a Gaussian, Bayesian lens. In their view, training a model tightens a probabilistic bound, and the learned density is what makes the bound tight. Mixture density networks use a bound for the marginal distribution, while autoencoders use a conditional one. The paper focuses on networks with multiple outputs, where each output is the center of a Gaussian component, and asks what objective such networks should really optimize.
Their first result equates the autoencoder objective with maximizing the trace of a Gaussian operator. When one-to-one matching is not required, maximizing the operator's nuclear norm instead would raise its overall rank. Their second construction adds a norm term that increases sample diversity and blocks output collapse. The paper is theoretical, offering new objectives for density modeling.
Catalog record
| CREATOR | Bo Hu, José C. Príncipe |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 180 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-12903