Direct Bethe Free Energy Minimization for Bayesian Neural Network
Pavel Prochazka · 2026
Read Direct Bethe Free Energy Minimization for Bayesian Neural Network by Pavel Prochazka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08446.txt.
Description
Pavel Procházka prices a hidden tax inside variational training. Fitting a belief over a network's last-layer weights by maximizing the ELBO feels principled. Under model misspecification, the best posterior need not give the best predictions. Procházka locates the exact term responsible for the gap.
The negative ELBO splits into three parts. One part holds predictive log scores. One is a prior penalty. One is a binding term, a sum of per-observation Jensen gaps. That term ties predictive spread to posterior covariance. Removing the binding recovers known predictive-score training. In a flat-leverage limit, the author computes the Gaussian cost of the binding outright.
Catalog record
| CREATOR | Pavel Prochazka |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08446