Direct Bethe Free Energy Minimization for Bayesian Neural Network

Pavel Prochazka · 2026

Read Direct Bethe Free Energy Minimization for Bayesian Neural Network by Pavel Prochazka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08446.txt.

Description

Pavel Procházka prices a hidden tax inside variational training. Fitting a belief over a network's last-layer weights by maximizing the ELBO feels principled. Under model misspecification, the best posterior need not give the best predictions. Procházka locates the exact term responsible for the gap.

The negative ELBO splits into three parts. One part holds predictive log scores. One is a prior penalty. One is a binding term, a sum of per-observation Jensen gaps. That term ties predictive spread to posterior covariance. Removing the binding recovers known predictive-score training. In a flat-leverage limit, the author computes the Gaussian cost of the binding outright.

Catalog record

CREATORPavel Prochazka
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08446

Browse the full library · About the Tower of Babel