Surprisal-Rényi Free Energy
Shion Matsumoto, Raul Castillo, Benjamin Prada, Ankur Arjun Mali · 2026
Read Surprisal-Rényi Free Energy by Shion Matsumoto, Raul Castillo, Benjamin Prada, Ankur Arjun Mali (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03405.txt.
Introduction
Shion Matsumoto and three colleagues introduce a new free-energy functional for learning and inference. Two classic objectives, forward and reverse KL divergence, push models in opposite directions, and averages alone cannot explain why. Their 2026 construction, the Surprisal-Rényi Free Energy, is built from log-moments of the likelihood ratio and sits outside the usual family of divergences.
Both KL divergences return as singular endpoint limits of the new functional. Expansions near those limits expose the variance of the log-likelihood ratio as a first-order correction, revealing an explicit mean-variance trade-off. The functional is also the unique minimizer of a weighted sum of KL terms, a Gibbs-style characterization. Finally, it directly controls large deviations of excess code length, giving a precise minimum description length reading. The results clarify the geometry behind the two KL regimes.
Catalog record
| CREATOR | Shion Matsumoto, Raul Castillo, Benjamin Prada, Ankur Arjun Mali |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89112 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.03405
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Surprisal-Rényi Free Energy Shion Matsumoto, Raul Castillo, Benjamin Prada, Ankur Arjun Mali Abstract The forward and reverse Kullback-Leibler (KL) divergences arise as limiting objectives in learning and inference yet induce markedly different inductive biases that cannot be explained at the level of expectations alone. In this work, we introduce the Surprisal-Rényi Free Energy (SRFE), a log-moment-based functional of the likelihood ratio that lies outside the class of f f -divergences. We show that SRFE recovers forward and reverse KL divergences as singular endpoint limits and derive local expansions around both limits in which the variance of the log-likelihood ratio appears as a first-order correction. This reveals an explicit mean-variance tradeoff governing departures from KL-dominated regimes. We further establish a Gibbs-type variational characterization of SRFE as the unique minimizer of a weighted sum of KL divergences and prove that SRFE directly controls large deviations of excess code-length via Chernoff-type bounds, yielding a precise Minimum Description Length interpretation. Together, these results identify SRFE as a variance- and tail-sensitive free-energy functional that clarifies the geometric and large-deviation structure underlying forward- and reverse-KL limits, without unifying or subsuming distinct learning frameworks. 1 Introduction Consider the task of approximating an intractable probability distribution p(x) with a tractable probability distribution q_{\theta}(x) parameterized by \theta . The goal is to find the set of parameters \theta\in\Theta that minimizes the divergence D(p\|q_{\theta}) . This basic formulation has been used in applications such as image classification [ 12 ] , variational inference [ 11 ] , generative adversarial networks [ 6 , 20 ] , knowledge distillation [ 8 ] , and reinforcement learning [ 23 , 24 , 21 ] . While large families of divergences exist, probabilistic machine learning has been dominated by the (forward) Kullback-Leibler (KL) divergence D_{\mathrm{KL}}\left(P\middle\|Q_{\theta}\right) [ 13 ] and its asymmetric counterpart, the reverse KL divergence D_{\mathrm{KL}}\left(Q_{\theta}\middle\|P\right) . Perhaps the most classical use of the forward KL divergence is in supervised learning, where cross entropy is used as a proxy. The objective discourages q_{\theta} from assigning small probability mass to samples it has observed, but in doing so, can assign probability mass to regions where samples do not exist in what is commonly referred to as mass-covering behavior [ 16 ] . In generative models, this can present itself in the generation of unrealistic samples with a high associated log-likelihood [ 9 , 25 ] . In contrast, the reverse KL divergence forces q_{\theta} to avoid assigning mass to regions where mass does not exist at the expense of failing to capture regions where mass does exist. This mode-seeking behavior results in q_{\theta} collapsing onto a single mode of the distribution and ignoring regions with lower probability [ 2 ] . A salient example of this is the training instability and collapse of generative adversarial networks towards a single point [ 22 ] 1 1 1 GANs originally used JSD as their objective but their collapsing behavior aligns with the mode-seeking behavior of the reverse KL divergence. . More recently, similar phenomena have been observed in reinforcement learning fine-tuning for large language models [ 14 ] , perhaps due to the inherent bias of reverse KL divergence-based objectives [ 24 ] . Fig. 1 illustrates the difference between mass-covering and mode-seeking behavior on a simple case of fitting a mixture of Gaussians with a single Gaussian. Figure 1: Gaussian Q that minimizes forward KL (blue) and reverse KL (red) with a mixture of Gaussians P with means \mu_{1},\mu_{2} where \mu_{2}\geq\mu_{1} and variance \sigma_{1}^{2}=\sigma_{2}^{2}=1 . Gaussians are equally weighted. We emphasize that the KL divergence is characterized by a fundamental asymmetry: the forward and reverse KL divergences encode different inductive biases despite sharing the same global minimizer. This binarization of the choice of objective presents a fundamental limitation as models are destined toward either extrema when the optimal solution may lie in the middle. Contribution Motivated by this limitation, we introduce the Surprisal-Rényi Free Energy (SRFE) – a risk-sensitive divergence between two probability distributions P and Q defined via a logarithmic interpolation between their densities. Formally, it is the scaled log moment-generating function (MGF) of the log-likelihood ratio \log p(x)/q(x) , which allows it to capture both average discrepancy and large-deviation behavior. A single parameter \tau\in(0,1) governs this interpolation: in the limit \tau\to 0 , SRFE recovers the forward KL divergence D_{\mathrm{KL}}\left(P\middle\|Q\right) ; in the limit \tau\to 1 , it recovers the reverse KL divergence D_{\mathrm{KL}}\left(Q\middle\|P\right) . For intermediate values of \tau , SRFE defines a smooth continuum between these two extremes, providing a tunable balance between average-case and tail-sensitive mismatch penalties. In summary, our contributions are as follows: 1. We introduce the SRFE, a normalized log-moment-generating functional of the log-likelihood ratio, and establish its fundamental properties, including nonnegativity, KL limits, variational characterization, and its distinction from classical f -divergences ( Section 3.1 ). 2. We perform a second-order analysis of SRFE around its KL limits, showing that while it shares local variance-of-surprisal corrections with power-divergence families, its curvature arises from a cumulant (log-MGF) structure rather than raw moment expansions ( Section 3.2 ). 3. We derive the exact gradient form of SRFE and show that it admits an escort-distribution expectation representation, yielding a continuum between forward and reverse KL divergences and inducing distinct optimization dynamics and improved gradient conditioning ( Section 3.3 ). 4. We prove that SRFE induces the Fisher-Rao Riemannian metric locally, demonstrating that it preserves the intrinsic statistical manifold structure while modifying the global divergence geometry ( Section 3.4 ). 5. We establish a minimum description length (MDL) interpretation of SRFE by deriving Chernoff-type tail bounds for excess codelength, showing that SRFE directly controls large-deviation behavior and risk-sensitive coding performance ( Section 3.5 ). 2 Preliminaries Notation. Let P and Q denote probability distributions over a measurable space ({\mathcal{X}},\Sigma) with densities p(x) and q_{\theta}(x) with respect to a \sigma -finite reference measure \mu . We consider the task of approximating p(x) using a parametric model q_{\theta}(x) drawn from a predefined family (e.g., Gaussian). Our objective is to minimize a divergence measure D(P\|Q_{\theta}) . The forward KL divergence is defined as D_{\mathrm{KL}}\left(P\middle\|Q\right)=\int_{\mathcal{X}}p(x)\log\frac{p(x)}{q_{\theta}(x)}\,d\mu(x),\,\text{when }P\ll Q, and the reverse KL divergence as D_{\mathrm{KL}}\left(Q\middle\|P\right)=\int_{\mathcal{X}}q_{\theta}(x)\log\frac{q_{\theta}(x)}{p(x)}\,d\mu(x),\,\text{when }Q\ll P. 2.1 f -divergences A broad class of divergences is given by the f -divergence family [ 4 ] . Definition 2.1 ( f -divergence) . Let f:(0,\infty)\to\mathbb{R} be convex with f(1)=0 . The f -divergence from P to Q is D_{f}(P\|Q)=\int_{\mathcal{X}}q(x)\,f\!\left(\frac{p(x)}{q(x)}\right)d\mu(x). (1) By appropriate choice of the generator f , this class recovers classical divergences such as the KL divergence, Jensen-Shannon divergence, \chi^{2} divergence, and Hellinger distance. 2.2 Cressie–Read power divergence family A notable one-parameter subset of f -divergences is the Cressie–Read (CR) power divergence family [ 3 ] . Definition 2.2 (Cressie–Read power divergence) . Let ({\mathcal{X}},\Sigma,\mu) be a measure space and let P and Q be probability measures with densities p=\frac{dP}{d\mu} and q=\frac{dQ}{d\mu} . Assume P\ll Q . For \lambda\in\mathbb{R}\setminus\{0,-1\} , D_{\mathrm{CR}}^{\lambda}\left(P\middle\|Q\right)=\frac{1}{\lambda(\lambda+1)}\int_{\mathcal{X}}p(x)\!\left[\left(\frac{p(x)}{q(x)}\right)^{\lambda}-1\right]d\mu(x). (2) The cases \lambda=0 and \lambda=-1 are defined by continuous limits. The CR family was originally introduced for goodness-of-fit testing and provides a continuous interpolation between several classical divergences. Of particular interest here is its connection to the forward and reverse KL divergences. Theorem 2.3 (CR limits to KL divergences) . Let P and