A note on the unique properties of the Kullback–Leibler divergence for sampling via gradient flows

Francesca Romana Crucinio · 2025

Read A note on the unique properties of the Kullback–Leibler divergence for sampling via gradient flows by Francesca Romana Crucinio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04330.txt.

Introduction

Francesca Crucinio proves that one famous distance measure holds a unique edge for drawing samples by flowing downhill, in a paper from 2025. Sampling from a known-up-to-a-constant distribution can be framed as shrinking a distance to the target. Solvers flow a cloud of candidate distributions down that slope.

The snag is the unknown constant in the target: many flows need it, which users do not have. Within a broad family of distances built from convex functions, only the Kullback-Leibler distance gives flows that never ask for the constant, across many common geometries. The note is short and general. It explains why that distance rules sampling practice.

Catalog record

CREATORFrancesca Romana Crucinio
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-04330

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A note on the unique properties of the Kullback–Leibler divergence for sampling via gradient flows Francesca Romana Crucinio Abstract We consider the problem of sampling from a probability distribution π \pi . It is well known that this can be written as an optimisation problem over the space of probability distribution in which we aim to minimise a divergence from π \pi . and The optimisation problem is normally solved through gradient flows in the space of probability distribution with an appropriate metric. We show that the Kullback–Leibler divergence is the only divergence in the family of Bregman divergences whose gradient flow w.r.t. many popular metrics does not require knowledge of the normalising constant of π \pi . 1 Introduction Sampling from a target probability distribution whose density is known up to a normalisation constant is a fundamental task in computational statistics and machine learning. A natural way to formulate this task is optimisation of a functional measuring the dissimilarity to the target probability distribution. Following this point of view, one can derive many popular sampling frameworks including variational inference ( Blei et al. 2017 ) , algorithms based on diffusions ( Roberts and Tweedie 1996 ; Durmus et al. 2019 ) and deterministic flows ( Liu 2017 ) , and algorithms based on importance sampling ( Chopin et al. 2024 ; Crucinio and Pathiraja 2025 ) . The connection between minimisation of a divergence and Monte Carlo algorithms is established through gradient flows over the space of probability measures (see, e.g., Chewi et al. 2025 ; Carrillo et al. 2024 for a recent review); with different metrics over this space leading to different differential equations whose discretisations correspond to many popular Monte Carlo algorithms. The most widely used divergence is the reverse Kullback–Leibler (KL) divergence whose gradient flow w.r.t. the Wasserstein-2 metric can be implemented by a Langevin diffusion ( Jordan et al. 1998 ) and easily discretised in time, resulting in the Unadjusted Langevin algorithm ( Roberts and Tweedie 1996 ) . Recently, gradient flows of the reverse Kullback–Leibler divergence w.r.t. the Fisher–Rao metric have also been considered ( Lu et al. 2019 ; Crucinio and Pathiraja 2025 ; Chen et al. 2023 ; Domingo-Enrich and Pooladian 2023 ) . The Fisher–Rao gradient flow is intimately tied to mirror descent dynamics ( Chopin et al. 2024 ) and can be efficiently implemented via importance sampling ( Crucinio and Pathiraja 2025 ; Korba and Portier 2022 ; Dai et al. 2016 ) . Combination of the Wasserstein and Fisher–Rao metric leads to the Wasserstein–Fisher–Rao gradient flow, which, in the case of the Kullback–Leibler divergence, corresponds to a combination of exploration moves using a Langevin diffusion and reweigthing ( Chewi et al. 2025 ; Lambert et al. 2022 ; Yan et al. 2024 ) , and can be implemented via sequential Monte Carlo ( Crucinio and Pathiraja 2025 ) . Minimisation of divergences (including the Kullback–Leibler) can also be obtained by deriving ordinary differential equations which follow velocity fields which reduce the KL ( Liu 2017 ; Korba et al. 2021 ; Nüsken 2024 ) . These algorithms can also be interpreted as gradient flows of the selected divergence w.r.t. kernelised metrics ( Duncan et al. 2023 ; Chewi et al. 2020 ) . Despite the widespread use of the KL divergence to derive gradient flows, it is well known that other families of divergences, e.g. f -divergences and Bregman divergences, are very flexible and allow to enforce desirable properties such as ability to capture tail behaviour or target the mode ( Regli and Silva 2018 ) . The rationale behind the choice of KL divergence is that this divergence leads to Monte Carlo sampling algorithms which do not require knowledge of the normalising constant of the target, and can therefore be implemented in general settings. In this note, we show that the Kullback–Leibler divergence is the only divergence within the class of Bregman divergences which leads to a gradient flow which does not depend on the normalising constant of \pi . This result complements that of Chen et al. 2023 which establishes an equivalent result for the class of f -divergences. Notation Fix a vector space of (signed) measures \mathcal{M}(\mathbb{R}^{d}) . Let \mathcal{M}^{*}(\mathbb{R}^{d}) the dual of \mathcal{M}(\mathbb{R}^{d}) . For \mu\in\mathcal{M}(\mathbb{R}^{d}) and f\in\mathcal{M}^{*}(\mathbb{R}^{d}) , we denote \langle f,\mu\rangle=\int_{\mathbb{R}^{d}}f(x)\mu(dx) . We denote by \mathcal{P}(\mathbb{R}^{d}) the set of probability measures over \mathcal{B}(\mathbb{R}^{d}) and endow this space with the topology of weak convergence. The subset of \mathcal{P}(\mathbb{R}^{d}) of probability measures admitting a density w.r.t. Lebesgue is denoted by \mathcal{P}_{ac}(\mathbb{R}^{d}) . 2 Background 2.1 Sampling as optimisation over measures Let us denote the target distribution over \mathbb{R}^{d} by \pi and let \mathcal{F}:\mathcal{M}(\mathbb{R}^{d})\rightarrow\mathbb{R}^{+} be a functional on \mathcal{M}(\mathbb{R}^{d}) measuring the distance between \mu and \pi . Consider the optimisation problem \min_{\mu\in\mathcal{P}(\mathbb{R}^{d})}\mathcal{F}(\mu), (1) that is the minimum of \mathcal{F} restricted to the space of probability measures, which recovers \pi if solved exactly. The optimisation problem ( 1 ) can be solved by following the direction of steepest descent of \mathcal{F} , given by the gradient flow partial differential equation (PDE) \displaystyle\partial_{t}\mu_{t}=-\textrm{grad}_{\mathtt{m}}\mathcal{F}(\mu_{t}), (2) where \textrm{grad}_{\mathtt{m}}\mathcal{F}(\mu) denotes the gradient of \mathcal{F} w.r.t. metric \mathtt{m} ( Ambrosio et al. 2008 ; Carrillo et al. 2024 ) . Depending on the choice of metric \mathtt{m} , the gradient \textrm{grad}_{\mathtt{m}}\mathcal{F}(\mu) takes different shapes; for most commonly used metrics the gradient is related to the natural gradient (also known as first variation) of \mathcal{F} ( Amari 1998 ) : Definition 1 . If it exists, the first variation of \mathcal{F} is the functional \frac{\delta\mathcal{F}}{\delta\mu}(\nu,x):\mathcal{M}(\mathbb{R}^{d})\times\mathbb{R}^{d}\rightarrow\mathbb{R} s. t. for any \mu\in\mathcal{M}(\mathbb{R}^{d}) , with \xi=\mu-\nu : \lim_{\epsilon\rightarrow 0}\nicefrac{{1}}{{\epsilon}}(\mathcal{F}(\nu+\epsilon\xi)-\mathcal{F}(\nu))=\langle\frac{\delta\mathcal{F}}{\delta\mu}(\nu,x),\xi\rangle (3) and is defined uniquely up to an additive constant. Restricting the optimisation problem in ( 1 ) to the set of probability measures with finite second moment, \mathcal{P}_{2}(\mathbb{R}^{d}) , and considering the 2 -Wasserstein distance as metric, one obtains the Wasserstein gradient flow ( Ambrosio et al. 2008 ) \displaystyle\partial_{t}\mu_{t}=-\textrm{grad}_{\mathtt{W}}\mathcal{F}(\mu_{t})=\nabla\cdot\left(\mu_{t}\nabla_{x}\frac{\delta\mathcal{F}}{\delta\mu}(\mu_{t},\cdot)\right)\quad t\in[0,\infty), where \nabla_{x} denotes the gradient in \mathbb{R}^{d} . Selecting \mathcal{F}(\mu)=\mathop{\mathrm{KL}}\nolimits(\mu|\pi) leads to the Fokker–Planck equation of the Langevin diffusion ( Jordan et al. 1998 ) \displaystyle dX_{t}=\nabla\log\pi(X_{t})dt+\sqrt{2}dB_{t}, (4) where (B_{t})_{t\geq 0} is a d -dimensional Brownian motion. The Euler–Maruyama discretisation of ( 4 ) is known as the unadjusted Langevin algorithm (ULA; Durmus et al. 2019 ), which leads to the Metropolis-adjusted Langevin algorithm (MALA; Roberts and Tweedie 1996 ) when combined with a Metropolis–Hastings accept/reject step. If instead one considers the Fisher–Rao metric, the gradient flow PDE ( 2 ) is ( Carrillo et al. 2024 ) \displaystyle\partial_{t}\mu_{t} \displaystyle=-\textrm{grad}_{\mathtt{FR}}\mathcal{F}(\mu_{t}) \displaystyle=\mu_{t}\left(-\frac{\delta\mathcal{F}}{\delta\mu}(\mu_{t},\cdot))+\mathbb{E}_{\mu_{t}}\left[\frac{\delta\mathcal{F}}{\delta\mu}(\mu_{t},\cdot))\right]\right)\quad t\in[0,\infty). Selecting \mathcal{F}(\mu)=\mathop{\mathrm{KL}}\nolimits(\mu|\pi) and observing that in this case \frac{\delta\mathcal{F}}{\delta\mu}(\mu)=\log(\mu/\pi) , one obtains the PDE \displaystyle\partial_{t}\mu_{t}=\mu_{t}\left(\log\left(\frac{\pi}{\mu_{t}}\right)-\mathbb{E}_{\mu_{t}}\left[\log\left(\frac{\pi}{\mu_{t}}\right)\right]\right)\quad t\in[0,\infty). This PDE is exactly solved by \mu_{t}\propto\pi^{1-e^{-t}}\mu_{0}^{e^{-t}} ( Chen et al. 2023 , Theorem 4.1) which corresponds to geometric tempering or annealing dynamics ( Domingo-Enrich and Pooladian 2023 ; Chopin et al. 2024 ) . A natural way to numerically approximate the Fisher–Rao flow of \mathop{\mathrm{KL}}\nolimits is to first discretise time and then use importance sampling ( Crucinio and Pathiraja 2025 ) : given \mu_{t} , one can obtain an approximation of \mu_{t+1} using \mu_{t} as proposal and weights \displaystyle w_{t+1}=\left(\frac{\pi}{\mu_{t}}\right)^{1-e^{-\gamma}}, where \gamma denotes the time discretisation step. Combining the Wasserstein and Fisher–Rao metrics one obtains the

Read the complete text