Properties and limitations of geometric tempering for gradient flow dynamics

Francesca Romana Crucinio [email protected], Sahani Pathiraja [email protected] · 2026

Read Properties and limitations of geometric tempering for gradient flow dynamics by Francesca Romana Crucinio [email protected], Sahani Pathiraja [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20301.txt.

Introduction

Crucinio and Pathiraja study sampling from a target law, framed as a search over laws that cuts Kullback-Leibler gap. They swap the fixed target for moving targets built by geometric tempering along Wasserstein and Fisher-Rao gradient flows. Convergence comes exponentially fast in continuous time, with new bounds in both cases, and popular time discretisations get the same treatment. In the Fisher-Rao case, mixing start and target laws by geometry never speeds convergence, in continuous or discrete time, and the work derives new adaptive tempering plans.

Catalog record

CREATORFrancesca Romana Crucinio [email protected], Sahani Pathiraja [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87846
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20301

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Properties and limitations of geometric tempering for gradient flow dynamics Francesca Romana Crucinio [email protected], Sahani Pathiraja [email protected] Abstract We consider the problem of sampling from a probability distribution π \pi . It is well known that this can be written as an optimisation problem over the space of probability distributions in which we aim to minimise the Kullback–Leibler divergence from π \pi . We consider the effect of replacing π \pi with a sequence of moving targets ( π t ) t ≥ 0 (\pi_{t})_{t\geq 0} defined via geometric tempering on the Wasserstein and Fisher–Rao gradient flows. We show that convergence occurs exponentially in continuous time, providing novel bounds in both cases. We also consider popular time discretisations and explore their convergence properties. We show that in the Fisher–Rao case, replacing the target distribution with a geometric mixture of initial and target distribution never leads to a convergence speed up both in continuous time and in discrete time. Finally, we explore the gradient flow structure of tempered dynamics and derive novel adaptive tempering schedules. 1 Introduction Sampling from a target probability distribution whose density is known up to a normalisation constant is a fundamental task in computational statistics and machine learning. A natural way to formulate this task is optimisation of a functional measuring the dissimilarity to the target probability distribution, typically the Kullback–Leibler (KL) divergence due to the fact that the KL does not require knowledge of the normalisation constant ( 5 and 12 ). When the space over which the optimisation is carried out is the Wasserstein space, i.e. probability distributions with bounded second moments equipped with the Wasserstein-2 distance ( 2 ) , the resulting gradient descent scheme is referred to as Wasserstein (W) gradient flow. A second class of gradient flows which has gained popularity in recent years is that of Fisher–Rao (FR) gradient flows, i.e. gradient descent w.r.t. the Fisher–Rao or Hellinger–Kakutani distance. Contrary to the Wasserstein metric, which leads to a diffusive behaviour, the Fisher–Rao metric leads to birth-death dynamics in which mass is created or removed ( 24 ; 25 ) . Convergence of the W gradient flow is guaranteed when the target distribution is sufficiently smooth and verifies a log-Sobolev assumption ( 30 ) . Exponential convergence of the FR gradient flow for KL does not require a log-Sobolev assumption and can be established under a bounded second moment assumption and mild regularity of the ratio between target and initial distribution ( 5 , Theorem 4.1 and Eq. (B.6)) , and therefore holds for a wider class of targets. Combinining the W and the FR gradient flows by making use of the Wasserstein–Fisher–Rao metric ( 18 ) leads to the Wasserstein–Fisher–Rao (WFR) gradient flow. The WFR flow combines the diffusive behaviour of the W gradient flow with the FR flow and enjoys more favourable convergence properties, combining the rate of convergence of the W flow with that of the FR flow ( 16 ; 24 ) ; its numerical approximation can considerably speed up the convergence of gradient based methods for multimodal targets ( 24 ; 25 ) . In this work we consider tempered versions of the Wasserstein, Fisher–Rao and Wasserstein–Fisher–Rao PDEs obtained by considering a time varying target \pi_{t}\propto\pi^{\lambda_{t}}\mu_{0}^{1-\lambda_{t}} for \lambda_{t}:\mathbb{R}_{+}\rightarrow[0,1] . This is inspired by recent work that has considered similar tempered targets in methods that locally transport mass ( 4 ; 26 ; 22 ; 21 ) . Our focus is both on methods based on transport as the Wasserstein gradient flow and on methods based on reweighting and birth–death dynamics as the Fisher–Rao gradient flow. While it is known that the FR gradient flow of the KL is intimately tied to tempering or annealing due to its unit time rescaling ( 16 ; 6 ) , it is not immediately obvious to what extent adding further tempering schedules improves or degrades convergence. We therefore study the continuous and discrete time convergence of the tempered W, FR and WFR flows (Section 2 ). We compare our results for the W flow with 4 and highlight similarities and improvements. In Section 3 we investigate the gradient flow structure of the tempered W and FR flow by jointly optimising tempering schedules and the evolution of the tempered flows. We compare the solution of the gradient flow approach with other approaches commonly used in the literature. Our main contributions are: • We provide novel results on the tempered W flow which clearly decompose the bound on the KL decay as the usual bound for the untempered W flow with a bias term which only disappears for \lambda_{t}\equiv 1 (Proposition 1 and 4 ). • We show convergence of the tempered FR dynamics both in continuous and in discrete time (Proposition 2 and 5 ). We also show that combining tempering with the FR dynamics never improves the KL decay (Proposition 3 ). • We explore the gradient flow structure of these tempered dynamics to obtain the steepest descent evolution for \lambda . This point of view allows us to compare the standard untempered flows with the tempered dynamics with the optimal tempering schedule. Theoretical and empirical results show that the steepest descent schedule does not improve over the untempered dynamics and is slower than other approaches commonly used in the literature (Section 3 and Section 4 ). Notation We define some notation that will be used throughout the manuscript. We endow \mathbb{R}^{d} with the Borel \sigma -field \mathcal{B}(\mathbb{R}^{d}) with respect to the Euclidean norm \|\cdot\| where d is clear from context. We denote by C^{1}(\mathbb{R}^{d}) the set of continuously differentiable functions, for all f\in C^{1}(\mathbb{R}^{d}) we denote the gradient by \nabla f . Throughout this manuscript we denote the target by \pi and assume \pi=\exp(-V_{\pi}) for some V_{\pi}\in C^{1}(\mathbb{R}^{d}) . The initial distribution for the tempering iterates is denoted by \mu_{0}=\exp(-V_{0}) with V_{0}\in C^{1}(\mathbb{R}^{d}) . Given \pi,\mu_{0} we define \pi_{t}\propto\pi^{\lambda_{t}}\mu_{0}^{1-\lambda_{t}} for \lambda_{t}:\mathbb{R}_{+}\rightarrow[0,1] . We denote by \mathcal{P}(\mathbb{R}^{d}) the set of probability measures over \mathcal{B}(\mathbb{R}^{d}) , and endow this space with the topology of weak convergence. For any p\in\mathbb{N} , we denote by \mathcal{P}_{p}(\mathbb{R}^{d})=\{\pi\in\mathcal{P}(\mathbb{R}^{d}):\int_{\mathbb{R}^{d}}\|x\|^{p}d\pi(x)<+\infty\} the set of probability measures over \mathcal{B}(\mathbb{R}^{d}) with finite p -th moment. We denote the subset of \mathcal{P}(\mathbb{R}^{d}) of measures which are absolutely continuous w.r.t. Lebesgue by \mathcal{P}^{ac}(\mathbb{R}^{d}) and let \mathcal{P}^{ac}_{p}(\mathbb{R}^{d}):=\mathcal{P}_{p}(\mathbb{R}^{d})\cap\mathcal{P}^{ac}(\mathbb{R}^{d}) . For any \mu,\nu\in\mathcal{P}_{p}(\mathbb{R}^{d}) we define the p -Wasserstein distance W_{p}(\mu,\nu) between \mu and \nu by W_{p}(\mu,\nu)=\left(\inf_{\gamma\in\mathbf{T}(\mu,\nu)}\int_{\mathbb{R}^{d}\times\mathbb{R}^{d}}\|x-y\|^{p}d\gamma(x,y)\right)^{1/p} where \mathbf{T}(\mu,\nu)=\{\gamma\in\mathcal{P}(\mathbb{R}^{d}\times\mathbb{R}^{d}):\gamma(A\times\mathbb{R}^{d})=\mu(A),\ \gamma(\mathbb{R}^{d}\times A)=\nu(A)\ \forall A\in\mathcal{B}(\mathbb{R}^{d})\} denotes the set of all transport plans between \mu and \nu . In the following, we metrise \mathcal{P}_{p}(\mathbb{R}^{d}) with W_{p} . 2 Tempered PDEs for sampling We consider a class of PDEs in which the target \pi is replaced by a sequence of moving targets (\pi_{t})_{t\geq 0} . We refer to these PDEs as tempered PDEs. The original non-tempered dynamics can be obtained by replacing \pi_{t} with \pi throughout. Samplers based on tempered dynamics have become popular in the literature with the hope that the introduction of moving targets will improve the convergence of the sampler ( 28 ; 1 ; 4 ; 22 ) . We focus here on the sequence (\pi_{t})_{t\geq 0} defined by the geometric tempering iteration \pi_{t}\propto\pi^{\lambda_{t}}\mu_{0}^{1-\lambda_{t}} where the tempering schedule \lambda_{t}:\mathbb{R}_{+}\rightarrow[0,1] is non-decreasing in t and weakly differentiable. This

Read the complete text