Anderson Mixing in Bures Wasserstein Space of Gaussian Measures

Vitalii Aksenov, Martin Eigel, Mathias Oster · 2026

Read Anderson Mixing in Bures Wasserstein Space of Gaussian Measures by Vitalii Aksenov, Martin Eigel, Mathias Oster (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22038.txt.

Introduction

Vitalii Aksenov, Martin Eigel, and Mathias Oster speed up sums that deal with chance curves. Many tasks in stats can be cast as fixed-point sums. You run the same step again and again till the result stops moving. One key use is to find the mean of a set of chance curves, in a sense set by the Wasserstein gap.

Plain runs of such steps can be slow. The team tries a speed-up trick called Anderson mixing. They test it in the space of bell-shaped curves, where gaps and means have neat closed forms. Fast solvers of this kind help in Bayes work and in training AI models. In both, these sums run deep in the loop, and their cost is paid on each run.

Catalog record

CREATORVitalii Aksenov, Martin Eigel, Mathias Oster
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22038

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Anderson Mixing in Bures Wasserstein Space of Gaussian Measures Vitalii Aksenov, Martin Eigel, Mathias Oster Abstract Various statistical tasks, including sampling or computing Wasserstein barycenters, can be reformulated as fixed-point problems for operators on probability distributions. Accelerating standard fixed-point iteration schemes provides a promising novel approach to the design of efficient numerical methods for these problems. The Wasserstein geometry on the space of probability measures, although not precisely Riemannian, allows us to define various useful Riemannian notions, such as tangent spaces, exponential maps and parallel transport, motivating the adaptation of Riemannian numerical methods. We demonstrate this by developing and implementing the Riemannian Anderson Mixing (RAM) method for Gaussian distributions. The method reuses the history of the residuals and improves the iteration complexity, and we argue that the additional costs, compared to Picard method, are negligible. We show that certain open balls in the Bures-Wasserstein manifold satisfy the requirements for convergence of RAM. The numerical experiments show a significant acceleration compared to a Picard iteration, and performance on par with Riemannian Gradient Descent and Conjugate Gradient methods. 1 Introduction There is a plethora of tasks in statistics, ranging from sampling given a Bayesian posterior to calculating Wasserstein barycenters and medians, that can be reformulated as a fixed-point problem in Wasserstein space. For example, the dynamics of the Wasserstein gradient flow of certain functionals is contractive and, thus, induces fix point schemes. To make this more precise, let \mathcal{P}_{2}(\mathbb{R}^{d}) be the space of probability measures with finite second moments \mathcal{P}_{2}(\mathbb{R}^{d}):=\left\{\mu:\mathbb{E}_{x\sim\mu}\|x\|^{2}_{2}<\infty\right\} endowed with the 2-Wasserstein distance W^{2}_{2}(\mu_{1},\mu_{2}):=\min_{\pi\in\Pi(\mu_{1},\mu_{2})}\int\|x-y\|^{2}\mathrm{d}\pi(x,y), where \Pi(\mu_{1},\mu_{2}) denotes the set of probability distributions on \mathbb{R}^{d}\times\mathbb{R}^{d} with marginals \mu_{1} and \mu_{2} . As shown in [ 6 ] , if some functional \mathcal{E}:\mathcal{P}_{2}(\mathbb{R}^{d})\to\mathbb{R}\cup\{\infty\} is \lambda -convex along generalized geodesics for \lambda>0 , then for any \mu\in\overline{D(\mathcal{E})}:=\overline{\{\mu\in\mathcal{P}_{2}(\mathbb{R}^{d}):\mathcal{E}(\mu)<\infty\}} there exists a unique locally Lipschitz curve S[\mu](t):(0,+\infty)\to\mathcal{P}_{2}(\mathbb{R}^{d}) , which is a gradient flow of \mathcal{E} with initial value \mu . For a fixed time \tau , the operator defined by G_{\tau}(\mu)=S[\mu](\tau) satisfies the following contraction estimate: W_{2}(G_{\tau}(\mu_{1}),G_{\tau}(\mu_{2}))\leq e^{-\tau\lambda}W_{2}(\mu_{1},\mu_{2}). If \mathcal{E}=\operatorname{KL}(\cdot|\rho_{*}) is the Kullback-Leibler divergence and \rho_{*}\sim e^{-V} for some coercive potential V , then \lambda -convexity along generalized geodesics is equivalent to \lambda -convexity of V and \rho_{*} is a fixed point of G . The case of \operatorname{KL} divergence is of particular interest, as a multitude of numerical algorithms for sampling can be interpreted as relaxations and approximations of this gradient flow. Important examples are Langevin dynamics, see, e.g., [ 41 ] , and Stein variational gradient descent, see, e.g., [ 26 , 12 ] . Furthermore, MCMC methods such as Metropolis-Hastings ( [ 18 ] ) or Metropolis-adjusted Langevin dynamics ( [ 30 ] ) have by design the target distribution as an invariant distribution of the Markov chain. Thus, they can also be seen as a fixed-point iteration. In addition, fixed-point iterations can be used to find the Wasserstein Barycenter as noted in [ 5 ] . Finally, it is known that for a convex function f:\mathbb{R}^{d}\to\mathbb{R} , a fixed point of \mathrm{Id}-h\nabla f is the root of the gradient and, therefore, the global minimizer. Thus, a fixed-point method can be used as a first-order optimization method and can be applied to a variety of optimization problems. However, in various instances fixed-point schemes based on a contraction principle suffer from a slow convergence rate, for example if the contraction constant is close to unity. In order to speed up the calculations one can generalize Euclidean or Riemannian acceleration schemes to Wasserstein space by exploiting the manifold-like structure. Previously, similar ideas have been applied in gradient-based optimization. In [ 22 ] , the Frank-Wolfe method is adapted to the Wasserstein setting. A Riemannian Frank-Wolfe method was applied to Wasserstein the barycenter problem in [ 37 ] . A counterpart of Nesterov acceleration in Wasserstein space is considered in [ 24 , 8 ] . Riemannian minimization on the Bures-Wasserstein manifold is studied in [ 17 ] . It is known that the space of probability measures on \mathbb{R}^{d} can be endowed with the 2 -Wasserstein metric and then forms a complete separable metric space ( [ 6 ] Proposition 7.1.5). This space is not exactly Riemannian (i.e. there is no atlas of maps to subsets of a fixed linear space), but nevertheless all the important geometric notions, such as geodesics, tangent space, exponential mapping and parallel transport, can be defined. However, the deviations of the geometry from standard Riemannian assumptions, e.g. the exponential map having a vanishing injectivity radius [ 16 ] , pose hurdles in designing schemes in Wasserstein space beyond the Gaussian case based on already known algorithms. Contributions: We adapt the Riemanian Anderson Mixing (RAM) algorithm introduced in [ 23 ] to Gaussian measures in Bures-Wasserstein manifolds (referred to as BWRAM), verifying some assumptions for the convergence analysis of RAM and identifying the limits of generalizing the methods to general measures in Wasserstein space. The convergence result that we prove can be summarized as follows: Theorem 1 (Convergence of BWRAM (informal statement)) . Let G(\Sigma)=\operatorname{Exp}_{\Sigma}(-F(\Sigma)) be a contractive mapping on the Bures-Wasserstein manifold and \Sigma_{*}\succeq\lambda\operatorname{Id} its fixed point with \lambda>0 . If the initial iterate \Sigma_{0}\in B_{W_{2}}(\Sigma_{*},r) for a sufficiently small r , and under additional smoothness assumptions on F , then the following convergence rate holds W_{2}(\Sigma_{k},G(\Sigma_{k}))\leq\left[(1-\beta_{k})+\kappa\beta_{k}\right]\|r_{k}\|+\sum_{i=0}^{\max\{m,k\}}\mathcal{O}(\|r_{k-i}\|^{2}),\\ (1) where \operatorname{Exp} denotes the Riemannian exponential map, r_{k}=-F(x_{k}) the fixed-point residual, m the maximal number of historical vectors and \beta_{k}>0 is the relaxation parameter. The theoretical estimate says that, up to higher-order terms, the method is guaranteed to converge as good as Picard iteration. We argue that the additional computation costs incurred by the method are negligible in comparison with the operator evaluation, and thus overall speedup of the solution is provided. The improved convergence behavior of BWRAM in comparison to Picard is illustrated with various numerical examples such as estimation of the steady-state of the Ornstein-Uhlenbeck process, minimization of the Kullback-Leibler divergence, and accelerated solution of averaging problems, such as Wasserstein barycenters and geometric medians. We also demonstrate that our method outperforms other well-known Riemannian minimization methods, such as Riemannian Gradient descent and Conjugate Gradient. Limitations: The method achieves convergence speed superior to Picard iteration at the expense of additional memory costs and a solution of a small-scale minimization problem. The performance of the method is sensitive to the choice of hyperparameters, such as the number of history vectors. Although the Picard method is outperformed robustly for an arbitrary selection, the maximal performance is only achieved for particular values, which cannot be known in advance. In \mathbb{R}^{d} , there exist strategies to mitigate the issue, for example, adaptive restarting, see, e.g., [ 38 ] and regularization, see, e.g., [ 39 ] . Their adaptation is left out to future work. Outline: The paper is structures as follows: In section 2 we recall the Anderson mixing algorithm in Euclidean and Riemannian spaces. Thereafter, we analyze the convergence behavior of the Anderson mixing on Bures-Wasserstein space based on [ 23 ] while providing the proofs in the appendix. Finally, in section 5 numerical experiments are shown. The implementation of the method can be accessed at https://github.com/viviaxenov/fpw 2 Anderson mixing in Euclidean space and for Riemannian manifolds with bounded sectional curvature Anderson mixing

Read the complete text