Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization
Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef Mroueh · 2026
Read Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization by Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef Mroueh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27767.txt.
Introduction
Optimising over probability measures is now routine in machine learning. The Wasserstein space gives such problems a geometry. It measures how far apart two distributions are. Many objectives are non-convex along its geodesics, though. So standard methods resist analysis here. Clément Bonet, Pierre-Cyril Aubin-Frankowski, and Youssef Mroueh study a structured subclass in 2026. Their objectives split as a difference of two convex parts.
The team lifts the classical convex-concave procedure into this space. Under smoothness assumptions, they prove near-stationarity along the iterates. Two widely used functionals get explicit decompositions. They are Maximum Mean Discrepancy and Energy Distance. Both compare distributions in learning tasks. Local convergence of the scheme is also established. Tests compare against Wasserstein gradient descent. Well-chosen decompositions converge faster and more stably. The choice of decomposition matters greatly, the authors note. Designing decompositions automatically is left as future work.
Catalog record
| CREATOR | Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef Mroueh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 121 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.27767
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef Mroueh Abstract Optimizing functionals over the space of probability measures is now ubiquitous in machine learning. A widely used approach is to perform the optimization directly over the Wasserstein space, but many objective functionals of practical interest are non-convex along Wasserstein geodesics, making the analysis of standard first-order methods challenging. In this work, we study a class of objectives over the Wasserstein space that admit a difference-of-convex (DC) decomposition and we lift the classical convex-concave procedure (CCCP) to this setting. Under smoothness and strong convexity assumptions on the convex components of the decomposition, we prove almost stationarity along the iterates of the resulting algorithm. Our main focus is on the Maximum Mean Discrepancy (MMD) and the Energy Distance (ED) functionals, for which we develop explicit Wasserstein DC decompositions, and establish local convergence of the scheme under mild assumptions. Empirically, we show that well-chosen DC decompositions yield faster and more stable convergence than Wasserstein gradient descent on these MMD objectives. 1 Introduction Optimizing over the space of probability measures is an important problem in machine learning, which has received attention to solve problems ranging from variational inference ( 15 ; 46 ; 64 ) to generative modeling ( 29 ; 82 ; 24 ) , reinforcement learning ( 93 ; 65 ) , optimization of neural networks ( 56 ; 27 ) or for modeling dynamics of cells ( 23 ; 79 ; 63 ) . One prominent way to optimize over the space of probability measures is to equip it with the Wasserstein distance ( 83 ; 70 ) , and to discretize the associated Wasserstein gradient flows ( 43 ; 86 ) . This allows to design many optimization algorithms as counterparts of their Euclidean version, such as gradient descent ( 86 ) , proximal point and gradient algorithms ( 43 ; 69 ) , coordinate descent ( 89 ) or mirror descent ( 18 ; 72 ) . While these optimization methods have demonstrated good results in minimizing several objectives such as the Kullback-Leibler divergence (KL) ( 86 ) , f -divergences ( 35 ; 5 ; 51 ) , the Energy Distance ( 41 ) or the Sliced-Wasserstein distance ( 52 ; 30 ; 17 ) , most of them are tailored for convex functionals along (generalized) geodesics. However, many popular functionals are known to be non-convex. This is the case for instance of the squared Wasserstein distance itself ( 4 , Chapter 9) , the Sliced-Wasserstein distance ( 20 ) , the KL with non-log-concave target ( 55 ) , the Energy distance ( 40 ) or the Maximum Mean Discrepancy (MMD) ( 6 ) . An alternative to the convexity assumption is to consider Polyak-Łojaziewicz inequalities ( 14 ; 50 ; 94 ) , however these are so far only known to hold in restrictive cases such as trivially for the squared Wasserstein distance, the KL w.r.t. a measure satisfying the log-Sobolev inequality ( 83 , Chapter 21) , Sliced-Wasserstein distance over Gaussians ( 80 ) , the MMD with Coulomb kernels and smooth initializations ( 21 , Section 2) . For the MMD specifically, to address its non-convexity in the Wasserstein space, 6 proposed injecting noise to the gradient. However, tuning the amount of noise remains delicate. 37 performed instead the optimization of MMD in the Wasserstein-MMD space, where MMD is convex. Their method improves performance, but it requires to change weights of the distribution, mixing two implicit steps in squared Wasserstein and MMD. More recently , 12 optimized it in the Wasserstein Fisher-Rao space through a fixed-point algorithm, also changing weights. We propose instead a new particle-based algorithm on the Wasserstein space to handle non-convex functionals. To achieve such a goal, a candidate class of analogous algorithms over \mathbb{R}^{d} is the family of methods for objectives that can be written as a difference-of-convex functions (DC) ( 49 ; 66 ) . On \mathbb{R}^{d} this includes a large class of functions of interest, in particular twice continuously differentiable functions ( 91 ; 42 ) . One key such algorithm is the Convex-Concave Procedure ( 91 ) . These DC algorithms have already been used for many applications in machine learning, such as kernel selection ( 7 ) , clustering ( 78 ) , dictionary learning ( 84 ) , optimal transport ( 81 ) , or neural network optimization ( 9 ; 8 ) to name a few. However, except for a few works that studied DC algorithms on Riemannian manifolds ( 73 ; 85 ; 13 ; 33 ) , and recently on the Wasserstein space ( 55 ; 54 ) , optimization algorithms for DC functionals have mostly been studied on Euclidean spaces. Contributions. In this work, we focus on developing an optimization scheme on the Wasserstein space tailored for the non-convex objectives that can be written as a difference of convex objectives. To do so, we introduce the Wasserstein Convex-Concave Procedure (WCCCP), and analyze its theoretical convergence, proving almost stationary along its iterates. The closest work to ours is ( 55 ) which used a Wasserstein Proximal Gradient algorithm ( 69 ) to solve some DC problems, but which restricted their applications to functionals whose concave part is a potential energy, i.e. linear. We argue that it is too restrictive to handle objectives such as Maximum Mean Discrepancies, which can be decomposed as a sum of two non-convex quadratic and linear terms. Thus, we deal with the more general case, where the concave part can be any differentiable functional over the Wasserstein space. Then, for several kernels, we show that with a well-chosen decomposition obtained by splitting the kernel, the introduced scheme can better optimize the MMD than the Wasserstein Gradient Descent. In particular, we provide experiments on the Energy distance and the MMD with Gaussian kernel. Notation. We denote by \mathcal{P}_{2}(\mathbb{R}^{d}) the space of probability distributions with second finite moments, and by \mathcal{P}_{\mathrm{ac}}(\mathbb{R}^{d}) its restriction to absolutely continuous measures with respect to the Lebesgue measure. Given \mu,\nu\in\mathcal{P}_{2}(\mathbb{R}^{d}) , we denote by \mathrm{W}_{2}^{2}(\mu,\nu)=\inf_{\gamma\in\Pi(\mu,\nu)}\ \int\|x-y\|_{2}^{2}\ \mathrm{d}\gamma(x,y) the squared-Wasserstein distance, where \Pi(\mu,\nu) is the set of couplings between \mu and \nu , and \Pi_{o}(\mu,\nu) is its subset of optimal couplings. The metric space (\mathcal{P}_{2}(\mathbb{R}^{d}),\mathrm{W}_{2}) is called the Wasserstein space. For any \mu\in\mathcal{P}_{2}(\mathbb{R}^{d}) , we denote by L^{2}(\mu) the Hilbert space of functions f:\mathbb{R}^{d}\to\mathbb{R}^{d} such that \int\|f(x)\|_{2}^{2}\ \mathrm{d}\mu(x)<\infty equipped with the norm \|f\|_{L^{2}(\mu)}^{2}=\int\|f(x)\|_{2}^{2}\ \mathrm{d}\mu(x) and with inner product \langle\cdot,\cdot\rangle_{L^{2}(\mu)} . Given \mathrm{T}:\mathbb{R}^{d}\to\mathbb{R}^{d} , \mathrm{T}_{\#}\mu\in\mathcal{P}_{2}(\mathbb{R}^{d}) is the pushforward measure of \mu . 2 Background We begin by recalling a few facts on optimization on the Wasserstein space. More precisely, we recall the notion of Wasserstein gradient, of total convexity on the Wasserstein space, and some classical optimization schemes on (\mathcal{P}_{2}(\mathbb{R}^{d}),\mathrm{W}_{2}) . For more details about Wasserstein gradient flows, we refer to e.g. ( 4 ; 48 ) . Then, we provide a brief introduction to the convex-concave procedure on Euclidean spaces. Wasserstein gradient. Let \mathcal{F}:\mathcal{P}_{2}(\mathbb{R}^{d})\to\mathbb{R} be a functional. It admits a Wasserstein gradient \nabla_{\mathrm{W}_{2}}\mathcal{F}(\mu)\in L^{2}(\mu) at \mu\in\mathcal{P}_{2}(\mathbb{R}^{d}) if for all \nu\in\mathcal{P}_{2}(\mathbb{R}^{d}) , \gamma\in\Pi_{o}(\mu,\nu) , the following first order Taylor expansion is satisfied ( 19 ; 48 )