Thinned Mean Field Langevin Dynamics
Zonghao Chen, Heishiro Kanagawa, François-Xavier Briol, Chris J. Oates, Lester Mackey · 2026
Read Thinned Mean Field Langevin Dynamics by Zonghao Chen, Heishiro Kanagawa, François-Xavier Briol, Chris J. Oates, Lester Mackey (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28589.txt.
Introduction
Zonghao Chen and four colleagues cut the cost of mean-field Langevin dynamics with kernel thinning. Learning tasks minimize an entropy-regularized goal over distributions, with the distribution as the fixed point of a McKean-Vlasov process run with N particles at N squared cost. Their KT-MFLD lets particles link only to a coreset of square root N size, dropping cost to N to the three halves. Under mild regularity it keeps the convergence guarantees within log factors, confirmed on neural-network training, quantization, and posterior computation.
Catalog record
| CREATOR | Zonghao Chen, Heishiro Kanagawa, François-Xavier Briol, Chris J. Oates, Lester Mackey |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 112935 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.28589
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Thinned Mean Field Langevin Dynamics Zonghao Chen, Heishiro Kanagawa, François-Xavier Briol, Chris J. Oates, Lester Mackey Abstract Several important learning tasks can be formulated as minimizing an entropy-regularized objective over an appropriate space of probability distributions. Mean-field Langevin dynamics (MFLD) facilitate computation in this general context, casting the minimizer as the invariant distribution of a McKean–Vlasov process, which can be numerically discretized using N N particles and thus simulated. However, simulating this interacting particle system has computational complexity of order N 2 N^{2} . Motivated by recent research into kernel thinning , we propose KT-MFLD , in which each particle interacts only with a thinned particle coreset of size 𝒪 ( N 1 2 ) \mathcal{O}(N^{\frac{1}{2}}) . KT-MFLD thus reduces the computational complexity to order N 3 2 N^{\frac{3}{2}} while, under mild regularity conditions, achieving the same convergence guarantees (up to logarithmic factors) as MFLD. Our theoretical analysis is empirically confirmed on tasks including the training of student-teacher neural networks, quantization with maximum mean discrepancy, and computation of predictively-oriented posteriors in a post-Bayesian framework. † † affiliationnotice: Equal contribution 1 Introduction Arguably the most popular regularizer in probabilistic machine learning is entropy ; several diverse tasks can be formulated as finding the distribution \pi which minimizes an entropy-regularized functional ( Suzuki et al. 2023 ; Chizat 2022 ; Chazal et al. 2025 ) ; \displaystyle\mathscr{F}(\mu):=F(\mu)-\sigma\mathrm{Ent}(\mu). (1) Here, \mathrm{Ent}(\mu)=-\int\log\mu(\boldsymbol{x})\mu(\mathrm{d}\boldsymbol{x}) denotes the differential entropy, and \sigma>0 controls the strength of entropic regularization. Here we overload notation so that the measure \mu and its density are represented by the same symbol. In settings where F is linear with respect to \mu , i.e. F(\mu)=\int V(\boldsymbol{x})\mathrm{d}\mu(\boldsymbol{x}) , the minimizer \pi can be written up to its normalization constant as \pi(\boldsymbol{x})\propto\exp(-V(\boldsymbol{x})/\sigma) ( Jordan et al. 1998 ; Murphy 2022 ) . In contrast, when F is nonlinear , no such explicit characterization of \pi is available, precluding standard sampling methods such as Markov chain Monte Carlo ( Neal 1993 ) . Prominent examples of the above formulation with nonlinear F include the optimal mean-field distributions of two-layer infinitely wide neural networks ( Chizat and Bach 2018 ; Hu et al. 2021 ; Rotskoff and Vanden-Eijnden 2022 ) , quantization with maximum mean discrepancy ( Arbel et al. 2019 ; Chen et al. 2025a ; Chen et al. 2026 ) , emerging post-Bayesian frameworks for inference ( Wild et al. 2023 ; McLatchie et al. 2025 ; Shen et al. 2025 ) , trajectory inference ( Chizat et al. 2022 ; Gu et al. 2025 ) , and sparse inverse problems ( Boyd et al. 2017 ) . One popular approach in the nonlinear setting is mean-field Langevin dynamics (MFLD), a descent scheme that minimizes \mathscr{F} in the steepest direction over \mathcal{P}_{2}(\mathbb{R}^{d}) , the space of probability distributions with finite second moment, with respect to the Wasserstein-2 metric ( Del Moral 2013 ; Hu et al. 2021 ; Suzuki et al. 2023 ; Chizat 2022 ) . MFLD can be interpreted via its dual formulation as a McKean–Vlasov process; given an initial distribution \mu_{0}\in\mathcal{P}_{2}(\mathbb{R}^{d}) , \displaystyle\mathrm{d}X_{s}=-\nabla F^{\prime}[\mu_{s}](X_{s})\mathrm{d}s+\sqrt{2\sigma}\,\mathrm{d}B_{s},\quad s>0, (2) where \mu_{s}=\textrm{Law}(X_{s}) , F^{\prime}[\mu_{s}]:\mathbb{R}^{d}\to\mathbb{R} is the first variation of F at \mu_{s} ( Santambrogio 2015 , Chapter 7) , \nabla denotes the Euclidean gradient, and B_{s} is a standard Brownian motion on \mathbb{R}^{d} . Figure 1 : Illustration of KT-MFLD . In MFLD , all N particles interact pairwise at each iteration t\in\{0,1,\ldots,T\} , resulting in a cost of \Omega(N^{2}) . In contrast, in KT-MFLD , all N particles ( ; ) interact only with a selected subset of M particles ( ), chosen via kernel thinning, leading to a reduced cost of \mathcal{O}(N^{3/2}) . For linear F , the first variation F^{\prime}[\mu_{s}](X_{s}) is independent of \mu_{s} , and we recover Langevin dynamics, which can be simulated using a single particle ( Durmus and Moulines 2019 ) . In contrast, for nonlinear F the first variation F^{\prime}[\mu_{s}](X_{s}) depends on \mu_{s} as well as on X_{s} . Since \mu_{s} is intractable, Eq. ( 2 ) cannot be directly simulated. A popular solution is to approximate Eq. ( 2 ) using a system of N interacting particles \mathscr{X}_{s}:=\{\boldsymbol{x}_{s}^{(i)}\}_{i=1}^{N} with associated empirical measure \mu_{\mathscr{X}_{s}}=\frac{1}{N}\sum_{i=1}^{N}\delta_{\boldsymbol{x}_{s}^{(i)}} , replacing the population-level drift \nabla F^{\prime}[\mu_{s}] by its empirical counterpart \nabla F^{\prime}[\mu_{\mathscr{X}_{s}}] . This interacting particle system can be simulated, for example using the Euler–Maruyama method at discrete time steps t=\{0,\ldots,T\} , and, under mild conditions on F , it has been proven that the empirical distribution \mu_{\mathscr{X}_{t}} converges to the minimizer \pi in Wasserstein-2 distance, as both T,N\to\infty ( Chizat 2022 ; Suzuki et al. 2023 ; Nitanda 2024 ; Nitanda et al. 2025 ) . Unfortunately, simulating the state \boldsymbol{x}_{t+1}^{(i)} of the i -th particle at time t+1 requires computing interaction terms involving all of the N particles at time t . This results in an \Omega(N) cost per particle and an overall \Omega(N^{2}) cost per iteration. This limitation becomes particularly severe when the target distribution \pi is high-dimensional and multimodal, as representing such distributions typically requires a large number of particles. Such computational constraints often force practitioners to work with relatively small particle populations. For instance, in mean-field neural networks ( Nitanda et al. 2022 ) and post-Bayesian inference ( Shen et al. 2025 ) , the number of particles is typically restricted to the order of 10^{2} . Our experiments in Section 4 demonstrate that increasing the number of particles to the order of 10^{3} consistently leads to improved performance, highlighting the importance of more scalable particle-based simulations ( Jin et al. 2020 ) . To reduce the \Omega{O}(N^{2}) per-iteration cost, we propose a new descent scheme that, at each iteration t , selects a representative subset \{\bar{\boldsymbol{x}}_{t}^{(i)}\}_{i=1}^{M}\subset\{\boldsymbol{x}_{t}^{(i)}\}_{i=1}^{N} of size M=\mathcal{O}(N^{1/2}) . We refer to this selection procedure as thinning . Each particle then interacts only with these M representative particles, reducing the per-iteration computational complexity from \Omega{O}(N^{2}) to \mathcal{O}(N^{3/2}) . The effectiveness of this approach depends on how the representative particles are selected. For this work, we exploit a new near-linear time algorithm called kt-split -Compress ( Shetty et al. 2022 ) and we call the corresponding descent scheme KT-MFLD . Our main theoretical contribution is to establish sufficient conditions under which KT-MFLD achieves accuracy comparable to MFLD, measured by the discrepancy between \mu_{\mathscr{X}_{T}} and target distribution \pi . We further demonstrate these conditions are satisfied in the applications of MFLD considered in this work. In particular, for a fixed large T and small step size, the finite-particle approximation error of KT-MFLD scales as \mathcal{O}(N^{-1}(\log N)^{2}) . This matches the approximation error of the original MFLD up to the logarithmic factor in N . By contrast, if the representative subset \{\bar{\boldsymbol{x}}_{t}^{(i)}\}_{i=1}^{M} was selected via random subsampling at each iteration, referred to as Random-MFLD , the resulting finite-particle approximation error would deteriorate to \Omega(N^{-\frac{1}{2}}) . Section 4 presents empirical results verifying that KT-MFLD outperforms Random-MFLD and other coreset-related acceleration methods ( Jin et al. 2020 ) over a range of tasks for which MFLD is commonly used. Notations: Boldface symbols (e.g. \boldsymbol{x} ) are used exclusively to denote particles, while all remaining variables are written in standard font. \|\cdot\| without any subscripts denotes the Euclidean norm. For a bivariate differentiable