Least squares variational inference
Yvann Le Fay, Nicolas Chopin, Simon Barthelmé · 2025
Read Least squares variational inference by Yvann Le Fay, Nicolas Chopin, Simon Barthelmé (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18475.txt.
Description
Le Fay, Chopin, and Barthelmé replace gradient machinery in variational inference with plain least squares. The task is to approximate a target probability distribution by the best member of a chosen family. When that family is exponential, they show the best member solves a fixed-point equation. Their method, LSVI, iterates that equation by Monte Carlo, and each step is an ordinary least-squares fit.
The scheme turns out to equal stochastic mirror descent, which yields convergence guarantees. For Gaussian families they add refinements bringing full-shape cost to cubic in dimension and mean-field cost to linear. Across examples LSVI beats leading methods while never computing a gradient, a relief for models where gradients are costly or unavailable.
Catalog record
| CREATOR | Yvann Le Fay, Nicolas Chopin, Simon Barthelmé |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 128 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-18475