Least squares variational inference

Yvann Le Fay, Nicolas Chopin, Simon Barthelmé · 2025

Read Least squares variational inference by Yvann Le Fay, Nicolas Chopin, Simon Barthelmé (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18475.txt.

Description

Le Fay, Chopin, and Barthelmé replace gradient machinery in variational inference with plain least squares. The task is to approximate a target probability distribution by the best member of a chosen family. When that family is exponential, they show the best member solves a fixed-point equation. Their method, LSVI, iterates that equation by Monte Carlo, and each step is an ordinary least-squares fit.

The scheme turns out to equal stochastic mirror descent, which yields convergence guarantees. For Gaussian families they add refinements bringing full-shape cost to cubic in dimension and mean-field cost to linear. Across examples LSVI beats leading methods while never computing a gradient, a relief for models where gradients are costly or unavailable.

Catalog record

CREATORYvann Le Fay, Nicolas Chopin, Simon Barthelmé
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE128 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-18475

Browse the full library · About the Tower of Babel