Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures
Luigi Fogliani, Bruno Loureiro, Marylou Gabrié · 2026
Read Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures by Luigi Fogliani, Bruno Loureiro, Marylou Gabrié (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12923.txt.
Description
Varying fitting often collapses onto one mode of a many-mode target, missing the rest entirely. Luigi Fogliani and two co-authors study annealing as a remedy in a manageable test case: learning a Gaussian mixture, where collapse is well understood. Using low-D summary stats, they describe exactly how starting temperature and annealing rate interact, deriving a sharp formula for the probability of collapse.
An appropriately chosen annealing schedule can robustly prevent collapse, the analysis shows. Number evidence suggests the same trade-offs carry over qualitatively to neural models, including RealNVP normalising flows. For practitioners, the paper offers guidance on designing annealing schedules that keep varying fitting honest about multimodality, rather than settling for a single, confident, incomplete answer.
Catalog record
| CREATOR | Luigi Fogliani, Bruno Loureiro, Marylou Gabrié |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-12923