Mode Collapse of Mean-Field Variational Inference

Shunan Sheng, Bohan Wu, Alberto González-Sanz · 2025

Read Mode Collapse of Mean-Field Variational Inference by Shunan Sheng, Bohan Wu, Alberto González-Sanz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17063.txt.

Description

Shunan Sheng, Bohan Wu, and Alberto González-Sanz explain a failure of a common approximation method in a 2025 paper. Mean-field variational inference approximates complex probability distributions with simpler ones. It replaces tangled distributions by products of independent pieces. Users often see its answers collapse onto one mode.

Mode collapse means the approximation hugs one peak and ignores others. The authors give the first theoretical explanation of this behavior. They measure component separation with a new separateness notion. Explicit bounds then limit the mass placed on each component. The analysis extends from two components to many. Collapse turns out to depend on the components' relative positions. Their fix adds a rotation to the approximation family. Numerical tests support both the theory and the remedy.

Catalog record

CREATORShunan Sheng, Bohan Wu, Alberto González-Sanz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17063

Browse the full library · About the Tower of Babel