Understanding Self-Supervised Learning via Latent Distribution Matching
Fabian A Mikulasch, Friedemann Zenke · 2026
Read Understanding Self-Supervised Learning via Latent Distribution Matching by Fabian A Mikulasch, Friedemann Zenke (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03517.txt.
Description
Fabian A. Mikulasch and Friedemann Zenke propose a single framework for self-supervised learning, the method that lets models learn useful representations from unlabeled data. They cast it as latent distribution matching: a representation should fit an assumed latent model well, yet stay spread out enough to avoid collapse. That pair of goals is called alignment and uniformity.
Their view unifies independent component analysis with contrastive, non-contrastive, and predictive methods, including stop-gradient approaches. From it they derive a sampling-free Bayesian filtering model with a Kalman-based predictor for high-dimensional time series. They also prove that predictive matching can recover identifiable latent representations under mild assumptions.
Catalog record
| CREATOR | Fabian A Mikulasch, Friedemann Zenke |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 104 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03517