Understanding Self-Supervised Learning via Latent Distribution Matching

Fabian A Mikulasch, Friedemann Zenke · 2026

Read Understanding Self-Supervised Learning via Latent Distribution Matching by Fabian A Mikulasch, Friedemann Zenke (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03517.txt.

Description

Fabian A. Mikulasch and Friedemann Zenke propose a single framework for self-supervised learning, the method that lets models learn useful representations from unlabeled data. They cast it as latent distribution matching: a representation should fit an assumed latent model well, yet stay spread out enough to avoid collapse. That pair of goals is called alignment and uniformity.

Their view unifies independent component analysis with contrastive, non-contrastive, and predictive methods, including stop-gradient approaches. From it they derive a sampling-free Bayesian filtering model with a Kalman-based predictor for high-dimensional time series. They also prove that predictive matching can recover identifiable latent representations under mild assumptions.

Catalog record

CREATORFabian A Mikulasch, Friedemann Zenke
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03517

Browse the full library · About the Tower of Babel