Your contrastive learning problem is secretly a distribution alignment problem

Zihao Chen, Chi-Heng Lin, Ran Liu, Jingyun Xiao, Eva L. Dyer * · 2025

Read Your contrastive learning problem is secretly a distribution alignment problem by Zihao Chen, Chi-Heng Lin, Ran Liu, Jingyun Xiao, Eva L. Dyer * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20141.txt.

Description

Zihao Chen and four co-authors reframe contrastive learning as a distribution matching problem. Their 2025 paper connects a popular self-teaching method to optimal transport. Contrastive learning builds image and text features by comparing pairs. Why it works has stayed murky despite wide success.

Losses used in contrastive training mirror alignment with entropic transport. That mirror yields new loss families and multi-step variants. Using the whole spread of hidden features beats pair-only views. Transport tools then allow losses that forgive noisy views. Constraints on the match reshape the feature space on purpose. The bridge imports a mature toolbox into self-supervised learning.

Catalog record

CREATORZihao Chen, Chi-Heng Lin, Ran Liu, Jingyun Xiao, Eva L. Dyer *
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE153 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20141

Browse the full library · About the Tower of Babel