Your contrastive learning problem is secretly a distribution alignment problem
Zihao Chen, Chi-Heng Lin, Ran Liu, Jingyun Xiao, Eva L. Dyer * · 2025
Read Your contrastive learning problem is secretly a distribution alignment problem by Zihao Chen, Chi-Heng Lin, Ran Liu, Jingyun Xiao, Eva L. Dyer * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20141.txt.
Description
Zihao Chen and four co-authors reframe contrastive learning as a distribution matching problem. Their 2025 paper connects a popular self-teaching method to optimal transport. Contrastive learning builds image and text features by comparing pairs. Why it works has stayed murky despite wide success.
Losses used in contrastive training mirror alignment with entropic transport. That mirror yields new loss families and multi-step variants. Using the whole spread of hidden features beats pair-only views. Transport tools then allow losses that forgive noisy views. Constraints on the match reshape the feature space on purpose. The bridge imports a mature toolbox into self-supervised learning.
Catalog record
| CREATOR | Zihao Chen, Chi-Heng Lin, Ran Liu, Jingyun Xiao, Eva L. Dyer * |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 153 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20141