Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning
Tillmann Rheude, Stefan Hegselmann, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin · 2026
Read Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning by Tillmann Rheude, Stefan Hegselmann, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05834.txt.
Description
Rheude and three colleagues expose weak spots in multi-modal contrastive learning, where one weak, skewed, or missing input warps retrieval via multiplicative mixing. Their Gated Symile tunes each input's weight per candidate, sliding toward neutral directions with an explicit NULL choice. Across synthetic and three real three-modality sets it tops tuned leading baselines on top-1 retrieval.
Catalog record
| CREATOR | Tillmann Rheude, Stefan Hegselmann, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50885 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.05834