Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning

Tillmann Rheude, Stefan Hegselmann, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin · 2026

Read Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning by Tillmann Rheude, Stefan Hegselmann, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05834.txt.

Description

Rheude and three colleagues expose weak spots in multi-modal contrastive learning, where one weak, skewed, or missing input warps retrieval via multiplicative mixing. Their Gated Symile tunes each input's weight per candidate, sliding toward neutral directions with an explicit NULL choice. Across synthetic and three real three-modality sets it tops tuned leading baselines on top-1 retrieval.

Catalog record

CREATORTillmann Rheude, Stefan Hegselmann, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50885
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05834

Browse the full library · About the Tower of Babel