Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
Shezheng Song, Shasha Li, Jie Yu · 2026
Read Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention by Shezheng Song, Shasha Li, Jie Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08151.txt.
Description
Shezheng Song, Shasha Li, and Jie Yu ask where vision meets language inside multimodal models. Their 2026 arXiv paper notes that multimodal large language models now score well on vision-language tasks. Yet how visual information is integrated across layers remains poorly understood.
The authors run layer-wise masking experiments to trace that integration step by step. Masking layers shows which stages carry visual detail into the language side. Their title points to contrastive attention as a tool for refining the fusion process. For model builders, a clearer map of visual fusion could guide leaner designs and better fixes when images are misread.
Catalog record
| CREATOR | Shezheng Song, Shasha Li, Jie Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-08151