Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs
Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao, Xuelong Li · 2026
Read Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs by Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao, Xuelong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08241.txt.
Description
Chain-of-thought reasoning helps multimodal models, but their visual attention often starts misaligned and never recovers, so errors propagate through the whole reasoning chain. Siqu Ou and four co-authors trace this to poor credit assignment for visual attention during training.
Their model, SAYO, trains with reinforcement learning using a region-level visual attention reward that ties optimization signals directly to visually grounded reasoning steps. The reward teaches the model to look at the right regions and keep looking there as reasoning proceeds. Across multiple multimodal benchmarks, SAYO improves performance consistently over baselines. Vision-language researchers gain a training signal aimed at seeing, not just talking about seeing.
Catalog record
| CREATOR | Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao, Xuelong Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-08241