VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories
Jiachen Yu, Yufei Zhan ,, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu · 2025
Read VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories by Jiachen Yu, Yufei Zhan ,, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11571.txt.
Description
Jiachen Yu and co-authors ask whether big vision-language models truly reason from the image in front of them or lean on stored memory, in a 2025 paper. Long chain-of-thought tricks lift such models' scores, but how much of the gain is real sight was unclear. The team built a tool that edits set visual cues in test images on its own.
Edited images flip the right answer while keeping the question the same. Score gaps before and after edits expose how much a model leans on sight versus memory. The test set, VFaith-Bench, holds more than seven hundred entries in five groups plus a human-marked sight task. Top closed and open models were put through it in depth.
Catalog record
| CREATOR | Jiachen Yu, Yufei Zhan ,, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-11571