VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories

Jiachen Yu, Yufei Zhan ,, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu · 2025

Read VFaith: Do Large Multimodal Models Really Reason on Seen Images Rather than Previous Memories by Jiachen Yu, Yufei Zhan ,, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11571.txt.

Description

Jiachen Yu and co-authors ask whether big vision-language models truly reason from the image in front of them or lean on stored memory, in a 2025 paper. Long chain-of-thought tricks lift such models' scores, but how much of the gain is real sight was unclear. The team built a tool that edits set visual cues in test images on its own.

Edited images flip the right answer while keeping the question the same. Score gaps before and after edits expose how much a model leans on sight versus memory. The test set, VFaith-Bench, holds more than seven hundred entries in five groups plus a human-marked sight task. Top closed and open models were put through it in depth.

Catalog record

CREATORJiachen Yu, Yufei Zhan ,, Ziheng Wu, Yousong Zhu, Jinqiao Wang, Minghui Qiu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11571

Browse the full library · About the Tower of Babel