SurgCheck: Do Vision–Language Models Really Look at Images in Surgical VQA?
Jongmin Shin †, Ka Young Kim †, Eunki Cho, Seong Tae Kim ∗, Namkee Oh ∗ · 2026
Read SurgCheck: Do Vision–Language Models Really Look at Images in Surgical VQA? by Jongmin Shin †, Ka Young Kim †, Eunki Cho, Seong Tae Kim ∗, Namkee Oh ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01911.txt.
Description
Jongmin Shin and colleagues test whether vision-language models truly look at surgical images when they answer questions about them. Many surgical question sets give clues in the wording, so a model may score well from language alone. The team builds SurgCheck, a paired test where each image keeps the same answer but the question is rewritten without entity names.
To keep the new questions clear, they add cues such as a box, an arrow, position, or a short description. Across five models, scores fell on the less biased questions, even though the images did not change. Text-only tests also held up for some tasks. The finding shows that high scores need not mean faithful visual reading.
Catalog record
| CREATOR | Jongmin Shin †, Ka Young Kim †, Eunki Cho, Seong Tae Kim ∗, Namkee Oh ∗ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01911