SurgCheck: Do Vision–Language Models Really Look at Images in Surgical VQA?

Jongmin Shin †, Ka Young Kim †, Eunki Cho, Seong Tae Kim ∗, Namkee Oh ∗ · 2026

Read SurgCheck: Do Vision–Language Models Really Look at Images in Surgical VQA? by Jongmin Shin †, Ka Young Kim †, Eunki Cho, Seong Tae Kim ∗, Namkee Oh ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01911.txt.

Description

Jongmin Shin and colleagues test whether vision-language models truly look at surgical images when they answer questions about them. Many surgical question sets give clues in the wording, so a model may score well from language alone. The team builds SurgCheck, a paired test where each image keeps the same answer but the question is rewritten without entity names.

To keep the new questions clear, they add cues such as a box, an arrow, position, or a short description. Across five models, scores fell on the less biased questions, even though the images did not change. Text-only tests also held up for some tasks. The finding shows that high scores need not mean faithful visual reading.

Catalog record

CREATORJongmin Shin †, Ka Young Kim †, Eunki Cho, Seong Tae Kim ∗, Namkee Oh ∗
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01911

Browse the full library · About the Tower of Babel