Responses Fall Short of Understanding: Revealing the Gap between Internal Representations and Responses in Visual Document Understanding

Haruka Kawasaki, Ryota Tanaka, Kyosuke Nishida · 2026

Read Responses Fall Short of Understanding: Revealing the Gap between Internal Representations and Responses in Visual Document Understanding by Haruka Kawasaki, Ryota Tanaka, Kyosuke Nishida (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04411.txt.

Description

Kawasaki, Tanaka and Nishida probe how visual document understanding is stored inside vision-language models. Visual document understanding demands sight, text recognition and reasoning over layouts. Yet models are usually judged only on generated replies, which may hide whether required information was truly held. Using linear probing across layers, the authors find a clear gap between inner states and generated replies. Required information often encodes more plainly from middle layers than from the final layer. Fine-tuning middle layers improves probing and reply accuracy while closing the gap.

Catalog record

CREATORHaruka Kawasaki, Ryota Tanaka, Kyosuke Nishida
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36215
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04411

Browse the full library · About the Tower of Babel