What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models

Dasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, Teabin Lim, Eungyeol Ahn, Jungwhan Kim, Seunghyeok Hong, Youngsook Song · 2026

Read What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models by Dasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, Teabin Lim, Eungyeol Ahn, Jungwhan Kim, Seunghyeok Hong, Youngsook Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06165.txt.

Description

Real users ask vague questions and let the picture fill the gaps. Dasol Choi and 9 co-authors show how much that hurts vision-language models in a 2026 benchmark paper. Standard benchmarks use clear, explicit prompts, which flatter the models. Everyday queries from Korean online communities look nothing like that.

Their HAERAE-Vision benchmark holds 653 real visual questions, filtered from about 86,000 candidates, each paired with an explicit rewrite. Among 39 models tested, even the strongest scored under half on the original wording. Rewriting the query clearly added large gains, especially for smaller models. Web search did not close the gap. Much apparent model weakness, the authors conclude, is really query underspecification.

Catalog record

CREATORDasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, Teabin Lim, Eungyeol Ahn, Jungwhan Kim, Seunghyeok Hong, Youngsook Song
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-06165

Browse the full library · About the Tower of Babel