Eye-Q : A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning

Ali Najar, Alireza Mirrokni, Arshia Izadyari, Sadegh Mohammadian, Amir Homayoon Sharifizade, Asal Meskin, Mobin Bagherian, Ehsaneddin Asgari · 2026

Read Eye-Q : A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning by Ali Najar, Alireza Mirrokni, Arshia Izadyari, Sadegh Mohammadian, Amir Homayoon Sharifizade, Asal Meskin, Mobin Bagherian, Ehsaneddin Asgari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03400.txt.

Description

Ali Najar and seven co-authors introduce Eye-Q, a multilingual benchmark of visual word puzzles for testing whether vision-language models truly reason about images. Standard benchmarks reward surface recognition, but these puzzles demand more: spotting implicit visual cues, forming and revising hypotheses, and mapping what is seen to a non-literal word or phrase. Shortcuts through literal grounding, text recognition, or retrieval matching are deliberately blocked.

Eye-Q contains 1,343 puzzles, each a conceptually dense scene with a brief description and a specific target word or phrase to infer. Distractors and contextual relationships force selective attention, abstraction, and associative inference. The set spans English, Persian, and Arabic puzzles, plus cross-lingual ones. The authors evaluate leading vision-language models with an open-ended protocol aligned to human judgment.

Catalog record

CREATORAli Najar, Alireza Mirrokni, Arshia Izadyari, Sadegh Mohammadian, Amir Homayoon Sharifizade, Asal Meskin, Mobin Bagherian, Ehsaneddin Asgari
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03400

Browse the full library · About the Tower of Babel