Eye-Q : A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning
Ali Najar, Alireza Mirrokni, Arshia Izadyari, Sadegh Mohammadian, Amir Homayoon Sharifizade, Asal Meskin, Mobin Bagherian, Ehsaneddin Asgari · 2026
Read Eye-Q : A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning by Ali Najar, Alireza Mirrokni, Arshia Izadyari, Sadegh Mohammadian, Amir Homayoon Sharifizade, Asal Meskin, Mobin Bagherian, Ehsaneddin Asgari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03400.txt.
Description
Ali Najar and seven co-authors introduce Eye-Q, a multilingual benchmark of visual word puzzles for testing whether vision-language models truly reason about images. Standard benchmarks reward surface recognition, but these puzzles demand more: spotting implicit visual cues, forming and revising hypotheses, and mapping what is seen to a non-literal word or phrase. Shortcuts through literal grounding, text recognition, or retrieval matching are deliberately blocked.
Eye-Q contains 1,343 puzzles, each a conceptually dense scene with a brief description and a specific target word or phrase to infer. Distractors and contextual relationships force selective attention, abstraction, and associative inference. The set spans English, Persian, and Arabic puzzles, plus cross-lingual ones. The authors evaluate leading vision-language models with an open-ended protocol aligned to human judgment.
Catalog record
| CREATOR | Ali Najar, Alireza Mirrokni, Arshia Izadyari, Sadegh Mohammadian, Amir Homayoon Sharifizade, Asal Meskin, Mobin Bagherian, Ehsaneddin Asgari |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 109 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03400