iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs
Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni · 2025
Read iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs by Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03214.txt.
Description
Julius Mayer and co-authors test whether vision-language models can reason about space by making them play a sliding tile puzzle. Their benchmark, iVISPAR, asks a model to act as an agent, planning moves across turns rather than answering once. The puzzle appears in three forms: three-dimensional views, flat two-dimensional views, and plain text.
A broad set of leading open and commercial models was compared against optimal solutions and a human baseline. Models handled the flat version best and stumbled on the three-dimensional and text forms. All fell short of people, especially on tangled configurations. The authors say the gap exposes how weakly today's models align what they see with where things are.
Catalog record
| CREATOR | Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-03214