iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs

Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni · 2025

Read iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs by Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03214.txt.

Description

Julius Mayer and co-authors test whether vision-language models can reason about space by making them play a sliding tile puzzle. Their benchmark, iVISPAR, asks a model to act as an agent, planning moves across turns rather than answering once. The puzzle appears in three forms: three-dimensional views, flat two-dimensional views, and plain text.

A broad set of leading open and commercial models was compared against optimal solutions and a human baseline. Models handled the flat version best and stumbled on the three-dimensional and text forms. All fell short of people, especially on tangled configurations. The authors say the gap exposes how weakly today's models align what they see with where things are.

Catalog record

CREATORJulius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03214

Browse the full library · About the Tower of Babel