SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models
Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl, Bartosz Zieliński, Marcin Przewięźlikowski · 2026
Read SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models by Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl, Bartosz Zieliński, Marcin Przewięźlikowski (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11729.txt.
Introduction
Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl, Bartosz Zieliński, and Marcin Przewięźlikowski built a test for space sense in large vision models. Models such as DINO and CLIP read the sense of an image well. Yet they are weak at space tasks, such as judging depth or place, which robots need.
Their test, named SpaRRTa in the work, is made from fake scenes built by code. It lets them check space skills in a clean, set way. Recent work adds 3D tasks to model training, so a fair test is now key. The test can show where a model still fails at space thought.
Catalog record
| CREATOR | Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl, Bartosz Zieliński, Marcin Przewięźlikowski |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-11729
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SpaRRTa: A Synthetic Benchmark for Evaluating Spatial Intelligence in Visual Foundation Models Turhan Can Kargin, Wojciech Jasiński, Adam Pardyl, Bartosz Zieliński, Marcin Przewięźlikowski Abstract Visual Foundation Models (VFMs), such as DINO and CLIP, excel in semantic understanding of images but exhibit limited spatial reasoning capabilities, which limits their applicability to embodied systems. As a result, recent work incorporates some 3D tasks (such as depth estimation) into VFM training. However, VFM performance remains inconsistent across other spatial tasks, raising the question of whether these models truly have spatial awareness or overfit to specific 3D objectives. To address this question, we introduce the Spatial Relation Recognition Task (SpaRRTa) benchmark, which evaluates the ability of VFMs to identify relative positions of objects in the image. Unlike traditional 3D objectives that focus on precise metric prediction (e.g., surface normal estimation), SpaRRTa probes a fundamental capability underpinning more advanced forms of human-like spatial understanding. SpaRRTa generates an arbitrary number of photorealistic images with diverse scenes and fully controllable object arrangements, along with freely accessible spatial annotations. Evaluating a range of state-of-the-art VFMs, we reveal significant disparities between their spatial reasoning abilities. Through our analysis, we provide insights into the mechanisms that support or hinder spatial awareness in modern VFMs. We hope that SpaRRTa will serve as a useful tool for guiding the development of future spatially aware visual models. keywords Visual Foundation Models, 3D awareness, Synthetic data 1 Introduction Visual Foundation Models (VFMs) such as DINO 12 ; 44 ; 45 and CLIP 50 demonstrate remarkable performance in semantic visual understanding. Hence, they are widely used for semantic perception tasks including image classification 19 , object recognition 35 , multimodal learning 3 ; 33 , and scene understanding, with applications in domains such as retail 69 and medical imaging 70 . As these models move beyond static perception and are increasingly deployed in embodied and interactive settings, their role expands from recognizing visual content to supporting downstream decision-making and interaction 75 ; 6 . Figure 1 : Effective spatial reasoning requires awareness of spatial relations between visible objects. To analyze such spatial awareness of Visual Foundation Models (VFMs), we introduce Spatial Relation Recognition Task (SpaRRTa). We generate images showing different spatial layouts of several objects, encode them with VFMs, and probe whether their latent representations reliably encode the information about the spatial relations between the objects from a selected perspective. However, effective deployment of VFMs in physical environments requires more than just semantic recognition. To act and navigate, embodied agents must be aware of the spatial properties of their surroundings, and reason about three-dimensional spatial relations between objects, such as their relative position and perspective (e.g., the truck is to the left of the tree from the human’s point of view ) 18 ; 36 . As a result, recent years have seen an increased interest in evaluating VFMs in terms of their performance in tasks that require spatial awareness, such as depth prediction, camera pose estimation, and surface normal estimation 5 ; 14 . Moreover, recent VFM training objectives increasingly incorporate geometric cues 78 ; 79 ; 81 ; 76 . While this has yielded improvements in tasks most related to cues used in pretraining, the overall improvements in 3D perception remain inconsistent 5 ; 14 . This raises a fundamental question: do contemporary VFMs acquire a general notion of spatial awareness, or do they instead overfit to specific, quantifiable geometric objectives without capturing abstractions that underpin human spatial understanding? To address this question, we introduce Spatial Relation Recognition Task (SpaRRTa), a benchmark designed to evaluate abstract spatial awareness in visual representations. Rather than focusing on precise 3D predictions such as distances or surface normals, SpaRRTa targets relational, human-like spatial concepts, including relative position between objects (see Figure 1 ). At a conceptual level, identifying relative object positions is a fundamental capability underlying more advanced forms of spatial reasoning 32 ; 31 ; 11 , providing a basis for detailed use cases like navigation, planning, and object manipulation. In practice, SpaRRTa evaluates the efficacy of decoding spatial relation information of given objects from frozen VFM latent representations. By decoupling relational spatial reasoning from precise geometric predictions, SpaRRTa provides a principled way to assess whether VFMs represent transferable spatial structure rather than task-specific geometric shortcuts. SpaRRTa is built with Unreal Engine 5 21 , enabling the generation of photorealistic scenes with a wide range of object categories, layouts, and background contexts (see Figure 2 ). This synthetic setup provides full control over object placement and precise annotation of spatial relations, allowing evaluation at a scale and diversity that would be prohibitively expensive to obtain from real-world data. By systematically varying object configurations and viewpoints across diverse environments, SpaRRTa enables a controlled and robust evaluation of spatial relation understanding in realistic visual settings, remaining in-distribution for common VFMs trained on natural images. Figure 2 : SpaRRTa evaluates the efficacy of encoding the spatial relations between objects in the image by VFM, either from the camera ( Ego centric) or an arbitrary object’s ( Allo centric) perspective (see the right-most image). We use Unreal Engine 5 to produce photorealistic test images that are in-distribution for contemporary VFMs, using a variety of scene environments and object assets. Using SpaRRTa, we evaluate a broad set of VFMs trained under different supervision regimes and analyze how they encode spatial relations. Our results show that self-supervised VFMs consistently outperform supervised and vision–language models, yet even the strongest models exhibit limitations in perspective-dependent and cluttered settings. By comparing different probing mechanisms, we further reveal that spatial information is primarily stored at the patch level and is largely obscured by global pooling, highlighting the importance of structured aggregation for uncovering spatial knowledge. We hope that SpaRRTa will serve as a useful diagnostic tool for studying spatial representations and for guiding the development of future models with stronger spatial awareness, which is essential for embodied and interactive applications. Our contributions can be summarized as follows: • The Tool: We introduce SpaRRTa, a photorealistic, synthetically generated evaluation environment built on Unreal Engine 5 for probing spatial reasoning in VFMs. • The Benchmark: We define two standardized challenge sets, SpaRRTa -ego (camera-centric) and SpaRRTa -allo (allocentric/perspective-taking), designed to disentangle spatial logic from semantic recognition. • The Insights: We benchmark varying families of VFMs (MIM, Joint-Embedding, Supervised, Vision-Language) and characterize which models provide spatial information in their representations and how these representations are structured. 2 Related Work Spatial awareness in visual representation learning. Mainstream self-supervised vision transformers: DINO 12 ; 44 ; 45 , MAE 28 , I-JEPA 1 are trained almost exclusively on 2D imagery. They achieve high accuracy on high-level perception tasks, but they can lag behind on depth and multiview geometry when probed or fine-tuned 5 ; 14 . To bridge this gap, several recent variants inject explicit 3D cues during pre-training: MultiMAE 2 , and EmbodiedMAE 20 reconstruct heterogeneous RGB, depth, and semantic segmentation maps; CroCo enforces cross-view completion 78 ; 79 ; SPA 81 trains with differentiable novel-view rendering; Finally, V-JEPA 6 learns to predict future image representations based on conditioned on masked video frames. In contrast to these self-supervised approaches, VGGT 76 adopts a fully supervised paradigm by training a large-scale