DSI-Bench: A Benchmark for Dynamic Spatial Intelligence

​Ziang Zhang, Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, ​ Minjie Hong, Zhou Zhao, Zhejiang University; Alibaba Group; Shanghai AI Lab; https://dsibench.github.io · 2025

Read DSI-Bench: A Benchmark for Dynamic Spatial Intelligence by ​Ziang Zhang, Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, ​ Minjie Hong, Zhou Zhao, Zhejiang University; Alibaba Group; Shanghai AI Lab; https://dsibench.github.io (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18873.txt.

Introduction

Ziang Zhang and his seven co-authors benchmark how well vision models understand scenes where both the camera and the objects are moving. Static and two-dimensional skills do not guarantee a model can track dynamic three-dimensional relationships. Their benchmark, DSI-Bench, contains nearly one thousand dynamic videos and more than one thousand seven hundred hand-annotated questions, organized around nine separated motion patterns for observers and objects.

Symmetric designs in space and time reduce biases and let evaluators isolate reasoning about self-motion from reasoning about object motion. Testing fourteen vision-language and specialist models exposed recurring failures: models confuse the observer's motion with the object's, fall back on semantic expectations, and misjudge relative relationships in moving scenes. The authors offer the benchmark as a diagnostic for building models with genuine dynamic spatial intelligence.

Catalog record

CREATOR​Ziang Zhang, Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, ​ Minjie Hong, Zhou Zhao, Zhejiang University; Alibaba Group; Shanghai AI Lab; https://dsibench.github.io
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-18873

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DSI-Bench: A Benchmark for Dynamic Spatial Intelligence ​Ziang Zhang, Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, ​ Minjie Hong, Zhou Zhao, Zhejiang University; Alibaba Group; Shanghai AI Lab; https://dsibench.github.io Abstract Reasoning about dynamic spatial relationships is essential, as both observers and objects often move simultaneously. Although vision-language models (VLMs) and visual expertise models excel in 2D tasks and static scenarios, their ability to fully understand dynamic 3D scenarios remains limited. We introduce D ynamic S patial I ntelligence and propose DSI-Bench, a benchmark with nearly 1,000 dynamic videos and over 1,700 manually annotated questions covering nine decoupled motion patterns of observers and objects. Spatially and temporally symmetric designs reduce biases and enable systematic evaluation of models’ reasoning about self-motion and object motion. Our evaluation of 14 VLMs and expert models reveals key limitations: models often conflate observer and object motion, exhibit semantic biases, and fail to accurately infer relative relationships in dynamic scenarios. Our DSI-Bench provides valuable findings and insights about the future development of general and expertise models with dynamic spatial intelligence. Figure 1: Dynamic Spatial Intelligence. Unlike static settings, dynamic scenarios involve evolving spatial relationships among the observer, observed objects, and the environment. Humans can intuitively perceive such changes in spatial relations, whereas Vision-Language Models (VLMs) often exhibit hallucinations and biases in dynamic spatial reasoning due to semantic misleadness and coupled motion understanding. 1 Introduction When early humans chased prey across open landscapes, they instinctively adjusted their running paths in response to the animals’ movements, helping them close the distance. Likewise, modern drivers adapt their direction and speed in response to the movements of other vehicles and pedestrians on the road. We live in a world where both the observer and the objects being observed are constantly in motion. For humans, understanding how our own position shifts and how other objects move in 3D space simultaneously is an intuitive ability rooted in basic spatial intelligence. In recent years, significant progress has been made in visual perception and understanding. Vision-language models (VLMs), which deeply integrate visual and linguistic modalities, have demonstrated remarkable cross-modal semantic alignment and reasoning abilities, achieving impressive results in open-domain perception and dialogue. State-of-the-art VLMs OpenAI 2025a ; OpenAI 2025b ; Comanici et al. 2025 ; Wang et al. 2025b ; Bai et al. 2025 ; ByteDance 2025 have achieved leading performance on 2D computer vision tasks such as detection and visual question answering, while also demonstrating a certain degree of understanding of temporal actions and static spatial relations. With the development of visual foundation models such as DINOv2 Oquab et al. 2023 , an increasing number of studies introduce expertise models that exhibit strong zero-shot capabilities in spatial perception. These models demonstrate superior performance and robustness in accomplishing specific tasks and perceiving task-relevant information. For example, Dust3R Wang et al. 2024a , VGGT Wang et al. 2025a enable robust estimation of camera poses and 3D motion trajectories, while the CoTracker Karaev et al. 2024 and SpatialTracker Xiao et al. 2025 families support pixel-level motion understanding, enabling highly accurate tracking of object and region trajectories in videos. These advances lay a critical foundation for building multimodal systems with spatial awareness and dynamic understanding. However, current methods are still largely limited to scenarios where either the observer or the objects remain static. In more realistic and practically relevant dynamic environments, where both the observer and the objects are in motion, their performance and adaptability have not yet been systematically explored. To fill this gap, we introduce the problem of D ynamic S patial I ntelligence, and conduct a thorough, decoupled analysis of the ability of state-of-the-art methods to understand different forms of motion. Having dynamic spatial intelligence entails the ability to decouple reasoning about both the agent’s own motion and that of other objects within a scene. This requires models to exhibit a range of sophisticated capabilities, including temporal reasoning, capturing spatially consistent elements during dynamic changes, and understanding spatial relationships, which further encompasses spatiotemporal understanding and reasoning. To this end, we propose DSI-bench, a VQA benchmark dedicated to Dynamic Spatial Intelligence. DSI-Bench comprises nearly 1000 dynamic scene videos which collected, cleaned, and clipped from diverse sources, covering 5 decoupled motion patterns of observers and objects. Focusing on the relationships among the observer, objects, and the scene, we designed 6 question types with over 1,700 VQA questions, all of which are manually annotated and verified. To mitigate biases in 3D space—such as left/right biases and correlations between object orientation and motion, we construct spatially and temporally symmetric versions of the same questions. Our evaluation of 14 widely used VLMs and expertise models on DSI-Bench yields several key findings. By analyzing the models’ self-explanations and visualization outputs, we observe that: 1) VLMs tend to conflate observer and the observed object’s motion rather than treating them as distinct. 2) Semantic biases associated with the observed object can distort visual perception, leading to hallucinations. 3) Classical 3D constraints fail to consistently characterize relative pose relationships in continuous dynamic scene videos. Our contribution can be summarized as: • We introduce a more realistic and practically valuable Dynamic Spatial Intelligence task and propose DSI-Bench, a benchmark specifically designed for the systematic evaluation of dynamic spatial reasoning. • We evaluate widely used general visual-language models and expertise visual foundation models on DSI-Bench and, through spatially and temporally symmetric sample designs, analyze their biases and hallucination tendencies in dynamic scenarios. • Finally, by examining failure cases of state-of-the-art spatial foundation models, we highlight the limitations of classical 3D constraints in dynamic spatial perception. 2 Related works 2.1 Large Vision-Language Model Beyond single-frame understanding, VLMs OpenAI 2025a ; OpenAI 2025b ; ByteDance 2025 ; Comanici et al. 2025 have advanced to capture both temporal information and static spatial relations in videos, achieving strong performance across tasks such as detection, open dialogue, and multimodal reasoning. For instance, the Qwen2.5-VL Bai et al. 2025 series demonstrates strong performance in long video understanding by incorporating absolute timestamp encoding. Its innovative visual encoder further allows dynamic adjustment of frame rates and video resolution, enhancing applicability across diverse video scenarios. Similarly, the InternVL-3.5 Wang et al. 2025b series employs a visual resolution router that dynamically focuses on fine-grained video details, significantly improving video reasoning efficiency while maintaining overall performance. 2.2 Benchmarks for Spatial Intelligence To evaluate and investigate the spatial intelligence of VLMs, a series of benchmarks have been proposed. VSI-Bench Yang et al. 2025a systematically measures model performance across 8 categories of spatial reasoning tasks and examines how cognitive maps may facilitate VLM inference. MMSI-Bench Yang et al. 2025b design problems that cannot be solved from a single viewpoint, highlighting the importance of multi-image dependency in spatial reasoning. VLM4D Zhou et al. 2025 extends this line of work by treating video as a four-dimensional modality, thereby exploring the models’ capacity to capture spatiotemporal correlations. Although these benchmarks provide a foundation for assessing spatial intelligence, they overlook issues of multimodal hallucination and bias in VLMs. Recently, 3DSR-Bench Ma et al. 2024 addressed this issue by introducing the Evalflip strategy and designing queries from uncommon perspectives to analyze hallucination in more diverse settings. Similarly, Ori-Bench Wang et al. 2024c revealed that VLMs are prone to orientation hallucinations when

Read the complete text