GroundSet: A Cadastral-Grounded Dataset for Spatial Understanding with Vector Data

Roger Ferrod, Maël Lecene, Krishna Sapkota, George Leifman, Vered Silverman, Genady Beryozkin, Sylvain Lobry · 2026

Read GroundSet: A Cadastral-Grounded Dataset for Spatial Understanding with Vector Data by Roger Ferrod, Maël Lecene, Krishna Sapkota, George Leifman, Vered Silverman, Genady Beryozkin, Sylvain Lobry (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14609.txt.

Introduction

Ferrod and co-authors ground a spatial reasoning dataset in cadastral vector records. Earth observation models still falter at fine spatial questions. GroundSet ties aerial images to checkable land registry data. The set holds 3.8 million labeled objects in 510,000 high-resolution images. Labels span 135 fine semantic classes. Seven spatial reasoning tasks form the test bench. A standard LLaVA model sets the baseline. Strong supervision lets plain models master fine grounding, they show, while zero-shot rivals lag.

Catalog record

CREATORRoger Ferrod, Maël Lecene, Krishna Sapkota, George Leifman, Vered Silverman, Genady Beryozkin, Sylvain Lobry
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51714
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14609

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GroundSet: A Cadastral-Grounded Dataset for Spatial Understanding with Vector Data Roger Ferrod, Maël Lecene, Krishna Sapkota, George Leifman, Vered Silverman, Genady Beryozkin, Sylvain Lobry Abstract Precise spatial understanding in Earth Observation is essential for translating raw aerial imagery into actionable insights for critical applications like urban planning, environmental monitoring and disaster management. However, Multimodal Large Language Models exhibit critical deficiencies in fine-grained spatial understanding within Remote Sensing, primarily due to a reliance on limited or repurposed legacy datasets. To bridge this gap, we introduce a large-scale dataset grounded in verifiable cadastral vector data, comprising 3.8 million annotated objects across 510k high-resolution images with 135 granular semantic categories. We validate this resource through a comprehensive instruction-tuning benchmark spanning seven spatial reasoning tasks. Our evaluation establishes a robust baseline using a standard LLaVA architecture. We show that while current RS-specialized and commercial models (e.g., Gemini) struggle in zero-shot settings, high-fidelity supervision effectively bridges this gap, enabling standard architectures to master fine-grained spatial grounding without complex architectural modifications. 1 Introduction Multimodal models have become ubiquitous, embedding themselves into any sort of modern application. In particular, the advent of Multimodal Large Language Models (MLLMs) [ 26 , 11 , 19 ] has fundamentally reshaped the AI landscape. Driven by unprecedented scaling, MLLMs have demonstrated remarkable zero-shot capabilities and broad world knowledge, enabling them to solve a diverse array of tasks with minimal supervision. Furthermore, when integrated into agentic frameworks, these models can orchestrate external tools to address complex, multi-step problems [ 10 , 15 ] . Despite these advancements, however, a critical limitation persists: current MLLMs exhibit a severe deficiency in fine-grained spatial understanding [ 5 , 1 ] . They significantly struggle with fundamental vision tasks – such as precise object detection and segmentation – which are prerequisites for deployment in high-stakes domains like Remote Sensing (RS), where sub-meter precision is critical and current MLLM inaccuracies actively prevent operational use. The adaptation of MLLMs to Earth Observation (EO) is compelling yet practically stalled by the scarcity of high-quality spatial annotations. While unlabelled satellite imagery is abundant, dense and accurate annotations remain a bottleneck: human labeling is cost-prohibitive and lacks the scalability required to support modern machine learning research. Consequently, recent literature (e.g., [ 14 , 20 , 17 , 22 ] ) has increasingly relied on repurposing legacy datasets originally designed for closed-set object detection (e.g., DOTA [ 24 ] , DIOR [ 16 ] ). These works typically transform older data by synthetically generating question-answer pairs to fit modern instruction-tuning formats. However, this approach is inherently bounded by the quality, limited semantic scope and nature of the underlying source data, failing to address the fundamental issues of scalability and the acquisition of novel domain knowledge. Figure 1 : Overview of our dataset. We release both the raw vector data and a derived instruction-tuning dataset. The cadastral vector layers (e.g., buildings, vegetation, roads) serve as the ground truth and are automatically transformed into instructions for downstream tasks including scene captioning, object detection, semantic segmentation, localized classification, Visual Question Answering (VQA) and Referring Expression Comprehension (REC). In this work, we propose a novel, large-scale dataset comprising 3.8M annotated objects across 510k high-resolution images, specifically tailored for spatial understanding in EO. The primary innovation lies in our data curation methodology: rather than relying on legacy sets or noisy crowd-sourcing, we leverage cadastral vector data as our source of ground truth. This approach ensures the acquisition of high-fidelity geometric data that is manually curated and legally verified by national agencies, effectively solving the scalability-quality trade-off. Moreover, this rigorous sourcing allows us to introduce novel imagery while offering a semantic depth previously unavailable in the field. Whereas existing datasets are typically restricted to a few dozen generic classes (e.g., [ 17 , 14 , 20 ] ), we provide 135 highly specific semantic labels. These include fine-grained categories such as power plants, heritage sites, specific crop types and diverse civil and industrial infrastructures. By exposing models to this level of administrative detail, we provide a challenging new benchmark for fine-grained visual recognition (Figure 1 ). Furthermore, we posit that this rigorous geometric foundation can support a broad spectrum of research applications beyond the tasks proposed in this paper (e.g., [ 3 , 4 , 2 ] ) To empirically demonstrate the utility of this vector data, we establish a comprehensive instruction fine-tuning benchmark comprising seven spatial reasoning tasks as a primary use case. Our experiments highlight a critical gap in the field: existing RS-specialized models (e.g., GeoChat [ 14 ] , SkySenseGPT [ 20 ] ) exhibit severe degradation in zero-shot generalization, sometimes underperforming relative to their base architectures. In contrast, we show that high-fidelity data enables standard architectures to master these tasks without complex architectural modifications. Our fine-tuned baseline significantly improves upon the zero-shot performance of specialized spatial models (e.g., Ferret [ 27 ] ) and commercial baselines. This performance gap serves as quantitative proof of the unique domain knowledge contained in our cadastral annotations—knowledge that is currently absent from web-scale pre-training. The main contributions of this work are: 1. The Data: A novel large-scale EO dataset distinguished by its verified cadastral origin, offering unprecedented semantic richness (135 classes) and density (3.8M objects). 2. The Benchmark: A comprehensive instruction fine-tuning suite comprising 7 spatial understanding tasks, derived from the vector data, to demonstrate its applicability to modern VLM training. 3. The Baseline: A fine-tuned model serving as a robust reference for future research, demonstrating that standard VLMs can master complex grounding tasks when trained on high-density verified vector data. 2 Related Work General-Purpose Spatial VLMs Recent advances in spatial understanding rely on modular architectures like LLaVA [ 19 ] and PaliGemma [ 7 ] . To enable grounding, pioneering works such as Shikra [ 9 ] , Kosmos-2 [ 21 ] and MiniGPT-v2 [ 8 ] treat spatial coordinates as textual tokens, effectively bootstrapping spatial skills from pre-trained LLMs. Ferret [ 27 ] further refined this by introducing a hybrid sampler for fine-grained feature extraction. These findings demonstrate the feasibility of embedding visual grounding directly into a single unified model, leveraging LLM’s established reasoning and knowledge retrieval capabilities. Remote Sensing MLLMs The Remote Sensing community adapted these paradigms via "continue pre-training" strategies. Early adopters GeoChat [ 14 ] , SkySenseGPT [ 20 ] , EarthGPT [ 29 ] and RSGPT [ 13 ] , followed by recent works like GeoGround [ 30 ] and GeoPixel [ 22 ] , fine-tune generic architectures on domain corpora. While validating RS VQA feasibility, these models remain bottlenecked by the quality and scale of training data. Datasets Data remains the primary constraint. Most benchmarks (e.g., VRSBench [ 17 ] , GeoChat [ 14 ] , GeoPixel [ 22 ] , RSVG [ 28 ] ) rely on legacy closed-set datasets like DOTA [ 24 ] and DIOR [ 16 ] , which are limited to generic categories. While large-scale initiatives like SkyScript [ 23 ] and SatLas [ 6 ] rely on OpenStreetMap (OSM) for global coverage, they often trade off resolution and precision. In contrast, our work leverages high-resolution (20cm) cadastral data to provide the dense, verified vector annotations required for deep spatial understanding. 3 Data To ensure precise

Read the complete text