MapTrace: Scalable Data Generation for Route Tracing on Maps
Artemis Panagopoulou, Aveek Purohit, Achin Kulshrestha, Soroosh Yazdani, Mohit Goyal □ \Box Google XR ∇ \nabla University of PennsylvaniaProject Page: https://artemisp.github.io/maptrace · 2025
Read MapTrace: Scalable Data Generation for Route Tracing on Maps by Artemis Panagopoulou, Aveek Purohit, Achin Kulshrestha, Soroosh Yazdani, Mohit Goyal □ \Box Google XR ∇ \nabla University of PennsylvaniaProject Page: https://artemisp.github.io/maptrace (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-19609.txt.
Introduction
Route tracing on maps turns out to be a weak spot for multimodal language models, the systems that reason over images and text together. Artemis Panagopoulou and four co-authors note that models often break basic path rules, partly because pixel-accurate route annotations are costly to collect by hand. Their answer, MapTrace, generates that training data synthetically instead.
MapTrace builds synthetic map images and parses them at pixel level to produce exact path labels automatically. From it the team assembled 23 thousand path samples spread over 4 thousand maps, then fine-tuned open and proprietary models with the data. On the MapBench benchmark, success rates rose by up to 6.4 points and path-tracing error fell, showing spatial skills can be taught with synthetic supervision.
Catalog record
| CREATOR | Artemis Panagopoulou, Aveek Purohit, Achin Kulshrestha, Soroosh Yazdani, Mohit Goyal □ \Box Google XR ∇ \nabla University of PennsylvaniaProject Page: https://artemisp.github.io/maptrace |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-19609
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MapTrace: Scalable Data Generation for Route Tracing on Maps Artemis Panagopoulou, Aveek Purohit, Achin Kulshrestha, Soroosh Yazdani, Mohit Goyal □ \Box Google XR ∇ \nabla University of PennsylvaniaProject Page: https://artemisp.github.io/maptrace Abstract While Multimodal Large Language Models have achieved human-like performance on many visual and textual reasoning tasks, their proficiency in fine-grained spatial understanding, such as route tracing on maps remains limited. Unlike humans, who can quickly learn to parse and navigate maps, current models often fail to respect fundamental path constraints, in part due to the prohibitive cost and difficulty of collecting large-scale, pixel-accurate path annotations. To address this, we introduce a scalable synthetic data generation pipeline that leverages synthetic map images and pixel-level parsing to automatically produce precise annotations for this challenging task. Using this pipeline, we construct a fine-tuning dataset of 23k path samples across 4k maps, enabling models to acquire more human-like spatial capabilities. Using this dataset, we fine-tune both open-source and proprietary MLLMs. Results on MapBench show that finetuning substantially improves robustness, raising success rates by up to 6.4 points, while also reducing path-tracing error (NDTW). These gains highlight that fine-grained spatial reasoning, absent in pretrained models, can be explicitly taught with synthetic supervision. 1 Introduction Multimodal Large Language Models (MLLMs) have achieved impressive progress in joint visual and textual understanding, demonstrating strong capabilities across recognition, reasoning, and instruction following tasks ( 25 ; 15 ; 1 ; 26 ) . Despite these advances, their spatial reasoning ability remains underdeveloped. While models excel at semantic tasks such as identifying objects and describing visual scenes, they perform poorly on tasks that require reasoning about spatial, geometric, and topological concepts ( 12 ; 21 ) . Figure 1: MapTrace: Given a start and end location, the model outputs a valid path that respects map constraints In contrast to humans, who quickly learn to parse and use maps, MLLMs often fail to produce valid routes, revealing a persistent limitation in their current training paradigms. To address this deficiency, we introduce MapTrace, a new task that requires a model to generate a precise sequence of coordinates forming a valid path between two points on a map (Figure 1 ). Unlike high-level navigation tasks that rely on language instructions ( 31 ; 20 ) , MapTrace isolates a model’s ability to reason directly about connectivity and traversability at the pixel level, grounding model’s understanding of maps. A major challenge towards effectively teaching MLLMs to trace routes on maps is the prohibitive cost of creating large-scale, pixel-accurate path annotations and the difficulty in obtaining usage permission of layout maps from commercial sources. To overcome this, we design a scalable synthetic data generation pipeline. The pipeline generates diverse map layouts, automatically constructs traversable path masks, converts them into graph structures, and produces optimal shortest-path annotations. This process yields 23k high-quality path samples across 4k maps, providing explicit supervision for connectivity, traversability, and pathfinding. Fine-tuning MLLMs on this dataset results in up to a 30% relative improvement on MapBench ( 31 ) . Our contributions are threefold: (1) We introduce MapTrace , the first task and dataset explicitly designed to probe coordinate-level spatial reasoning in MLLMs by requiring models to generate pixel-accurate paths on maps. (2) We present a scalable synthetic data generation pipeline that produces 23k path annotations over 4k diverse maps, leveraging LLM-based critics to ensure high fidelity. (3) We provide empirical insight into the spatial reasoning gap in current MLLMs, showing that fine-tuning on MapTrace substantially improves robustness (raising success rates by up to 6.4 points) and reduces path-tracing error (NDTW) in most map domains. 2 Related Work Spatial Reasoning. MLLMs have achieved strong results across recognition, reasoning, and instruction-following tasks ( 25 ; 15 ; 1 ; 26 ) , yet many studies highlight persistent weaknesses in spatial reasoning. Benchmarks targeting geometric relations, relative positioning, and layout understanding ( 29 ; 32 ; 9 ) consistently show that models falter when asked to reason about structure rather than appearance. For example, even when models can identify objects correctly, they struggle to judge whether one object is inside, adjacent, or aligned with another. These limitations suggest that current pre-training provides limited supervision for spatial structure ( 12 ) . Our work focuses on one fundamental instance of this challenge, path-tracing on maps, serving as a probe of fine-grained spatial reasoning. Map-Based Reasoning. Maps provide a natural testbed for spatial reasoning because they require both visual parsing and structured interpretation. Prior work has explored constrained domains such as floor plans with explicit labels ( 6 ; 11 ) , standardized geographic maps ( 34 ; 20 ) , and transit networks ( 8 ) . These tasks take the form of either question answering about landmarks, or generating high-level navigation instructions ( 13 ; 8 ) , with a smaller body of work attempting explicit path generation but only under simplified layouts ( 6 ; 11 ) . More complex commercial maps (e.g., malls, zoos, museums) have received less attention, with exceptions such as 3 , who evaluated MLLMs on three maps and found frequent failures in spatial grounding. MapBench ( 31 ) introduced a broader benchmark on commercial maps, measuring sequences of correct landmarks as a proxy for navigation. Our work complements this line by shifting evaluation to the coordinate level, where path quality is judged by geometric accuracy and adherence to map constraints. Synthetic Data Generation. The scarcity of large-scale, pixel-accurate path annotations poses a major challenge for training models on map reasoning. To address similar bottlenecks in other domains, researchers have increasingly turned to synthetic data generation, where LLMs and MLLMs act both as data producers ( 16 ; 2 ) and evaluators or “judges” ( 17 ) . This paradigm has been used to scale instruction-following datasets ( 14 ) , align models with human preferences ( 7 ) , and filter low-quality samples ( 10 ) . Synthetic data has also proven valuable for tasks that are expensive or ambiguous to annotate, such as compositional reasoning ( 19 ) . We extend these ideas to the spatial domain: our pipeline generates diverse synthetic maps, derives shortest-path annotations automatically, and uses MLLM critics for filtering low quality samples. This pipeline allows us to efficiently produce high-quality data for path-tracing, a capability that has been prohibitively expensive to collect data at scale due to licencing restrictions and high annotation costs. 3 Novel Task: MapTrace We introduce MapTrace , a task designed to evaluate the fine-grained spatial reasoning abilities of MLLMs. Unlike prior work that focuses on simplified layouts ( 11 ) or high-level instruction following ( 31 ) , MapTrace requires models to generate pixel-accurate, traversable paths on commercial wayfinding maps (e.g., malls, zoos, museums). Figure 1 illustrates the setup: given start and end locations, the model must output a valid route that respects the map’s connectivity constraints. Figure 2: MapTrace Synthetic data generation pipeline. A LLM generates various map descriptions, which are rendered into images by a text-to-image model. Candidate path masks are extracted via dominant-color selection and filtered by a Mask Critic. Valid masks are converted into a pixel-graph to compute shortest-path candidates, which are then judged by a Path Critic for quality and traversability. 3.1 Task