Can LLMs Learn to Map the World from Local Descriptions?
Sirui Xia, Aili Chen, Xintao Wang, Tinghui Zhu, Yikai Zhang, Jiangjie Chen, Yanghua Xiao · 2025
Read Can LLMs Learn to Map the World from Local Descriptions? by Sirui Xia, Aili Chen, Xintao Wang, Tinghui Zhu, Yikai Zhang, Jiangjie Chen, Yanghua Xiao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20874.txt.
Introduction
Sirui Xia and co-authors ask in a 2025 paper whether large language models can build a map of the world from local notes alone. The team tests two skills: finding a full layout from small links between places, and learning road links from travel paths to plan routes.
Tests in a made-up city show the models can guess links they were not shown. Their inner maps also line up with real place patterns. The models learn road links from path notes and plan good routes to new spots. Readers get a clear test of whether text models hold true space sense, not just word skill.
Catalog record
| CREATOR | Sirui Xia, Aili Chen, Xintao Wang, Tinghui Zhu, Yikai Zhang, Jiangjie Chen, Yanghua Xiao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20874
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can LLMs Learn to Map the World from Local Descriptions? Sirui Xia, Aili Chen, Xintao Wang, Tinghui Zhu, Yikai Zhang, Jiangjie Chen, Yanghua Xiao Abstract Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in tasks such as code and mathematics. However, their potential to internalize structured spatial knowledge remains underexplored. This study investigates whether LLMs, grounded in locally relative human observations, can construct coherent global spatial cognition by integrating fragmented relational descriptions. We focus on two core aspects of spatial cognition: spatial perception, where models infer consistent global layouts from local positional relationships, and spatial navigation, where models learn road connectivity from trajectory data and plan optimal paths between unconnected locations. Experiments conducted in a simulated urban environment demonstrate that LLMs not only generalize to unseen spatial relationships between points of interest (POIs) but also exhibit latent representations aligned with real-world spatial distributions. Furthermore, LLMs can learn road connectivity from trajectory descriptions, enabling accurate path planning and dynamic spatial awareness during navigation. 1 Introduction Figure 1: Summary of our research framework. Firsr, we construct a simulated environment and generate training data capturing relative spatial relations and shortest paths. Then, we apply continual pre-training to the LLM and evaluate its spatial cognition through explicit prediction tasks and latent representation analysis. Recent advances in large language models (LLMs) have demonstrated impressive performance across diverse tasks, including code generation, mathematical reasoning, and natural language generation ( Chen et al., 2021 ; Shao et al., 2024 ; Kojima et al., 2022 ) . LLMs are trained on vast amounts of human-generated text Achiam et al. (2023) ; Bai et al. (2023) , including structured resources such as Wikipedia and informal unstructured dialogues. Since human language inherently relies on local semantic relationships, this enables LLMs to excel at capturing these context-dependent associations. However, it remains unexplored whether they can implicitly acquire a deep, structured understanding of global information from large amounts of fragmented, localized data—and apply it to reasoning and planning tasks such as spatial reasoning, route optimization, or multi-step inference. A prime example of a domain requiring structured understanding is spatial cognition —the ability to construct coherent mental representations of physical environments. In human communication, spatial relationships are often conveyed through relational language ( e.g. , “The library is 100 meters southeast of the park”), which, though concise, encodes rich geometric information such as direction, distance, and topology. Humans seamlessly integrate such fragmented descriptions into unified mental maps, demonstrating a remarkable capacity to derive global spatial understanding from localized cues . This global cognition supports higher-level spatial reasoning, navigation, and planning. This reliance on human perspectives raises a fundamental yet underexplored question: To what extent can LLMs, grounded in locally relative human observations, develop a coherent understanding of global space? This challenge goes beyond processing coordinate data—it requires understanding of spatial language, integrate fragmented descriptions, and build consistent mental maps. The model must comprehend geometric relationships ( e.g. , direction, distance), synthesize incomplete information, and maintain logical coherence across global spatial contexts—all without relying on visual input or explicit coordinates. To explore this question, we conduct a comprehensive analysis focusing on two core aspects of spatial cognition: 1) Spatial Perception : We investigate whether LLMs can integrate fragmented relational descriptions into a cohesive mental representation of space. This evaluates the model’s ability to reconstruct static spatial understanding solely from linguistic input, without access to explicit coordinate information. 2) Spatial Navigation : We examine whether LLMs can extract topological knowledge from trajectory sequences and apply it to generate optimal paths between previously unconnected locations. This tests the model’s capacity for dynamic spatial reasoning and path planning in the absence of explicit connectivity. To enable a controlled investigation, we construct a simulated urban environment and introduce a two-stage training and analysis framework guided by two core research questions, which leverages two complementary data modalities: (1) relational spatial descriptions capturing pairwise distances and directions between points of interest (POIs); and (2) trajectory descriptions representing shortest paths across the environment. We analyze whether spatial cognition is formed and how it is expressed through three experimental paradigms: (1) Explicit spatial prediction , assessing task-level prediction; (2) Latent representation analysis , probing geometry in hidden states; and (3) Robustness evaluation , measuring stability under navigational perturbations. The key findings are as follows: • LLMs can construct global spatial cognition from local observations : LLMs demonstrate spatial perception by inferring unseen POI relationships, and spatial navigation by planning optimal paths between unconnected locations—revealing coherent global understanding emerging from fragmented linguistic input. • LLMs can develop structured, implicit spatial representations : LLMs encode absolute spatial coordinates within their latent space, aligned with real-world geometry, and dynamically track their position during navigation—indicating the emergence of implicit spatial abstraction without explicit coordinates. • LLM’s spatial navigation remains fragile under perturbation : LLMs exhibit limited robustness to path perturbations, with their recovery ability dependent on the distribution of training data, suggesting that their understanding of road spatial information is limited, lacking a continuous and precise representation. 2 Global Setup Simulation Environment. To facilitate controlled investigation and data collection, we construct a synthetic 100\times 100 grid map representing a simplified urban layout. Roads run along horizontal and vertical lines ( x=i or y=j , for 0\leq i,j\leq 100 ), with traversal weights w randomly sampled from [0.8,1.2] to simulate varying traffic conditions—higher weights indicate faster travel. We randomly place N_{POI}=1024 points of interest (POIs) on the grid, each assigned a unique identifier p_{k} ( k\in{1,2,\dots,1024} ). Each grid unit represents 1 kilometer, with the x -axis pointing east and the y -axis north. Problem Formulation. To explore whether LLMs can develop spatial cognition from natural language descriptions, we define two research tasks that capture key aspects of spatial cognition: (1) Global Spatial Perception — Can the model build a globally consistent understanding of spatial layouts based on local, relational language descriptions? (2) Dynamic Spatial Planning and Navigation — Can the model infer the structure of an underlying road network from local shortest-path descriptions, and use this knowledge to dynamically plan routes between previously unseen pairs of POIs? Data. (1) The Relational Spatial Dataset is used in the first stage to train the model to infer global spatial structure from local pairwise relations. Each sample computes the Euclidean distance d(p_{i},p_{j}) and azimuth \alpha(p_{i},p_{j})\in[-180^{\circ},180^{\circ}] between POIs (p_{i},p_{j}) , expressed through templated natural language ( e.g. , “The distance from p_{i} to p_{j} is 2.5 km, and the azimuth is 135 degrees.”). To enhance linguistic diversity, we vary the surface realizations of each template. (2) The Trajectory Dataset is used in the second stage to train dynamic spatial navigation. The road network is modeled as a weighted graph, and shortest paths between POIs are computed using Dijkstra’s algorithm. Each path is translated into multi-step natural language instructions Dijkstra (1959) ( e.g. , “Start at p_{i} , go east on r_{3} for 3 km, then north on r_{8} for 2 km to reach p_{j} ”), capturing both directional and topological structure. These datasets are introduced through continuous pre-training in two stages: first, to build coherent spatial representations from relational cues; and second, to acquire navigation capabilities based on learned connectivity. Model and Two-Stage CPT Training. We adopt