From Perception to Action: Spatial AI Agents and World Models Abstract While large language models have become the prevailing approach for agentic reasoning and planning ( Brown et al. 2020 ; OpenAI 2023 ; Touvron et al. 2023 ; Team and Google 2023 ; Anthropic 2024 ; Dubey et al. 2024 ; OpenAI 2023 ; Anil et al. 2023 ; Chowdhery et al. 2022 ; Jiang et al. 2023 ; Abdin et al. 2024 ; Grattafiori et al. 2024 ; Devlin et al. 2019 ) , their success in symbolic domains does not readily translate to the physical world. Spatial intelligence, the ability to perceive 3D structure, reason about object relationships, and act under physical constraints, is an orthogonal capability that proves important for embodied agents ( Chen et al. 2024a ; Yang et al. 2025 ; Duan et al. 2022 ; Amin and Kiela 2024 ; Cheng et al. 2025 ; Guo et al. 2024c ; Liu et al. 2024c ) . Existing surveys address either agentic architectures or spatial domains in isolation. None provide a unified framework connecting these complementary capabilities. This paper bridges that gap. Through a thorough review of over 2,000 papers, citing 742 works from top-tier venues, we introduce a unified three-axis taxonomy connecting agentic capabilities with spatial tasks across scales. Crucially, we distinguish spatial grounding (metric understanding of geometry and physics) from symbolic grounding (associating images with text), arguing that perception alone does not confer agency. Our analysis reveals three key findings mapped to these axes: (1) hierarchical memory systems (Capability axis) are important for long-horizon spatial tasks ( Packer et al. 2023 ; Banino et al. 2018 ; Xu et al. 2025 ; Zhang et al. 2025a ; Blundell et al. 2016 ; Pritzel et al. 2017 ) . (2) GNN-LLM integration (Task axis) is a promising approach for structured spatial reasoning ( Jin et al. 2023 ; Chen et al. 2024e ; Chen et al. 2024c ; Chai et al. 2023 ; Shehzad et al. 2024 ; Fatemi et al. 2023 ; Fatemi et al. 2024 ) . (3) World models (Scale axis) are essential for safe deployment across micro-to-macro spatial scales ( Hafner et al. 2023 ; Bruce et al. 2024b ; Ha and Schmidhuber 2018 ; Feng et al. 2025 ; Ding et al. 2024 ; Brooks et al. 2024 ; Hafner et al. 2020 ; Hafner et al. 2021 ; Schrittwieser et al. 2020 ) . We conclude by identifying six grand challenges and outlining directions for future research, including the need for unified evaluation frameworks to standardize cross-domain assessment. This taxonomy provides a foundation for unifying fragmented research efforts and enabling the next generation of spatially-aware autonomous systems in robotics, autonomous vehicles, and geospatial intelligence. 1 Introduction The pursuit of artificial general intelligence increasingly centers on creating agents that can perceive, reason about, and act within physical environments ( Brooks 1991 ; Russell and Norvig 2010 ) . Large language models excel at symbolic reasoning and planning ( Brown et al. 2020 ; OpenAI 2023 ) , yet they fail consistently in spatial contexts: navigation agents hallucinate non-existent paths, manipulation planners propose physically infeasible grasps, and embodied systems misjudge object distances by orders of magnitude ( Chen et al. 2024a ; Yang et al. 2025 ) . These failures stem from a fundamental gap that is distinct from the multimodal grounding problem: while vision-language models can associate images with text (symbolic grounding), spatial intelligence requires metric understanding of geometry, physics, and action consequences (spatial grounding). LLMs lack this latter capability, possessing no internal model of 3D structure, physical dynamics, or geometric constraints. Multimodal foundation models have accelerated visual understanding ( Radford et al. 2021 ; Liu et al. 2023b ; OpenAI 2023 ) , yet perception alone does not confer the ability to act: a model that can describe a scene in detail may still be unable to navigate through it or manipulate objects within it. Translating perception into effective, controllable spatial action remains the critical bottleneck for embodied AI ( Ahn et al. 2022 ; Brohan et al. 2023 ; Kawaharazuka et al. 2025 ) . This observation motivates our focus on agentic systems that close the loop from perception to action. We define Agentic AI as systems exhibiting goal-directed behavior through autonomous decision-making, characterized by three core capabilities that form our taxonomy’s Capability axis: memory for experience accumulation, planning (including self-reflection as meta-level planning for continuous improvement), and tool use for capability extension ( Wang et al. 2024b ; Xi et al. 2023 ; Yao et al. 2023b ; Shinn et al. 2023b ) . These agents operate through iterative cycles of perception, reasoning, action, and feedback ( Yao et al. 2023b ; Shinn et al. 2023b ) . Complementarily, Spatial Intelligence encompasses the ability to perceive 3D structure, reason about object relationships, navigate environments, and manipulate physical objects ( Chen et al. 2024a ; Marr 1982 ; Newcombe 2010 ) . Critically, spatial tasks span three scales that form our taxonomy’s Scale axis: micro-spatial (centimeter-scale manipulation and grasping), meso-spatial (meter-scale room navigation and scene understanding), and macro-spatial (kilometer-scale urban planning and geospatial analysis) ( Battaglia et al. 2018 ; Kipf and Welling 2017 ) . This scale distinction is not merely taxonomic but causal: agents trained at one scale fail when deployed at another because the relevant features, physics, and action spaces differ qualitatively. A grasping policy optimized for centimeter precision cannot plan city-scale routes, and a traffic predictor has no representation of contact forces. Scale mismatch is a primary source of transfer failure in spatial AI. The convergence of these domains is essential for real-world AI applications, which we organize by the Task axis of our taxonomy: Navigation (meso-to-macro scale): Autonomous vehicles must perceive dynamic environments and plan safe trajectories ( Hu et al. 2023b ; Caesar et al. 2020 ; Waymo 2023 ; Geiger et al. 2012 ; Chen et al. 2024d ; Waymo 2024 ; Tian et al. 2024 ) . Indoor robots require vision-language navigation through complex building layouts ( Anderson et al. 2018c ; Krantz et al. 2020 ) . Manipulation (micro-to-meso scale): Robotic assistants require understanding of object affordances and spatial relationships for grasping, placement, and tool use ( Brohan et al. 2023 ; Ahn et al. 2022 ; Team et al. 2024 ; Kim et al. 2024 ; Driess et al. 2023 ; Black et al. 2024 ; Collaboration 2023 ) . Macro-scale reasoning : Urban computing systems must model complex spatio-temporal dependencies for traffic prediction and resource allocation ( Jin et al. 2023 ; Li et al. 2018 ; Yu et al. 2018 ; Wu et al. 2019b ; Zheng et al. 2014 ) . Geospatial intelligence platforms must analyze satellite imagery and geographic data at planetary scale ( Jakubik et al. 2024 ; Cong et al. 2022 ; Mai et al. 2023 ; Janowicz et al. 2020 ; Li et al. 2025a ; Xiao et al. 2025 ) . Despite this importance, existing surveys treat these areas in isolation, lacking a unified framework connecting agentic architectures with spatial requirements. Contributions. This survey makes four primary contributions: 1. A unified three-axis taxonomy connecting agentic AI components (memory, planning, tool use) with spatial intelligence domains (navigation, scene understanding, manipulation, geospatial analysis) across spatial scales (micro, meso, macro). This framework enables clear identification of research gaps, guides architectural decisions for new systems, and provides a common vocabulary for cross-domain collaboration. 2. A comprehensive analysis of 742 cited works drawn from over 2,000 reviewed papers, revealing a significant scale imbalance: 68% of methods target meso-spatial tasks (room-scale navigation), while micro-spatial manipulation and macro-spatial geospatial reasoning remain underexplored despite their commercial importance. We identify GNN-LLM integration, vision-language-action models, and world model-based planning as key architectural patterns bridging this gap. 3. A detailed comparison with existing surveys (Table 1 ), quantifying coverage gaps and demonstrating how this work uniquely bridges agentic AI and spatial intelligence domains. 4. A forward-looking roadmap identifying six grand challenges that represent structural barriers to progress, not incremental improvements: unified cross-scale representation, grounded long-horizon planning, safety guarantees, sim-to-real transfer, multi-agent coordination, and edge deployment. Addressing these challenges requires fundamental advances in how agents represent and reason about space. 2 Methodology This survey follows a rigorous literature review methodology consistent with best practices in computer science ( Kitchenham 2004 ; Petersen et al. 2008 ; Wohlin 2014 ; Keele et al. 2007 ; Brereton et al. 2007 ; Dybå and Dingsøyr 2007 ; Moher et al. 2009 ; Okoli and Schabram 2010 ; Webster and Watson 2002 ; Jalali and Wohlin 2012 ; Snyder 2019 ; Tranfield et al. 2003 ) . We queried complementary academic databases: Google Scholar for breadth, arXiv for cutting-edge preprints, ACM Digital Library and IEEE Xplore for peer-reviewed systems research, Semantic Scholar for citation-aware ranking, and DBLP ( Ley 2002 ) for broad venue coverage. Keywords including “agentic AI,” “spatial intelligence,” “embodied AI,” “vision-language navigation,” “robot manipulation,” “geospatial AI,” “world models,” “graph neural networks,” “spatio-temporal learning,” “vision-language-action,” and “foundation models for robotics.” Our initial search yielded over 3,000 papers. We then applied a rigorous multi-stage filtering process: 1. Temporal Filtering: We selected papers published between 2018 and 2026. The 2018 lower bound marks a critical inflection point: the release of BERT ( Devlin et al. 2019 ) initiated the foundation model era, while Habitat ( Savva et al. 2019 ) and AI2-THOR ( Kolve et al. 2017 ) established standardized embodied AI benchmarks that enabled reproducible research. Pre-2018 foundational works (e.g., PointNet ( Qi et al. 2017a ) , GCN ( Kipf and Welling 2017 ) ) are included where they established methods still in active use. 2. Venue Filtering: We prioritized papers from top-tier venues including NeurIPS, ICML, ICLR, CVPR, ECCV, ICCV, CoRL, RSS, IROS, ICRA, ACM Computing Surveys, IEEE TPAMI, Nature, Science, Science Robotics, and leading arXiv preprints. 3. Quality Filtering: To mitigate citation-based selection bias that disadvantages recent work, we explicitly included low-citation papers introducing breakthrough approaches (e.g., early VLA models, novel GNN-LLM architectures) alongside high-citation foundational methods. This dual strategy balances established impact with emerging innovation. 4. Relevance Filtering: We ensured papers directly addressed the intersection of agentic capabilities and spatial intelligence. This process resulted in a final corpus of over 2,000 papers, from which 742 are directly cited, which were carefully analyzed to derive the taxonomy, identify key trends, and synthesize the findings presented in this survey. We employed a snowball sampling technique to ensure broad coverage of related works, following citation chains both forward and backward. Two independent reviewers validated the paper selection and taxonomy development, achieving 94% inter-annotator agreement on inclusion criteria. Disagreements fell into two categories: scope disputes (whether a paper’s spatial component was sufficiently central, 4% of cases) and taxonomy assignment (which axis a method primarily addressed, 2% of cases). All disagreements were resolved through discussion until consensus. 3 Related Work and Survey Comparison While several surveys have addressed aspects of agentic AI or spatial intelligence, no prior work has explicitly and systematically unified these domains within a single framework that spans from micro-scale manipulation to macro-scale geospatial reasoning. We review existing surveys across five categories and provide a detailed comparison in Table 1 . Agentic AI Surveys. Recent surveys on LLM-based agents ( Wang et al. 2024b ; Xi et al. 2023 ; Guo et al. 2024b ; Durante et al. 2024 ; Weng 2023 ; Mialon et al. 2023 ) focus on reasoning and tool use but do not address spatial capabilities. Sumers et al. 2024 provides a cognitive architecture perspective. The common limitation across these works is their treatment of agents as primarily symbolic reasoners operating over text and structured data, with no mechanism for maintaining spatial state representations (maps, 3D scenes, object poses) that persist across reasoning steps. This absence prevents these frameworks from supporting embodied tasks where geometric consistency is essential. Embodied AI Surveys. Embodied AI surveys ( Duan et al. 2022 ; Gupta et al. 2021 ; Francis et al. 2022 ; Savva et al. 2019 ; Anderson et al. 2018b ; Gervet et al. 2023 ) cover simulation environments and benchmarks but lack integration with agentic architectures. Kawaharazuka et al. 2025 surveys vision-language-action models specifically for robotics. Notably absent from these works is treatment of autonomous long-horizon planning: how agents decompose multi-step tasks, recover from failures, and accumulate experience across episodes without human intervention. Geospatial AI Surveys. Geospatial AI surveys ( Mai et al. 2023 ; Janowicz et al. 2020 ; Xiao et al. 2025 ) and spatio-temporal data mining reviews ( Jin et al. 2023 ; Atluri et al. 2018 ; Wang et al. 2020 ; Jiang and Luo 2022 ; Balachandar et al. 2025 ) are highly specialized. Critically, these works treat geospatial systems as passive prediction tools that output forecasts (traffic flow, land cover) without closing the loop: they do not address how predictions should trigger actions, how actions affect future observations, or how agents should adapt when predictions prove incorrect. This open-loop framing limits their applicability to autonomous systems that must act on geospatial intelligence. Graph Neural Network Surveys. GNN surveys ( Wu et al. 2020a ; Bronstein et al. 2021 ; Hamilton 2020 ; Battaglia et al. 2018 ; Zhou et al. 2020 ; Zhang et al. 2020 ; Veličković 2023 ) provide comprehensive coverage of graph learning but do not focus on spatial applications or agent integration. Surveys on GNNs for specific domains include traffic ( Jiang and Luo 2022 ) , urban computing ( Balachandar et al. 2025 ) , and spatio-temporal prediction ( Jin et al. 2023 ) . Vision-Language Model Surveys. Surveys on VLMs ( Zhang et al. 2024c ; Bordes et al. 2024 ) cover multimodal understanding but do not address spatial action or embodiment. Kawaharazuka et al. 2025 surveys vision-language-action models specifically for robotics. Table 1: Comparison with Existing Surveys. Symbols: ✓ = comprehensive coverage, \circ = partial coverage, blank = not covered. Our work provides the first unified coverage across all dimensions. Survey Agentic AI Embodied AI Spatial Reasoning Geospatial GNNs Industry Unified Taxonomy Wang et al. (2024) ( Wang et al. 2024b ) ✓ \circ \circ Xi et al. (2023) ( Xi et al. 2023 ) ✓ \circ Duan et al. (2022) ( Duan et al. 2022 ) ✓ \circ Kawaharazuka et al. (2025) ( Kawaharazuka et al. 2025 ) \circ ✓ \circ Jin et al. (2023) ( Jin et al. 2023 ) \circ \circ ✓ Mai et al. (2023) ( Mai et al. 2023 ) ✓ \circ Bronstein et al. (2021) ( Bronstein et al. 2021 ) \circ ✓ Zhang et al. (2024) ( Zhang et al. 2024c ) \circ \circ This Survey ✓ ✓ ✓ ✓ ✓ ✓ ✓ As Table 1 reveals, existing surveys cluster around either agentic reasoning (top rows) or domain-specific spatial methods (middle rows), with no prior work achieving coverage across all seven dimensions. The pattern is instructive: agentic surveys (Wang, Xi) show strong coverage of reasoning but blank cells for geospatial and GNNs. Spatial surveys (Jin, Mai, Bronstein) show the inverse. The consistent absence of checkmarks in the “Unified Taxonomy” column across all prior work reflects the field’s fragmentation into disconnected subcommunities. This gap motivates our unified taxonomy. 4 Unified Three-Axis Taxonomy We propose a three-axis taxonomy (Figure 1 ) that maps agentic capabilities to spatial task requirements across spatial scales. To read this framework: each method occupies a position along all three axes simultaneously. The Task axis specifies what the system does, the Capability axis specifies how it reasons and acts, and the Scale axis specifies the spatial granularity. This structure enables detailed comparison of methods and identification of underexplored regions in the design space. Crucially, we enforce a structural discipline: every method reviewed in this survey is explicitly mapped to all three axes (see Table 2 ), ensuring consistent characterization and enabling direct cross-method comparison. 4.1 Taxonomy Axes Axis 1: Spatial Task. We identify four primary spatial task categories: • Navigation : Goal-directed movement through environments spanning indoor and outdoor settings. Indoor tasks include point-goal ( Anderson et al. 2018a ; Wijmans et al. 2020 ; Savva et al. 2019 ) , object-goal ( Chaplot et al. 2020b ; Batra et al. 2020 ) , and vision-language navigation ( Anderson et al. 2018c ; Krantz et al. 2020 ) . Outdoor tasks include autonomous driving on highways and urban streets ( Hu et al. 2023b ; Caesar et al. 2020 ; Waymo 2023 ; Geiger et al. 2012 ) , off-road navigation ( Triest et al. 2024 ) , and pedestrian-scale outdoor wayfinding ( Mirowski et al. 2018 ) • Scene Understanding : Perceiving and reasoning about 3D structure, objects, and spatial relationships • Manipulation : Physical interaction with objects, including grasping ( Mahler et al. 2017 ; Morrison et al. 2018 ; Fang et al. 2020 ; ten Pas et al. 2017 ) , placement ( Zeng et al. 2021 ) , and tool use ( Qin et al. 2024a ; Qin et al. 2024b ) • Geospatial Analysis : Large-scale spatial reasoning including satellite imagery ( Christie et al. 2018a ; Christie et al. 2018b ; Demir et al. 2018 ; Xia et al. 2017 ; Sumbul et al. 2019 ) , urban computing ( Zheng et al. 2014 ; Yuan et al. 2020b ; Zheng 2015 ) , and geographic information systems ( Longley et al. 2015 ; Goodchild 2007 ) Axis 2: Agentic Capability. We identify three core agentic capabilities: • Memory : Short-term (in-context), long-term (retrieval-augmented), episodic, and spatial memory systems • Planning : Reactive, hierarchical, search-based, and world model-based planning approaches. These span a spectrum from reactive policies that map observations directly to actions (fast but brittle to novel situations) to model-based planners that simulate future states before acting (slower but more robust to distribution shift). Self-reflection ( Shinn et al. 2023b ) operates as a meta-capability that spans memory (storing past failures) and planning (revising future actions) • Tool Use & Action : API integration, code generation, physical action primitives, and skill libraries Axis 3: Spatial Scale. We distinguish three spatial scales with boundaries grounded in sensing and actuation constraints: • Micro-spatial ( < 1m): Pose estimation, grasping, fine manipulation at centimeter precision. This boundary reflects the effective range of tactile sensors and the workspace of typical robot arms. • Meso-spatial (1m–100m): Room navigation, building exploration, indoor/outdoor local scenes. This range corresponds to the effective field of view of onboard cameras and lidar, where metric SLAM remains tractable. • Macro-spatial ( > 100m): City-scale planning, satellite imagery, infrastructure networks spanning kilometers. Beyond 100m, agents must rely on external sensing (satellites, city-wide sensor networks) and cannot directly actuate the environment. Figure 1: A unified three-axis taxonomy connecting Agentic AI capabilities with Spatial Intelligence domains across spatial scales. The intersection of these dimensions defines the design space for autonomous spatial intelligence systems. Design trade-off : No existing method achieves strong performance across all three axes simultaneously. Micro-scale manipulation systems (bottom-left) achieve centimeter precision but cannot plan beyond the immediate workspace. Geospatial models (top-right) reason at planetary scale but lack closed-loop action capabilities. The sparsely populated regions of this space (e.g., macro-scale manipulation, micro-scale geospatial) represent open research opportunities where cross-axis integration could yield high-impact advances. 4.2 Methods-Taxonomy Mapping Table 2 maps representative methods to our three-axis taxonomy, demonstrating how the framework organizes the field. Table 2: Representative Methods Mapped to the Three-Axis Taxonomy. Representation types: Symbolic (language, code), Metric (geometric, visual), Latent (learned embeddings), Multimodal (combined). Method Spatial Task Agentic Capability Scale Repr. Primary Failure Mode VLN-BERT ( Hong et al. 2021 ) Navigation Memory + Planning Meso Multimodal Instruction grounding errors SayCan ( Ahn et al. 2022 ) Manipulation Planning + Tool Use Micro–Meso Symbolic Affordance mismatch RT-2 ( Brohan et al. 2023 ) Manipulation Tool Use Micro Metric Out-of-distribution objects VLMaps ( Huang et al. 2023a ) Navigation Memory Meso Metric Semantic drift over time Voyager ( Wang et al. 2023a ) Navigation + Manip. Memory + Planning Meso Symbolic Code execution failures DCRNN ( Li et al. 2018 ) Geospatial Memory (low planning) Macro Latent Non-stationary dynamics Graph WaveNet ( Wu et al. 2019b ) Geospatial Memory (low planning) Macro Latent Sparse graph regions Prithvi ( Jakubik et al. 2024 ) Geospatial Memory only Macro Metric No action capability DreamerV3 ( Hafner et al. 2023 ) Navigation + Manip. Planning (World Model) Micro–Meso Latent Model compounding error PaLM-E ( Driess et al. 2023 ) Manipulation Planning + Tool Use Micro–Meso Multimodal Hallucinated actions OpenVLA ( Kim et al. 2024 ) Manipulation Tool Use Micro Metric Limited generalization LLaGA ( Chen et al. 2024e ) Scene Understanding Memory Meso Multimodal Graph construction noise Key Takeaways: Taxonomy • The three-axis taxonomy (Task \times Capability \times Scale) provides a comprehensive framework for organizing spatial AI research • Most methods address meso-spatial scales. Micro and macro scales remain underexplored • Memory systems are critical across all spatial tasks but implementations vary significantly by scale • Critical gap: Macro-scale geospatial methods are memory-dominant with minimal planning capabilities. No existing system combines city-scale reasoning with autonomous goal-directed behavior • The intersection of GNN-based methods with agentic capabilities represents an emerging frontier 5 Agentic AI Components for Spatial Intelligence This section examines how agentic capabilities enable spatial intelligence, organized around the core scientific question: How do agents internally represent, reason about, and act within spatial environments? 5.1 Memory Systems: How Do Agents Remember Spatial Information? Memory enables agents to accumulate and retrieve experiential knowledge. While cognitive science provides foundational theories ( Tulving 1972 ; Baddeley 2003 ) , our focus is on computational instantiations that enable AI systems to maintain persistent spatial knowledge. The central challenge is: How can agents maintain persistent spatial knowledge across varying time horizons and scales? Short-Term Memory. In-context learning ( Brown et al. 2020 ; Dong et al. 2022 ; Olsson et al. 2022 ; Akyurek et al. 2023 ; Dai et al. 2023a ; Min et al. 2022 ; Xie et al. 2022 ; Wei et al. 2023 ; Chan et al. 2022 ) allows models to adapt to new tasks through examples in the prompt. This mechanism enables rapid adaptation without parameter updates, leveraging the attention mechanism to condition on provided demonstrations. Working memory mechanisms ( Graves et al. 2014 ; Weston et al. 2015 ; Sukhbaatar et al. 2015 ; Kumar et al. 2016 ; Santoro et al. 2016 ; Graves et al. 2016 ; Munkhdalai and Yu 2017 ) enable temporary information storage during reasoning, supporting multi-step computations that exceed single forward pass capabilities. To illustrate the distinction from spatial memory: short-term memory might retain the instruction “go to the kitchen and find a red cup,” while spatial memory encodes the metric layout of the kitchen itself, including doorway locations and cabinet positions. Long-Term Memory. Retrieval-augmented generation ( Lewis et al. 2020 ; Packer et al. 2023 ; Guu et al. 2020 ; Borgeaud et al. 2022 ; Asai et al. 2023 ; Trivedi et al. 2023 ; Izacard et al. 2023 ; Shi et al. 2023 ; Ram et al. 2023 ; Khandelwal et al. 2020 ) enables knowledge persistence beyond context limits. MemGPT ( Packer et al. 2023 ) introduces hierarchical memory management for extended conversations. AMEM ( Xu et al. 2025 ) provides agentic memory for LLMs. MemEvolve ( Zhang et al. 2025a ) enables meta-evolution of agent memory. Vector databases ( Johnson et al. 2019 ; Malkov and Yashunin 2018 ; Douze et al. 2024 ; Wang et al. 2021 ; Pinecone 2023 ; Jegou et al. 2011 ; Ge et al. 2014 ; Guo et al. 2020 ) provide efficient similarity search for memory retrieval, enabling agents to access relevant past experiences. Episodic Memory. Episodic memory stores specific experiences and events ( what happened ), enabling agents to learn from past interactions ( Blundell et al. 2016 ; Pritzel et al. 2017 ; Banino et al. 2018 ; Ritter et al. 2018 ; Fortunato et al. 2019 ; Botvinick et al. 2019 ; Gershman and Daw 2017 ) . This type of memory is critical for spatial agents that must remember visited locations, encountered objects, and successful action sequences ( Savinov et al. 2018 ; Chaplot et al. 2020c ; Fang et al. 2019 ; Ramakrishnan et al. 2022 ; Ye et al. 2021 ; Chen et al. 2022f ) . The distinction from spatial memory is important: episodic memory records that “the robot collided with a chair at 3pm,” while spatial memory encodes the persistent geometric structure (the chair’s location relative to the table) independent of any specific event. Spatial Memory. While episodic memory records what happened where , spatial memory encodes the structure of where itself : the geometric and topological relationships that persist independent of specific events. Specialized implementations include cognitive maps ( Tolman 1948 ; O’Keefe and Nadel 1978 ; Moser et al. 2008 ; Hafting et al. 2005 ) , topological representations ( Kuipers 2000 ; Choset and Nagatani 2001 ; Thrun 1998 ; Kuipers and Byun 1991 ) , and metric maps ( Thrun et al. 2005 ; Durrant-Whyte and Bailey 2006 ; Cadena et al. 2016 ; Mur-Artal et al. 2015 ; Mur-Artal and Tardós 2017 ; Campos et al. 2021 ; Engel et al. 2017 ; Engel et al. 2014 ) . Neural approaches to spatial memory include Neural SLAM ( Chaplot et al. 2020c ; Chaplot et al. 2020d ; Chaplot et al. 2020b ; Chaplot et al. 2021 ) , semantic maps ( Huang et al. 2023a ; Henriques and Vedaldi 2018 ; Shah et al. 2023b ; Shah et al. 2023a ; Huang et al. 2023c ; Chen et al. 2023a ) , and scene graphs ( Armeni et al. 2019 ; Rosinol et al. 2020 ; Hughes et al. 2022 ; Gu et al. 2024 ; Wu et al. 2021 ; Wald et al. 2020 ; Kim et al. 2019 ) . Spatial Failure Modes. Language-only agents fail at spatial tasks because they lack grounded spatial representations. These failures cluster into four categories, each traceable to a specific representational gap: (1) spatial hallucination , where agents describe impossible spatial configurations (GPT-4V fails on 40% of spatial relationship questions in SpatialBench ( Chen et al. 2024a ) ). (2) reference frame confusion , where agents conflate egocentric and allocentric coordinates (VLN agents show 15-20% error rates from frame misalignment ( Anderson et al. 2018c ) ). (3) scale insensitivity , where agents fail to distinguish micro, meso, and macro-scale reasoning (SayCan’s affordance model fails when object scales differ from training ( Ahn et al. 2022 ) ). (4) temporal drift , where spatial memory degrades over long horizons (VLMaps shows semantic drift after 100+ steps without map updates ( Huang et al. 2023a ) ). 5.2 Planning Systems: How Do Agents Plan Over Spatial Horizons? Planning decomposes goals into executable action sequences, enabling complex task completion ( Russell and Norvig 2010 ; Ghallab et al. 2004 ; LaValle 2006 ; Fikes and Nilsson 1971 ; Sacerdoti 1974 ; Nau et al. 2003 ; Bylander 1994 ; Geffner and Bonet 2013 ; Kautz and Selman 1992 ; Helmert 2006 ) . The central challenge is: How can agents decompose spatial goals into feasible action sequences while accounting for geometric constraints? Chain-of-Thought Reasoning. Step-by-step reasoning ( Wei et al. 2022 ; Kojima et al. 2022 ; Wang et al. 2022 ; Zhou et al. 2023a ; Fu et al. 2023 ; Chen et al. 2023c ; Zhang et al. 2023 ; Khot et al. 2023 ; Diao et al. 2023 ) enables structured problem decomposition. Self-consistency ( Wang et al. 2022 ) improves reliability through multiple reasoning paths. Zero-shot chain-of-thought ( Kojima et al. 2022 ) enables reasoning without demonstrations. Tree-Based Search. Tree of Thoughts ( Yao et al. 2023a ; Long 2023 ; Xie et al. 2023 ; Hulbert et al. 2024 ) explores multiple solution branches through deliberate search. Graph of Thoughts ( Besta et al. 2023 ; Lei et al. 2023 ; Yao et al. 2024 ) enables more complex reasoning structures with arbitrary connections. RAP ( Hao et al. 2023 ; Shridhar et al. 2020b ; Zhao et al. 2024 ; Liu et al. 2024d ) combines reasoning with acting in a planning framework. Monte Carlo Tree Search variants ( Silver et al. 2016 ; Schrittwieser et al. 2020 ; Browne et al. 2012 ; Anthony et al. 2017 ; Silver et al. 2017 ; Silver et al. 2018 ; Vinyals et al. 2019 ; Kocsis and Szepesvári 2006 ) provide principled exploration with theoretical guarantees. Hierarchical Planning. LLM-Planner ( Song et al. 2023 ) enables few-shot grounded planning for embodied agents. Inner Monologue ( Huang et al. 2022a ) provides feedback-driven planning through internal dialogue. HiPlan ( Li et al. 2025b ) introduces hierarchical planning with LLMs. Hierarchical RL approaches ( Nachum et al. 2018 ; Vezhnevets et al. 2017 ; Bacon et al. 2017 ; Pertsch et al. 2021 ; Kulkarni et al. 2016 ; Gupta et al. 2019 ; Levy et al. 2019 ; Li et al. 2020 ; Zhang et al. 2021b ; Sutton et al. 1999 ) decompose tasks into subtasks with temporal abstraction. Task and Motion Planning. TAMP ( Garrett et al. 2021 ; Kaelbling and Lozano-Pérez 2011 ; Toussaint 2015 ; Dantam et al. 2016 ; Srivastava et al. 2014 ; Garrett et al. 2020 ; Driess et al. 2020 ) integrates symbolic planning with continuous motion planning for robotic applications. This approach combines the expressiveness of symbolic reasoning with the precision of geometric planning. From Symbolic to Neural Planning. The methods above rely on explicit symbolic representations that offer formal guarantees: plans can be verified against preconditions, and failures can be diagnosed by inspecting intermediate states. Recent work explores whether large neural models can subsume these capabilities, trading hand-crafted structure for learned representations that generalize more broadly but sacrifice interpretability. This shift raises fundamental questions about plan verifiability and failure recovery, particularly in safety-critical applications where understanding why a plan failed is as important as whether it succeeded. LLM-Based Planning. Recent work uses LLMs directly for planning ( Huang et al. 2022b ; Valmeekam et al. 2023a ; Song et al. 2023 ; Silver et al. 2024 ; Liu et al. 2023a ; Kambhampati et al. 2024 ; Valmeekam et al. 2023b ) . SayCan ( Ahn et al. 2022 ) grounds language models in affordances. Code as Policies ( Liang et al. 2023b ) generates executable robot code. ProgPrompt ( Singh et al. 2023 ) uses programmatic prompting for task planning. A critical caveat: LLM-generated plans cannot be formally verified for correctness or safety. Unlike classical planners that guarantee precondition satisfaction, LLM planners may produce fluent but physically impossible action sequences. This limitation is particularly concerning for safety-critical applications (e.g., surgical robotics, autonomous vehicles) where plan failures have irreversible consequences. Spatial Planning Failure Modes. LLM-based planners fail when: (1) geometric constraints are violated : on BEHAVIOR-1K ( Li et al. 2023a ) , LLM planners achieve only 12% success due to collision-ignoring plans. (2) action preconditions are unmet : VirtualHome ( Puig et al. 2018 ) shows 35% of LLM plans fail on precondition violations. (3) long-horizon credit assignment fails : ALFRED ( Shridhar et al. 2020a ) success drops from 65% to 18% as task horizon increases from 5 to 20 steps. (4) dynamic replanning is absent : RoboTHOR ( Deitke et al. 2020 ) shows 40% failure rate when unexpected obstacles appear. 5.3 Tool Use and Action: How Do Agents Ground Language in Geometry? Tool use extends agent capabilities through external interfaces and physical actions ( Osiurak and Badets 2016 ; Vaesen 2012 ; Beck et al. 2011 ; Shumaker et al. 2011 ; Seed and Byrne 2010 ; Tomasello 1999 ) . The central challenge is: How can language-based reasoning be translated into precise geometric actions? We distinguish two fundamentally different categories: symbolic tools (APIs, code execution, database queries) that operate in digital space with deterministic outcomes, and physical actuation (grasping, locomotion, manipulation) that operates in continuous physical space with stochastic dynamics and irreversible consequences. This distinction matters because symbolic tool errors can be caught and retried, while physical action errors may cause permanent damage. API Integration. Toolformer ( Schick et al. 2023 ; Parisi et al. 2022 ; Mialon et al. 2023 ; Nakano et al. 2021 ) enables self-supervised tool learning. Gorilla ( Patil et al. 2023 ; Li et al. 2023e ; Tang et al. 2023 ) specializes in API calling with retrieval augmentation. ToolLLM ( Qin et al. 2024a ; Hao et al. 2024 ; Qin et al. 2024b ; Xu et al. 2023d ) provides comprehensive tool use benchmarks. TaskMatrix ( Liang et al. 2023c ; Lu et al. 2023 ; Yang et al. 2023b ) connects foundation models with millions of APIs. TALM ( Parisi et al. 2022 ) augments language models with tool use. Additional tool-use frameworks include HuggingGPT ( Shen et al. 2023b ) , ViperGPT ( Surís et al. 2023 ) , Visual ChatGPT ( Wu et al. 2023a ) , and MM-REACT ( Yang et al. 2023b ) . Code Generation. PAL ( Gao et al. 2023 ; Wang et al. 2024a ) uses code for reasoning. Code as Policies ( Liang et al. 2023b ) generates executable robot code from language. Codex ( Chen et al. 2021a ) , StarCoder ( Li et al. 2023f ) , CodeLlama ( Roziere et al. 2023 ) , DeepSeek-Coder ( Guo et al. 2024a ) , and WizardCoder ( Luo et al. 2023 ) provide code generation capabilities. ProgPrompt ( Singh et al. 2023 ) uses programmatic prompting for robotics. Self-debugging ( Chen et al. 2023f ; Olausson et al. 2023 ) improves code quality through iterative refinement. ReAct Architecture. ReAct ( Yao et al. 2023b ; Yao et al. 2023c ; Liu et al. 2023c ) interleaves reasoning with action execution, enabling agents to think before acting. Reflexion ( Shinn et al. 2023b ; Shinn et al. 2023a ; Kim et al. 2023 ) adds self-reflection for improvement through verbal reinforcement. These architectures form the foundation for many spatial agents. Physical Action. For embodied agents, tool use extends to physical manipulation ( Zeng et al. 2021 ; Brohan et al. 2022 ; Brohan et al. 2023 ; Shridhar et al. 2022 ; Shridhar et al. 2023 ) . Action primitives ( Dalal et al. 2021 ; Nasiriany et al. 2022 ; Mandlekar et al. 2021 ) provide reusable building blocks. Skill libraries ( Wang et al. 2023a ; Lynch et al. 2020 ; Pertsch et al. 2021 ; Singh et al. 2021 ; Xu et al. 2023a ) enable compositional action. Key Takeaways: Agentic Components • Memory systems must be explicitly spatial: cognitive maps, semantic maps, and scene graphs outperform generic retrieval for spatial tasks • Hierarchical planning with geometric grounding addresses the gap between high-level language goals and low-level motor commands • Tool use bridges language and action through code generation, API calls, and learned action primitives • Key failure modes stem from lack of spatial grounding: hallucination, reference frame confusion, and geometric constraint violation 6 Spatial Intelligence Domains Micro-summary: This section surveys methods for agent navigation, scene understanding, manipulation, and geospatial reasoning. We explicitly tie these task domains to agentic planning and highlight key benchmark gaps. This section examines the four primary spatial task domains, organized around the question: What spatial capabilities must agents possess to operate in the physical world? 6.1 Navigation: How Do Agents Move Through Space? Micro-summary: This subsection covers visual and language-guided navigation, from classical robotics to modern learning-based systems. We highlight the evolution from explicit mapping to implicit spatial reasoning. For agents, navigation is not merely path execution but the foundation for all downstream spatial tasks. Navigation requires agents to perceive environments, plan paths, and execute locomotion toward goals ( Bonin-Font et al. 2008 ; DeSouza and Kak 2002 ; Thrun 2002 ; Siegwart et al. 2011 ; Choset et al. 2005 ; Cadena et al. 2016 ; Engel et al. 2017 ; Fuentes-Pacheco et al. 2015 ; Durrant-Whyte and Bailey 2006 ; Konolige et al. 2008 ) . Vision-Language Navigation. VLN tasks require agents to follow natural language instructions in visual environments ( Anderson et al. 2018c ; Qi et al. 2020 ; Krantz et al. 2020 ; Fried et al. 2018 ; Chen et al. 2022h ; Shah et al. 2023b ; Hong et al. 2020 ; Chen et al. 2021c ; An et al. 2023 ; Wang et al. 2019b ; Ma et al. 2019a ; Tan et al. 2019 ; Zhu et al. 2020 ; Hong et al. 2021 ; Qiao et al. 2022 ; Chen et al. 2022g ; Guhur et al. 2021 ) . R2R ( Anderson et al. 2018c ) introduced the paradigm. REVERIE ( Qi et al. 2020 ) adds object grounding. VLN-CE ( Krantz et al. 2020 ) extends to continuous environments. Object-Goal Navigation. ObjectNav requires navigating to object categories ( Batra et al. 2020 ; Chaplot et al. 2020a ; Majumdar et al. 2022 ; Gadre et al. 2022 ; Gadre et al. 2023 ; Dorbala et al. 2022 ; Ramakrishnan et al. 2022 ; Ye et al. 2021 ; Khandelwal et al. 2022 ; Yokoyama et al. 2024 ) . ZSON ( Majumdar et al. 2022 ) enables zero-shot navigation. CLIP-Nav ( Dorbala et al. 2022 ) uses vision-language models. CoW ( Gadre et al. 2022 ) uses CLIP on wheels for semantic navigation. Audio-Visual Navigation. Audio cues guide navigation in SoundSpaces ( Chen et al. 2020 ; Chen et al. 2022d ; Gan et al. 2020 ; Chen et al. 2021b ; Majumder et al. 2022 ; Chen et al. 2022c ) . This modality is critical for finding sound-emitting targets. Embodied Question Answering. EQA requires navigation to answer questions ( Das et al. 2018b ; Gordon et al. 2018 ; Wijmans et al. 2019 ; Yu et al. 2019 ; Das et al. 2018a ; Gordon et al. 2019 ) . 3D-QA ( Azuma et al. 2022 ; Ma et al. 2022a ; Hong et al. 2023b ; Chen et al. 2024f ; Zhu et al. 2023 ; Huang et al. 2024b ) extends to 3D scene understanding. 6.2 Scene Understanding: How Do Agents Perceive 3D Structure? Micro-summary: This subsection reviews techniques for 3D scene understanding, connecting these representations to their use in downstream agentic planning. Scene representations determine what actions an agent can plan: richer representations enable more sophisticated reasoning about object affordances and spatial constraints. Scene understanding encompasses perceiving 3D geometry ( Hartley and Zisserman 2003 ; Szeliski 2022 ; Forsyth and Ponce 2011 ; Marr 1982 ; Prince 2012 ; Faugeras 1993 ) , recognizing objects ( Krizhevsky et al. 2012 ; He et al. 2016 ; Simonyan and Zisserman 2015 ; Szegedy et al. 2015 ; Huang et al. 2017 ; Tan and Le 2019 ) , and reasoning about spatial relationships ( Johnson et al. 2015 ; Krishna et al. 2017 ; Lu et al. 2016 ; Xu et al. 2017 ; Zellers et al. 2019 ) . Neural Scene Representations. NeRF ( Mildenhall et al. 2020 ; Barron et al. 2022 ; Müller et al. 2022 ; Park et al. 2019 ; Mescheder et al. 2019 ; Barron et al. 2023 ; Martin-Brualla et al. 2021 ; Tancik et al. 2022 ; Chen et al. 2022a ; Fridovich-Keil et al. 2022 ) revolutionized 3D reconstruction. Mip-NeRF ( Barron et al. 2022 ) handles multi-scale rendering. 3D Gaussian Splatting ( Kerbl et al. 2023 ; Luiten et al. 2023 ; Fan et al. 2024 ; Yu et al. 2024 ; Huang et al. 2024a ) enables real-time rendering. Integration with SLAM ( Sucar et al. 2021 ; Zhu et al. 2022 ; Keetha et al. 2024 ; Bird et al. 2025 ; Teed and Deng 2021 ; Teed et al. 2024 ; Mur-Artal and Tardós 2017 ; Campos et al. 2021 ) enables online reconstruction. Point Cloud Processing. Point cloud methods ( Qi et al. 2017a ; Qi et al. 2017b ; Wang et al. 2019c ; Thomas et al. 2019 ; Zhao et al. 2021 ; Wu et al. 2019a ; Liu et al. 2019 ; Ma et al. 2022b ) process raw 3D data. Point-BERT ( Yu et al. 2022b ) , Point-MAE ( Pang et al. 2022 ) , PointGPT ( Chen et al. 2024b ) , Point-Bind ( Guo et al. 2023 ) , and Uni3D ( Zhou et al. 2024 ) introduce self-supervised pretraining. 3D object detection ( Shi et al. 2019 ; Qi et al. 2019 ; Shi et al. 2020 ; Chen et al. 2023g ; Lang et al. 2019 ; Yin et al. 2021 ; Fan et al. 2022 ; Zhou and Tuzel 2018 ) enables scene parsing. Depth Estimation. Monocular depth estimation ( Godard et al. 2019 ; Ranftl et al. 2021 ; Ranftl et al. 2020 ; Yang et al. 2024c ; Fu et al. 2024b ; Bhat et al. 2021 ; Li et al. 2022 ; Yuan et al. 2022 ; Eigen et al. 2014 ) provides geometric understanding from single images. Depth Anything ( Yang et al. 2024c ) achieves strong zero-shot transfer. Metric3D ( Yin et al. 2023 ) recovers metric depth. Semantic Segmentation. Semantic segmentation ( Long et al. 2015 ; Chen et al. 2017 ; Kirillov et al. 2023 ; Peng et al. 2023 ; Chen et al. 2023b ; Xie et al. 2021 ; Cheng et al. 2022 ; Jain et al. 2023 ) enables scene parsing. SAM ( Kirillov et al. 2023 ) provides promptable segmentation. Open-vocabulary methods ( Ghiasi et al. 2022 ; Liang et al. 2023a ; Chen et al. 2023a ; Ding et al. 2022 ; Xu et al. 2023b ; Zhou et al. 2022 ) enable zero-shot recognition. Connection to Agentic Planning. These representations are not ends in themselves but inputs to downstream planning. A NeRF provides the geometric substrate for collision checking. A semantic map tells the planner which regions are traversable. A scene graph encodes the relational structure needed for task decomposition. The choice of representation directly constrains what plans an agent can generate and verify. 6.3 Manipulation: How Do Agents Interact with Objects? Micro-summary: This subsection focuses on robot manipulation, covering grasping, pre-grasp manipulation, and long-horizon sequential tasks. For agents, manipulation is where planning meets physical reality: each action has irreversible consequences that constrain all future possibilities. Manipulation requires understanding object affordances ( Gibson 1979 ; Do et al. 2018 ; Nagarajan et al. 2019 ; Fang et al. 2018 ; Zhu et al. 2015 ; Myers et al. 2015 ) , planning contact-rich interactions ( Chitnis et al. 2020 ; Kroemer et al. 2021 ) , and executing precise motor commands ( Argall et al. 2009 ; Billard et al. 2008 ; Ravichandar et al. 2020 ) . Vision-Language-Action Models. VLA models ( Brohan et al. 2022 ; Brohan et al. 2023 ; Team et al. 2024 ; Kim et al. 2024 ; Black et al. 2024 ; Driess et al. 2023 ; Collaboration 2023 ; Zitkovich et al. 2023 ; Padalkar et al. 2023 ) directly map visual observations and language instructions to actions. RT-1 ( Brohan et al. 2022 ) introduced large-scale robot learning. RT-2 ( Brohan et al. 2023 ) demonstrated web-scale pretraining transfer. RT-X ( Collaboration 2023 ) enables cross-embodiment learning. Octo ( Team et al. 2024 ) provides an open generalist policy. OpenVLA ( Kim et al. 2024 ) offers open-source VLA. \pi_{0} ( Black et al. 2024 ) introduces flow matching for robot learning. RoboCat ( Bousmalis et al. 2023 ) demonstrates self-improvement. Imitation Learning. Behavior cloning ( Pomerleau 1988 ; Chi et al. 2023 ; Zhao et al. 2023b ; Chi et al. 2024 ; Florence et al. 2022 ; Mandlekar et al. 2021 ; Mandlekar et al. 2022 ) learns from demonstrations. Diffusion Policy ( Chi et al. 2023 ) applies diffusion models to action generation. ACT ( Zhao et al. 2023b ) uses action chunking with transformers. Learning from play ( Lynch et al. 2020 ) enables unstructured learning. Reinforcement Learning. RL for manipulation ( Kalashnikov et al. 2018 ; Levine et al. 2018 ; Haarnoja et al. 2018 ; Schulman et al. 2017 ; Fujimoto et al. 2018 ; Lillicrap et al. 2016 ; Mnih et al. 2015 ; Mnih et al. 2016 ) enables learning from interaction. QT-Opt ( Kalashnikov et al. 2018 ) scales to real-world grasping. SAC ( Haarnoja et al. 2018 ) provides sample-efficient learning. Simulation Environments. Simulation platforms ( James et al. 2020 ; Yu et al. 2020 ; Makoviychuk et al. 2021 ; Savva et al. 2019 ; Kolve et al. 2017 ; Gu et al. 2023 ; Deitke et al. 2023 ; Xiang et al. 2020 ; Mu et al. 2021 ; Ehsani et al. 2021 ; Szot et al. 2021 ; Li et al. 2021 ; Shen et al. 2021 ) provide training environments. RLBench ( James et al. 2020 ) offers diverse manipulation tasks. Meta-World ( Yu et al. 2020 ) provides multi-task benchmarks. Isaac Gym ( Makoviychuk et al. 2021 ) enables GPU-accelerated simulation. Long-Horizon Task Composition. Real-world manipulation rarely involves isolated actions. Making a sandwich requires opening a drawer, retrieving bread, closing the drawer, opening the refrigerator, and so forth, with each action creating preconditions for subsequent steps. This sequential dependency structure distinguishes manipulation from navigation: a wrong turn can be corrected, but a broken egg cannot be unbroken. Current benchmarks like BEHAVIOR-1K ( Li et al. 2023a ) and ARNOLD ( Gong et al. 2023 ) explicitly test this compositional structure, revealing that success rates drop precipitously (often by 50% or more) as task horizon increases from 5 to 20 steps. 6.4 Geospatial Analysis: How Do Agents Reason at Planetary Scale? Micro-summary: This subsection covers methods for macro-scale geospatial analysis, providing intuitive explanations for key equations. For agents operating at city or planetary scale, these methods enable reasoning about phenomena that unfold over hours to years, requiring fundamentally different planning horizons than room-scale tasks. Geospatial analysis requires processing satellite imagery ( Zhu et al. 2017 ; Ma et al. 2019b ; Yuan et al. 2020a ; Zhang et al. 2016 ; Tuia et al. 2016 ; Camps-Valls et al. 2014 ) , modeling urban dynamics ( Bibri and Krogstie 2017 ; Batty 2013 ; Kitchin 2014 ; Jiang et al. 2021 ) , and reasoning about geographic relationships ( Egenhofer and Franzosa 1991 ; Cohn and Renz 2008 ; Randell et al. 1992 ) . Remote Sensing Foundation Models. Prithvi ( Jakubik et al. 2024 ) provides geospatial foundation models trained on Harmonized Landsat Sentinel-2 data. SatMAE ( Cong et al. 2022 ) introduces masked autoencoders for satellite imagery. Satlas ( Bastani et al. 2023 ) enables large-scale geospatial understanding. GeoAI ( Janowicz et al. 2020 ; Mai et al. 2023 ) surveys the field. CROMA ( Fuller et al. 2024 ) and microestimates ( Chi et al. 2022 ) advance remote sensing analysis. Spatio-Temporal Graph Neural Networks. STGNNs model complex urban dynamics through graph-structured representations ( Jin et al. 2023 ; Atluri et al. 2018 ; Wang et al. 2020 ; Jiang and Luo 2022 ; Balachandar et al. 2025 ) . The core intuition is that spatial locations (intersections, sensors, neighborhoods) form a graph where edges encode proximity or connectivity, and the state of each node evolves over time. The general formulation combines spatial message passing (how information flows between connected locations) with temporal convolution (how each location’s state evolves over time): \mathbf{H}^{(l+1)}=\sigma\left(\mathbf{A}\mathbf{H}^{(l)}\mathbf{W}^{(l)}+\text{TemporalConv}(\mathbf{H}^{(l)})\right) (1) Here, \mathbf{A} captures spatial structure (which locations influence which), \mathbf{H} represents node states, and the temporal convolution captures how patterns evolve over time. DCRNN ( Li et al. 2018 ) models traffic as diffusion on graphs, capturing the intuition that congestion spreads through a road network much like heat diffuses through a material: \mathbf{H}^{(l)}=\sum_{k=0}^{K}\left(\mathbf{P}_{f}^{k}\mathbf{X}\mathbf{W}_{k,1}+\mathbf{P}_{b}^{k}\mathbf{X}\mathbf{W}_{k,2}\right) (2) The forward matrix \mathbf{P}_{f} models downstream propagation (how congestion at an intersection affects roads leading away from it), while \mathbf{P}_{b} models upstream effects (how downstream congestion backs up into feeding roads). STGCN ( Yu et al. 2018 ) combines graph and temporal convolutions through a sandwiched structure. Graph WaveNet ( Wu et al. 2019b ) addresses a key limitation: predefined adjacency matrices (based on physical road connections) may miss important dependencies. Two distant locations might be strongly correlated due to shared commuter patterns even without direct road connection. Graph WaveNet learns the graph structure from data: \tilde{\mathbf{A}}=\text{SoftMax}\left(\text{ReLU}\left(\mathbf{E}_{1}\mathbf{E}_{2}^{T}\right)\right) (3) The learnable embeddings \mathbf{E}_{1},\mathbf{E}_{2} discover which locations should be connected based on their traffic patterns, not just their physical proximity. AGCRN ( Bai et al. 2020 ; Song et al. 2020 ) introduces node-specific patterns through adaptive modules. ASTGCN ( Guo et al. 2019 ; Guo et al. 2021 ) adds spatial and temporal attention mechanisms. GMAN ( Zheng et al. 2020 ; Park et al. 2020 ) uses graph multi-attention with transform attention for long-range dependencies. STGRAT ( Choi et al. 2022 ) advances the field. Urban Computing. Urban computing ( Zheng et al. 2014 ; Yuan et al. 2020b ; Zheng 2015 ) applies AI to city-scale challenges. ST-LLM ( Liu et al. 2024a ) and UniST ( Yuan et al. 2024 ) integrate language models with spatio-temporal reasoning. Traffic prediction ( Li et al. 2018 ; Yu et al. 2018 ; Wu et al. 2019b ) and demand forecasting ( Geng et al. 2019 ; Yao et al. 2018 ; Zhang et al. 2017 ) represent key applications. Key Takeaways: Spatial Domains • Navigation has progressed from point-goal to language-guided and zero-shot paradigms through vision-language integration • Scene understanding benefits from neural implicit representations (NeRF, 3DGS) combined with semantic grounding • Manipulation is being transformed by VLA models that transfer web-scale knowledge to robotic control • Geospatial analysis increasingly uses foundation models and GNNs for planetary-scale reasoning 7 Enabling Technologies Micro-summary: This section reviews the three core technologies that underpin modern spatial AI systems: Graph Neural Networks for relational reasoning, World Models for predictive simulation, and Multimodal Foundation Models for perception and grounding. We explain each technology with intuitive examples and connect them to the taxonomy. 7.1 Graph Neural Networks: How Do Agents Reason About Relationships? Micro-summary: This subsection explains how GNNs model spatial relationships, providing an intuitive explanation of the message-passing mechanism and its application across micro- and macro-scale spatial reasoning. Why include GNNs in a survey on spatial agents? The answer lies in their inductive bias. Transformers treat inputs as unordered sets (or sequences with learned positional encodings), making them agnostic to the relational structure inherent in spatial data. GNNs, by contrast, explicitly encode relationships as edges, providing a natural representation for spatial constraints: objects in a scene, intersections in a road network, or joints in a robot arm. This relational bias makes GNNs particularly effective for tasks where the structure of connections matters as much as the content of nodes ( Kipf and Welling 2017 ; Veličković et al. 2018 ; Xu et al. 2019 ; Hamilton et al. 2017 ; Wu et al. 2020b ; Wu et al. 2020a ; Zhou et al. 2020 ; Zhang et al. 2020 ; Li et al. 2016 ; Defferrard et al. 2016 ; Bruna et al. 2014 ) . Message Passing Framework. The general GNN formulation follows the message passing paradigm ( Gilmer et al. 2017 ; Scarselli et al. 2009 ; Battaglia et al. 2018 ; Xu et al. 2018 ; Morris et al. 2019 ) : \displaystyle\mathbf{m}_{v}^{(l)} \displaystyle=\text{AGGREGATE}^{(l)}\left(\left\{\mathbf{h}_{u}^{(l-1)}:u\in\mathcal{N}(v)\right\}\right) (4) \displaystyle\mathbf{h}_{v}^{(l)} \displaystyle=\text{UPDATE}^{(l)}\left(\mathbf{h}_{v}^{(l-1)},\mathbf{m}_{v}^{(l)}\right) (5) where \mathcal{N}(v) denotes the neighbors of node v , and AGGREGATE and UPDATE are learnable functions. Geometrically, this can be understood as each node ”listening” to its neighbors, gathering their information, and then updating its own representation based on what it heard, much like how a robot joint might update its understanding of the arm’s configuration by sensing the positions of adjacent joints. GNN-LLM Integration. A key conceptual insight: graphs can serve as externalized spatial memory for LLM-based agents. While LLMs excel at semantic reasoning, they lack persistent, structured representations of spatial relationships. By encoding scene graphs, road networks, or object configurations as graphs, and processing them through GNNs, agents can maintain spatial state that persists across reasoning steps and survives context window limitations. Emerging work combines GNNs with LLMs for this structured spatial reasoning ( Chen et al. 2024e ; Tang et al. 2024 ; Fatemi et al. 2023 ; Fatemi et al. 2024 ; Gowda et al. 2025 ; Ye et al. 2024 ; Zhao et al. 2023a ; Huang et al. 2024c ) . Graph instruction tuning ( Zhang et al. 2024b ) further enhances this capability. LLaGA ( Chen et al. 2024e ) provides language-graph alignment. GraphGPT ( Tang et al. 2024 ) enables graph reasoning through language models. Geometric Deep Learning. Geometric deep learning ( Bronstein et al. 2021 ) provides theoretical foundations for spatial reasoning on non-Euclidean domains. Equivariant networks ( Cohen and Welling 2016 ; Fuchs et al. 2020 ; Satorras et al. 2021 ; Weiler and Cesa 2019 ; Thomas et al. 2018 ; Kondor et al. 2018 ) respect spatial symmetries through: f(T_{g}\cdot x)=T_{g}\cdot f(x) (6) where T_{g} is a group transformation. Graph transformers ( Ying et al. 2021 ; Dwivedi et al. 2023 ; Rampášek et al. 2022 ; Kreuzer et al. 2021 ; Chen et al. 2022e ) combine attention with graph structure. E3NN ( Batzner et al. 2022 ) , geometric message passing ( Brandstetter et al. 2022 ) , and SchNet ( Schütt et al. 2017 ) advance equivariant architectures. 7.2 World Models: How Do Agents Predict the Future? Micro-summary: This subsection covers world models, which enable agents to simulate future outcomes of their actions. We contrast model-based and model-free approaches and discuss the role of world models in safe and efficient planning. For an agent to act safely and effectively in the physical world, it must be able to predict the consequences of its actions. World models provide this predictive capability, enabling planning and foresight by learning a model of how the world behaves ( LeCun 2022 ; Schmidhuber 2015 ; Matsuo et al. 2022 ; LeCun 2024b ; LeCun 2024a ; Moerland et al. 2023 ; Sutton 1991 ; Deisenroth and Rasmussen 2011 ) . This is essential for spatial safety, as it allows an agent to avoid physically dangerous or irreversible states. Latent Dynamics Models. The core idea is simple: rather than predicting future images directly (computationally expensive and prone to compounding errors), world models compress observations into a compact latent space and predict dynamics there. An encoder compresses high-dimensional observations (e.g., images) into a compact latent state. A dynamics model then predicts the next latent state given the current state and an action, effectively simulating the future in a compressed space. A decoder can reconstruct observations when needed. Formally: Encoder: \displaystyle\mathbf{z}_{t}=q_{\phi}(\mathbf{z}_{t}|\mathbf{o}_{\leq t},\mathbf{a}_{