Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoning

Yubo Zhao 1 Qi Wu 1 Yifan Wang 1 Yu-Wing Tai 2 Chi-Keung Tang 1 1 The Hong Kong University of Science and Technology 2 Dartmouth College {yzhaodx,qwuaz,ywangpa}@connect.ust.hk, [email protected], [email protected] · 2025

Read Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoning by Yubo Zhao 1 Qi Wu 1 Yifan Wang 1 Yu-Wing Tai 2 Chi-Keung Tang 1 1 The Hong Kong University of Science and Technology 2 Dartmouth College {yzhaodx,qwuaz,ywangpa}@connect.ust.hk, [email protected], [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07323.txt.

Introduction

Yubo Zhao, Qi Wu, and colleagues let language models steer moving agents through crowded spaces in a 2025 study. Earlier tests of model spatial reasoning confined agents to four directions in simple, still rooms with a single mover. Real navigation involves many agents, moving obstacles, and floor plans with clutter.

The authors encode real indoor layouts and dynamic agents in a uniform format the model can read. Language model reasoning then plans routes through the changing scene. Gains over earlier limited studies would show that text-trained models hold usable spatial sense. Such reasoners could sit above low-level controllers in service robots and games.

Catalog record

CREATORYubo Zhao 1 Qi Wu 1 Yifan Wang 1 Yu-Wing Tai 2 Chi-Keung Tang 1 1 The Hong Kong University of Science and Technology 2 Dartmouth College {yzhaodx,qwuaz,ywangpa}@connect.ust.hk, [email protected], [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-07323

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Navigating Motion Agents in Dynamic and Cluttered Environments through LLM Reasoning Yubo Zhao 1 Qi Wu 1 Yifan Wang 1 Yu-Wing Tai 2 Chi-Keung Tang 1 1 The Hong Kong University of Science and Technology 2 Dartmouth College {yzhaodx,qwuaz,ywangpa}@connect.ust.hk, [email protected], [email protected] Abstract This paper advances motion agents empowered by large language models (LLMs) toward autonomous navigation in dynamic and cluttered environments, significantly surpassing first and recent seminal but limited studies on LLM’s spatial reasoning, where movements are restricted in four directions in simple, static environments in the presence of only single agents much less multiple agents. Specifically, we investigate LLMs as spatial reasoners to overcome these limitations by uniformly encoding environments (e.g., real indoor floorplans), agents which can be dynamic obstacles and their paths as discrete tokens akin to language tokens. Our training-free framework supports multi-agent coordination, closed-loop replanning, and dynamic obstacle avoidance without retraining or fine-tuning. We show that LLMs can generalize across agents, tasks, and environments using only text-based interactions, opening new possibilities for semantically grounded, interactive navigation in both simulation and embodied systems. Figure 1 : Single agent (top) navigating a real floor plan in a hide-and-seek scenario. Multiple agents (bottom) simultaneously move toward their destinations, the LLM autonomously resolves potential collisions. Our LLM-based system serves as an autonomous path navigator with zero-shot spatial reasoning capabilities, effectively handling obstacle avoidance and collision resolution in dynamic environments. 1 Introduction Large Language Models (LLMs) Achiam et al. (2023) ; Team et al. (2024) ; Guo et al. (2025) ; Touvron et al. (2023) have made significant strides in reasoning and planning across various domains. These models exhibit powerful generalization capabilities, enabling them to handle unseen scenarios with remarkable flexibility and comprehension of world knowledge Sun et al. (2023) ; Wu et al. (2024c) ; Wu et al. (2024a) ; Wu et al. (2024b) ; Singh et al. (2023) ; Hendrycks et al. (2021) ; Sun et al. (2024) . This paper investigates the spatial reasoning ability inherent in pre-trained LLMs in zero-shot path navigation under dynamic environments. To make navigation tasks amenable to LLMs, we uniformly represent the environment (e.g., floor maps), agents, and paths as tokens that interact within a shared space. This token-based representation enables efficient spatial reasoning, providing robust and flexible solutions for pathfinding in complex environments. Specifically, we represent paths as sequences of sparse anchor points, allowing agents to move flexibly to any position within the space, rather than relying on predefined movement sets (e.g., 4 or 8 adjacent positions), as seen in previous works Wu et al. (2024b) ; Yang et al. (2024) ; Aghzal et al. (2024) ; Aghzal et al. (2023) ; Martorell (2025) . This approach aligns more closely with human intuition for maneuvering and obstacle avoidance, while also reducing the processing burden on the LLM. Thus, this work represents a departure from the conventional approach in path navigation using reinforcement learning (RL) which, while performing well in controlled settings, requires extensive data collection and training and often struggles to generalize to novel scenarios—challenges where LLMs demonstrate promising zero-shot adaptability. Despite that, we believe LLM and RL are not at odds with each other; rather, their synergy provides complementary advantages. With the rise of LLMs, we believe the dataset and insight we develop in this paper will be very useful in extending cognitive and language reasoning into embodied spatial domains, offering a flexible and scalable alternative in settings where previous approaches face practical constraints. We evaluate our LLM-powered, training-free system through experiments on real floorplans on various modern LLMs in both single-agent and multi-agent settings. Our results, e.g., Figure 1 , show that this approach generalizes effectively to previously unseen environments and tasks. Additionally, we showcase the system’s ability to solve dynamic problems in closed-loop environments, where agent communication and coordination are essential to ensure robust navigation and motion. Agents can autonomously adjust their plans and avoid collisions in real-time, highlighting the system’s ability to handle complex, unpredictable scenarios. Finally, we showcase a proof - of - concept extension to humanoid motion generation, demonstrating how the system can generate realistic and contextually appropriate movements for agents in a wide range of dynamic environments. We believe this work will inspire real-world applications in areas such as autonomous robotics, virtual reality, and interactive human-robot interaction, where agents must navigate and collaborate in complex, dynamic spaces. 2 Related Work 2.1 Spatial Understanding and Reasoning Effective navigation and planning in space require a fundamental understanding of the environment, a critical cognitive ability for both humans and intelligent systems. With the advancement of LLMs, spatial reasoning has become an emerging research focus. Studies such as Wu and Guo () ; Mirzaee et al. (2021) ; Mirzaee and Kordjamshidi (2022) ; Momennejad et al. (2023) investigate LLMs’ spatial reasoning capabilities through verbal reasoning tasks, such as question answering (QA). Other works like Mirchandani et al. (2023) explore LLMs’ ability to recognize patterns, while Hong et al. (2023) focuses on enhancing LLMs’ 3D reasoning capabilities. Additionally, Wu et al. (2024b) evaluates LLMs’ performance in solving QA-based tiling puzzles. However, these studies largely focus on static or symbolic reasoning, and do not consider grounded, embodied spatial tasks that require interaction with dynamic environments. We believe modern LLMs’ powerful language reasoning can be extended into spatial reasoning with proper token representation. In this paper, we validate this idea in extensive quantitative and qualitative evaluations, demonstrating their effectiveness in understanding spatial environments and generating feasible solutions. Moreover, we showcase their downstream applications, such as humanoid motion, extending beyond theoretical or virtual scenarios to more practical settings. 2.2 Using LLMs for Navigation Recent LLM studies on path navigation apply to restricted scenarios. For example, Wu et al. (2024b) proposes a protocol to evaluate LLMs’ ability to navigate in simple environments with a single possible route; Aghzal et al. (2024) ; Aghzal et al. (2023) investigate LLMs’ pathfinding capabilities in square grids with simplified, synthetic environments; Martorell (2025) explores using LLMs to navigate a single agent in a 5x5 grid without obstacles. All these studies focus on navigation within unreal environments with movements restricted to four directions (up, down, left, right), limiting their real-world applicability. Other works Zhou et al. (2023) ; Yang et al. (2024) ; Liu et al. (2025) ; Yuan et al. (2024) ; Yu et al. (2023) employ language models for individual navigation tasks with camera inputs. However, these efforts are often limited to perception-conditioned action selection, without broader path planning, spatial negotiation, or multi-agent interaction. In contrast, our approach enables global path planning over realistic environments with multiple agents, supporting continuous movement, interaction, and closed-loop adaptation. Unlike prior works constrained by low-dimensional grids or simple mazes, we demonstrate that LLMs can perform robust spatial reasoning in human-scale spaces, coordinating complex navigation tasks with no fine-tuning or retraining. 2.3 Environment-Aware Agents Path planning, especially for multiple interacting agents, is an active research area in robotics and AI, commonly studied as Multi-Agent Path Finding (MAPF) Stern et al. (2019) ; Li et al. (2021) ; Shaoul et al. (2024) . Classical planning methods, such as the A* algorithm Hassan et al. (2021) ; Zhao et al. (2023) , effectively solve single-agent navigation tasks in static environments but often face challenges

Read the complete text