Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance
Lingfeng Zhang 1,2,3, Xiaoshuai Hao 3,†,‡, Xizhou Bu 3,4, Yingbo Tang 5, Hongsheng Li 1, Jinghui Lu 3, Xiu-shen Wei 6, Jiayi Ma 7, Yu Liu 8, Jing Zhang 7, Hangjun Ye 3, Xiaojun Liang 2, Long Chen 3, Wenbo Ding 1,† · 2026
Read Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance by Lingfeng Zhang 1,2,3, Xiaoshuai Hao 3,†,‡, Xizhou Bu 3,4, Yingbo Tang 5, Hongsheng Li 1, Jinghui Lu 3, Xiu-shen Wei 6, Jiayi Ma 7, Yu Liu 8, Jing Zhang 7, Hangjun Ye 3, Xiaojun Liang 2, Long Chen 3, Wenbo Ding 1,† (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26839.txt.
Introduction
Zhang and thirteen colleagues propose Walk with Me, a map-free system guiding robots that assist people outdoors over long distances. It uses GPS context and public-map points of interest to turn spoken intentions into destinations and waypoint routes. A high-level vision-language model handles coarse planning and hard cases such as crowded crossings, while a low-level vision-language-action policy drives routine segments and pauses when unsafe. The split lets the robot follow social norms without costly prebuilt high-definition maps.
Catalog record
| CREATOR | Lingfeng Zhang 1,2,3, Xiaoshuai Hao 3,†,‡, Xizhou Bu 3,4, Yingbo Tang 5, Hongsheng Li 1, Jinghui Lu 3, Xiu-shen Wei 6, Jiayi Ma 7, Yu Liu 8, Jing Zhang 7, Hangjun Ye 3, Xiaojun Liang 2, Long Chen 3, Wenbo Ding 1,† |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42087 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.26839
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance Lingfeng Zhang 1,2,3, Xiaoshuai Hao 3,†,‡, Xizhou Bu 3,4, Yingbo Tang 5, Hongsheng Li 1, Jinghui Lu 3, Xiu-shen Wei 6, Jiayi Ma 7, Yu Liu 8, Jing Zhang 7, Hangjun Ye 3, Xiaojun Liang 2, Long Chen 3, Wenbo Ding 1,† Abstract Assisting humans in open-world outdoor environments requires a robot to translate high-level natural-language intentions into safe, long-horizon, and socially compliant navigation behavior. This capability is essential for human-centric applications such as last-mile delivery and blind guidance, yet remains challenging in complex outdoor scenarios. Existing map-based methods typically rely on costly pre-built HD maps, which are difficult to construct and maintain at scale, while still offering limited semantic understanding of high-level human intent. In contrast, learning-based navigation policies are mostly restricted to indoor and short-horizon settings. To bridge this gap, we propose Walk with Me , a map-free framework for long-horizon social navigation from high-level human instructions. Instead of requiring a pre-built HD map, Walk with Me only leverages GPS context and lightweight candidate points-of-interest from a public map API for semantic destination grounding and waypoint proposal. Specifically, a High-Level Vision-Language Model first grounds the user’s abstract instruction into a concrete destination and plans a coarse waypoint sequence. During execution, an observation-aware routing mechanism decides whether the current situation can be handled by the Low-Level Vision-Language-Action policy or should be escalated back to the High-Level VLM for explicit safety reasoning. Routine navigation segments are executed by the Low-Level VLA, which predicts socially compliant navigation actions from visual observations, the next waypoint, and recent trajectory context, while complex situations such as crowded crossings trigger high-level reasoning and stop-and-wait behavior when unsafe. The robot then executes the predicted actions, updates its state, and iterates until reaching the destination. By combining semantic intent grounding, map-free long-horizon planning, safety-aware high-level reasoning, and low-level action generation, Walk with Me enables practical outdoor social navigation for human-centric assistance. 1]Tsinghua University 2]Pengcheng Laboratory 3]Xiaomi EV 4]Fudan University 5 Institute of Automation, Chinese Academy of Sciences 6]Southeast University 7 Wuhan University 8]Hefei University of Technology \contribution [†]Corresponding authors \contribution [‡]Project leader \website https://linglingxiansen.github.io/Walk-with-Me/ Figure 1 : Motivation of Walk with Me . Previous navigation methods typically rely on pre-built HD maps, manually specified goals, or short-horizon indoor settings, making them difficult to apply to open-world human-centric assistance. In contrast, our framework grounds high-level human instructions into real-world destinations, plans long-horizon routes with lightweight public map priors, and performs socially compliant navigation with explicit safety reasoning. This enables practical outdoor applications such as last-mile delivery and blind guidance. 1 Introduction As embodied agents move from controlled laboratories into everyday human environments, they are increasingly expected not only to navigate to predefined coordinates, but also to walk with humans toward destinations described in natural language. Human-centric applications such as last-mile delivery and blind guidance require a robot to understand an abstract instruction such as “I want to go for a walk”, infer a meaningful real-world destination, and accompany the user there in a safe and socially compliant manner [ 19 , 11 , 50 , 44 , 51 , 26 ] . We refer to this setting as outdoor social navigation from high-level human instructions . Unlike conventional point-goal navigation or short-horizon indoor navigation, this problem simultaneously requires open-vocabulary intent grounding, long-horizon route reasoning, and socially aware local decision-making in dynamic outdoor scenes involving crosswalks, traffic, and crowds. Existing navigation systems are not well suited to this setting. Classical map-based pipelines [ 38 , 27 , 5 , 12 ] typically depend on pre-built high-definition (HD) maps that provide accurate metric geometry for localization and planning. Although such maps are effective in structured domains, they are costly to construct, difficult to maintain under frequent urban changes, and hard to scale to the long-tail outdoor environments that a human-assistance robot may encounter. More importantly, HD maps mainly encode geometry rather than human intent: they do not explain where a user wants to go when given an abstract request such as “take me somewhere to relax”. In contrast, recent learning-based navigation policies, including end-to-end visual navigation models [ 32 , 33 , 35 , 43 , 46 , 22 ] and vision-language-action (VLA) models [ 3 , 4 , 40 , 17 ] , have shown promising closed-loop behaviors, but most of them remain limited to indoor environments, short-horizon routes, or low-level goal specifications. Without explicit long-horizon grounding and safety reasoning, these policies struggle to scale to city-level outdoor navigation and may produce socially disruptive behaviors in complex pedestrian scenes [ 28 , 8 , 47 , 41 ] . This raises a central question: how can a robot convert high-level human intent into long-horizon, socially compliant outdoor navigation without relying on pre-built HD maps? Our key observation is that the information needed for global intent grounding and coarse route planning is already available in public map services. Such services provide lightweight geographic priors, including GPS locations, points of interest (POIs), and walking-route APIs, through a standard interface. This is fundamentally different from requiring the robot or the system operator to build and maintain an HD metric map. In this work, we use map-free to mean that the robot does not depend on a pre-built HD map for navigation. Instead, it uses public map services only as a lightweight semantic and geographic prior for destination grounding and coarse waypoint proposal, while relying on onboard perception and learned policies for local navigation. Based on this observation, we propose Walk with Me , a hierarchical framework for map-free, long-horizon outdoor social navigation from high-level human instructions. The core idea is to decouple where to go from how to walk there . A High-Level Vision-Language Model (VLM) interprets the user’s abstract instruction together with GPS context and candidate POIs from a public map service, selects a concrete destination, and generates a coarse sequence of geo-referenced waypoints. A Low-Level Vision-Language-Action (VLA) policy then takes the current visual observation, the next waypoint, and recent trajectory context to generate socially compliant local navigation actions. Between these two modules, an observation-aware router monitors the current scene and upcoming route segment. Routine segments are handled by the Low-Level VLA, while complex situations such as crosswalks, dense crowds, or ambiguous traffic conditions are escalated back to the High-Level VLM for explicit safety reasoning. When the scene is judged unsafe, the robot executes a stop-and-wait behavior until it is safe to proceed. The overall pipeline forms a closed perception–reasoning–action loop. Given a user instruction, Walk with Me first grounds the instruction into a destination and coarse route using GPS and public map-service candidates. During execution, the robot continuously updates its visual observation, pose, trajectory history, and next waypoint. The router decides whether the current step should be solved by low-level trajectory generation or high-level safety reasoning. The robot executes the resulting action, updates its state, and repeats this process until the destination is reached. This design allows the system to retain the scalability of public geographic services, the semantic reasoning ability of VLMs, and the closed-loop control capability of VLA policies, without requiring a pre-built HD map. We deploy Walk with Me on a commodity wheeled robot and evaluate it