SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planning

Zebin Han, Xudong Wang, Baichen Liu, Qi Lyu, Zhenduo Shang, Jiahua Dong, Lianqing Liu, Zhi Han · 2026

Read SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planning by Zebin Han, Xudong Wang, Baichen Liu, Qi Lyu, Zhenduo Shang, Jiahua Dong, Lianqing Liu, Zhi Han (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-04699.txt.

Introduction

Zebin Han and seven co-authors send a navigation agent on errands in sequence rather than one trip at a time, in a 2026 arXiv paper. In sequential-horizon vision-and-language navigation, a single long instruction bundles several tasks, and standard navigation models degrade sharply: the information overload stops the agent attending to the visual details that matter right now.

SeqWalker's answer is hierarchy. A high-level planner watches the current scene and selects which sub-instruction is active, shrinking the agent's cognitive load to the errand in progress. A low-level planner runs an exploration-verification loop that exploits the instruction's logical structure to catch and correct trajectory errors. To measure this neglected setting, the team also extended an existing indoor navigation dataset into a benchmark for the sequential task. Experiments across it show SeqWalker clearly ahead of current navigation models.

Catalog record

CREATORZebin Han, Xudong Wang, Baichen Liu, Qi Lyu, Zhenduo Shang, Jiahua Dong, Lianqing Liu, Zhi Han
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-04699

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planning Zebin Han, Xudong Wang, Baichen Liu, Qi Lyu, Zhenduo Shang, Jiahua Dong, Lianqing Liu, Zhi Han Abstract Sequential-Horizon Vision-and-Language Navigation (SH-VLN) presents a challenging scenario where agents should sequentially execute multi-task navigation guided by complex, long-horizon language instructions. Current vision-and-language navigation models exhibit significant performance degradation with such multi-task instructions, as information overload impairs the agent’s ability to attend to observationally relevant details. To address this problem, we propose SeqWalker, a navigation model built on a hierarchical planning framework. Our SeqWalker features: i) A High-Level Planner that dynamically selects global instructions into contextually relevant sub-instructions based on the agent’s current visual observations, thus reducing cognitive load; ii) A Low-Level Planner incorporating an Exploration-Verification strategy that leverages the inherent logical structure of instructions for trajectory error correction. To evaluate SH-VLN performance, we also extend the IVLN dataset and establish a new benchmark. Extensive experiments are performed to demonstrate the superiority of the proposed SeqWalker. Code — https://seqwalker.github.io/seqwalker/ Introduction Vision-and-Language Navigation (VLN) has emerged as a critical component for embodied AI, requiring intelligent agents to interpret natural language instructions and navigate through complex environments ( 40 ; 14 ; 7 ) . This capability holds significant promise for diverse applications, such as autonomous robotics ( 39 ; 35 ) , home robots ( 12 ) , and robots operating in complex dynamic environments ( 10 ; 32 ; 13 ) . While recent advances ( 5 ; 50 ) have demonstrated remarkable progress in single-trajectory VLN scenarios, the more practical challenge of sequential multi-task navigation remains largely unaddressed. In practical applications, agents are often required to execute multiple sub-tasks sequentially within a single deployment session ( 28 ; 17 ) . For example, on an usual morning, a student asks an agent to help him get up and go to school. This requires the agent to complete a series of tasks in sequence: i) take the clothes from the wardrobe, ii) take me to the bathroom for washing up, iii) prepare breakfast, iv) put the umbrella in my schoolbag, as illustrated in Fig. 5 . The navigation agent is required to complete the user’s tasks sequentially and reach the destination. Different from the traditional VLN task, where the user describes one navigation trajectory with one instruction, the user only needs to provide the agent with one sequential-horizon instruction to enable the agent to complete multiple navigation tasks. This is more time-effective for the user, but it also poses more challenges. The sequential-horizon navigation means larger navigation scenes with more complex user instructions . Current VLN methods ( 29 ; 41 ) are dedicated to persistent and efficient navigation in large scenes. Some methods ( 44 ; 51 ) store the historical state of previous processes and feed it into the agent’s current state. Iterative Vision-and-Language Navigation (IVLN) ( 19 ) proposes a map-based navigation method that maps the scene to maps during navigation. These advancements improve the navigation performance in persistent scenes and bring VLN research closer to large-scale scene applications. On the other hand, as the navigation scene expands, the complexity of the navigation task increases further, which means that the user describes multiple navigation tasks with one instruction, and the user inevitably describes multi-task trajectories with more complex and longer sequential-horizon instructions. However, the performance of existing VLN models is severely degraded with sequential-horizon instructions, as the excessive information hinders the agent from focusing on details relevant to its current observations. Figure 1: Illustration of the proposed Sequential-Horizon Vision-and-Language Navigation (SH-VLN) task and our SeqWalker model. Different from traditional VLN, SH-VLN requires navigation agents to follow a sequential multi-task trajectory navigation, while users tend to provide complex long instructions, posing greater challenges. Our SeqWalker adopts a Hierarchical Planning strategy, where the High-Level Planner selects sub-tasks and instruction phrases based on the agent’s observations, and the Low-Level Planner provides actions for robust navigation using a proposed Exploration and Verification strategy. To enable sequential multi-task navigation in large scenes, we introduce a new VLN task called S equential- H orizon V ision-and- L anguage N avigation (SH-VLN). This task emphasizes navigating long distances in large scenes using sequential-horizon instructions, which presents three main challenges: i) Scene Persistence. In real-world applications, a deployed navigation agent often operates within the same environment over extended periods ( 16 ; 4 ) . To ensure efficient operation, the agent should be able to save scene information rather than treat each episode as a new exploration. ii) Complex Long Instructions. Different from a single-period navigation task, describing multi-task navigation in large scenes inevitably requires more complex and longer instructions to capture navigation trajectories. However, long and complex instructions interfere with the traditional navigation agents’ understanding of instructions and make it difficult to match them to the current observation, leading to navigation failures. iii) Multi-Trajectory Robustness. Sequential-horizon navigation often means longer navigation trajectories. Sequential navigation in long trajectories is more likely to make errors, and once an error occurs, the trajectory is likely to worsen with it, so there are high demands on the robustness of navigation. An intuitive way to solve the long instruction problem is to directly leverage the powerful long sequence reasoning capability of large language models (LLMs). However, directly applying LLMs is ineffective. Although LLMs ( 24 ; 47 ) with several hundred billion parameters handle long sequences well, and models with smaller parameters that can be embedded in robots have shortcomings for long sequences. Additionally, directly applying LLM cannot guide accurate actions due to the lack of specific knowledge about the navigation scene. To solve the SH-VLN task, we propose a SeqWalker, a novel VLN model built on a hierarchical planning framework. SeqWalker is capable of hierarchically understanding the user sequential long instructions step-by-step, while explicitly saving the scene maps to achieve efficient sequential-horizon navigation in large scenes, as shown in the Fig. 5 . Specifically, in High-Level Planner , SeqWalker performs local segmentation on global long instructions using pre-trained CLIP models to obtain the most relevant segmented instruction with the agent’s current scenes. Compared to global long instructions, the segmentation instructions not only reduce the difficulty of focusing information but also enable corrections for navigation trajectory errors. In Low-Level Planner , we develop an Exploration and Verification strategy to achieve forward progress toward destinations or to leverage the inherent logical order of instructions to dynamically correct the current navigation trajectories, which further improves the robustness of long navigation. To evaluate the proposed SH-VLN task, we extend the existing IVLN dataset ( 19 ) and propose a new benchmark. Extensive experiments are performed to demonstrate the effectiveness and superiority of our proposed SeqWalker. The main contributions of this work are as follows: • We introduce Sequential-Horizon Vision-and-Language Navigation (SH-VLN), a new VLN task where agents follow sequential-horizon instructions to complete a sequential multi-task navigation trajectory. We also provide a new benchmark for evaluating the SH-VLN task. • We propose a SeqWalker model, which is built on a hierarchical planning framework. Our SeqWalker performs local segmentation on long instructions to understand instructions step-by-step while explicitly saving scene maps for sequential-horizon navigation in large scenes. • We develop an Exploration and Verification strategy. Based on this, SeqWalker navigation

Read the complete text