Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving

Zhihua Hua, Junli Wang, Pengfei Li, Qihao Jin, Bo Zhang, Kehua Sheng, Yilun Chen, Zhongxue Gan, Wenchao Ding · 2026

Read Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving by Zhihua Hua, Junli Wang, Pengfei Li, Qihao Jin, Bo Zhang, Kehua Sheng, Yilun Chen, Zhongxue Gan, Wenchao Ding (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12208.txt.

Introduction

Hua and eight colleagues find end-to-end driving systems lean too hard on local scene understanding while underusing global navigation input, so planning correlates weakly with navigation. Their Sequential Navigation Guidance framework encodes navigation paths for long-term trajectories plus turn-by-turn cues for real-time choices, with a matching question-answering dataset. Their fused model leads the field through precise navigation modeling, needing no extra perception losses.

Catalog record

CREATORZhihua Hua, Junli Wang, Pengfei Li, Qihao Jin, Bo Zhang, Kehua Sheng, Yilun Chen, Zhongxue Gan, Wenchao Ding
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32314
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12208

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving Zhihua Hua, Junli Wang, Pengfei Li, Qihao Jin, Bo Zhang, Kehua Sheng, Yilun Chen, Zhongxue Gan, Wenchao Ding Abstract Global navigation information and local scene understanding are two crucial components of autonomous driving systems. However, our experimental results indicate that many end-to-end autonomous driving systems tend to over-rely on local scene understanding while failing to utilize global navigation information. These systems exhibit weak correlation between their planning capabilities and navigation input, and struggle to perform navigation-following in complex scenarios. To overcome this limitation, we propose the Sequential Navigation Guidance (SNG) framework, an efficient representation of global navigation information based on real-world navigation patterns. The SNG encompasses both navigation paths for constraining long-term trajectories and turn-by-turn (TBT) information for real-time decision-making logic. We constructed the SNG-QA dataset, a visual question answering (VQA) dataset based on SNG that aligns global and local planning. Additionally, we introduce an efficient model SNG-VLA that fuses local planning with global planning. The SNG-VLA achieves state-of-the-art performance through precise navigation information modeling without requiring auxiliary loss functions from perception tasks. Project page: SNG-VLA I Introduction In recent years, end-to-end autonomous driving systems have garnered significant attention from researchers [ 34 , 35 ] . The end-to-end paradigm simplifies traditional modular systems, is better suited to data-driven training approaches, and demonstrates enhanced generalization performance [ 4 ] . Global Navigation information plays a pivotal role in end-to-end autonomous driving systems [ 18 ] , providing essential directional references for trajectory planning. Unlike prediction [ 7 ] , which generates multimodal trajectory forecasts, planning requires explicit navigation inputs to produce deterministic driving trajectories [ 17 , 11 ] . Despite the critical role of global navigation information in end-to-end autonomous driving systems, we have made a surprising observation: removing or corrupting navigation information in existing end-to-end driving methods minimally affects planning performance and, in some cases, even improves performance . For instance, as illustrated in Fig. 1 , our experiments with the Transfuser [ 6 ] on the NAVSIM [ 8 ] benchmark demonstrate that complete removal of navigation information paradoxically yields superior results. This phenomenon contradicts basic driving logic, as one would anticipate a substantial decline in planner performance when explicit navigation information is absent. This unexpected outcome raises a critical question: Do current end-to-end autonomous driving systems truly understand and utilize global navigation information? Fig. 1 : We demonstrate the impact of introducing perturbations to the driving command and the effectiveness of the SNG. The command ablation experiments comprise five distinct experimental settings, with Transfuser-command-ablation representing the statistical results across all five experiments. Detailed data are presented in Table IV . The utilization of random or erroneous navigation information demonstrates minimal impact on model performance. SNG exhibits significant performance improvement over the baseline approach. Our answer is unequivocally negative. Current research [ 32 , 29 , 41 ] predominantly employs driving commands (such as “Turn Left”, “Go Forward”, “Turn Right”, “None”) to represent global navigation information, utilizing one-hot encoding to discretize driving behaviors into finite categories. However, as shown in Fig. 2 , this approach exhibits the following limitations: (1) the annotation process relies on a fixed temporal horizon or spatial intervals [ 17 , 8 ] , which can lead to ambiguous interpretations in complex scenarios. In the roundabout, the significant lateral displacement of the vehicle going forward caused the driving command to be incorrectly labeled as a “Turn Left”. (2) this representation suffers from oversimplification. In beyond visual range (BVR) scenarios [ 26 ] , a vehicle must change lanes in advance to execute a turn at a distant intersection. However, as the global navigation command is labeled as “Go Forward”, it creates causal confusion in the model when encountering lane-changing behaviors present in expert trajectories. Consequently, numerous end-to-end autonomous driving systems fail to effectively utilize navigation information, and their performance likely stems from overfitting to specific input channels [ 39 , 23 ] . To address these limitations and enhance the navigation semantic comprehension capabilities of end-to-end autonomous driving systems, we propose a novel paradigm of Sequential Navigation Guidance (SNG), inspired by real-world navigation patterns [ 31 ] . The SNG effectively represents navigation information by integrating static global path planning with dynamic high-level guidance: (1) Navigation Path: A predefined trajectory segment extracted from the global path, serving as a reference line for planning; (2) Real-time Turn-by-Turn (TBT) Information: A comprehensive set of high-level guidance cues comprising current driving actions with associated distance and time estimations, future actions, and corresponding supplementary actions, which collectively inform the planning process. Notably, both types of information can be conveniently acquired through navigation APIs and are readily available off-the-shelf in practical deployment. To further align local planning with global planning, we propose SNG-QA, which imposes additional constraints on local planning within both the reasoning and action spaces. The SNG-QA dataset comprises 100K QA pairs that decompose the reasoning process into hierarchical planning components. The local planning reasoning process integrates both global planning outcomes and local scene understanding. We utilize NAVSIM [ 8 ] and its annotations to construct the SNG-QA pairs according to predefined task formats. We introduce an efficient model SNG-VLA. The model employs multimodal fusion encoder and a unified transformer backbone, which can efficiently handle the constraints between global navigation information and local scene inputs. The model autoregressively generates textual reasoning and planning trajectories. The proposed method demonstrates remarkable efficacy in modeling global navigation information, offering a plug-and-play solution that significantly enhances the planning capabilities of end-to-end autonomous driving systems. Our model also achieves state-of-the-art performance on both the Bench2Drive [ 15 ] a closed-loop benchmark based on Carla [ 9 ] and the NAVSIM [ 8 ] a real-world evaluation benchmark. Our contributions can be summarized as follows: 1. To address the limitations of current global navigation representation, we propose a novel Sequential Navigation Guidance approach to structure global navigation information, offering both long-term trajectory constraints and real-time decision-making logic. 2. We propose the SNG-QA dataset, which partitions the reasoning process into global and local planning components to enhance local planning rationality and ensure consistency between local and global planning. 3. We develop an effective model that, without incorporating auxiliary tasks and utilizing precise navigation information, achieves state-of-the-art (SOTA) performance in both Bench2Drive and NAVSIM benchmarks. Fig. 2 : We demonstrate erroneous examples in the annotation process of driving commands. (a) Incorrect annotation occurs in roundabouts due to longitudinal displacement. (b) The annotation information fails to comprehend lane-changing behavior, resulting in causal confusion. Fig. 3 : Overview of our pipeline. Sequential navigation guidance is consists of navigation path and TBT information. SNG-QA comprises three subtasks: global planning, local planning, and trajectory planning. The architecture of our model is divided into two parts: the multimodal feature fusion encoder and the unified transformer backbone. II Related Work II-A End-to-end autonomous driving Traditional autonomous driving systems are often composed of multiple modular components [ 22 , 25 ] , whereas end-to-end autonomous driving enables a direct mapping from raw sensor data to planning trajectories [ 12 , 3 ] . Most methods [ 33 , 13 ] and dataset [ 2 ] utilize driving commands as global

Read the complete text