OpenVLN: Open-world Aerial Vision-Language Navigation

Peican Lin, Gan Sun, Chenxi Liu, Fazeng Li, Weihong Ren, Yang Cong · 2025

Read OpenVLN: Open-world Aerial Vision-Language Navigation by Peican Lin, Gan Sun, Chenxi Liu, Fazeng Li, Weihong Ren, Yang Cong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06182.txt.

Introduction

Peican Lin and co-authors present OpenVLN, a framework that lets drones follow spoken or written directions in open outdoor settings. Vision-language models already guide ground robots, but aerial flight brings harder data collection and much longer routes. OpenVLN aims to handle language-guided flight while training on only limited data.

A reshaped reinforcement learning setup fine-tunes the vision-language model for drone navigation using rule-based policies. A separate long-horizon planner builds flight paths step by step, guided by value-based rewards. On the TravelUAV benchmark, the method raised success rate by up to about four percentage points over baselines, with similar gains on path quality measures.

Catalog record

CREATORPeican Lin, Gan Sun, Chenxi Liu, Fazeng Li, Weihong Ren, Yang Cong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-06182

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OpenVLN: Open-world Aerial Vision-Language Navigation Peican Lin, Gan Sun, Chenxi Liu, Fazeng Li, Weihong Ren, Yang Cong Abstract Vision-language models (VLMs) have been widely-applied in ground-based vision-language navigation (VLN). However, the vast complexity of outdoor aerial environments compounds data acquisition challenges and imposes long-horizon trajectory planning requirements on Unmanned Aerial Vehicles (UAVs), introducing novel complexities for aerial VLN. To address these challenges, we propose a data-efficient Open -world aerial V ision- L anguage N avigation ( i.e., OpenVLN) framework, which could execute language-guided flight with limited data constraints and enhance long-horizon trajectory planning capabilities in complex aerial environments. Specifically, we reconfigure a reinforcement learning framework to optimize the VLM for UAV navigation tasks, which can efficiently fine-tune VLM by using rule-based policies under limited training data. Concurrently, we introduce a long-horizon planner for trajectory synthesis that dynamically generates precise UAV actions via value-based rewards. To the end, we conduct sufficient navigation experiments on the TravelUAV benchmark with dataset scaling across diverse reward settings. Our method demonstrates consistent performance gains of up to 4.34% in Success Rate, 6.19% in Oracle Success Rate, and 4.07% in Success weighted by Path Length over baseline methods, validating its deployment efficacy for long-horizon UAV navigation in complex aerial environments. I INTRODUCTION Vision-language navigation (VLN) [ 1 ] is a cornerstone task for embodied agents, it demands that agents traverse intricate, real-world environments solely via following natural-language instructions. Recent VLN research has grew significantly, with notable proliferation of novel methodologies in this domain. A central focus of VLN research involves constructing world models that enable agents to comprehend their surrounding environments [ 6 ] . Simultaneously, the emergence of large pre-trained vision-language models (VLMs) has spurred a rise of VLN methodologies utilizing fune-tuning techniques [ 13 ] . As the performance of pre-trained VLMs improves, researchers aim to develop agents capable of reasoning and planning trajectories from human instructions. Fig. 1: Challenge illustration for long-distance Vision-Language Navigation task, where drone needs to navigate to the destination according to the given instruction. Extended planning trajectories propagate environmental uncertainty, compounding localization error accumulation. Although current research has achieved substantial progress in grounded navigation [ 30 , 3 ] , aerial navigation [ 28 ] for unmanned aerial vehicles (UAVs) remains critically underexplored. Compared to ground-based VLN tasks, UAV-based VLN confronts significantly more complex environments. Compounding this challenge, practical aerial data collection difficulties result in scarcity of adequate UAV-VLN datasets that impede real-world implementation. Furthermore, conventional approaches fail to maintain a precise planning trajectory during long-horizon navigation , especially in unstructured aerial domains. While AerialVLN [ 15 ] and OpenUAV [ 23 ] establish vision-language navigation benchmarks for UAVs, their methodologies necessitate fine-tuning on large-scale simulated datasets. This constraint is unfavorable for real-world deployment due to the inherent challenges of aerial data acquisition [ 22 ] . Furthermore, these approaches demonstrate limited long-horizon planning capabilities, resulting in pronounced performance degradation when confronted with extended navigation tasks. To further analyze these practical challenges, we investigate aerial VLN for UAVs under dual constraints of data scarcity and long-horizon planning requirements . Specifically: (1) Data scarcity impedes cross-modal representation learning, degrading the accuracy of navigation task under sparse training dataset; (2) Extended planning trajectory propagate environmental uncertainty, compounding localization error accumulation across extended trajectories. Conventional approaches [ 7 , 5 , 24 ] fail to jointly address these issues as supervised VLM adaptation lacks policy generalization and classical planners face limitations with mission-scale contingencies. Fig. 2: Overall architecture of our OpenVLN framework. (a) The VLN-CE replanner that controls flight actions; (b) and (c) together form an Reinforcement Learning framework for data-efficient VLM fine-tuning under data scarcity, including a verifiable reward, value model reward at waypoints and an RL fine-tuning optimizer; and (d) a vision-language navigation model with a sensing encoder, multi-modal grounding module [ 14 ] , and an action decoder that predicts the next waypoint, producing the planned trajectory and the UAV pose. To resolve the challenges above, in this paper, we develop an Open -world aerial V ision- L anguage N avigation (OpenVLN) framework. Our framework introduces a novel approach to the OpenUAV benchmark, leveraging reinforcement learning and a value reward mechanism to enable efficient VLM fine-tuning under data scarcity constraints. On the one hand, to efficiently fine-tune the VLM, we design a rule-based reinforcement learning method that optimizes VLM updates through a novel reward function. Departing from conventional reward mechanisms, this approach streamlines policy construction while enabling self-evolution during model fine-tuning, extracting robust task-aligned feature representations from minimal data samples. This approach substantially outperforms conventional supervised learning methods, demonstrating enhanced robustness when only sparse training data are available. On the other hand, we tackle long-horizon navigation challenges by introducing a value-based reward optimizer that synthesizes extended trajectory sequences. Unlike conventional VLN methods that supervise trajectories only at the final waypoint, our approach provides dense, per-waypoint supervision: every waypoint along the trajectory is evaluated by a learned value model, yielding verifiable intermediate rewards, better credit assignment over long horizons, and improved stability. The value optimizer evaluates waypoint quality and provides dense feedback signals by fusing visual observations with instruction features during planning. A value model is utilized to assess state quality and guide optimal path selection under environmental uncertainty in long-range trajectory planning. This enhances the accuracy of UAV trajectory planning, enabling the generation of smoother, more robust trajectories. Consequently, it facilitates efficient path planning for complex navigation tasks during UAV operations. The main contributions of this paper are as follows: • We propose an Open -world aerial V ision- l anguage N avigation ( i.e., OpenVLN) framework, a novel architecture that synergizes reinforcement learning for efficient VLM fine-tuning with long-horizon trajectory VLN-CE planning for UAV navigation. • We develop a self-evolving reinforcement learning optimization strategy with task-aligned rewards, enabling robust VLM fine-tuning using minimal aerial data while outperforming supervised methods. • We introduce a value-model–guided long-horizon reward mechanism that combines dense per-waypoint reward-estimates with a verifiable reward, guiding optimal waypoint selection and improving accuracy and stability in long-horizon UAV navigation. II RELATED WORK II-A Vision-Language Navigation Researchers commonly employ pretrained vision-language models (VLMs) and fine-tune them for vision-language navigation (VLN), yet real-world UAV scenarios suffer from severe data scarcity . Early VLN datasets, such as R2R [ 1 ] , REVERIE [ 19 ] , and SOON [ 29 ] , primarily focused on indoor environments, equipping agents with complex scene exploration abilities following natural language instructions. In , AerialVLN [ 15 ] first brings discrete-action, long-trajectory navigation to aerial scenes; CityNav [ 12 ] scales it to real-world maps yet still relies on fixed action sets and pre-computed global plans. However, their reliance on fixed action sets and precomputed long-range plans limits adaptability to diverse outdoor distributions. CMA-based models [ 5 ] require large, diverse paired image-instruction examples to learn stable alignments but often overfit or forget rare visual cues in limited UAV datasets. End-to-end policies [ 20 ] , which map raw pixels and language directly to low-level controls, demand heavy supervision and extensive training, leading to slow convergence and brittleness to imperfect localization and distribution shifts . To address these limitations, recent work has transitioned from discrete nav-graphs to continuous settings, exposing agents to low-level control

Read the complete text