Contextualized Autonomous Drone Navigation using LLMs Deployed in Edge-Cloud Computing

Hongqian Chen, Yun Tang, Antonios Tsourdos, Weisi Guo · 2025

Read Contextualized Autonomous Drone Navigation using LLMs Deployed in Edge-Cloud Computing by Hongqian Chen, Yun Tang, Antonios Tsourdos, Weisi Guo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00607.txt.

Introduction

Hongqian Chen, Yun Tang, Antonios Tsourdos, and Weisi Guo give drones a language model that rewrites their navigation on the fly. Drone training cannot cover every surprise the real world holds, yet a human operator can describe a surprise in words. Their 2025 study deploys large language models inside the navigation loop, adjusting map settings and even deriving new task instructions from spoken context.

The open question the paper tackles is placement: these models are heavy, and future 6G networks split computing between the device, the edge, and the cloud. The team tests which models handle the two jobs well and where each should sit in that architecture. The aim is a drone that hears 'the field is flooded on the east side' and reshapes its route the way a human pilot would. The authors frame it as a design map for language-guided flight.

Catalog record

CREATORHongqian Chen, Yun Tang, Antonios Tsourdos, Weisi Guo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-00607

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Contextualized Autonomous Drone Navigation using LLMs Deployed in Edge-Cloud Computing Hongqian Chen, Yun Tang, Antonios Tsourdos, Weisi Guo Abstract Autonomous navigation is usually trained offline in diverse scenarios and fine-tuned online subject to real-world experiences. However, the real world is dynamic and changeable, and many environmental encounters/effects are not accounted for in real-time due to difficulties in describing them within offline training data or hard to describe even in online scenarios. However, we know that the human operator can describe these dynamic environmental encounters through natural language, adding semantic context. The research is to deploy Large Language Models (LLMs) to perform real-time contextual code adjustment to autonomous navigation. The challenge not evaluated in literature is what LLMs are appropriate and where should these computationally heavy algorithms sit in the computation-communication edge-cloud computing architectures. In this paper, we evaluate how different LLMs can adjust both the navigation map parameters dynamically (e.g., contour map shaping) and also derive navigation task instruction sets. We then evaluate which LLMs are most suitable and where they should sit in future edge-cloud of 6G telecommunication architectures. I Introduction Smart cities will increasingly integrate autonomous vehicles in the ground and in the air (eVTOLs and air taxis) for a variety of purposes. Autonomous navigation has been a subject of intensive research and development in recent decades, especially in urban and congested areas [ 1 , 2 ] . Dynamically adjusting autonomous navigation in response to emerging situations, effects, and values remains hard encoded. For example, a global path planning algorithm (e.g., A-star) may toggle between a set of penalty functions (e.g., low carbon footprint route, fastest route, smoothest route, avoid certain locations route) based on user requirements. Although these reflect certain user values, the algorithms cannot consider all personal preferences and situational contexts at the offline programming and training stage. The opportunity is that LLMs can enhance navigation performance by personalizing the code to adapt to emerging real-world changes [ 3 , 4 ] . For example, as shown in Fig. 1 , the human user may interpret certain obstacles differently depending on context or situation of that day, this is difficult to pre-code into the algorithm. As such, LLMs can help translate user reactions in real-time into potential fields that augment the navigation algorithm. I-A Review of LLM-informed Navigation The attention-based transformer architecture in LLMs utilizes a self-attention mechanism to process input data. Unlike previous models such as RNNs and LSTMs, which process data sequentially, transformers handle input data in parallel. This parallel processing not only enhances model efficiency but also strengthens the model’s ability to understand the context within language. During the training of an LLM, the model typically undergoes two main phases: pre-training and fine-tuning: • In the pre-training phase, the model is trained on a large amount of text using unsupervised learning techniques to gain a general understanding of language and context. • In the fine-tuning phase, the model is adjusted on a smaller, task-specific dataset to perform well on specific tasks such as answering questions or classification. LLMs as such have the strong ability to semantically contextualize a wide range of algorithms. For example, in [ 5 ] the research explores the potential of utilizing GPT-3.5-turbo from OpenAI, for robotic path planning. In recent years [ 6 ] , work has been done to use LLMs to generate rare scenarios so that offline training in autonomous navigation can achieve safer and more contextualized path planning and improve trustworthy [ 7 ] . The path planning element maybe an optimisation route based on globally known parameters (e.g., A-star) and/or adjust locally to emerging challenges using an offline trained data-driven reinforcement learning (RL) algorithm (e.g., toggle policies in RL [ 8 ] and improve motion control [ 9 ] ). In our recent work [ 10 ] , we encoded different philosophical and social values in how path planning algorithms react to emerging hazards and challenges such that users can perform real-time online soft-encoding of algorithms during a navigation process using natural language. More recent advances have also added multi-modal data such as language-image associations to better understand landmarks and obstacles in navigation [ 11 ] and interact with them [ 12 ] . Most existing work in this emerging area has largely focused on a single navigation process (e.g., route planning around obstacles) and not consider multiple sub-task-based navigation (e.g., take-off, track target, and land). Neither has existing work considered the computation requirements of LLMs in realistic distributed mobile systems and has largely used cloud-based services with high latency and a strong wired communication network requirement. I-B Gap and Novelty The research is to deploy Large Language Models (LLMs) to perform real-time contextual code adjustment to autonomous navigation. LLMs translate user natural language into formal code generation, modifications, or parameterizations in real time. What is not sufficient acknowledged are two important areas: (1) how can LLMs generate instruction sets for autonomous navigation that requires several sub-tasks, and (2) which LLMs are most suitable for mobile agents and where they should sit in future edge-fog-cloud continuum of 6G telecommunication architectures. We aim to show how we can create realistic urban drone navigation scenarios via a simulation platform and: • Using LLMs to contextualize path planning - digesting situational contexts and new obstacle data. • Using Small language models to generate sequence of sub-tasks and tactical control commands for a planned path; • Evaluate the suitability of different LLMs (GPT, LLama, Moonshot, Qwen) in terms of computation and latency and where they should sit architecturally to enable the above functionalities; Combined, they offer a new level of LLM-navigation architecture for real-time navigation situations for future urban mobility. Fig. 1: System Diagram of LLM-navigation: (A) 3D city map, (B) Mission definition, (C) LLM generated instruction set for sub-tasks, (D) Navigation via way points and potential fields, and (E) Contextualized potential field adjustment using LLMs. II System Setup and Methodology The overall system architecture shown in Fig. 1 is formed of several intertwined components that aims at navigating whilst minimizing a cost function related to obstacle and situational potential fields: (A) 3D city map, (B) Contextualized potential field adjustment using LLMs to enable (C) Path planning via way points and potential fields, and LLM generated instruction set for drone flight sub-tasks for the path planned. Fig. 2: A) Map parsing with descriptive inputs, B) Potential field adjustment with contextual inputs. Fig. 3: Native-AI implemented inside ORAN architecture to achieve near real-time path planning using LLMs and real-time tactical control of sub-tasks using smaller language models. II-A LLM-informed Navigation: Path Planning Level For the LLM to adjust navigation algorithms (Fig. 1 A-C), it is first necessary to inform the LLM of the format of the original map, allowing the LLM to understand and analyze the meaning of the JSON-formatted map. Subsequently, the changes in the map must be conveyed to the LLM in human language, and the LLM is required to comprehend the updated information about the map. It should then alter the map according to the construction method of the original map, outputting the new map in the JSON format of the original map. Finally, the map outputted by the LLM needs to be passed to path

Read the complete text