Next-Generation LLM for UAV: From Natural Language to Autonomous Flight
Liangqi Yuan, Chuhao Deng, Dong-Jun Han, Inseok Hwang, Sabine Brunswicker, Christopher G. Brinton · 2025
Read Next-Generation LLM for UAV: From Natural Language to Autonomous Flight by Liangqi Yuan, Chuhao Deng, Dong-Jun Han, Inseok Hwang, Sabine Brunswicker, Christopher G. Brinton (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21739.txt.
Introduction
Liangqi Yuan and five co-authors chart how large language models can run drone operations end to end, from a spoken order to a flown mission. Research so far has toyed with small drones and isolated pieces like path planning. Real medium and long-range aircraft bring airport procedures and dense regulations. They also carry unforgiving mission standards that toys never face.
Their NeLV system chains five components. A language model parses the instruction, and a route planner picks points of interest. A path planner lays waypoints, and a control platform flies the trajectory. Monitoring watches the execution. Demonstrations cover multi-drone patrol, multi-stop delivery, and multi-hop relocation. Beyond the working system, a five-level taxonomy maps the road from today's parser role to a fully autonomous autopilot. It names the technical prerequisites and open challenges at each rung.
Catalog record
| CREATOR | Liangqi Yuan, Chuhao Deng, Dong-Jun Han, Inseok Hwang, Sabine Brunswicker, Christopher G. Brinton |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21739
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Next-Generation LLM for UAV: From Natural Language to Autonomous Flight Liangqi Yuan, Chuhao Deng, Dong-Jun Han, Inseok Hwang, Sabine Brunswicker, Christopher G. Brinton Abstract With the rapid advancement of Large Language Models (LLMs), their capabilities in various automation domains, particularly Unmanned Aerial Vehicle (UAV) operations, have garnered increasing attention. Current research remains predominantly constrained to small-scale UAV applications, with most studies focusing on isolated components such as path planning for toy drones, while lacking comprehensive investigation of medium- and long-range UAV systems in real-world operational contexts. Larger UAV platforms introduce distinct challenges, including stringent requirements for airport-based take-off and landing procedures, adherence to complex regulatory frameworks, and specialized operational capabilities with elevated mission expectations. This position paper presents the Next-Generation LLM for UAV ( NeLV ) system—a comprehensive demonstration and automation roadmap for integrating LLMs into multi-scale UAV operations. The NeLV system processes natural language instructions to orchestrate short-, medium-, and long-range UAV missions through five key technical components: (i) LLM-as-Parser for instruction interpretation, (ii) Route Planner for Points of Interest (POI) determination, (iii) Path Planner for waypoint generation, (iv) Control Platform for executable trajectory implementation, and (v) UAV monitoring. We demonstrate the system’s feasibility through three representative use cases spanning different operational scales: multi-UAV patrol, multi-POI delivery, and multi-hop relocation. Beyond the current implementation, we establish a five-level automation taxonomy that charts the evolution from current LLM-as-Parser capabilities (Level 1) to fully autonomous LLM-as-Autopilot systems (Level 5), identifying technical prerequisites and research challenges at each stage. Project page with code and videos: https://liangqiyuan.github.io/NeLV/ . I Introduction The rise of Large Language Models (LLMs) has transformed numerous domains, such as mobile services, vehicles, and robotics [ 1 , 2 , 3 ] . These fields have become increasingly intelligent and user-friendly through LLM integration, enabling command and control through natural language. This conversation-based control between humans and LLMs improves both the ability of LLMs to interpret context and the convenience with which humans can direct LLMs to perform actions [ 4 ] . LLMs fulfill diverse roles within these systems. LLM-as-Router can orchestrate task allocation and model selection for human pilots, LLM-as-Agent can execute actions on behalf of humans, and LLM-as-Judge can conduct evaluations in place of human judgment. The increasing specialization of LLMs in distinct roles throughout systems optimizes their performance through clear responsibility allocation. Consequently, LLMs are becoming an essential component of next generation autonomous control and self-driving technologies. Fig. 1: Overview of the NeLV System. TABLE I: Comparison of Related Literature of LLM for UAV System Literature Year Type Key Topic Implementation Case Study Short-Range Medium-Range Long-Range [ 5 ] 2024 Survey LLMs for UAVs - - - - [ 6 ] 2025 Survey LLMs for UAVs - - - - [ 7 ] 2024 Survey Generative AI for UAVs - - - - [ 8 ] 2024 Survey Generative AI for UAV Networks - - - - [ 9 ] 2024 Survey Generative AI for UAV Networks - - - - [ 10 ] 2024 Benchmark Simulation Platform Simulator City UAV Simulator - - [ 11 ] 2023 Research LLM for UAV Planning Indoor Small UAV Mission Planning - - [ 12 ] 2023 Research LLM for UAV Planning Simulator Mission Planning - - [ 13 ] 2024 Research LLM for UAV Planning Indoor Small UAV Object Detection - - [ 14 ] 2024 Research LLM for UAV Planning Indoor Small UAV Obstacle Prediction - - [ 15 ] 2024 Research LLM for UAV Planning Indoor Small UAV Mission Planning - - [ 16 ] 2024 Research LLM for UAV Planning Outdoor Small UAV Swarm Formation - - Ours 2025 Position Paper • LLM-as-Parser • Route Planning • Path Planning • Control Platform • UAV Monitoring Outdoor Medium UAV Multi-UAV Patrol Multi-POI Delivery Multi-Hop Relocation Unmanned Aerial Vehicles (UAVs) represent an important component of next generation transportation [ 17 , 18 ] . According to data from the Federal Aviation Administration (FAA), registered UAVs in the United States exceed 1 million as of March 2025, with 427,335 remote pilots certified [ 19 ] . As UAV numbers increase, this human pilot workforce proves inadequate and represents one of the limitations preventing UAVs from assuming a more substantial role in transportation. How to enable fewer human pilots to operate a single UAV or allow one human pilot to control multiple UAVs remains an ongoing research question. This challenge necessitates assistance from powerful automation systems to help human pilots process information, comprehend data, suggest operations, execute actions, and communicate effectively. LLMs emerge as strong candidates not only because they accept language input, which proves considerably more efficient than alternative command inputs, but also because their robust reasoning capabilities enable them to process UAV-related information effectively. Although considerable literature focuses on LLM for UAV systems, most studies remain confined to singular aspects, such as using LLMs solely for small toy UAV planning, as detailed in Table I . A significant research gap exists regarding a comprehensive system that progresses from human language input to determining potential path nodes, developing detailed paths, integrating with UAV platforms, and ultimately controlling UAVs. Most of the literature exclusively addresses the planning of small UAVs without considering medium- and long-range UAV planning. These categories exhibit substantial differences. Small UAVs, typically multirotors, can take off from any location and perform short-distance missions, but possess limited cargo capacity and lower performance capabilities. Medium- and long-range UAVs, typically fixed-wings, require airport infrastructure and the adherence to strict take-off and landing procedures. They must also comply with FAA regulations, including prohibitions against flying over crowds and requirements to avoid air traffic control zones. These considerations present new and unique challenges for LLM for UAV systems. In this paper, we present the Next-Generation LLM for UAV ( NeLV ) system, illustrated in Figure 1 . This LLM-powered UAV system translates human language input into autonomous control of short-, medium-, and long-range UAVs that perform various missions. The system incorporates multiple key technical components, including LLM-as-Parser, route planning, path planning, and control platform. Human pilots can access the input for each component and intervene to control or modify operations. While we envision future systems where LLMs play central roles in planning, control, and decision-making, current LLM capabilities remain limited for safety-critical UAV operations. Therefore, this position paper serves dual purposes: (1) System Demonstration: We present a working implementation where LLMs handle natural language interpretation (Level 1 of our taxonomy) while traditional algorithms perform route planning, path planning, and control; and (2) Automation Roadmap: We establish a five-level taxonomy that defines technical requirements, identifies key challenges, and charts development pathways from current LLM-as-Parser capabilities to future LLM-as-Autopilot systems capable of fully autonomous decision-making. The contributions of this position paper are as follows. • End-to-End System Framework and Implementation: We introduce NeLV , the first complete framework integrating LLMs with multi-scale UAV operations from natural language input to executable flight trajectories. To our knowledge, NeLV represents the first LLM-powered UAV system capable of supporting short-, medium-, and long-range operations, as well as complex missions involving multi-UAV patrol, multi-POI delivery, and multi-hop relocation scenarios. • Five-Stage Pipeline Architecture: We develop a comprehensive pipeline encompassing: (i) LLM-as-Parser that engages in conversational interaction with human