Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning
Federica Tonti, Ricardo Vinuesa · 2025
Read Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning by Federica Tonti, Ricardo Vinuesa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25679.txt.
Introduction
Federica Tonti and Ricardo Vinuesa navigate drones in urban flows. Their 2025 paper uses deep reinforcement learning for navigation. Urban flow is 3D and highly turbulent. Recirculation zones complicate flight through city canyons.
They develop an optimal navigation strategy using learning. A high-fidelity run represents the urban environment. Their method is flow-aware proximal policy tuning. A gated transformer architecture gives richer flow information. Results compare against several rival navigation methods. A memory-based variant and classic Zermelo method competed. Their flow-aware method increased success rate greatly. Crash rates fell compared with all rivals tested. Turbulence awareness helps drones exploit or avoid flows. Completely reimagined drone landscapes may follow from this. Urban delivery routes could become safer and quick.
Catalog record
| CREATOR | Federica Tonti, Ricardo Vinuesa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-25679
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning Federica Tonti, Ricardo Vinuesa Abstract Unmanned Aerial Vehicles (UAVs) are increasingly populating urban areas for delivery and surveillance purposes. In this work, we develop an optimal navigation strategy based on Deep Reinforcement Learning. The environment is represented by a three-dimensional high-fidelity simulation of an urban flow, characterized by turbulence and recirculation zones. The algorithm presented here is a flow-aware Proximal Policy Optimization (PPO) combined with a Gated Transformer eXtra Large (GTrXL) architecture, giving the agent richer information about the turbulent flow field in which it navigates. The results are compared with a PPO+GTrXL without the secondary prediction tasks, a PPO combined with Long Short Term Memory (LSTM) cells and a traditional navigation algorithm. The obtained results show a significant increase in the success rate (SR) and a lower crash rate (CR) compared to a PPO+LSTM, PPO+GTrXL and the classical Zermelo’s navigation algorithm, paving the way to a completely reimagined UAV landscape in complex urban environments. K eywords Deep Reinforcement Learning, UAV, Navigation, Urban environment Introduction The rapidly increasing number of Unmanned Aerial Vehicles (UAVs) in urban environments is due to the number of tasks they can perform, from surveillance and traffic monitoring to package delivery and their ability to reach locations that can be challenging for other aerial vehicles, such as helicopters. This also presents problems related to acoustic pollution and the risk of accidents. It is crucial to develop an efficient strategy for the autonomous navigation of UAVs in complex environments, not only to perform the aforementioned tasks but also to reduce their environmental impact related to acoustic pollution. Path planning becomes essential in order to reduce navigation time and minimize energy consumption. Navigation in urban environments is extremely challenging due to the complexity of the environment itself, with the presence of buildings and complex three-dimensional wind velocity distributions. The velocity field is characterized by the presence of turbulent wakes, vortex shedding, turbulent velocity fluctuations, gusts, recirculation zones and complex interaction phenomena due to the presence of obstacles [ 7 , 4 ] . Traditional path planning employs deterministic algorithms and heuristic methods. Popular algorithms are potential-field methods [ 17 ] , grid-based algorithms [ 26 , 30 , 6 ] , and sampling-based methods such as Rapidly-exploring Random Trees (RRT) [ 21 , 45 , 23 , 43 ] , as well as Probabilistic Roadmaps (PRM) [ 20 , 2 ] . Although these algorithms have been shown to be successful in environments which do not exhibit uncertainties, they yield poor performance in dynamic environments. Simultaneously Localization And Mapping (SLAM) algorithms include obstacle detection and avoidance by including a mapping of the environment [ 10 , 22 , 28 , 9 ] , but in large-scale environments they exhibit degraded efficiency since building a map of the whole environment is practically unfeasible. Deep-Learning (DL)-based methods have recently received significant attention. DL can significantly improve obstacle avoidance and path planning for UAVs by using neural networks to process and interpret sensory data, such as images from cameras, or LiDAR signals [ 34 , 24 , 29 ] or ground-based stations [ 42 ] . These methods allow UAVs to detect and avoid obstacles more efficiently by recognizing patterns and predicting potential collisions in real time. Deep-Learning methods are extremely efficient for environments featuring small variations, as they are based on labels and sensitive to environmental changes. They are suitable for closed geometries but less reliable for urban environments, where the conditions typically change very rapidly. This brings the necessity to develop reinforcement-learning (RL) methods to understand and automate decision-making processes, in which the agent learns by trial-and-error [ 33 ] . Deep Reinforcement Learning (DRL) has been successfully applied to UAV navigation tasks in complex environments. Bouhamed et al. [ 5 ] used a Deep Deterministic Policy Gradient (DDPG) algorithm to train quadrocopters to navigate in a three-dimensional environment avoiding collisions with static and moving obstacles. Wang et al. [ 40 ] implemented a Faster Regions with Convolutional Neural Network Feature (Faster R-CNN) algorithm for obstacle detection, improved upon a traditional Deep-Q Network (DQN) approach. Collision avoidance capability was significant in unknown or dynamically changing environments. Sheng et al. [ 32 ] used a Twin Delayed Deep Deterministic Policy Gradient (TD3) to train a UAV to navigate a highly dynamic environment with multiple obstacles achieving high success rates and efficient path planning. Wang et al. [ 41 ] proposed a Distributed Privileged Reinforcement Learning (DPRL) framework to address partial observability in navigation tasks, outperforming conventional vision-based methods. AlMahamid and Grolinger [ 1 ] introduced Agile DQN, an adaptive deep recurrent attention reinforcement learning method that enhances UAV obstacle avoidance by processing only the most relevant visual regions, achieving faster training and higher performance in simulated 3D environments. Raj and Kos [ 27 ] applied a DQN-based approach for mobile robot navigation in unknown 2D environments, showing effective obstacle avoidance and improved navigation performance. Zhao et al. [ 46 ] proposed an elastic adaptive DRL strategy to stabilize training in autonomous navigation tasks, demonstrating superior collision avoidance in multi-agent traffic scenarios. Berg et al. [ 3 ] combined deep reinforcement learning with nonlinear model predictive control (NMPC) for autonomous surface vessels, improving digital twin synchronization and obstacle avoidance through adaptive control policies. Recently, DRL has been applied to autonomous navigation tasks where the environment is given by high-fidelity simulations. Gunnarson et al. [ 13 ] applied a V-Racer algorithm with a Remember and Forget Experience Replay to discover time-efficient navigation policies to steer a fixed-speed swimmer through unsteady two-dimensional flow fields. Jiao et al. [ 19 ] showed that gradient sensing was critical for a navigation task in unsteady wake scenarios, applying DRL in a 2D unsteady flow field. The geocentric agent (with access to global direction) could reach the goal by sensing local flow velocity alone. However, the egocentric agent consistently failed with only local velocity sensors. An egocentric navigator needed the extra information on how the flow changed around it to break symmetries and reliably navigate the wake. Tonti et al. [ 36 ] used a Proximal Policy Optimization (PPO) algorithm [ 31 ] in combination with Long Short-Term Memory (LSTM) cells [ 15 ] for UAV navigation in a two-dimensional urban-like turbulent flow field obtained from a high-fidelity simulation. This study included random start and target locations, as well as random start snapshots, showing the adaptability of the agent in complex flow fields. The present study is an application to a three-dimensional urban environment, represented by a high-fidelity simulation, comparing three different architectures: PPO with LSTM cells, PPO with Gated Transformer XL (GTrXL) [ 25 ] , and PPO with GTrXL with an auxiliary task for flow prediction, in a multi-objective reinforcement learning framework, so that the UAV can further optimize its trajectory exploiting the prediction of the flow field of the next snapshot. Proximal policy optimization combined with GTrXL has recently been used for navigation tasks. Huang et al. [ 16 ] applied PPO+GTrXL in an indoor environment with obstacles, comparing the results with a Soft Actor-Critic (SAC) algorithm [ 14 ] combined with a GTrXL. Federici et al. [ 8 ] applied PPO+GTrXL to a