GRaD-Nav: Efficiently Learning Visual Drone Nav igation with G aussian R adiance Fields and D ifferentiable Dynamics
Qianzhong Chen, Jiankai Sun, Naixiang Gao, JunEn Low, Timothy Chen, Mac Schwager · 2025
Read GRaD-Nav: Efficiently Learning Visual Drone Nav igation with G aussian R adiance Fields and D ifferentiable Dynamics by Qianzhong Chen, Jiankai Sun, Naixiang Gao, JunEn Low, Timothy Chen, Mac Schwager (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03984.txt.
Introduction
Qianzhong Chen and co-authors present a 2025 paper on teaching drones to fly by sight. Learning by trial can be slow, and skills learned in a model world often fail in the real one. Drones make this worse, since flight is shaky and vision and control affect each other. The team joins a rich three-dimensional scene method with learning that can be tuned by gradients.
Training runs through varied scenes, which helps the drone face new gates and clutter. A small extra network reads the scene at flight time and adjusts the policy. Tests on real drone hardware showed fast training and a clean jump from model to real flight. No extra tuning was needed for the real flights.
Catalog record
| CREATOR | Qianzhong Chen, Jiankai Sun, Naixiang Gao, JunEn Low, Timothy Chen, Mac Schwager |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-03984
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GRaD-Nav: Efficiently Learning Visual Drone Nav igation with G aussian R adiance Fields and D ifferentiable Dynamics Qianzhong Chen, Jiankai Sun, Naixiang Gao, JunEn Low, Timothy Chen, Mac Schwager Abstract Autonomous visual navigation is an essential element in robot autonomy. Reinforcement learning (RL) offers a promising policy training paradigm. However existing RL methods suffer from high sample complexity, poor sim-to-real transfer, and limited runtime adaptability to navigation scenarios not seen during training. These problems are particularly challenging for drones, with complex nonlinear and unstable dynamics, and strong dynamic coupling between control and perception. In this paper, we propose a novel framework that integrates 3D Gaussian Splatting (3DGS) with differentiable deep reinforcement learning (DDRL) to train vision-based drone navigation policies. By leveraging high-fidelity 3D scene representations and differentiable simulation, our method improves sample efficiency and sim-to-real transfer. Additionally, we incorporate a C ontext-aided E stimator Net work (CENet) to adapt to environmental variations at runtime. Moreover, by curriculum training in a mixture of different surrounding environments, we achieve in-task generalization, the ability to solve new instances of a task not seen during training. Drone hardware experiments demonstrate our method’s high training efficiency compared to state-of-the-art RL methods, zero shot sim-to-real transfer for real robot deployment without fine tuning, and ability to adapt to new instances within the same task class (e.g. to fly through a gate at different locations with different distractors in the environment). I Introduction Autonomous drones have the potential to improve practices in agriculture, environmental management, and search and rescue [ 1 , 2 , 3 ] . A critical aspect of these robots’ functionality is their efficient navigation and control in complex, unstructured environments [ 4 ] . Traditional approaches to this problem have predominantly relied on a stack of different modules including perception, localization, mapping, planning, and control [ 5 ] , [ 6 ] . However, the integration of these different modules has many issues, including high system complexity and computational overhead, communication latency between modules, multiple points of failure, and difficult-to-characterize error propagation between modules. Recent advancements in machine learning have offered alternative solutions to these challenges. Deep reinforcement learning (DRL) has been proven to be successful in drone racing [ 7 ] . By leveraging DRL, robots can potentially learn to directly map sensor inputs to control outputs, bypassing the need for explicit modular separation [ 8 ] . While promising, so far, these DRL solutions have been mostly limited to drone racing-like tasks but have not proved successful in navigating complex and unstructured environments. To tackle the challenge, one of the most important bottlenecks lies on the difficulty in getting high-quality perception data when training the policy in conventional simulators [ 9 , 10 ] . Synthetic visual data still struggles to accurately capture fine details, often leading to a substantial sim-to-real gap [ 11 ] . Fig. 1 : Our GRaD-Nav architecture combines a visual+dynamics context encoder (CENet) within an Actor-Critic framework, trained end-to-end using a differentiable drone dynamics model and 3D Gaussian Splatting scene representation for photo-realistic visuals at training time. The policy transfers zero-shot to drone hardware and adapts to new navigation task instances at runtime. 3D reconstruction has demonstrated exceptional capability in generating high-fidelity 3D scene representations from sparse views [ 12 , 13 ] , making it an attractive approach for creating an environment model [ 14 , 15 , 16 ] . Flying in 3D reconstruction has been proven successful for multiple drone tasks including trajectory planning, pose estimation, and visual navigation [ 17 , 18 , 19 , 20 , 21 , 22 ] . Moreover, the recent work SOUS VIDE [ 23 ] utilizes Flying in a Gaussian Splat (FiGS) to train end-to-end drone navigation policies, and demonstrated successful sim-to-real transfer. However, the works above are based on imitation learning, which requires a large amount of high-quality expert pilot data, long training time, and suffers from a lack of generalization to new task instances. On the other hand, differentiable simulators [ 24 , 25 , 26 , 27 ] and differentiable deep reinforcement learning (DDRL) algorithms [ 28 ] , [ 29 ] alleviate another critical issue of sample efficiency for conventional DRL. Differentiable simulators not only provide the simulated object’s next state, but also delivers the gradient of the simulated state with respect to the action and previous state, enhancing model training with gradient knowledge rather than simply trial and error. By leveraging a custom smoothing critic function and truncated learning window, the Short Horizon Actor-Critic (SHAC) algorithm [ 29 ] alleviates the problems of local minima and exploding/vanishing gradients that commonly occur in DDRL training. Even with exteroceptive sensors like RGB cameras, it is still a significant challenge for drone policies to understand the complex surrounding environment, especially the spatial relationship between drone and obstacle. To address this, we introduce a context-aided estimator network (CENet) [ 30 ] positioned before the policy network, enabling the agent to infer crucial environmental properties at runtime. While previous approaches have primarily relied on proprioceptive sensing, we extend this idea by including RGB camera sensor inputs, leading to improved in-task adaptability in navigation scenarios. Concretely, CENet outputs a latent embedding at runtime summarizing visual and dynamical context. The policy is conditioned on this code, enabling the drone to modify its behavior in response to the sensed runtime environment. To achieve the goal of visual-motor navigation, we propose a novel approach that leverages 3DGS in conjunction with DDRL, using SHAC-like training algorithm and a policy architecture augmented with a CENet module for in-context runtime adaptation. By utilizing this combination, we develop a visual navigation policy learning framework that transfers to real-world drone navigation performance. Our main contributions are: • We introduce a simulator for training robot visuomotor policies by integrating 3DGS for high-fidelity visuals with a differentiable dynamics model for computing end-to-end gradients. • We propose GRaD-Nav, a DDRL algorithm designed for efficient end-to-end visual navigation policy learning, achieving higher sample efficiency than previous approaches. • We demonstrate in-task generalization by training a single policy for the drone to fly through gates at different locations with different distractors in the environment not seen at training time. II Background II-A Differentiable Simulation and Differentiable Reinforcement Learning II-A 1 Differentiable Dynamics Simulation Conceptually, we view the simulator as a differentiable abstract function, \mathbf{s}_{t+1}=\mathcal{F}\left(\mathbf{s}_{t},\mathbf{a}_{t}\right) , which transitions a state \mathbf{s} from time t to t+1 . Here, \mathbf{a}_{t} represents the control input during that time-step. Given a differentiable scalar loss function \mathcal{L} and its adjoint \mathcal{L}^{*}=\frac{\partial\mathcal{L}}{\partial\mathbf{s}_{t+1}} , the backward pass of the simulator computes: \displaystyle\frac{\partial\mathcal{L}}{\partial\mathbf{s}_{t}}=\left(\frac{\partial\mathcal{L}}{\partial\mathbf{s}_{t+1}}\right)\left(\frac{\partial\mathcal{F}}{\partial\mathbf{s}_{t}}\right),\hskip 9.24994pt\frac{\partial\mathcal{L}}{\partial\mathbf{a}_{t}}=\left(\frac{\partial\mathcal{L}}{\partial\mathbf{s}_{t+1}}\right)\left(\frac{\partial\mathcal{F}}{\partial\mathbf{a}_{t}}\right). (1) II-A 2 Short-Horizon Actor-Critic The Short-Horizon Actor-Critic method (SHAC) [ 29 ] was introduced to address the challenges associated with gradient-based policy learning. This method simultaneously trains a policy network (actor) \pi_{\theta} and a value network (critic) V_{\phi} during task execution, dividing the entire task horizon into several smaller sub-windows across learning episodes. Differentiable simulation allows for backpropagation of the gradient through states and actions within the sub-windows, providing an accurate policy gradient. The SHAC method operates in an on-policy manner as follows: In each learning episode, the algorithm samples N trajectories \left\{\tau_{i}\right\} of short horizon h\ll H in parallel from the simulation,