3D RL-DWA: A Hybrid Reinforcement Learning and Dynamic Window Approach for Goal-Directed Local Navigation in Multi-DoF Robots
1 st Chiara Castellani, 2 nd Enrico Turco, 3 rd Domenico Prattichizzo · 2026
Read 3D RL-DWA: A Hybrid Reinforcement Learning and Dynamic Window Approach for Goal-Directed Local Navigation in Multi-DoF Robots by 1 st Chiara Castellani, 2 nd Enrico Turco, 3 rd Domenico Prattichizzo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12689.txt.
Introduction
Castellani, Turco and Prattichizzo combine reinforcement learning with the Dynamic Window Approach for 3D local navigation of many-degree robots. Sparse point cloud data let the method adjust both motion and shape of a deformable microrobot, steering toward a goal in complex, constrained spaces while maximising occupied volume. Tests in a simulated vascular network, over 1080 trials, show the hybrid beats pure reinforcement learning and model-based methods on deformation and path completion.
Catalog record
| CREATOR | 1 st Chiara Castellani, 2 nd Enrico Turco, 3 rd Domenico Prattichizzo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38961 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12689
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
3D RL-DWA: A Hybrid Reinforcement Learning and Dynamic Window Approach for Goal-Directed Local Navigation in Multi-DoF Robots Thanks: This work was supported by the European Union’s Horizon Europe Research and Innovation Programme under Grant Agreement #101070066, project REGO. 1 st Chiara Castellani, 2 nd Enrico Turco, 3 rd Domenico Prattichizzo Abstract In this paper, we present a novel hybrid approach that combines Reinforcement Learning (RL) with Dynamic Window Approach (DWA) for adaptive 3D local navigation of high-degree-of-freedom robotic systems. Our method leverages sparse point cloud data to dynamically adjust both the motion and the shape of a deformable microrobot, enabling the system to navigate toward a goal in complex, constrained environments while maximizing the occupied volume. We evaluate our framework in a simulated vascular network. Experimental results, based on 1080 trials, indicate that integrating RL with a DWA-based local planner significantly enhances both deformation and navigation capabilities compared to pure RL and model-based methods. In particular, the proposed autonomous controller consistently achieves high deformation and near-perfect path completion during training and maintains robust performance in unseen scenarios. These findings highlight the potential of hybrid planning strategies for efficient and adaptive 3D navigation under sparse sensory conditions. I Introduction Autonomous navigation is typically achieved using a combination of global and local planning approaches [ 14 ] . Global path planning computes a trajectory from the start to the goal pose while considering obstacles and robot kinematics. However, these strategies assume a fully known and static environment, making them impractical for real-world applications where conditions are often partially known and dynamically changing [ 11 ] . In contrast, local path planning plays a crucial role in enabling robots to adapt in real-time, responding to environmental changes through onboard sensor data (e.g., LiDAR or cameras). This allows for reactive obstacle avoidance and continuous trajectory refinement. Navigation in 3D environments is highly complex, especially for robots with high degrees of freedom (DoF) [ 8 , 9 ] . These systems must efficiently coordinate multiple motion variables while ensuring collision-free navigation. Moreover, holonomic and external constraints further complicate real-time motion planning, requiring adaptive control strategies. Fig. 1: Proposed framework for adaptive 3D local navigation. The autonomous controller combines RL and DWA within a closed-loop system. At each time step, the controller receives an observation vector \boldsymbol{o}=[\boldsymbol{s},\boldsymbol{l}] from the environment, where \boldsymbol{s} denotes the current task state and \boldsymbol{l} contains the laser-scan readings. The RL agent dynamically adjusts the DWA cost-function weights ( \alpha,\beta,\gamma,\zeta ) and outputs angular \omega_{x} and deformation velocities \boldsymbol{\dot{\delta}} . The DWA module then selects the optimal linear velocity \boldsymbol{v} from the admissible velocity set. One of the most widely used local navigation techniques is the Dynamic Window Approach (DWA) [ 6 ] . This method optimizes the planned path by evaluating a cost function that balances factors such as distance to obstacles and alignment with the goal. While effective, conventional DWA has limitations, particularly its reliance on fixed parameter tuning, which makes adaptation to dynamic environments difficult. Traditional DWA requires extensive manual tuning of cost function weights to adapt to different scenarios, such as navigating narrow gaps or open spaces. This limits adaptability in dynamic environments and increases computational costs in high-dimensional motion spaces, complicating real-time motion planning. Recent advancements in Reinforcement Learning (RL) have shown promise in adaptive navigation, enabling robots to dynamically adjust their control policies based on environmental conditions [ 32 ] . Hybrid approaches combining RL with classical planners, such as DWA [ 10 , 13 , 4 ] , have demonstrated improved adaptability and robustness. However, existing work has primarily focused on 2D motion planning in structured environments, leaving a gap in solutions for 3D motion control in high-DoF robotic systems. In this work, we extend these emerging approaches and propose a novel framework, namely the 3D RL-DWA (shown in Fig. 1 ), for the navigation of a 9-DoF deformable robot. Our system combines RL-driven decision-making with a DWA-based local control, allowing the robot to dynamically adjust its linear, angular, and deformation velocities. We specifically address goal-directed local navigation, where the global route is defined as a limited sequence of waypoints. Our contribution is an adaptive local controller that guides the robot toward each waypoint while simultaneously optimizing its deformation, leveraging sparse sensory data. The robot is required to navigate in a complex and constrained environment, consisting of a simulated vascular network, toward a selected target while maximizing the occupied volume and avoiding collisions with the vessels. Robot actuation is described through a simplified kinematic model that captures both motion and deformation, and is inspired by recent advances in ferrofluidic robotic systems [ 28 , 7 , 29 ] . However, in this work, we focus on high-level control and do not explicitly model magnetic forces and material dynamics. The effectiveness of our approach is validated through extensive experimental trials, where we compare performance against a pure RL and a model-based navigation method. II Related Works Research on autonomous navigation has traditionally focused on 2D motion for mobile robots, extending to 3D applications in domains such as unmanned aerial vehicles (UAVs) and underwater robots [ 17 ] . However, as robotic systems operate in increasingly complex and dynamic environments, there is a need for advanced navigation strategies with adaptive decision-making and real-time reconfigurability. In such scenarios, local path planning plays a crucial role in ensuring efficient and collision-free navigation. Among local path planning techniques, artificial potential field, optimization-based methods, and learning-based strategies have been widely employed [ 14 , 21 ] . In particular, DWA has proven highly effective in dynamic environments, as it optimizes velocity profiles through a cost function that balances target heading, obstacle avoidance, and motion continuity. Extensions of traditional DWA to 3D applications were explored in UAVs [ 2 ] and underwater robots [ 25 ] . However, DWA relies on fixed parameter tuning, which limits adaptability in dynamic and unstructured environments and often requires extensive experimentation [ 2 ] . To address this limitation, several works have explored hybrid approaches that integrate adaptive parameter tuning into DWA-based navigation. Neuro-fuzzy and fuzzy-logic approaches have been used to adjust the DWA parameters online according to environmental conditions [ 23 , 1 , 27 ] . In parallel, RL techniques have been integrated with DWA to improve adaptability. In [ 26 ] , DWA was used to refine reward signals in a Proximal Policy Optimization (PPO) RL framework, while [ 10 ] introduced an RL-based controller to fine-tune the DWA velocity outputs. Further developments have included the TD3-DWA approach [ 13 ] for collision-free motion planning, and DWA-RL [ 19 ] , which integrates deep RL with DWA to improve navigation among mobile obstacles. Additionally, Dobrevski and Skočaj [ 4 ] proposed an RL framework that leverages high-dimensional laser scan observations to dynamically adjust DWA’s parameters, while in [ 3 ] , Q-learning has been employed for DWA parameter optimization. Despite these advancements, most RL-DWA methods remain focused on 2D navigation [ 4 , 19 , 10 , 13 , 3 ] and do not address high-DoF systems operating in constrained 3D spaces. Other hybrid methods integrate RL with Model Predictive Control (MPC). Romero et al. [ 21 ] proposed an Actor-Critic MPC architecture