Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots
Ashutosh Mishra, Shreya Santra, Elian Neppel, Edoardo M. Rossi Lombardi, Shamistan Karimov, Kentaro Uno, Kazuya Yoshida · 2025
Read Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots by Ashutosh Mishra, Shreya Santra, Elian Neppel, Edoardo M. Rossi Lombardi, Shamistan Karimov, Kentaro Uno, Kazuya Yoshida (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20347.txt.
Introduction
Ashutosh Mishra and six co-authors propose decentralized reinforcement learning for modular reconfigurable robots aimed at lunar operations. Such robots can reshape themselves for specific space tasks, but the combinatorial growth of possible morphologies blocks unified control. In their scheme each module learns its own policy: wheel modules use Soft Actor-Critic for locomotion, while seven-degree-of-freedom limbs use Proximal Policy Optimization for steering and manipulation.
In simulation, the steering policy tracks desired angles closely, the manipulation policy plateaus at a strong success rate, and the wheel policy cuts average motor torque dramatically against a baseline. Lunar-analogue field tests confirm zero-shot integration for locomotion, steering, and early reconfiguration alignment. The system switches smoothly among execution modes without idle states or control conflicts.
Catalog record
| CREATOR | Ashutosh Mishra, Shreya Santra, Elian Neppel, Edoardo M. Rossi Lombardi, Shamistan Karimov, Kentaro Uno, Kazuya Yoshida |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20347
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots Ashutosh Mishra, Shreya Santra, Elian Neppel, Edoardo M. Rossi Lombardi, Shamistan Karimov, Kentaro Uno, Kazuya Yoshida Abstract Modular reconfigurable robots suit task-specific space operations, but the combinatorial growth of morphologies hinders unified control. We propose a decentralized reinforcement learning (Dec-RL) scheme where each module learns its own policy: wheel modules use Soft Actor-Critic (SAC) for locomotion and 7-DoF limbs use Proximal Policy Optimization (PPO) for steering and manipulation, enabling zero-shot generalization to unseen configurations. In simulation, the steering policy achieved a mean absolute error of 3.63 ° 3.63\text{\,}\mathrm{\SIUnitSymbolDegree} between desired and induced angles; the manipulation policy plateaued at 84.6 % 84.6\text{\,}\mathrm{\%} success on a target-offset criterion; and the wheel policy cut average motor torque by 95.4 % 95.4\text{\,}\mathrm{\%} relative to baseline while maintaining 99.6 % 99.6\text{\,}\mathrm{\%} success. Lunar-analogue field tests validated zero-shot integration for autonomous locomotion, steering, and preliminary alignment for reconfiguration. The system transitioned smoothly among synchronous, parallel, and sequential modes for Policy Execution, without idle states or control conflicts, indicating a scalable, reusable, and robust approach for modular lunar robots. I Introduction Modular robotics presents an exciting frontier for space exploration, where mission success depends on reliability, ease of repair, and adaptability. In line with the Japan Science and Technology Agency’s Moonshot Project (Goal 3) [ 1 ] , our long-term aim is to develop modular robots that autonomously learn, adapt, evolve in intelligence, and collaborate with humans on the lunar surface by 2050. To achieve this goal, we developed “MoonBots,” a modular and reconfigurable platform designed for autonomous and cooperative operations in assembling and maintaining future lunar infrastructure [ 2 ] . While, teleoperation and algorithmic assembly show promise [ 3 , 4 , 5 ] , autonomous self-reconfiguration is essential for long missions. Modular robots combine standard units into task-specific morphologies. Although hardware advances allow rapid reconfiguration, learning control policies remains challenging when varied combinations alter kinematics and observation–action spaces [ 6 , 7 ] . Fig. 1: MoonBot modules and policy: Wheel modules use SAC (Soft Actor-Critic) for locomotion and 7-DoF limbs use PPO (Proximal Policy Optimization) for manipulation and steering. As shown in Fig. 1 , MoonBots consist of interchangeable modules capable of reconfiguration to transport, assembly, and manipulation. However, learning separate policies for each configuration is inefficient, while a single monolithic policy struggles to generalize due to conflicting optimization signals and lack of structural inductive bias [ 7 , 8 ] . A promising solution is decentralized reinforcement learning (Dec-RL), where each module learns its own policy from local observations while coordinating with neighbors to achieve global goals [ 9 , 10 , 11 ] . This work proposes a decentralized architecture for modular, reconfigurable robots, allowing for reusable and plug-and-play policies that generalize to new morphologies without requiring retraining, thereby enabling rapid adaptation to new hardware layouts in unpredictable lunar environments. The key contributions of this work are: 1. A fully decentralized RL architecture where each module learns independently with task-specific algorithms, enabling scalable control and zero-shot adaptation to new configurations without retraining. 2. Enhanced sample efficiency and robustness over centralized approaches, maintaining performance across varied configurations. 3. Experimental validation with experiments conducted in lab and lunar-like Space Exploration Field at JAXA [ 12 ] , demonstrating autonomous reconfiguration and complex coordinated task execution. II Related Work Lunar operations are challenged by high-latency, low-bandwidth communications and unpredictable environments. Advances in AI and robotics are enabling greater autonomy and reducing reliance on Earth-based teleoperation. Deep Reinforcement Learning (DRL) enables adaptive autonomy in space robots by learning optimal actions in stochastic, partially observable environments typical of planetary surfaces [ 9 ] . Recent efforts have also aimed to establish standardized evaluation frameworks for robotic manipulation in space, highlighting the importance of reproducible benchmarks for comparing learning-based and classical controllers [ 13 ] . Unlike rule-based algorithmic systems, RL controllers generalize across varied and unseen conditions, aiding anomaly recovery, re-planning, and context-aware exploration while reducing human intervention [ 7 , 14 , 15 ] . As system complexity grows - as in a multi-modular reconfigurable robotic system - centralized RL becomes impractical, less scalable, and more vulnerable to failures [ 11 , 9 ] . Dec-RL aims to overcome these challenges by distributing learning and execution, enabling each module to act independently with partial observability while achieving coordinated goals [ 16 , 17 ] . Early work showed Dec-RL can outperform centralized baselines in sample and computational efficiency [ 11 , 18 ] , and recent studies have applied it to multi-robot systems for scalable, robust cooperation [ 9 , 16 ] . Examples include Sartoretti et al. [ 8 ] using decentralized policies for articulated mobile robots, Zhang et al. [ 19 ] enabling cooperative object transport via decentralized Deep Q-Learning (DQN) without a central controller, and Farivarnejad et al. [ 20 ] implementing a decentralized Proportional–Derivative (PD) controller. for payload transport in microgravity with minimal information sharing. Whitman et al. [ 7 ] proposed a policy structure for modular robotics, where policy graphs match hardware graphs, sharing networks for module types to enable transfer to unseen configurations. However, their approach relies on a configuration-conditioned global policy and is not fully decentralized. Our proposed architecture addresses this gap where locomotion and manipulation modules autonomously acquire reusable policies that can be composed at runtime. We validate our method with simulations and hardware experiments across diverse mission tasks and robot morphologies. III Modular Robot System and Decentralized Policy Learning \begin{overpic}[./fig/sim_wheel.png] \put(3.0,3.0){\fcolorbox{black}{black}{\textcolor{white}{\textbf{(a)}}}} \end{overpic} \begin{overpic}[./fig/sim_manip.jpg] \put(3.0,3.0){\fcolorbox{black}{black}{\textcolor{white}{\textbf{(b)}}}} \end{overpic} \begin{overpic}[./fig/sim_steer.jpg] \put(3.0,3.0){\fcolorbox{black}{black}{\textcolor{white}{\textbf{(c)}}}} \end{overpic} \begin{overpic}[./fig/sim_vehicle.jpg] \put(3.0,3.0){\fcolorbox{black}{black}{\textcolor{white}{\textbf{(d)}}}} \end{overpic} Fig. 2: Simulation environment for modular robot components, used for individual policy learning and testing. Wheel locomotion and Dragon-mode steering were learned in MuJoCo, while manipulation and Vehicle-mode steering tasks were implemented in NVIDIA Isaac Lab. (a) Wheel dynamics simulation. (b) Manipulator grasping. (c) Dragon steering. (d) Vehicle limb-steering. The robotic platform comprises primary modular units that integrate a two-motor wheel base with a 7-DoF robotic limbs as shown in Fig. 1 (a), supporting flexible morphologies for lunar construction and manipulation. We focus on two configurations: Vehicle mode shown in Fig. 1 (b), which employs two wheel modules and a single limb for active steering; and Dragon mode Fig. 1 (c), consisting of two serially connected wheel-limb modules, with a front limb for manipulation and a rear limb for stabilization and maneuvering, enabling concurrent locomotion and task execution. These configurations, along with the corresponding simulation environments used for their policy learning, are shown in Fig. 2 . While the manipulation and locomotion tasks could be solved using several alternative reinforcement learning algorithms, our aim is not to compare specific algorithms, but to demonstrate how heterogeneous policies can coexist and cooperate within a single modular architecture. PPO for manipulation and SAC for locomotion were chosen owing to their proven stability and effectiveness in continuous-control tasks [ 21 , 22 ] . This combination enables the evaluation of a multi‑modal decentralized learning framework, where each module type learns its own control strategy yet can be composed dynamically at runtime. The key contribution lies in showing that such independently trained, task‑specific policies can coordinate seamlessly on modular lunar hardware, supporting adaptability rather than algorithm novelty. Coordinating these heterogeneous modules