Learning Decentralized Routing Policies via Graph Attention-based Multi-Agent Reinforcement Learning in Lunar Delay-Tolerant Networks
Federico Lozano-Cuadra, Beatriz Soret, Marc Sanchez Net, Abhishek Cauligi, Federico Rossi · 2025
Read Learning Decentralized Routing Policies via Graph Attention-based Multi-Agent Reinforcement Learning in Lunar Delay-Tolerant Networks by Federico Lozano-Cuadra, Beatriz Soret, Marc Sanchez Net, Abhishek Cauligi, Federico Rossi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20436.txt.
Introduction
Federico Lozano-Cuadra and four co-authors route data for a team of Moon rovers that must pass finds back to a lander. Links come and go, and rover paths are not known in advance. The task is cast as a part-seen Markov choice task. Their plan learns with graph attention and multi-agent steps, trained as a group but run by each rover on its own.
The plan uses just local views. It needs no world map refresh and no floods of spare copies, unlike path-based and flood-based plans. In Monte Carlo runs with random scout fields, it lifts the share of packs that arrive. It makes no twin packs and loses fewer packs. Short path forecasts help it, and it scales to bigger teams.
Catalog record
| CREATOR | Federico Lozano-Cuadra, Beatriz Soret, Marc Sanchez Net, Abhishek Cauligi, Federico Rossi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20436
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Decentralized Routing Policies via Graph Attention-based Multi-Agent Reinforcement Learning in Lunar Delay-Tolerant Networks Federico Lozano-Cuadra, Beatriz Soret, Marc Sanchez Net, Abhishek Cauligi, Federico Rossi Abstract We present a fully decentralized routing framework for multi-robot exploration missions operating under the constraints of a Lunar Delay-Tolerant Network (LDTN) . In this setting, autonomous rovers must relay collected data to a lander under intermittent connectivity and unknown mobility patterns. We formulate the problem as a Partially Observable Markov Decision Problem (POMDP) and propose a Graph Attention-based Multi-Agent Reinforcement Learning (GAT-MARL) policy that performs Centralized Training, Decentralized Execution (CTDE) . Our method relies only on local observations and does not require global topology updates or packet replication, unlike classical approaches such as shortest path and controlled flooding-based algorithms. Through Monte Carlo simulations in randomized exploration environments, GAT-MARL provides higher delivery rates, no duplications, and fewer packet losses, and is able to leverage short-term mobility forecasts; offering a scalable solution for future space robotic systems for planetary exploration, as demonstrated by successful generalization to larger rover teams. I Introduction The renewed interest in planetary and lunar surface exploration has accelerated the development of autonomous multi-robot systems. Missions such as Cooperative Autonomous Distributed Robotic Exploration (CADRE) [ 1 ] , led by NASA’s Jet Propulsion Laboratory (JPL) , aim to demonstrate high Technology Readiness Level (TRL) for cooperative autonomy in space robotics. In particular, CADRE plans to deploy a team of three rovers to explore the Reiner Gamma region of the Moon around a lander, which will serve as a base station and gateway to Earth. However, these initial demonstrations assume that the exploring agents remain within the communication range of the lander, thereby simplifying network design because nodes (rovers and lander) can simply broadcast the information and assume all-to-all communication. In future science-driven missions, exploring agents will need to leave the lander’s coverage area to reach farther, uncharted regions. This shift introduces new challenges: nodes must operate autonomously as a Delay-Tolerant Network (DTN) [ 2 ] , making independent routing and storage decisions under intermittent connectivity. This task is especially challenging in an unknown and complex environment like the lunar surface, further complicating the problem of reliable transmission of data packets and effective network management. As such, there is a pressing need to develop new routing algorithms for the next generation of autonomous networks and distributed robotic explorers. In this paper, we address these challenges by proposing a decentralized, learning-based communication policy where each rover makes local routing decisions based on limited neighborhood observations (Fig. 1 ), enabling robust multi-agent collaboration under DTN conditions. The problem of routing in time-varying networks has seen significant interest from the community: Fig. 1 : Snapshot of a Lunar Delay Tolerant Network (LDTN) with three rovers \mathfrak{r}\in\mathfrak{R} and a lander \mathcal{L} , which serves as the base station. Rovers \mathfrak{r}_{i} and \mathfrak{r}_{k} have direct connection with \mathcal{L} and with one another, whereas \mathfrak{r}_{j} can only communicate with \mathfrak{r}_{k} . Every rover explores a different sub-area of the map \mathcal{M} by defining objectives \mathbf{o} . Each \mathfrak{r} collects a local observation \mathbf{X}_{r}=[\mathbf{x}_{\mathfrak{n}}]_{\mathfrak{n}\in\mathcal{N}_{r}} from its neighborhood (e.g., \mathcal{N}_{i}=\{\mathfrak{r}_{i},\mathfrak{r}_{k},\mathcal{L}\} for \mathfrak{r}_{i} ), which serves as input to a pre-trained policy \pi that determines the routing decisions. Delay Tolerant Networks: DTNs are communication networks designed for environments with long propagation delays and intermittent or disrupted connectivity. Their architectures introduce store-and-forward mechanisms, enabling data packets to be buffered at the source or at intermediate nodes until a suitable communication opportunity arises [ 2 ] . This paradigm is particularly relevant for the networks found in interplanetary exploration [ 3 ] , where continuous End-to-End (E2E) paths rarely exist and link availability is highly dynamic. The extensive literature on the topic of routing in DTNs can be classified according to the available information oracles [ 4 ] . The first category comprises policies that rely solely on local knowledge of 1-hop neighbors. Here, random algorithms such as Hot Potato [ 5 ] , controlled flooding algorithms like Spray and Wait [ 6 ] , and opportunistic routing [ 7 ] are applicable. A second category includes algorithms that require knowledge of the network topology and link availability. Classical shortest path algorithms like Dijkstra [ 8 ] belong to this category, potentially including queueing information if available. Another family of policies depends on knowledge of node mobility. For example, Contact Graph Routing (CGR) predicts E2E paths for packets using pre-computed contact opportunities between all nodes in the network [ 9 ] . Finally, genie-aided oracles have full knowledge of the system, including current and future traffic demands in addition to future contact opportunities. In such cases, linear programming provides the optimal solution [ 4 ] . However, in autonomous robotic exploration scenarios, access to such information oracles is typically infeasible due to intermittent connectivity, unknown mobility patterns, and the need for decentralized decision-making, which arises naturally from these properties of the communication network. While traditional DTN routing methods perform well under specific assumptions, they face limitations in scenarios with partial observability, decentralized operation, or highly dynamic topologies. Data-driven methods can provide additional performance by learning patterns in traffic demand and contact availability, improving performance even when explicit models are unavailable or inaccurate. Machine Learning for distributed space systems: Machine Learning (ML) techniques are increasingly being explored for autonomous decision-making in space communication systems. For planetary exploration scenarios, [ 10 ] studies the use of Federated Learning (FL) to merge the maps generated by exploring rovers in a mission concept inspired by CADRE ’s concept of operations. In [ 11 ] , a Reinforcement Learning (RL) algorithm is proposed to autonomously manage the buffer of one node in a DTN . In [ 12 , 13 ] multi-agent RL is proposed to solve the routing problem in satellite constellations, where each node of the network is an independent learning agent that makes its own autonomous routing decisions. However, the works in [ 12 , 13 ] assume continuous E2E connectivity and do not incorporate store-and-forward mechanisms, rendering them unsuitable for DTNs . Additionally, their models do not support variable-sized input graphs, limiting their adaptability in dynamic multi-robot settings in which rovers may split and regroup as they explore. As a result, direct comparison with existing ML -based routing methods is not straightforward, since most rely on continuous connectivity or centralized knowledge. In contrast, Graph Attention Networks (GATs) [ 14 ] naturally accommodate changing neighborhood sizes, making them well suited for decentralized cooperation in evolving topologies. Building on graph-based methods, [ 15 ] proposes a multi-agent RL approach that processes graph-structured state information to enable decentralized satellite coordination under limited communication. Statement of Contributions: The main contributions of this paper are: (1) we propose a rigorous decision-theoretic model of routing over a Lunar Delay-Tolerant Network (LDTN) in the context of planetary exploration missions, applicable to any beyond-line-of-sight robotic networks; (2) we propose a novel Graph Attention-based Multi-Agent Reinforcement Learning (GAT-MARL) policy that performs decentralized routing under intermittent connectivity and partial observability; (3) we demonstrate through extensive simulations in a custom Python-based packet-level simulator that GAT-MARL outperforms classical DTN strategies, including Spray and Wait and Greedy Forwarding , while demonstrating scalability and relying only on locally observed information.