PANAMA: A Network-Aware MARL Framework for Multi-Agent Path Finding in Digital Twin Ecosystems

Arman Dogru, R. Irem Bor-Yaliniz, and Nimal Gamini Senarath · 2025

Read PANAMA: A Network-Aware MARL Framework for Multi-Agent Path Finding in Digital Twin Ecosystems by Arman Dogru, R. Irem Bor-Yaliniz, and Nimal Gamini Senarath (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06767.txt.

Introduction

Fleets of robots need routes that respect both each other and the networks carrying their data. Arman Dogru, Irem Bor-Yaliniz, and Nimal Gamini Senarath introduce a framework in 2025 for multi-agent path finding inside digital twin systems, virtual copies of real operations. Their method trains agents together but lets each act on its own, with priority rules that account for network load.

In simulations, the framework beat existing benchmarks on accuracy, speed, and scale. The authors also draw out data-sharing strategies meant to keep automated systems resilient in complex settings. They present the work as a bridge between network-aware choices and sturdy coordination for embodied machines.

Catalog record

CREATORArman Dogru, R. Irem Bor-Yaliniz, and Nimal Gamini Senarath
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-06767

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PANAMA: A Network-Aware MARL Framework for Multi-Agent Path Finding in Digital Twin Ecosystems Arman Dogru, R. Irem Bor-Yaliniz, and Nimal Gamini Senarath Abstract Digital Twins (DTs) are transforming industries through advanced data processing and analysis, positioning the world of DTs, Digital World , as a cornerstone of next-generation technologies including embodied AI. As robotics and automated systems scale, efficient data-sharing frameworks and robust algorithms become critical. We explore the pivotal role of data handling in next-gen networks, focusing on dynamics between application and network providers (AP/NP) in DT ecosystems. We introduce PANAMA, a novel algorithm with Priority Asymmetry for Network Aware Multi-agent Reinforcement Learning (MARL) based multi-agent path finding (MAPF). By adopting a Centralized Training with Decentralized Execution (CTDE) framework and asynchronous actor-learner architectures, PANAMA accelerates training while enabling autonomous task execution by embodied AI. Our approach demonstrates superior pathfinding performance in accuracy, speed, and scalability compared to existing benchmarks. Through simulations, we highlight optimized data-sharing strategies for scalable, automated systems, ensuring resilience in complex, real-world environments. PANAMA bridges the gap between network-aware decision-making and robust multi-agent coordination, advancing the synergy between DTs, wireless networks, and AI-driven automation. I Introduction Embodied artificial intelligence (Embodied AI) serves as a key enabler for achieving high levels of automation, as it enables physical entities, such as robots, to perceive and dynamically interact with their environment [ 1 ] . 6G and beyond networks can play a key role for embodied AI to support communications, sensing and computation offloading [ 2 ] . Another enabler can be the Digital Twins, which can provide substantial validation, testing and prediction capabilites for high-level automation in complex environments [ 3 , 4 ] . At the intersection of both methods, lies data. In fact, as the 6G standardization unfolds, data framework of 3GPP, such as data handling capabilities and even the boundaries of 3GPP data are under discussion. Although agentic AI, personal connectivity assistance and several robotics-related use cases are already accepted in 3GPP 22.280 v031, data exposure issues encircling these applications remain ambiguous, making role of 6G questionable in enabling automation. One one hand, assume that network data is exposed to the application provider (AP) for the AP to derive necessary insights. On the other, assume that the AP data is exposed to the network provider (NP) for NP to derive related information. Neither one of these scenarios is favorable from both practical and regulatory aspects. In this paper, we argue that DTs can play a key role in creating buffer zones for data exposure and, by proposing a novel multi-agent reinforcement learning (MARL) based multi-agent path finding (MAPF) algorithm (PANAMA), we show how both NP and AP can benefit from a balanced data exposure, which increases communication efficiency and reliability simultaneously. First, we propose a DT ecosystem where DTs belonging to various real-world entities, such as humans, robots, and networks. DTs can collect data, provide information, control actuators, and interact with other DTs and real-world entities. In Fig. 1 we show a subset of this ecosystem with DTs of robots (D-Robot), a factory (D-Factory) and a network (D-Net). These DTs collaborate to train D-Robots and then use inference to test the performance of operation continuously. Note that networks provide integrated sensing and communications (ISAC) services to maintain DTs and enable analysis for the operation. The presented architecture can be used in several frameworks. For example, the DTs in this framework can be act as agents in an agentic AI operation in a fully automated warehouse scenario. Proposed Digital World Control Function (DWCF) and Digital World Data Processing Function (DWDPF) provide control and data plan functionalities, respectively, to enable dynamic creation and operation of DT networks (DTN). The benefits of this ecosystem become evident when scaled to solve complex and critical scenarios. The cooperative MAPF problem is a fundamental challenge in robotics and AI with wide-ranging applications, from warehouse automation and traffic control to video games and drone swarm coordination [ 1 ] . We formally model this complex, interactive decision-making scenario as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). In order to improve capability to scale up and comply with the proposed network architecture, we employ a Centralized Training with Decentralized Execution (CTDE) paradigm [ 1 ] . Training of a central, mutual policy comes with challenges inherent in MARL, where the policy fails to co-operate with copies of itself. Our novel asymetric priority based observations force the observed state information to be asymetrical and facilitates cooperation with a centrally trained policy. Considering realistic communication constraints by utilizing D-Net, we finally obtain the PANAMA, which not only demonstrates network awareness but also has superior performance in terms of training efficiency and scalability compared to the state-of-the-art benchmarks. II System Model Fig. 1: The asynchronous actor-learner architecture. Multiple ‘Actor‘ processes collect experience in parallel and send it to the ‘Central Learner‘. The learner optimizes the shared policy and broadcasts updated weights back to the actors. The first component of the system model is the DT ecosystem and D-Net modeling. The second one is the real-world environment and how it is represented in the PANAMA framework. II-A Network Architecture 6G networks can enable automation by improving data handling and connectivity capabilities, e.g., by supporting Model Context Protocol and improving anonymous communication capabilities. For example, DWCF can provide a network slice for the specific DTN, while DWDPF provide data processing capabilities. Fig. 1 shows how D-Robots share location and planned movement with D-Factory, and D-Factory provides FoV. In addition, D-Net receives D-Robot paths, factory map and other relevant information from D-Factory to provide corresponding network analysis. The data exposure is managed by DWCF and DWDPF, as well as internal DT functions, while satisfying privacy and security constraints. Ideally, D-Net is the digital twin of the wireless network, which represents not only the equipment but also channel and other complex entities. In this study, the D-Net comprises a physical layer model determining the signal quality across the map. We model a multi-sector cellular deployment where agents correspond to users. The received signal power in dBm, P_{\text{rx,dB}}(i,s) , for an agent i from a base station sector s is given by: P_{\text{rx,dB}}(i,s)=P_{\text{tx,dB}}+G_{\text{ant}}(i,s)-\text{PL}(i,s), (1) where P_{\text{tx,dB}} is the sector’s transmit power in dBm, G_{\text{ant}}(i,s) is the directional antenna gain in dBi from sector s towards agent i , and \text{PL}(i,s) is the total path loss in dB, based on 3GPP TR 36.837 for Urban Micro (UMi) environments, which includes a shadowing component. An agent i is served by the sector providing the strongest signal: s_{i}^{*}=\arg\max_{s}P_{\text{rx,dB}}(i,s) . The received power in linear scale from any sector s is P_{\text{rx,lin}}(i,s)=10^{P_{\text{rx,dB}}(i,s)/10} . The Signal-to-Interference-plus-Noise Ratio (SINR) for agent i at time t , denoted as \gamma_{i}(t) , is then: \gamma_{i}(t)=\frac{P_{\text{rx,lin}}(i,s_{i}^{*}(t))}{\sum_{s^{\prime}\neq s_{i}^{*}(t)}P_{\text{rx,lin}}(i,s^{\prime})+N_{0}}, (2) where the denominator sums the interference from all non-serving sectors and adds noise power, N_{0} . An agent’s potential data rate, R_{i}(t) , is determined from its SINR using a standard Modulation and Coding Scheme (MCS) lookup table: R_{i}(t)=\text{MCSLookup}(\gamma_{i}(t)) [ 5 ] . The SINR map, normalized to a [0,1] range as \bar{\gamma}_{i}(t) , is provided as the fourth channel in each agent’s spatial observation. II-B MARL Framework and Environment To allow for a fair comparison, our environment is designed to

Read the complete text