Unreal-MAP: Unreal-Engine-Based General Platform for Multi-Agent Reinforcement Learning
Tianyi Hu, Qingxu Fu, Zhiqiang Pu, Yuan Wang, Tenghai Qiu · 2025
Read Unreal-MAP: Unreal-Engine-Based General Platform for Multi-Agent Reinforcement Learning by Tianyi Hu, Qingxu Fu, Zhiqiang Pu, Yuan Wang, Tenghai Qiu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15947.txt.
Introduction
Tianyi Hu and co-authors built Unreal-MAP in 2025, a playground for training teams of learning agents. Multi-agent learning needs rich test worlds, but building them is slow and costly. A game engine offers vast visual and physical resources ready for use.
Their platform runs on Unreal Engine and lets users craft custom team tasks with ease. It supports setup, editing, and viewing, and all parts are open source. A test framework works with both rule-based and learning-based methods from other toolkits. The team ran several leading algorithms on sample tasks and reported how each performed. They hope the tool speeds progress in team learning research.
Catalog record
| CREATOR | Tianyi Hu, Qingxu Fu, Zhiqiang Pu, Yuan Wang, Tenghai Qiu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15947
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Unreal-MAP: Unreal-Engine-Based General Platform for Multi-Agent Reinforcement Learning Tianyi Hu, Qingxu Fu, Zhiqiang Pu, Yuan Wang, Tenghai Qiu Abstract In this paper, we propose Unreal Multi-Agent Playground (Unreal-MAP), an MARL general platform based on the Unreal-Engine (UE). Unreal-MAP allows users to freely create multi-agent tasks using the vast visual and physical resources available in the UE community, and deploy state-of-the-art (SOTA) MARL algorithms within them. Unreal-MAP is user-friendly in terms of deployment, modification, and visualization, and all its components are open-source. We also develop an experimental framework compatible with algorithms ranging from rule-based to learning-based provided by third-party frameworks. Lastly, we deploy several SOTA algorithms in example tasks developed via Unreal-MAP, and conduct corresponding experimental analyses. We believe Unreal-MAP can play an important role in the MARL field by closely integrating existing algorithms with user-customized tasks, thus advancing the field of MARL. The source code for our project is available at github.com/binary-husky/unreal-map and github.com/binary-husky/hmp2g . 1 Introduction Figure 1: The research workflow for using Unreal-MAP. For novice users, Unreal-MAP provides direct access to built-in tasks, and offers comprehensive algorithm deployment functions and result visualization capabilities. For advanced users, Unreal-MAP enables the modification of built-in tasks or the creation of new tasks to test research ideas. Multi-agent reinforcement learning (MARL) has demonstrated remarkable potential in many practical fields, including swarm robotic control ( Kalashnikov et al., 2018 ; Chen et al., 2020 ) , autonomous vehicles ( Peng et al., 2021b ) , and video games ( Vinyals et al., 2019 ) . There are many classical and practical algorithms that have emerged in the field of MARL, such as QMIX ( Rashid et al., 2020b ) , QPLEX ( Wang et al., 2020 ) , MAPPO ( Yu et al., 2022 ) and HAPPO ( Zhong et al., 2024 ) . The development and success of these algorithms could not be achieved without the support of numerous well-designed simulation environments. These environments provide ample simulated data for testing and comparing algorithms, thereby fostering a variety of more advanced algorithms. Obviously, the progression of MARL algorithms will not stop at mere comparison of performance; their ultimate aspiration lies in the deployment within practical applications. However, when considering rapid deployment in real-world applications, current popular simulation environments cannot be freely and effectively customized according to user needs. Additionally, crafting new domain-specific environments for particular problems frequently encounters challenges in swiftly integrating with existing MARL algorithm libraries 1 1 1 We refer to MARL libraries as experimental frameworks, characterized by their interfaces for both common environments and various algorithms. , thus hindering the deployment of state-of-the-art (SOTA) algorithms to meet practical requirements ( Oroojlooy & Hajinezhad, 2023 ) . The existence of modern game engines offers a possible solution to the aforementioned issues. Game engines are software frameworks originally designed to simplify the development of video games ( Boyd & Barbosa, 2017 ) . Over years of evolution, modern game engines have established vast development communities, along with rich rendering resources and physics engine materials ( Wheeler, 2023 ) . If possible, these potential resources could greatly facilitate the application deployment within the MARL domain. Moreover, game engines are closely linked with the rapidly developing generative AI, which has the potential to quickly transform user needs into real products. Some works have utilized simulation data from game engines for the training of generative AI ( Lu et al., 2024 ) , whereas others have established a feedback loop, incorporating these trained generative models into game engine plugins to facilitate scene creation ( NVIDIA, 2024 ) . This suggests that combining game engines with MARL could have a very promising future. However, there remains a significant gap between the game development community associated with game engines and the MARL community. Although some efforts, such as URLT ( Sapio & Ratini, 2022 ) and Unity-ML Agents ( Juliani, 2018 ) , have built frameworks for freely constructing RL training scenarios using game engines 2 2 2 In this paper, we refer to the works that allow for the free construction of RL training scenarios as general platforms ( Juliani, 2018 ) . , they still face issues in training efficiency and scaling complexity ( Kaup et al., 2024 ) . To address this, we propose Unreal-MAP, an MARL general platform based on the Unreal Engine 3 3 3 https://www.unrealengine.com . . Compared to existing solutions, Unreal-MAP possesses the following main features: (1) Fully Open-Source and Easily Modifiable , Unreal-MAP utilizes a layered design, and all components from the bottom-level engine to the top-level interfaces are open-sourced. Users can easily modify all elements of an MARL task by focusing only on the higher-level operational layers. (2) Optimized Specifically for MARL , the underlying engine of Unreal-MAP has been optimized to enhance efficiency in large-scale agent simulations and data transmission. This optimization allows users to develop simulations with heterogeneous, large-scale, multi-team settings that showcase distinctive multi-agent features through Unreal-MAP. (3) Parallel Multi-Process Execution and Controllable Single-Process Time Flow , Unreal-MAP supports the parallel execution of multiple simulation processes as well as the adjustment of the simulation time flow speed in a single process. Users can accelerate simulations to speed up training or decelerate simulations for detailed slow-motion analysis. To fully utilize the capabilities of Unreal-MAP, we also develop an MARL experimental framework known as the Hybrid Multi-Agent Playground (HMAP). This framework includes implementations of rule-based algorithms, built-in learning-based algorithms, and algorithms from third-party frameworks such as PyMARL2 ( Hu et al., 2021 ) and HARL ( Zhong et al., 2024 ) . By leveraging Unreal-MAP and HMAP, users can rapidly customize environments and deploy algorithms, validate new research ideas, and apply them in practical scenarios. The overview of the research workflow for using Unreal-MAP is depicted in Figure 1 and more details can be found in Appendix C.4 . The contributions of this work are summarized as follows: firstly, an MARL general platform based on the Unreal Engine; secondly, an accompanying modular MARL experimental framework; thirdly, a collection of highly extensible example scenarios based on Unreal-MAP and related experimental analyses. We believe Unreal-MAP can serve as a comprehensive tool to advance the development of MARL and ultimately facilitate their application in real-world scenarios. 2 Related Work Simulation Environments for MARL. Existing MARL environments can be broadly divided into domain-specific environments and environment-suites . The former includes a series of tasks designed around the same domain, which usually share common genre and similar benchmark metrics. Notable examples of these works include MPE ( Mordatch & Abbeel, 2017 ) , SMAC ( Samvelyan et al., 2019 ) , GRF ( Kurach et al., 2020 ) , MAMuJoCo ( Peng et al., 2021a ) , and GoBigger ( Zhang et al., 2022 ) . Due to architectural constraints, domain-specific environments cannot be freely modified according to user needs. For instance, a StarCraft-based SMAC scenario can never be reconfigured into soccer-style tasks. On the other hand, environment suites consist of sets of environments packaged together, commonly used to more conveniently benchmark the performance of algorithms. Some works also redesign built-in environments to enhance training speed. Typical works in this category include PettingZoo ( Terry et al., 2021 ) and JaxMARL (