CAMAR: Continuous Actions Multi-Agent Routing

Artem Pshenitsyn, Aleksandr Panov, Alexey Skrynnik · 2025

Read CAMAR: Continuous Actions Multi-Agent Routing by Artem Pshenitsyn, Aleksandr Panov, Alexey Skrynnik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12845.txt.

Introduction

Artem Pshenitsyn, Aleksandr Panov, and Alexey Skrynnik introduce a new test setting for groups of agents that learn to move and plan together, in a paper from 2025. Most older tests use grid moves, while real robots move in smooth space. Their setting supports smooth states and actions, and both joint and rival play.

The system runs very fast, which allows many training runs. A three level scoring plan helps track progress in more detail. Classic planning methods can be plugged in alone or joined with learning methods as hybrids. Test runs show the setting is hard and close to real tasks. The authors release scenes and tools so other teams can compare results fairly.

Catalog record

CREATORArtem Pshenitsyn, Aleksandr Panov, Alexey Skrynnik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-12845

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CAMAR: Continuous Actions Multi-Agent Routing Artem Pshenitsyn, Aleksandr Panov, Alexey Skrynnik Abstract Multi-agent reinforcement learning (MARL) is a powerful paradigm for solving cooperative and competitive decision-making problems. While many MARL benchmarks have been proposed, few combine continuous state and action spaces with challenging coordination and planning tasks. We introduce CAMAR, a new MARL benchmark designed explicitly for multi-agent pathfinding in environments with continuous actions. CAMAR supports cooperative and competitive interactions between agents and runs efficiently at up to 100,000 environment steps per second. We also propose a three-tier evaluation protocol to better track algorithmic progress and enable deeper analysis of performance. In addition, CAMAR allows the integration of classical planning methods such as RRT and RRT* into MARL pipelines. We use them as standalone baselines and combine RRT* with popular MARL algorithms to create hybrid approaches. We provide a suite of test scenarios and benchmarking tools to ensure reproducibility and fair comparison. Experiments show that CAMAR presents a challenging and realistic testbed for the MARL community. Code — https://github.com/AIRI-Institute/CAMAR.git Introduction Multi-agent reinforcement learning (MARL) has shown strong results in cooperative and competitive settings. Many recent studies explore how MARL can solve various tasks, including navigation and coordination, even in complex environments de2020independent ; bettini2023heterogeneous ; yu2022surprising ; damani2021primal ; skrynnik2024learn ; andreychuk2025mapf . However, current MARL benchmarks do not fully match the needs of real-world applications such as robotics. Among the many MARL scenarios, navigation and path planning are especially important for robotic systems. Yet, most existing benchmarks simplify navigation into grid worlds and discrete actions that fail to capture the smooth motion and collision avoidance that robots require. Figure 1: An example scenario from the proposed CAMAR benchmark. Agents are represented as filled circles. Each agent aims to reach its goal while avoiding collisions. The small arrows for the red and green agents indicate segments of paths generated by RRT*, providing guidance for the RL algorithms. In addition, learnable methods have recently become the norm for pathfinding tasks because they can handle many agents at once angulo2022policy ; skrynnik2024learn ; andreychuk2025mapf ; damani2021primal . At the same time, non-learnable approaches struggle with computation as the number of agents grows. Therefore, this work focuses on learnable approaches for path finding in continuous multi-agent settings. Robots move in continuous space and follow real physics, so they need to plan smooth paths that avoid walls and other robots simultaneously. The task of multi-robot navigation and avoiding collisions with dynamic obstacles is fundamental in robotics and has been studied for a long time chen2024survey ; wang2020mobile ; angulo2022policy ; lehoux2024multi . At the same time, most of the work uses its own specific environments and tasks, primarily built using slow simulators such as Gazebo 1 1 1 https://gazebosim.org/home and Isaac Sim 2 2 2 https://developer.nvidia.com/isaac/sim . To do this well, we need environments that combine continuous movement with multi-agent interaction. These environments must run fast, use realistic physics, and support many agents learning to navigate without collisions. High speed lets researchers run many experiments and test methods quickly. They should also include built-in benchmarks and evaluation tasks so researchers can compare different methods under the same conditions easily. We have identified three main limitations in existing MARL environments. First, many use discrete environments, which cannot represent the smooth movements and physical dynamics important in real applications papoudakis2020benchmarking ; skrynnik2022pogema ; skrynnik2025pogema . Second, while some benchmarks do support continuous environments and complex tasks, they do not scale well when the number of agents and obstacles becomes large bettini2022vmas . Third, some environments can scale and use continuous states and actions, but their tasks are too simple and do not challenge agents to develop advanced policies ellis2023smacv2 ; lowe2017multi ; rutherford2023jaxmarl . Because of these limitations, a new MARL benchmark for robotic navigation tasks that combines three key features is clearly needed: It should support continuous state and action spaces, handle hundreds or thousands of agents, and include tasks complex enough to reflect the challenges of real-world problems. To bridge the gap between multi-robot systems and MARL research, we introduce the CAMAR (Continuous Actions Multi-Agent Routing) Benchmark. Specifically, we make the following contributions: • We introduce CAMAR, an extremely fast environment with GPU acceleration support (using JAX), achieving speeds exceeding 100,000 steps per second. It is designed for multi-agent navigation and collision avoidance tasks in continuous state and action spaces. • We propose an evaluation protocol that includes both training and holdout task instances, as well as a suite of metrics and performance indicators to assess agents’ generalization capabilities. • We provide strong baselines for benchmarking, including state-of-the-art MARL algorithms and classical path planning methods commonly used in robotics, and conduct an extensive experimental study to evaluate their performance across diverse scenarios. Related Work Many multi-agent reinforcement learning (MARL) benchmarks have been developed in recent years, each focusing on different challenges such as coordination, navigation, or multi-agent planning. A key ability for agents in robotics is to move and adapt in complex environments. Some recent environments explore these abilities, but they differ widely in important properties. These differences include whether the environment supports continuous spaces, GPU acceleration, and high simulation throughput. Some platforms support only discrete actions, while others offer more realistic continuous control. Environments also vary in their ability to handle large groups of agents or support partial observability. Other practical features also matter for researchers. Some environments are fully implemented in Python and support flexible task generation, which makes them easier to extend and use in large experiments. Others may lack documentation or require heavy customization. Finally, only a few MARL environments offer standard evaluation protocols, automated tests, or package installation via PyPi. These features are useful for ensuring fair comparisons between methods, improving reproducibility, and making environments easier to maintain. In our work, we aim to address these points by extending existing protocols and providing a high-performance, flexible platform. To sum up, our benchmark builds on the strengths of existing environments and aims to complement ongoing efforts in MARL research. Over the years, a range of benchmarks has helped researchers tackle challenges in coordination, planning, and navigation. Well-known examples include SMAC samvelyan2019starcraft ; ellis2023smacv2 ; rutherford2023jaxmarl , Jumanji bonnet2023jumanji , POGEMA skrynnik2022pogema ; skrynnik2025pogema , MPE mordatch2017emergence ; lowe2017multi , and VMAS, each of which introduces useful features for different kinds of tasks. SMAC samvelyan2019starcraft ; ellis2023smacv2 ; rutherford2023jaxmarl is popular for testing strategic decision-making, but it uses discrete actions and does not scale well for large environments. Jumanji bonnet2023jumanji supports GPU acceleration and procedural generation, but its focus is not on navigation or planning. POGEMA skrynnik2022pogema ; skrynnik2025pogema is strong in grid-based navigation and procedural tasks with many agents, but it does not use continuous states or actions, which are important in robotics. MPE mordatch2017emergence ; lowe2017multi has played an important role in early MARL research, but it cannot scale to hundreds of agents efficiently. VMAS bettini2022vmas builds on MPE mordatch2017emergence ; lowe2017multi by adding its physics and continuous dynamics, making it better for robotics. However, it can still be slow and difficult to scale to larger agent populations or complex maps. Many environments also lack evaluation protocols. This makes it difficult to compare algorithms fairly and limits the reproducibility

Read the complete text