An Efficient Open World Environment for Multi-Agent Social Learning

Eric Ye, Ren Tao, Natasha Jaques · 2025

Read An Efficient Open World Environment for Multi-Agent Social Learning by Eric Ye, Ren Tao, Natasha Jaques (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15679.txt.

Introduction

Eric Ye, Ren Tao, and Natasha Jaques open a playground for studying how AI agents learn from each other. Real settings are social: many self-driven agents, some of them expert, share one world. The authors argue that a learner surrounded by experts can pick up skills faster, but open-ended worlds for testing that idea were missing. Their environment lets many agents chase separate, complex goals at once.

Agents can profit from watching experts, and they may drift into cooperation against shared enemies, or into building and sharing tools for long projects. The team uses the world to measure how social learning changes performance, and whether rivalry or teamwork helps more. They offer the environment as a testbed for socially intelligent agents that must learn the way people do, from the crowd around them.

Catalog record

CREATOREric Ye, Ren Tao, Natasha Jaques
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15679

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An Efficient Open World Environment for Multi-Agent Social Learning Eric Ye, Ren Tao, Natasha Jaques Abstract Many challenges remain before AI agents can be deployed in real-world environments. However, one virtue of such environments is that they are inherently multi-agent and contain human experts. Using advanced social intelligence in such an environment can help an AI agent learn adaptive skills and behaviors that a known expert exhibits. While social intelligence could accelerate training, it is currently difficult to study due to the lack of open-ended multi-agent environments. In this work, we present an environment in which multiple self-interested agents can pursue complex and independent goals, reflective of real world challenges. This environment will enable research into the development of socially intelligent AI agents in open-ended multi-agent settings, where agents may be implicitly incentivized to cooperate to defeat common enemies, build and share tools, and achieve long horizon goals. In this work, we investigate the impact on agent performance due to social learning in the presence of experts and implicit cooperation such as emergent collaborative tool use, and whether agents can benefit from either cooperation or competition in this environment. 1 Introduction The real world is inherently multi-agent. For AI systems ranging from autonomous cars to household and digital assistants to be useful to people, they require social intelligence to effectively navigate human multi-agent environments. Despite additional challenges like coordinating with other agents, multi-agent systems also present new opportunities. As intelligent (human) agents complete their independent tasks in the real world, they demonstrate highly effective behaviors honed by a lifetime of experience ( Rendell et al. 2010 ) . If AI agents could effectively leverage this information, they could not only learn to rapidly acquire complex skills, but also adapt online to environmental changes, addressing fundamental challenges in AI relating to learning and generalization. In this paper, we introduce Multi-Agent Craftax (MAC) , an open-ended MARL environment designed to test the social intelligence of AI agents in a highly performant manner. Our environment is capable of investigating questions related to social learning, collaborative tool use, the emergence of complex skills, and how these factors are affected by incentives to cooperate or compete. Built on the high-performance, JAX-based Craftax environment ( Matthews et al. 2024 ) , MAC is capable of running 100 million training steps in less than one hour on a single GPU. In this dynamic environment, multiple agents can pursue independent goals, which each require complex, long-horizon planning in a large, partially-observable, sparse reward environment. Importantly, the goals share underlying subtasks, such as tool crafting, so agents with social learning and cooperation skills can more effectively achieve their goals. Thus, MAC is designed to facilitate research into methods that improve agents’ abilities to learn, interact, and adapt in the presence of other intelligent agents. Figure 1: Sample pixel observations from a single agent in MAC. We compare the performance of agents trained with others to their single-agent counterparts to study whether the presence of other agents accelerates learning. We find that agents do not benefit from existing social learning methods, pointing to the need for more sophisticated algorithms. Yet, agents use tools crafted by other agents 90% of the time, highlighting implicit cooperation in the shared environment. Social Learning —learning how to learn from other agents—could improve generalization of AI agents by enabling them to rapidly acquire new skills from experts present in their environment ( Ndousse et al. 2021 , Filos et al. 2021 , Bhoopchand et al. 2023 ) . Consider an office robot tasked with processing trash. Initially unaware of the trash categorizations and bin locations, it can rapidly learn these by observing where workers throw trash into which bins. Exploration is costly, creating conditions where social learning is especially beneficial ( Laland 2004 ) . In this benchmark, we use recently proposed metrics ( Bhoopchand et al. 2023 ) , to measure the performance of state-of-the-art MARL algorithms ( De Witt et al. 2020 , Yu et al. 2022 ) and recently proposed social learning methods ( Ndousse et al. 2021 ) . Emergent Collaborative Tool Use. Tool use is a core component of human social learning and our cultural and technological evolution ( Van Schaik and Pradhan 2003 , Humphrey 1976 , Boyd et al. 2011 ) . A person creating and leaving a tool in a shared environment facilitates social learning by structuring the environment so others can more easily discover how to use and manufacture tools ( Henrich and Tennie 2017 ) . MAC uniquely presents an opportunity to study how agents share tools in an environment where one agent’s progress fundamentally alters the environment to benefit others. Cooperation vs. Competition in Mixed-Motive Settings. We study how competitive and cooperative pressures within MAC affect social learning, tool sharing, and agent performance. The environment has mixed-motive incentives, where agents can benefit both from cooperating (by sharing tools or teaming up against common enemies) and competing (by consuming resources or attacking others). By varying the environment’s reward structure, researchers can study how agents balance cooperation and competition and its impact on social learning and emergent behaviors. While we find interesting evidence of shared tool use, we find little indication that existing methods are effective at using social learning to improve performance on the demanding, long horizon tasks in MAC. This suggests further research into improving the ability of state-of-the-art MARL algorithms to rapidly acquire skills from others using social learning is needed. Multi-Agent Craftax provides a new way to study this, and particularly examine how agents benefit from sharing an environment with other intelligent agents who restructure it to be safer and provide shared tools. 2 Related Works Multi-Agent Environments. Many MARL environments (e.g. MADDPG ( Lowe et al. 2017 ) , Starcraft Multi-Agent Challenge (SMAC) ( Samvelyan et al. 2019 ) , and Melting Pot ( Leibo et al. 2021 ) ) focus on agent cooperation or coordination, but few share our focus on learning from independent agents with long-horizon, hard-exploration tasks. While Neural MMO is a massively-multiplayer open-ended environment, Suarez et al. 2019 mainly investigate how competition drives the emergence of intelligent behavior. We test both competition and cooperation, but primarily focus on studying and improving the ability of agents to learn socially from others and implicitly cooperate. Minecraft Research. Many environments based on Minecraft ( Mojang Studios 2009 , Guss et al. 2019 , Mao et al. 2021 ) , such as Minecraft Building Assistance Game ( Laidlaw et al. 2024 ) and VillagerBench ( Dong et al. 2024 ) , have been created because of its long horizon, sparse rewards, and potential for emergence of complex behavior. One such case is Crafter ( Hafner 2021 ) , which is designed to be challenging, with achievements representing significant milestones in performance. However, because it can only be simulated on a CPU due to being written in pure Python, the benchmark is limited to only 1 million environment interactions. JAX-based environments. JAX ( Bradbury et al. 2018 ) is a Python library that compiles and optimizes array computation and automatic differentiation to run on hardware accelerators such as GPUs and TPUs, enabling speedup of many magnitudes ( Rutherford et al. 2024 )

Read the complete text