EmCoop: A Framework and Benchmark for Embodied Cooperation Among LLM Agents
Hanqing Yang, Shiyu Chen, Narjes Nourzad, Marie Siew, Jingdi Chen, Carlee Joe-Wong · 2026
Read EmCoop: A Framework and Benchmark for Embodied Cooperation Among LLM Agents by Hanqing Yang, Shiyu Chen, Narjes Nourzad, Marie Siew, Jingdi Chen, Carlee Joe-Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00349.txt.
Introduction
Yang and five colleagues study how teams of language model agents work together, arguing that final task success alone hides where joint work breaks down. Their COOP framework ties high level team conduct to visible progress in the task setting and sets checkable joint requirements, so failures can be traced over time. A repair tool built on it forecasts requirement failures from group plans and opens focused channels for fixes. Trials in two settings show gains in success and requirement keeping, at the price of added choice and message load.
Catalog record
| CREATOR | Hanqing Yang, Shiyu Chen, Narjes Nourzad, Marie Siew, Jingdi Chen, Carlee Joe-Wong |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102133 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.00349
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EmCoop: A Framework and Benchmark for Embodied Cooperation Among LLM Agents Hanqing Yang, Shiyu Chen, Narjes Nourzad, Marie Siew, Jingdi Chen, Carlee Joe-Wong Abstract Real-world scenarios increasingly require multiple embodied agents to collaborate in dynamic environments under embodied constraints, as many tasks exceed the capabilities of any single agent. Recent advances in large language models (LLMs) enable high-level cognitive coordination through reasoning, planning, and natural language communication. However, fine-grained analyses of how such collaboration emerges, unfolds, and contributes to task success in embodied multi-agent systems are difficult to conduct with existing benchmarks. In this paper, we introduce EmCoop, a benchmark framework for studying cooperation in LLM-based embodied multi-agent systems. Our framework separates a high-level cognitive layer from a low-level embodied interaction layer, allowing us to characterize agent cooperation through their interleaved dynamics over time. Given a cooperation-constrained embodied task, we propose generalizable, process-level metrics that diagnose collaboration quality and failure modes, beyond final task success. We instantiate our framework in two embodied environments that scale to arbitrary numbers of agents and support diverse communication topologies, and use these instantiations to demonstrate how EmCoop enables systematic analysis of cooperation dynamics across team sizes and task settings. The project web page can be found at: https://happyeureka.github.io/emcoop . † † affiliationnotice: Equal contribution 1 Introduction Embodied agents provide a fundamental paradigm for studying artificial intelligence in realistic settings that require continuous interaction with a dynamic physical environment ( 23 ) . Unlike purely symbolic or text-based systems, embodied agents must perceive, act, and adapt under physical and temporal constraints, enabling learning from causal feedback and long-horizon decision-making. In practice, most real-world embodied tasks cannot be solved by a single agent operating in isolation. Instead, they require multiple agents to communicate and coordinate their actions in shared environments to complete a shared task under real-world constraints like partial observability ( 5 ; 8 ; 7 ) , limited resources and mobility per agent ( 6 ; 4 ) , and asynchronous execution ( 15 ) . Examples include robotic teams, traffic systems, distributed sensing, and human–AI collaboration, where performance fundamentally depends on collective rather than individual behavior ( 10 ; 3 ; 22 ) . Figure 1 : Overview of EmCoop. (1) Agents operate through a dual-layer interface that bridges high-level planning (in Cognitive Layer ) and low-level execution (in Primitive Layer ) (Sec 4.1 ). (2) We instantiate our generalizable task designs (Sec 4.2 ) in our benchmark (Sec 6 ), which contains two embodied environments (MA-CRAFTER & CUBE) and diverse cooperative tasks. (3) Within the Multi-Agent Environment Interaction Loop (MAEIL), agents make plans and communicate asynchronously in Layer I , with primitive action execution handled by Layer II . (4) Through (1), (2), and (3), we make cooperation observable and quantifiable with cooperative metrics that enable systematic analysis of cooperative dynamics (Sec 5 ). This need for agent collaboration raises the first core challenge of this paper: how can we systematically study and evaluate collaboration in embodied multi-agent systems (MAS)? Existing benchmarks either focus on single-agent embodiment ( 19 ; 18 ; 11 ) or simplified multi-agent settings that fail to capture realistic collaboration dynamics ( 39 ) . Moreover, cooperation is often treated as an implicit factor in task success, rather than as an explicitly observable and analyzable process of repeated agent-agent or agent-environment interactions ( 40 ) . To address this gap, a benchmark for collaboration embodied MAS collaboration should (i) make collaboration transparent and human-readable, enabling visualization and auditing during the cooperation process, and (ii) support explicit communication topologies and interaction structures that can be analyzed and compared across environment settings. Large language models (LLMs) provide powerful cognitive capabilities for embodied agents, enabling symbolic task understanding, high-level reasoning, and long-horizon planning ( 26 ; 12 ) . Thus, LLMs are well-suited for open-ended, multi-modal, and complex embodied tasks that require abstraction and communication ( 2 ; 15 ) . However, real-world tasks require these cognitive and communicative decisions to influence low-level embodied execution and subsequent environment transitions under embodied constraints. This cognitive–embodied gap becomes even more pronounced in multi-agent settings, where multiple agents must align their internal reasoning, share knowledge, and coordinate physical actions in a shared environment. Understanding how cognitive intelligence is translated into collective embodied behavior, therefore, constitutes a second fundamental challenge for LLM-based embodied multi-agent systems. Motivated by the challenges of grounding cognitive intelligence into collective embodied behavior, we introduce EmCoop, a benchmark framework for studying cooperation in LLM-based multi-agent systems through embodied agents (Figure 1 ). EmCoop explicitly connects a high-level cognitive layer , where LLM agents perform symbolic reasoning, planning, and communication, with a low-level embodied layer , where agents execute primitive actions under physical and temporal constraints. This dual-layer design directly addresses our second fundamental challenge and, together with a formal specification of agent interactions, makes the entire cooperation process transparent, interpretable, and human-readable, addressing our first fundamental challenge. Unlike prior work that focuses on single-agent grounding or model-based world dynamics, EmCoop is model-free and emphasizes observable interaction processes, allowing it to generalize across diverse environment scenarios. After reviewing related work in Section 2 and formalizing embodied cooperation in Section 3 , we make the following contributions : (1) Dual-layer cognitive–embodied interface (Section 4.1 ). We formalize a dual-layer environment interface and multi-agent interaction loop for embodied LLM agents, which bridges high-level cognitive representations and low-level embodied interaction dynamics. By grounding LLM models into executable primitive actions, this interface provides a principled abstraction for translating cognitive decisions into embodied multi-agent behavior, thus allowing us to analyze agents’ cooperation process. (2) Cooperation-constrained task design (Section 4.2 ). We define cooperative tasks as temporally extended embodied processes whose progress requires sustained multi-agent collaboration. These tasks incorporate generalizable cooperation constraints that reflect the embodied environment. By requiring agents to cooperate and scaling to accommodate multiple numbers of agents, they provide a vehicle to study agent cooperation. (3) Process-level cooperation metrics and diagnostics (Section 5 ). We propose a suite of generalizable metrics that quantify cooperation quality beyond task success, including communication efficiency, coordination structure, temporal dependency, and failure modes. These metrics enable real-time diagnosis and visualization of cooperation dynamics in a reward-free and model-agnostic manner. (4) A scalable benchmark for cooperative intelligence (Section 6 ). We release a benchmark with two embodied environment instantiations of EmCoop that support evaluation on tasks with arbitrary numbers of agents, scalable difficulty levels, and diverse communication topologies. (5) Empirical analysis of cooperation dynamics (Section 7 ). We design controlled experiments to study how cooperation generalizes and evolves, addressing research questions (RQs) on the effect of communication topology on cooperation performance ( RQ1 ), the stages at which cooperation typically breaks down ( RQ2 ), and whether explicit cooperation feedback improves task performance ( RQ3 ). We conclude by discussing future work in Section 8 . Together , these contributions establish a unified and extensible benchmark for analyzing and designing collaborative LLM-based embodied multi-agent systems. 2 Related Work LLM-based multi-agent systems (MAS). Recent work explored LLM-based MAS through human-defined workflows, automatic workflow discovery, or multi-agent optimization ( 14 ) , including AutoGen ( 31 ) , MetaGPT ( 17 ) , GPT-Swarm ( 42 ) , and AgentVerse ( 9 )