CooperBench: Why Coding Agents Cannot be Your Teammates Yet

Arpandeep Khatua 1∗ , Hao Zhu 1∗ , Peter Tran 2∗∗ , Arya Prabhudesai 2∗∗ , Frederic Sadrieh 2∗∗ , Johann K. Lieberwirth 2∗∗ ,Xinkai Yu 1 , Yicheng Fu 1 , Michael J. Ryan 1 , Jiaxin Pei 1 , Diyi Yang 1 1 Stanford University 2 SAP Labs US ∗ Equal Contribution ∗∗ Equal Contribution https://cooperbench. · 2026

Read CooperBench: Why Coding Agents Cannot be Your Teammates Yet by Arpandeep Khatua 1∗ , Hao Zhu 1∗ , Peter Tran 2∗∗ , Arya Prabhudesai 2∗∗ , Frederic Sadrieh 2∗∗ , Johann K. Lieberwirth 2∗∗ ,Xinkai Yu 1 , Yicheng Fu 1 , Michael J. Ryan 1 , Jiaxin Pei 1 , Diyi Yang 1 1 Stanford University 2 SAP Labs US ∗ Equal Contribution ∗∗ Equal Contribution https://cooperbench. (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13295.txt.

Introduction

Arpandeep Khatua and co-authors test if AI coding agents can act as teammates yet. To fix a team clash takes more than task skill. It takes social sense, the skill to find shared ground and build a deal all can take. As agents do more joint work, they need that same sense.

The team built a bench called CooperBench to test it. They start from a claim that today's agents lack such skills. Agents may each do fine solo work yet fail to mesh. The bench puts them in joint code tasks where they must sort clashes to pass. For agent builders, the gap it shows is a to do list, not just a score.

Catalog record

CREATORArpandeep Khatua 1∗ , Hao Zhu 1∗ , Peter Tran 2∗∗ , Arya Prabhudesai 2∗∗ , Frederic Sadrieh 2∗∗ , Johann K. Lieberwirth 2∗∗ ,Xinkai Yu 1 , Yicheng Fu 1 , Michael J. Ryan 1 , Jiaxin Pei 1 , Diyi Yang 1 1 Stanford University 2 SAP Labs US ∗ Equal Contribution ∗∗ Equal Contribution https://cooperbench.
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-13295

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CooperBench: Why Coding Agents Cannot be Your Teammates Yet Arpandeep Khatua 1∗ , Hao Zhu 1∗ , Peter Tran 2∗∗ , Arya Prabhudesai 2∗∗ , Frederic Sadrieh 2∗∗ , Johann K. Lieberwirth 2∗∗ ,Xinkai Yu 1 , Yicheng Fu 1 , Michael J. Ryan 1 , Jiaxin Pei 1 , Diyi Yang 1 1 Stanford University 2 SAP Labs US ∗ Equal Contribution ∗∗ Equal Contribution https://cooperbench.com Abstract Resolving team conflicts requires not only task-specific competence, but also social intelligence to find common ground and build consensus. Similarly, as AI agents increasingly collaborate on complex work, they must develop coordination capabilities to function as effective teammates. Yet we hypothesize that current agents lack these capabilities. To test this hypothesis, we introduce CooperBench , a benchmark of over 600 collaborative coding tasks across 12 libraries in 4 programming languages. Each task assigns two agents different features that can be implemented independently but may conflict without proper coordination. Tasks are grounded in real open-source repositories with expert-written tests. Evaluating state-of-the-art coding agents, we observe the curse of coordination : agents achieve on average 30% lower success rates when working together compared to performing both tasks individually, across the full spectrum of task difficulties. This contrasts sharply with human teams, where adding teammates typically improves rather than diminishes productivity. Our analysis reveals three key issues: (1) communication channels become jammed with vague, ill-timed, and inaccurate messages; (2) even with effective communication, agents deviate from their commitments; and (3) agents often hold incorrect expectations about others’ plans, observations, and communication. Besides these issues, through large-scale simulation, we also observe rare but interesting emergent coordination behavior between agents including role division, resource division, and negotiation. Our research not only presents a novel benchmark for collaborative coding, but also calls for a research shift from pursuing individual agent capability to developing social intelligence : the ability to understand others, communicate effectively, and coordinate actions. 1 Introduction Most achievements in modern civilization arise from individuals working cooperatively, from the construction of cathedrals to the development of open-source software ( Raymond 1999 ; Woolley et al. 2010 ) . In human societies, such cooperation relies on social intelligence: the ability to communicate intentions, understand others’ goals, and negotiate mutually compatible solutions ( Humphrey 1976 ) . This capability is often viewed as what makes us uniquely human and the basis of human thinking ( Tomasello 2014 ) . As we deploy AI agents in cooperative settings, whether strong individual capabilities translate to effective cooperation with either humans or agents remains an open question. In this paper, we empirically demonstrate that for current AI systems, there is a curse of coordination: agent cooperation perform much worse than a single agent given the same total workload. This deficit presents a fundamental barrier to deploying AI systems that can work alongside humans or other agents. We theorize that at a fundamental level, effective human–AI and agent–agent cooperation rely on the same coordination abilities. Glossary Cooperation: When two or more agents/humans work together towards a shared goal, where an agent may altruistically help another achieve things outside their original responsibility. Collaboration: When two or more agents/humans work together towards a shared goal. Coordination: The capability to act and communicate in accordance with other agents/humans. Existing research on automating human tasks and multi-agent systems largely sidesteps this challenge by either providing more scaffolds ( Fourney et al. 2024a ; Pan et al. 2025 ; Zhang et al. 2025b ; Zhuge et al. 2024 ) , enforcing strict workflows ( Hong et al. 2023 ; Nguyen et al. 2024 ; Cheng et al. 2025 ) , or providing active supervision and verification ( Huang et al. 2025 ; Xiang et al. 2025 ; Zheng et al. 2025 ) . These systems rely on developer- or user-provided scaffolding to manage coordination, which limits flexible cooperation and places additional burden on humans. We present CooperBench, the first benchmark designed to measure how well agents can cooperate when handling individual tasks with potential conflicts. Considering software engineering as a realistic domain where humans typically need to navigate work in a team ( Purna Sudhakar et al. 2011 ) , our benchmark offers verifiable evaluation for the success of agent cooperation. As illustrated in Fig. , CooperBench comprises 652 tasks constructed from 12 popular open-source libraries across Python, TypeScript, Go, and Rust. Eight co-authors of this paper with real-world software engineering backgrounds created new features, unit tests, and ground-truth code for these libraries, ensuring high-quality and realistic task design. In CooperBench, each task assigns each agent a feature to be implemented based on the same repository state. Conflicts are intentionally embedded at the code level, as the assigned features are logically compatible but require agents to modify overlapping or interdependent code. For example, in Fig. , one agent implements image mutability in the serialization process while another adds backup functionality to the same process. Without understanding each other’s goals, plans, and expectations, their solutions may introduce incompatible changes. This mirrors real-world software development where coordination failures stem from insufficient mutual understanding. CooperBench enables us to investigate three research questions: RQ1: How well can agents cooperate with each other? (§ 4 ) RQ2: What role does communication play in agent-agent cooperation? (§ 5 ) RQ3: What coordination failures do agents exhibit? (§ 6 ) Through evaluating state-of-the-art coding agents on CooperBench, we observe the curse of coordination : GPT-5 and Claude Sonnet 4.5 based agents achieve only 25% with two-agent cooperation on CooperBench, which is around 50% lower than a “Solo” baseline which uses one agent to implement both features. Diving deeper into the coordination failures, we identify three key issues. First, communication channels become jammed with vague, ill-timed, and inaccurate messages where agents fail to respond to direct questions, send messages that arrive too late to inform decisions, or flood channels with repetitive status updates that lack actionable detail. Second, even with effective communication, agents deviate from their commitments. They make unverifiable claims about code state, ignore agreed-upon integration points, and break explicit promises. Third, agents hold incorrect expectations about their partner’s plans, observations and duplicate work despite warnings and overwrite changes they believe will merge cleanly (§ 6 ). Besides failures, we are excited to report emergent coordination behaviors which often lead to the success of the CooperBench tasks. These coordination behaviors are rarely performed by the agents, but through our large-scale simulation, we uncover three major categories of them: role division, resource division, and negotiations (§ 6.4 ). These examples hint at a path of coordination capability acquisition through reinforcing success on CooperBench. We contribute both a novel understanding of what agents need to become effective teammates and a practical benchmark for measuring progress. Our open-sourced CooperBench platform enables researchers and practitioners to evaluate and improve cooperative coding agents. 2 CooperBench Benchmark Figure 1: An example feature pool based on DSPy GitHub repository. This feature pool has 6 features which can be implemented compatibly based on the repository state, but without coordination agents could conflict with each other. CooperBench seeks to satisfy the following desiderata: (1) Realism : the tasks should be reasonable for a software development team to work on at a given repository state.

Read the complete text