Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents
Yun Hua, Haosheng Chen, Shiqin Wang, Wenhao Li, Xiangfeng Wang, Jun Luo · 2025
Read Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents by Yun Hua, Haosheng Chen, Shiqin Wang, Wenhao Li, Xiangfeng Wang, Jun Luo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07388.txt.
Introduction
Yun Hua and co-authors propose a way to make self interested language model agents cooperate, in a paper from 2025. Groups of such agents do well when roles and steps are fixed in advance. In open settings with no set rules, though, each agent tends to chase its own goals, and fair teamwork breaks down.
The hard part is credit assignment: judging what each agent added and paying for it in a way that aligns their aims. Hua and co-authors borrow from human work life, where short term hiring and subcontracting solve a like problem. Their workflow, Shapley-Coop, prices each agent's added value using marginal contribution, a classic idea from game theory. Structured talks between agents then turn those prices into working deals.
Catalog record
| CREATOR | Yun Hua, Haosheng Chen, Shiqin Wang, Wenhao Li, Xiangfeng Wang, Jun Luo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07388
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents Yun Hua, Haosheng Chen, Shiqin Wang, Wenhao Li, Xiangfeng Wang, Jun Luo Abstract Large Language Models (LLMs) show strong collaborative performance in multi-agent systems with predefined roles and workflows. However, in open-ended environments lacking coordination rules, agents tend to act in self-interested ways. The central challenge in achieving coordination lies in credit assignment —fairly evaluating each agent’s contribution and designing pricing mechanisms that align their heterogeneous goals. This problem is critical as LLMs increasingly participate in complex human-AI collaborations, where fair compensation and accountability rely on effective pricing mechanisms. Inspired by how human societies address similar coordination challenges (e.g., via temporary collaborations like employment or subcontracting), a cooperative workflow Shapley-Coop is proposed. Shapley-Coop integrates Shapley Chain-of-Thought—leveraging marginal contributions as a principled basis for pricing—with structured negotiation protocols for effective price matching, enabling LLM agents to coordinate through rational task-time pricing and post-task reward redistribution. This approach aligns agent incentives, fosters cooperation, and maintains autonomy. We evaluate Shapley-Coop across two multi-agent games and a software engineering simulation, demonstrating that it consistently enhances LLM agent collaboration and facilitates equitable credit assignment. These results highlight the effectiveness of Shapley-Coop’s pricing mechanisms in accurately reflecting individual contributions during task execution. 1 Introduction Large Language Models (LLMs) are increasingly deployed as autonomous agents in multi-agent systems, demonstrating remarkable effectiveness across diverse real-world scenarios including multi-player games piatti2024cooperate ; tan2024true , software development tasks qian2024chatdev ; hong2024metagpt , medical care applications luukkonen2023fingpt , education chu2025llm and etc. However, despite their success in structured settings, achieving spontaneous cooperation among self-interested LLM agents remains challenging in open-ended environments, where explicit rules and predefined roles are absent and agents’ goals can be inherently conflicting piatti2024cooperate . In such situations, agents acting purely in self-interest typically encounter social dilemmas willis2025will , leading to suboptimal collective outcomes. Contemporary approaches to multi-agent cooperation involving LLMs can broadly be categorized into three paradigms: (1) rule-oriented methods, which impose strict behavioral constraints but compromise agents’ autonomy chen2023multi ; zhuang2023pose ; zhang2024exploring ; (2) role-oriented methods, which assign static roles limiting adaptability in dynamic environments chen2024agentverse ; hong2024metagpt ; qian2024chatdev ; schmidgall2025agent ; and (3) model-oriented methods, which assume alignment of goals and thus fail to handle natural conflicts of interest effectively xu2024magic ; mu2023runtime ; li2023theory . Although effective in narrow, task-specific contexts, most frameworks have yet to consider the critical challenge of aligning heterogeneous goals and fairly credit assignment that are essential for spontaneous cooperation in more open-ended multi-agent LLMs interactions. The central challenge for effective LLM agent coordination in open-ended environments lies in credit assignment —fairly evaluating each agent’s individual contributions—and designing principled pricing mechanisms - an incentive-aware value distribution system for multi-agent coordination capable of aligning their heterogeneous objectives . Addressing this challenge involves two critical questions: 1 . How can we establish effective pricing mechanisms that align the inherently heterogeneous goals of self-interested agents, thus enabling spontaneous emergence of cooperative behaviors? 2 . Once aligned, how can we guarantee fair and accurate credit assignment, ensuring the allocated rewards accurately reflect each agent’s actual contribution during task execution? Social scientists have historically navigated similar coordination challenges by developing sophisticated institutional mechanisms. Employment contracts, subcontracting agreements, and structured negotiations explicitly define the terms of cooperation, aligning individual incentives through clear pricing and ensuring fairness through transparent evaluation of contributions. Social sciences and managerial economics have formalized these practices into rigorous theoretical frameworks. Notably, classical economic theories such as Pigovian taxes blaug2011welfare and the Coase theorem farrell1987information provide structured solutions for managing externalities. Inspired by these established economic and managerial practices, we propose a novel cooperative workflow designed to coordinate self-interested LLM agents in open-ended multi-agent environments: Shapley-Coop . Our framework integrates Shapley Chain-of-Thought reasoning, which leverages marginal contributions as a rigorous pricing foundation, with structured negotiation protocols to facilitate effective and autonomous price matching . Shapley-Coop enables spontaneous cooperation through rational task-time pricing and transparent post-task reward redistribution, thus achieving alignment of agents’ heterogeneous goals in open-ended environments and effective credit assignment. We empirically validate Shapley-Coop across three distinct experimental environments: (1) In a simplified "Escape Room" social dilemma scenario, we demonstrate that conventional negotiation methods fail to resolve reward allocation conflicts adequately, whereas Shapley-Coop effectively enables agents to recognize and negotiate fair rewards, achieving successful cooperative outcomes. (2) In the more complex multi-step "Raid Battle Game," Shapley-Coop successfully balances competing individual incentives against collective success, enabling effective cooperation and equitable reward distribution. (3) Lastly, within the challenging "ChatDEV" software-development simulation, we show that precise credit assignment enabled by Shapley-Coop significantly enhances cooperative dynamics among diverse, self-interested LLM agents, highlighting the practical value of our approach for real-world collaborative productivity. In summary, the primary contributions of this paper are: - Introducing a Pricing-Based Perspective for Multi-LLM Cooperation: We explicitly address the critical challenge of aligning heterogeneous goals among self-interested LLM agents through principled pricing mechanisms inspired by cooperative game theory, thus enabling spontaneous emergence of cooperation in open-ended scenarios. - Proposing Shapley-Coop, a cooperative workflow: We propose Shapley-Coop, a cooperative framework that integrates Shapley Chain-of-Thought reasoning and structured negotiation protocols, facilitating fair and effective credit assignment among self-interested LLM agents, aligning incentives, and maintaining autonomy. - Empirical Validation Demonstrating Robust Cooperation and Practical Relevance: Through comprehensive experiments across diverse social-dilemma scenarios and realistic software-engineering tasks, we validate that Shapley-Coop consistently fosters emergent cooperation, equitably allocates rewards, and significantly improves collaborative dynamics—demonstrating its practical applicability and robustness in realistic human-AI collaborative environments. 2 Cooperation among Self-Interested LLM Agents In a multi-agent system composed of N Large Language Model (LLM) agents, each agent \pi_{\theta^{i}} observes a local state s^{i} , takes an action a^{i} , and aims to maximize its own local reward function r^{i} ( i=1,\cdots,N , while the global reward is defined as R(\pi_{\theta^{1}},\ldots,\pi_{\theta^{N}}) . Figure 1 : Escape room: One agent pulls a lever ( -1 ) to let the other escape through a door ( +10 ). Cooperation is necessary. This setup creates a misalignment between individual and collective incentives due to heterogeneous goals - agents optimizie local rewards while neglecting their impact on the global reward. Such misalignments often induce social dilemmas , where private optimization creates spillover effects on system-level outcomes. Consider the Escape Room scenario (Figure 1 ), where two agents have divergent but complementary goals: one needs to pay -1 to pull a lever, while the other gains +10 for opening the door. Without coordination, their selfish policies lead to a Nash equilibrium where neither acts (global payoff 0 ). However, by introducing a transfer payment through pricing mechanism which adds fees or bonuses to fix conflicts between selfish and group goals, their goals can be aligned, enabling successful escape and achieving Pareto-superior outcomes. Proper pricing mechanism design requires fair credit assignment based on agents’ marginal performance contributions. We introduce the Shapley value from cooperative game theory as a principled tool for measuring marginal contributions. Table 1 : Original payoff matrix for the escape room game Agent_{2} : door Agent_{2} : lever Agent_{1} : door (-1,-1) (10,-1) Agent_{1} : lever (-1,10) (-1,-1) The core idea of Shapley value is simple: instead of evaluating an agent in isolation,it considers how much value it adds to all possible