Adaptive Punishment for Cooperation in Mixed-Motive Games

Min Tang, Fanqi Kong, Linyuan Lü, Xue Feng · 2026

Read Adaptive Punishment for Cooperation in Mixed-Motive Games by Min Tang, Fanqi Kong, Linyuan Lü, Xue Feng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24516.txt.

Introduction

Min Tang and three colleagues propose Adaptive Punishment for Cooperation, a 2026 plan for teamwork among self-interested agents. Punishment deters defection, but punishing all the time costs the punisher, so the plan scales its force with how bad each defection is. A module that rates each defection, guided by game rewards, sets how likely punishment is. Tests in iterated public-goods games and sequential social dilemmas show the plan beating earlier methods.

Catalog record

CREATORMin Tang, Fanqi Kong, Linyuan Lü, Xue Feng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29499
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24516

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Punishment for Cooperation in Mixed-Motive Games Min Tang, Fanqi Kong, Linyuan Lü, Xue Feng Abstract Mixed-motive scenarios are ubiquitous in real-world multi-agent interactions, where self-interested agents often defect for immediate rewards, overlooking the potential of altruistic cooperation to improve long-term gains and collective welfare. Peer punishment can deter defection, but as costly second-order altruism, its persistent imposition may undermine the punisher’s interests. Existing approaches often struggle to effectively implement punishment to promote cooperation. To balance the efficacy and cost of punishment, we propose Adaptive Punishment for Cooperation (APC), a distributed method that determines punishment intensity based on both a dynamic punishment probability and the severity of defection. This dynamic probability substantially reduces costly and ineffective punishment while also promotes cooperation. To accurately assess defection and its severity, we use a defection awareness module, whose learning is guided by game reward. Theoretical analysis and empirical results show APC performs effectively in iterated public goods game. Empirically, APC also significantly outperforms existing baselines across sequential social dilemmas, learning rational and effective punishment policies that foster cooperation by strategically deterring defection. 2 2 footnotetext: Corresponding author I Introduction Multi-Agent Reinforcement Learning (MARL) allows agents to interact in shared environments, yet mixed-motive scenarios [ 13 , 8 ] pose unique challenges compared to purely cooperative or competitive tasks [ 16 ] . In these settings, dynamic agent relationships often lead to social dilemmas where short-term individual gains supersede long-term collective welfare. Traditional training paradigms often fail to resolve these issues: Centralized Training with Decentralized Execution (CTDE) [ 24 ] relies on unrealistic assumptions of joint optimization among self-interested agents, while Decentralized Training (DTDE) frequently converges to suboptimal local equilibria (see Fig. 2(a) ). Consequently, there is a critical need for methods that guide decentralized agents out of these dilemmas toward socially beneficial outcomes. To promote cooperation in mixed-motive games, it is crucial to both deter defection and incentivize collective welfare. Inspired by human societies, punishment raises the cost of defection, discouraging short-term selfish behavior and guiding agents toward long-term group benefits [ 6 ] . However, peer punishment imposes a penalty on others while incurring a cost for the punisher (e.g., time and effort spent stopping someone from smoking in public). That is, punishment is second-order altruism. Excessive punishment may harm the punisher’s own interest, and agents may avoid punishing due to its cost, failing to achieve cooperation [ 12 ] . To address these challenges, we propose a distributed decision-making method named APC (Adaptive Punishment for Cooperation), which is context-sensitive and adaptively adjusts the probability and severity of punishment. Guided by reward signals, APC learns a defection awareness module to evaluate the defectiveness of others’ actions, which determines the probability and intensity of punishment. Notably, to prevent ineffective punishment and avoidable cost, punishment probability is dynamically adjusted according to the historical effectiveness of punishment, based on the reduction in defections. The main contributions of this work are: (1) We propose a novel opponent-adaptive punishment method, APC, that dynamically adjusts punishment intensity to avoid ineffective punishment and excessive costs, thereby promoting cooperation in mixed-motive games. (2) We develop a self-learning defection awareness algorithm for detecting different degrees of defection in multi-agent interactions. (3) We provide theoretical analysis and demonstrate empirically that APC outperforms existing baselines across Iterated Public Goods Game and Sequential Social Dilemmas (SSDs). II Related Work Multi-agent reinforcement learning has been widely applied to address problems related to social dilemmas. Various reward mechanisms, including centralized redistribution [ 5 ] and decentralized peer-to-peer rewarding [ 11 ] , have been proposed to promote cooperation by aligning individual and collective interests in mixed-motive games. Punishment, which promote cooperation without requiring additional reward resources, have gained increasing attention. In evolutionary game theory, punishment is widely regarded as a key factor in achieving stable cooperation [ 23 , 7 ] . Some approaches [ 15 , 25 ] rely on centralized controllers or self-organized structures among individuals to implement punishment, further investigating how punishment influence the evolution and stability of cooperative behaviors. Other methods [ 3 , 22 , 18 ] adopt conditional punishment policy, where the decision to punish depends on conditions such as the difference between an individual’s own payoff and the average payoff of its neighbors, thereby improving the specificity and effectiveness of punishment. In MARL, although some studies have attempted to incorporate punishment method into multi-agent systems, some designs remain relatively simple—typically adding a punishment action into the action space [ 13 , 9 , 19 ] . However, experimental results, shown as in Figure 3 , have shown that merely relying on such punishment action combined with standard Independent MARL methods often fails to promote cooperation in SSDs. Other approaches [ 26 , 1 ] depend on auxiliary mechanisms or specific environmental assumptions to support the effectiveness of punishment, such as introducing reputation systems or assuming that punishment behaviors can generate positive incentives. Some methods further assume that punishment can directly yield positive rewards for the punisher [ 21 , 27 ] , which is clearly unrealistic. We propose APC, which does not rely on environmental assumptions but instead identifies defection behaviors through a defection predictor network and applies targeted punishment. This guides agents to learn reasonable and effective punishment policy, thereby promoting cooperation more effectively. III Preliminaries An N -player Partially Observable Markov Game (POMG) is defined as \mathcal{M}=\langle N,\mathcal{S},\{\mathcal{O}^{i}\},\{\mathcal{A}^{i}\},T,\{R^{i}\}\rangle . Here, \mathcal{S} is the state space, while \mathcal{O}^{i} and \mathcal{A}^{i} denote the observation and action spaces for agent i , respectively. The state transition function T:\mathcal{S}\times\mathcal{A}^{1}\times\dots\times\mathcal{A}^{N}\times\mathcal{S}\to[0,1] defines the probability distribution over \mathcal{S} , representing the probability of transitioning to state s^{\prime} given the current state s and joint action \vec{a}=(a^{1},\dots,a^{N}) . Each agent i follows a policy \pi^{i}(a^{i}|o^{i}) to maximize its expected discounted return: V_{i}^{\pi^{i}}(s_{0})=\mathbb{E}_{\vec{a}_{t}\sim\pi,s_{t+1}\sim T}\left[\sum_{t=0}^{\infty}\gamma^{t}R^{i}(s_{t},\vec{a}_{t})\right], (1) where \gamma\in[0,1) is the discount factor. In our decentralized MARL framework, agents are trained using the Advantage Actor-Critic (A2C) method. The policy \pi^{i} is parameterized by \theta^{i} , and the critic evaluates the actor via the TD-error: \delta_{t}^{i}=R^{i}(s_{t},\vec{a}_{t})+\gamma V^{\pi_{\theta^{i}}}(\vec{o}_{t+1})-V^{\pi_{\theta^{i}}}(\vec{o}_{t}) , where \vec{o}_{t} represents the joint observation at timestep t . IV Methodology Fig. 1: Overview of the APC Framework. The framework consists of two core components: Defection Awareness and Adaptive Punishment. Any Agent i interacts with the environment to collect trajectories, which are stored in a replay buffer to train the defection predictor network \mu^{i} . \mu^{i} is trained using o^{i},a^{-j} , and r^{i} to predict the probability distribution \sigma^{ij}=p_{\mu^{i}}(\cdot|o^{i},a^{-j}) over \mathcal{A}^{j} . A higher-probability action a^{j} indicates a greater degree of defection. Adaptive punishment computes the probability p^{ij} (Eq.( 3 )) based on the defection frequency f^{ij} over the past L timesteps. The punishment intensity weight w^{ij} is then determined: if \sigma^{ij}(a^{j})>\tfrac{1}{|\mathcal{A}^{j}|} , w^{ij} is set proportional to \sigma^{ij}(a^{j}) with probability p^{ij} ; otherwise, w^{ij}=0 . Finally, agent i ’s total reward for policy update is r^{i,\text{tot}}=r^{i}-\sum_{j=1,j\neq i}^{N}w^{ij}c-\sum_{j=1,j\neq i}^{N}w^{ji}\delta , where c and \delta are unit cost and penalty. To address the dilemma of cooperation and defection in multi-agent systems, we propose a distributed MARL method, named Adaptive Punishment for Cooperation (APC). As shown in Fig. 1 , APC consists of two core modules: defection awareness and adaptive punishment. The defection awareness module takes agents’ local observation trajectories as input

Read the complete text