Learning Reward Functions for Cooperative Resilience in Multi-Agent Systems

Manuela Chacon-Chamorro, Luis Felipe Giraldo, Nicanor Quijano · 2026

Read Learning Reward Functions for Cooperative Resilience in Multi-Agent Systems by Manuela Chacon-Chamorro, Luis Felipe Giraldo, Nicanor Quijano (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22292.txt.

Introduction

Manuela Chacon-Chamorro, Luis Felipe Giraldo, and Nicanor Quijano study how groups of agents can stay strong when things go wrong. Their 2026 preprint centers on shared resilience: the power to foresee, resist, bounce back from, and adapt to shocks. Mixed groups, where each agent has its own aims, face the strain most.

The team focus on how to learn reward rules that build that shared strength. Their setting is learning by trial and error, with many agents at once. Strong rules could help fleets of robots or grids ride out faults. The work names a goal the field has largely skipped.

Catalog record

CREATORManuela Chacon-Chamorro, Luis Felipe Giraldo, Nicanor Quijano
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22292

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Reward Functions for Cooperative Resilience in Multi-Agent Systems Manuela Chacon-Chamorro, Luis Felipe Giraldo, Nicanor Quijano Abstract Multi-agent systems often operate in dynamic and uncertain environments, where agents must not only pursue individual goals but also safeguard collective functionality. This challenge is especially acute in mixed-motive multi-agent systems. This work focuses on cooperative resilience —the ability of agents to anticipate, resist, recover, and transform in the face of disruptions—a critical yet underexplored property in Multi-Agent Reinforcement Learning. We study how reward function design influences resilience in mixed-motive settings and introduce a novel framework that learns reward functions from ranked trajectories, guided by a cooperative resilience metric. Agents are trained in a suite of social dilemma environments using three reward strategies: (i) traditional individual reward; (ii) resilience-inferred reward; and (iii) hybrid that balance both. We explore three reward parameterizations—linear models, hand-crafted features, and neural networks—and employ two preference-based learning algorithms to infer rewards from behavioral rankings. Our results demonstrate that hybrid strategy significantly improve robustness under disruptions without degrading task performance and reduce catastrophic outcomes like resource overuse. These findings underscore the importance of reward design in fostering resilient cooperation, and represent a step toward developing robust multi-agent systems capable of sustaining cooperation in uncertain environments. Keywords: Multi-agent systems, Inverse reinforcement learning, Preference-based learning, Reward design, Cooperative resilience, Social dilemmas. 1 Introduction Multi-agent systems often operate in dynamic and uncertain environments, where disruptions threaten overall functionality [ 37 , 42 ] . In particular, in mixed-motive multi-agent systems, agents must do more than simply optimize individual performance, they must collectively adapt and recover from disruptions to preserve system-level well-being. Disruptions, whether internal (e.g., system failures), external (e.g., environmental shocks), or adversarial (e.g., targeted attacks), can compromise system performance, underscoring the need for adaptive recovery mechanisms [ 37 ] . This motivates recent studies of resilience in multi-agent systems [ 9 , 34 ] , in particular the concept of cooperative resilience , defined as the ability of agents to sustain system-level well-being by anticipating, resisting, recovering, and transforming under disruption [ 9 ] . Unlike traditional notions of stability, equilibrium, or robustness, cooperative resilience captures the dynamic, temporal, and distributed nature of multi-agent systems operating in uncertain and failure-prone environments. Cooperative resilience represents a critical yet underexplored dimension in Multi-Agent Reinforcement Learning (MARL). Conventional MARL methods, including value factorization approaches such as QMIX [ 31 ] and policy gradient methods such as PPO [ 33 ] , rely on the specification of a coherent reward function. This reward must capture the interdependent nature of the joint task shaped by agents’ interactions. However, how the reward design influences the ability of agents to cooperate, adapt, and persist under adverse conditions is still not sufficiently understood. This limitation becomes particularly evident in mixed-motive settings, where agents must balance individual goals with collective outcomes, making the design of effective incentive mechanisms especially challenging [ 26 , 21 ] . In this work, we address this challenge using Inverse Reinforcement Learning (IRL) [ 16 , 4 , 40 ] , a principled framework for inferring reward functions from observed agent behavior. IRL recovers latent reward functions that are assumed to have generated trajectories exhibiting desirable or near-optimal responses to disruptions. This approach enables the discovery of incentive structures that support emergent properties such as cooperative resilience, without requiring explicit encoding of such behaviors in the reward design. This perspective is complementary to recent work on group resilience through collaboration protocols [ 34 ] , which seeks to improve resilience by designing mechanisms and interaction structures that make agents more robust to perturbations. In contrast, our goal is to infer a reward function that captures cooperative resilience directly from behavioral evidence. Protocol-based approaches enrich agents’ capabilities, while resilience-aligned reward inference uncovers the underlying incentive structure that can support resilient collective behavior. Building on this idea, we introduce an approach for learning reward functions from ranked agent trajectories scored with a cooperative resilience metric. This metric quantitatively evaluates how well trajectories preserve collective welfare in the presence of disruptions. By using it, we generate preference rankings over observed behaviors and feed them into a preference-based IRL pipeline to infer reward functions that implicitly encode cooperative resilience. In this way, our main contribution is a reward function design method that leverages cooperative resilience to infer a collective reward component, steering agents toward sustained system performance under disruptions. The learned reward is, in principle, compatible with different MARL algorithms, since it can be used as a drop-in replacement for the underlying reward signal. In this sense, our framework offers a flexible reward-design component that can complement existing methods. We validate our approach in a mixed-motive social dilemma inspired by the Commons Harvest scenario from Melting Pot [ 28 , 2 ] . The environment captures the tension between individual incentives to maximize resource consumption and the collective need to preserve shared resources. Our results show that resilience-inferred rewards foster adaptive behaviors under disruption, extending sustainability and improving collective outcomes compared to baselines PPO and QMIX. To assess scalability, we extended the evaluation to larger multi-agent environments with more agents and resource limitations, where our method enhanced cooperative resilience and, beyond that, improved overall system behavior by extending sustainability and collective well-being over time, without sacrificing individual task performance. This highlights our broader insight: reward functions can be treated as a form of a priori knowledge, extracted from trajectory analysis under a system-level cooperative resilience metric. The remainder of this paper is organized as follows. Section 2 reviews related work, Section 3 presents our framework and Section 4 details the experimental setup and key results. Section 5 concludes with a summary and future directions. The appendix provides implementation details, extended results, and reproducibility information for experiments. 2 Background and Related Work 2.1 Cooperative AI and Resilience Cooperative AI studies the design of multi-agent systems that achieve outcomes benefiting the group as a whole [ 12 , 17 ] . In mixed-motive environments, where agents must balance individual objectives with collective welfare, designing mechanisms that foster cooperation is particularly challenging [ 17 ] . Standard reinforcement learning approaches often emphasize individual performance, which can lead to selfish behavior and the degradation of shared resources, especially in the presence of social dilemmas [ 32 , 36 , 13 , 21 ] . The introduction of disruptions, such as resource scarcity, unsustainable behaviors, or abrupt environmental changes, further complicates cooperation [ 20 , 27 ] . In these contexts, resilience becomes essential to sustain joint welfare. We focus on cooperative resilience [ 9 , 10 ] , similar to the concept of group resilience [ 34 ] , a system-level property that quantifies how well a group of agents can maintain collective well-being under stress. Building on approaches from ecology, infrastructure, and economic networks [ 5 , 11 , 15 ] , the methodology in [ 9 ] evaluates resilience by comparing disrupted vs. baseline performance across indicators of collective well-being, producing a score of this property. This score provides a quantitative basis for comparing systems and analyzing the behaviors and incentives that lead to resilient outcomes. It can also inform the design of new agents and reward structures that promote both cooperation

Read the complete text