Homeostatic Coupling for Prosocial Behavior
Naoto Yoshida, Kingson Man · 2025
Read Homeostatic Coupling for Prosocial Behavior by Naoto Yoshida, Kingson Man (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12894.txt.
Introduction
Naoto Yoshida and Kingson Man ask what makes artificial agents care for each other in this 2025 paper. Humans in pain stir distress in those watching, and that distress often drives help. The team built agents that must keep their own internal balance, like living things. Each agent could either watch a partner's state or have its own state wired to the partner's.
In three simple shared worlds, helping appeared only under direct wiring. Watching alone never moved agents to help. Empathy could also be learned: agents decoded a partner's outward signals to infer hidden states. The authors show prosocial behavior emerging when agents feel, in their own balance, what others feel.
Catalog record
| CREATOR | Naoto Yoshida, Kingson Man |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-12894
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Homeostatic Coupling for Prosocial Behavior Naoto Yoshida, Kingson Man Abstract When regarding the suffering of others, we often experience personal distress and feel compelled to help 1 1 1 Preprint. Under review. . Inspired by living systems, we investigate the emergence of prosocial behavior among autonomous agents that are motivated by homeostatic self-regulation. We perform multi-agent reinforcement learning, treating each agent as a vulnerable homeostat charged with maintaining its own well-being. We introduce an empathy-like mechanism to share homeostatic states between agents: an agent can either observe their partner’s internal state ( cognitive empathy ) or the agent’s internal state can be directly coupled to that of their partner ( affective empathy ). In three simple multi-agent environments, we show that prosocial behavior arises only under homeostatic coupling – when the distress of a partner can affect one’s own well-being. Additionally, we show that empathy can be learned: agents can “decode” their partner’s external emotive states to infer the partner’s internal homeostatic states. Assuming some level of physiological similarity, agents reference their own emotion-generation functions to invert the mapping from outward display to internal state. Overall, we demonstrate the emergence of prosocial behavior when homeostatic agents learn to “read” the emotions of others and then to empathize, or feel as they feel. Submission type: Full Paper Code available at: https://anonymous.4open.science/r/empathy_homeostatic_rl-7E8F Introduction For humans and other social animals, it is often distressing to regard the suffering of others. We feel empathy, sharing in the feelings of others rapidly and automatically through emotional contagion ( Hsee et al., 1993 ) . Such feelings can provide a strong motivation to reduce the suffering of others, even if it comes at a cost to the self. It has been proposed that tying one’s own welfare to the welfare of others can form the basis of prosocial behavior ( De Waal, 2008 ) . As artificial systems gain in abilities and integrate further into human society, it becomes more urgent to understand the natural mechanisms that implement empathy and which lead to the emergence of altruistic behavior, and to model them in artificial systems. Figure 1: A : An experimental setup used in behavioral experiments to test the altruism of monkeys (illustration created based on ( De Waal, 1997 ) ). B : A minimal HRRL environment inspired by behavioral experiments for studying prosocial behavior (food-sharing). Emotions and feelings, whether self- or other-directed, are theorized to arise from homeostasis, the regulation of internal body states within a range compatible with life ( Damasio, 1999 ) . Self-regulatory mechanisms have previously been implemented as a source of external motivation ( Oudeyer and Kaplan, 2007 ; Baldassarre, 2011 ) . Here we regard homeostasis ( Cannon, 1939 ) as an intrinsic and obligatory motivation of all living creatures. Homeostatic-like processes have recently been implemented in reinforcement learning (RL) agents and have resulted in the emergence of integrated behaviors ( Yoshida, 2017 ; Yoshida et al., 2023 ; Yoshida et al., 2024a ; Yoshida et al., 2024b ) . Here we start from minimal scenarios of prosocial behavior, originally arising from the field of animal behavior. We model social behavior using multi-agent environments, where each agent is formulated as a homeostat ( Ashby, 1952 ; Seth, 2014 ; Man and Damasio, 2019 ) . We begin with the analysis of a multi-agent toy model inspired by behavioral experiments on food sharing in monkeys (Figure 1 ). We next propose some requirements for prosocial behavior. In several simulations, we compare the effects of different implementations of empathy ( Christov-Moore et al., 2023 ) , including cognitive empathy, in which an agent can observe the internal state of a partner, and affective empathy, in which an agent’s own internal states are coupled to their partner’s internal states. We contribute the following findings: 1) Even in a very simple system, prosocial behavior was not acquired when each agent aimed for homeostasis only within its own body. 2) Prosocial behavior also did not emerge when an agent directly observed the internal states of other agents (cognitive empathy). 3) Prosocial behavior was observed when the agent’s internal state was directly coupled to the internal states of other agents (affective empathy). 4) Prosocial behavior was acquired as agents learned to better decode the homeostatic states of partner agents from the partner’s outward expressions. Our results suggest that, for homeostats motivated by self-regulation, it is necessary to incorporate an additional homeostatic coupling mechanism for prosocial behavior to arise. Homeostatic Reinforcement Learning RL provides a framework to learn behavior in dynamic environments that maximizes the sum of future rewards emitted from the environment ( Sutton and Barto, 2018 ) . The objective of RL is to obtain a policy \pi:{\cal S}\rightarrow{\cal A} that maximizes the expected value of the weighted cumulative sum of future rewards \sum_{t=0}^{\infty}\gamma^{t}r_{t} for all states s\in{\cal S} , based on the experience of the interactions with an environment. Here, t\in{\mathbb{Z}} is the time step, r is the reward signal, {\cal S} is the set of states in the environment, {\cal A} is the set of actions of the agent. 0\leq\gamma<1 is a positive constant called the discount factor. Homeostatic RL ( Keramati and Gutkin, 2011 ; Keramati and Gutkin, 2014 ; Hulme et al., 2019 ) integrates principles from physiological homeostasis by defining reward as the internally perceived reduction of deviations from homeostatic set points ( Keramati et al., 2017 ; Juechems and Summerfield, 2019 ; Uchida et al., 2022 ; Duriez et al., 2023 ) . Concretely, the reward is defined as a quantity proportional to the sequential difference of the drive D , as \displaystyle r_{t+1}=\beta(D_{t}-D_{t+1}), (1) where \beta>0 is the scaling constant ( Keramati and Gutkin, 2011 ) . The drive function D(s^{i}) returns a value greater than or equal to zero, such as the distance between s^{i} and s^{*} . Here s^{i} is interoception that monitors the internal state of the agent’s body ( Sherrington, 1906 ; Craig, 2002 ) and s^{*} is the set point of the interoception. Homeostatic parameters are not arbitrarily defined, but are fundamental to the viability and functionality of the agent ( Keramati and Gutkin, 2014 ; Man and Damasio, 2019 ) . In our conception, homeostatic RL asserts that the agent has a vulnerable body. Vulnerability is defined as the circular causality by which homeostatic states can affect the agent’s ability to regulate those states (i.e., it gets harder to take care of oneself as one falls apart). Figure 2: Basic network architecture of the agent in this study. Observations include information about the external environment (Exteroception), the agent’s own internal conditions (Self-Interoception), and, in the cognitive empathy condition (Other-Interoception), the partner agent’s internal conditions. Agent Architectures in Experiments In all of our computer experiments, we used Proximal Policy Optimization (PPO, ( Schulman et al., 2016 ; Schulman et al., 2017 ) ) as the agent’s optimizer. The agent’s policy model consisted of an encoder of inputs using a multi-layer perceptron and a recurrent connection using LSTM ( Hochreiter and Schmidhuber, 1997 ) in all experiments (Figure 2 ), and the action selection probability, \pi , was calculated by applying the softmax function to the affine transformation from the hidden state. Value