Noncooperative Human AI Agent Dynamics
Dylan Waldner, Vyacheslav Kungurtsev, Mitchelle Ashimosi · 2026
Read Noncooperative Human AI Agent Dynamics by Dylan Waldner, Vyacheslav Kungurtsev, Mitchelle Ashimosi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16916.txt.
Introduction
Waldner, Kungurtsev, and Ashimosi simulate clashes between AI agents and human decision makers in classic games. Humans are modeled with Prospect Theory, which captures reference dependence and stronger reactions to losses than gains. AI agents instead maximize expected utility. The team runs many mixes of agent types across classic matrix games. Some games were designed to expose gaps between the two preference styles. Simulations cover AI players, informed humans, and learning prospect agents. The authors report outcomes from near identical behavior to clear prospect anomalies, plus some surprises.
Catalog record
| CREATOR | Dylan Waldner, Vyacheslav Kungurtsev, Mitchelle Ashimosi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73378 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16916
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Noncooperative Human AI Agent Dynamics Dylan Waldner, Vyacheslav Kungurtsev, Mitchelle Ashimosi Abstract This paper investigates the dynamics of noncooperative interactions between artificial intelligence agents and human decision-makers in strategic environments. In particular, motivated by extensive literature in behavioral Economics, human agents are more faithfully modeled with respect to the state of the art using Prospect Theoretic preferences, while AI agents are modeled with standard expected utility maximization. Prospect Theory incorporates known cognitive heuristics employed by humans, including reference dependence and greater loss aversion relative to utility to relative gains. This paper runs different combinations of expected utility and prospect theoretic agents in a number of classic matrix games as well as examples specialized to tease out distinctions in strategic behavior with respect to preference functions, to explore the emergent behaviors from mixed population (human vs. AI) competition. Extensive numerical simulations are performed across AI, aware humans (those with full knowledge of the game structure and payoffs), and learning Prospect Agents (i.e., for AIs representing humans). A number of interesting observations and patterns show up, spanning barely distinguishable behavior, behavior corroborating Prospect preference anomalies in the theoretical literature, and unexpected surprises. Code can be found at this link . 1 Introduction The proliferation of research and development in AI Agents has yielded an explosion of literature studying the nature and possible outcomes associated with their mutual influence. The potential for noncooperative interactions includes pursuit-evasion and tactics in war, bots automating economic activity and trading, and any circumstance wherein an automated AI system is representing some human actor or institution, with unaligned desiderata relative to other stakeholders. This yields the challenging problem of Multi Agent Noncooperative Reinforcement Learning (RL) and RL Nash Games [ 37 ] . However, a natural question that has not been considered with reality-faithful modeling in this domain is what if humans and AI agents are both involved in these interactions? As far as epistemically valid models of human agents, we consider the field of behavioral economics, which has summarily rebutted the Homo Economicus model of human decision making as being accurately modeled by maximizing a closed form utility function. While a number of alternatives have been proposed, in this work, we consider Prospect Theory (PT) [ 14 ] and its extensions, on account of the fact that it can be considered the leading model, and one whose properties are the most thoroughly axiomatized mathematically, and equilibrium properties understood. This paper focuses on empirically exploring the outcomes of noncooperative game play across permutations of a conventional reward-maximizing AI Learning Agent, a Prospect-maximizing Human with a priori complete knowledge of the game’s payoffs, and a Prospect-maximizing Learning Agent representing a Human in the game. With comprehensive experiments across a set of classical noncooperative games as well as some games in the PT literature notable for their peculiarities in the presence of Prospect maximization. The paper is the first paper to study the emergent asymptotic behaviors of mixed populations (PT vs. EU) in noncooperative 2\times 2 games, making the following contributions: 1. Models humans and AIs as PT and EU preference agents. 2. Discerns between learning and aware agents. 3. Formulation and study of Q learning for multiple reference point regimes. Section 2 introduces background material and preliminaries for the mathematical notation used throughout the paper. Section 3 introduces the games suite used for the paper, including the payoff matrices and existing equilibria sets that exist in the literature. Section 4 presents the experiment structure, including the three types of agents and the algorithm used to define them. Numerical results are presented in Section 5 , and we conclude with a discussion and future work in Section 6 . 2 Background and Preliminaries 2.1 Nash Reinforcement Learning Nash reinforcement learning (RL) is the field concerned with learning policies that converge towards equilibria. The general framework Nash RL builds on is a Markov game, also known as a stochastic game [ 32 , 24 ] , which is a multi-agent generalization of a Markov Decision Process (MDP), a formal framework for defining agent interactions with an environment. In this section, we will first introduce notation for an MDP, and then build the notation and intuition of a Markov game from the MDP foundation. Then, we will provide some background on Nash RL and what approaches have been conducted to this point. An MDP models an agent’s interactions with an environment. Time is tracked as discrete time steps, t=0,1,2,3,... . At each step the agent is presented an environment’s state, S_{t}\in\mathcal{S} , and in response selects an action, A_{t}\in\mathcal{A} . As a consequence of that action, a reward R_{t+1}\in\mathcal{R}\subset\mathbb{R} and the next state S_{t+1}\in\mathcal{S} are generated with the transition function [ 35 ] : (S_{t+1},R_{t+1})\sim P(\cdot\mid S_{t},A_{t}) A trajectory, or sequence, through an MDP can then be written as [ 35 ] : (S_{0},A_{0},R_{1},S_{1},A_{1},R_{2},\dots) . Agents learn policies, A_{t}\sim\pi(A_{t}\mid S_{t}) , that maximize their value functions, which quantify the expected return that comes from being in a state and following a policy. Formally, this is written with the Bellman equation [ 4 ] : V_{\pi}(S_{t})=\mathbb{E}_{\pi}[R_{t+1}+\gamma V_{\pi}(S_{t+1})] where \gamma\in[0,1) is a discount factor. Markov Games are defined by a set of states \mathcal{S} and a collection of action sets (\mathcal{A}_{1},\mathcal{A}_{2},\dots,\mathcal{A}_{k}) . Actions are denoted by A_{t,i}\in\mathcal{A}_{i} , and the joint action set can be defined as \mathbf{A}_{t}=({A_{t,1}},{A_{t,2}},\dots,{A_{t,k}}) , where k is the number of agents in the environment [ 20 ] . Each agent has their own reward function, R_{i}:\mathcal{S}\times\mathcal{A}_{1}\times\mathcal{A}_{2}\times\dots\times\mathcal{A}_{k}\rightarrow\mathbb{R} , with the joint rewards at time t defined as \mathbf{R}_{t}=(R_{t,1},R_{t,2},\dots,R_{t,k}) . State-reward transitions can then defined as: (S_{t+1},\mathbf{R}_{t+1})\sim P(\cdot\mid S_{t},\mathbf{A}_{t}). Agents then learn a policy, \pi_{i} that they follow to maximize their value function V_{\pi_{i}} . Formally, where \boldsymbol{\pi}=(\pi_{1},\pi_{2},\dots,\pi_{k}) V_{\pi_{i}}(S_{t})=\mathbb{E}_{\boldsymbol{\pi}}\left[R_{t,i}+\gamma V_{\pi_{i}}(S_{t+1})\right] where \gamma\in[0,1) is the discount factor. We use \mathbf{\pi} as the joint policy because each agent’s return depends on every other agent’s actions [ 32 ] . The Nash RL framework provides the theoretical foundation for analyzing strategic interactions among rational expected-utility maximizing agents, which we extend to incorporate human agents with Prospect Theoretic preferences in the next section. We use the RL Nash framework to study whether interactions between humans (modeled by Prospect Theory) and AIs (modeled by Expected Utility) converge to an equilibrium with the intention of forecasting potential competitive advantages each party may have over the other. The full formulation is discussed in Section 2.3 . 2.2 Prospect Theory Behavioral Economics, using experiments of human decision making under scarcity, has revealed that the maximization of closed form utility functions is an incorrect model of human decision making. While the field of cognitive heuristics and biases is rich in a variety of sub-rational phenomena and alternative models, the most well respected theoretical framework comes from Prospect Theory as developed by Kahneman and Tversky, see, e.g. [ 14 ] . Prospect Theory (PT) emerged from extensive experimental evidence demonstrating systematic violations of expected utility theory. The theory introduces several key psychological principles: decisions are made relative to reference points rather than final states; losses loom larger than equivalent gains (loss aversion); people exhibit diminishing sensitivity to changes further from reference points; and probabilities are transformed through weighting functions that overweight small probabilities and underweight large ones. Mathematically, the properties of