Peer-Aware Cost Estimation in Nonlinear General-Sum Dynamic Games for Mutual Learning and Intent Inference
Seyed Yousef Soltanian, Wenlong Zhang · 2025
Read Peer-Aware Cost Estimation in Nonlinear General-Sum Dynamic Games for Mutual Learning and Intent Inference by Seyed Yousef Soltanian, Wenlong Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17129.txt.
Introduction
Seyed Yousef Soltanian and Wenlong Zhang proposed in 2025 an algorithm for two agents in a dynamic game who must learn each other's goals while they act. In human-robot settings, neither side may know the other's objective, and older methods wrongly assume one agent is a fully informed expert, which biases learning and breaks coordination.
Their method, N-PACE, gives each agent a model of how its peer learns. Using repeated linear-quadratic approximations of the game, each agent infers the peer's objective and updates its own policy live. That yields faster, unbiased learning of the unknown goal, and the peer model can even carry intent signals. Tests beat baselines that ignore peer learning.
Catalog record
| CREATOR | Seyed Yousef Soltanian, Wenlong Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-17129
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Peer-Aware Cost Estimation in Nonlinear General-Sum Dynamic Games for Mutual Learning and Intent Inference Seyed Yousef Soltanian, Wenlong Zhang Abstract Human-robot interactions can be modeled as incomplete-information general-sum dynamic games since the objective functions of both agents are not explicitly known to each other. However, solving for equilibrium policies for such games presents a major challenge, especially if the games involve nonlinear underlying dynamics. To simplify the problem, existing work often assumes that one agent is an expert with complete information about its peer, which can lead to biased estimates and failures in coordination. To address this challenge, we propose a nonlinear peer-aware cost estimation (N-PACE) algorithm for general-sum dynamic games. In N-PACE, using iterative linear quadratic (LQ) approximation of the nonlinear general-sum game, each agent explicitly models the learning dynamics of its peer agent while inferring their objective functions, leading to unbiased fast learning in inferring the unknown objective function of the peer agent, which is critical for task completion and safety assurance. Additionally, we demonstrate how N-PACE enables intent communication in such multi-agent systems by explicitly modeling the peer’s learning dynamics. I INTRODUCTION General-sum dynamic games offer a powerful framework for modeling multi-agent interactions [ 1 , 2 ] , with applications in autonomous driving [ 3 ] , assistive robots [ 4 ] , and social navigation [ 5 ] . However, the process of finding the feedback Nash equilibrium (FBNE) solution of such games, which can be achieved by solving a set of Hamilton-Jacobi-Bellman (HJB) equations [ 6 ] , is inherently complex, as traditional dynamic programming approaches suffer from the curse of dimensionality [ 7 ] . This problem becomes even more challenging when agents do not know each other’s objective function and must perform inference online while making decisions, resulting in an incomplete-information differential game, a common scenario in human-robot interaction (HRI) [ 4 , 8 , 9 , 10 ] and autonomous vehicle interaction [ 11 , 12 , 13 ] . Fig. 1: Demonstration of the core idea in N-PACE using a simulated general-sum game between two autonomous cars at an intersection. In (a), each agent assumes its peer knows its aggressiveness level, leading to biased inferences and possible collision. In (b), N-PACE allows agents to learn their peer’s aggressiveness while accounting for mutual learning, ensuring safe intersection crossing. Existing approaches to solve such games often rely on the assumption that one agent is an expert which knows the true intent of other agents [ 14 , 15 , 1 ] . Despite some empirical success of these approaches, in scenarios where all agents are learning simultaneously and inferring each other’s objective functions (which is defined as their intent in this paper), these assumptions can lead to biased estimates, poor coordination, and failure to achieve each agent’s goal [ 16 ] . This is because while doing the inference, they fail to account for the fact that other agents can be non-stationary learners, whose policies are conditioned on previously observed actions [ 9 ] . It is even more important to consider such mutual learning in a competitive setting where an agent has no information of the others’ true intent. In this paper, we highlight the importance of modeling the learning dynamics of peer agents while inferring their objectives, addressing critical limitations in existing approaches. We achieve this goal by developing a peer-aware cost estimation algorithm for general-sum nonlinear games (N-PACE), where each agent leverages prior knowledge about its peer agent’s learning dynamics (e.g., gradient-based learner and Bayesian learner) to condition its current inference of the other’s objective on their parameter learning, thereby removing the bias in their inferences. The N-PACE algorithm employs iterative linear quadratic (iLQ) approximation of the nonlinear general-sum game [ 5 ] , both in policy update and intent inference phases. We also demonstrate how (iLQ) enables efficient approximation during the inference phase of our algorithm. Contributions. This paper makes the following contributions. (1) We introduce the N-PACE algorithm for mutual intent inference in multi-agent interaction where all agents are making inference online. (2) We empirically demonstrate that N-PACE is critical for task completion and safety in multi-agent systems where all agents are learners, compared to the incorrect assumption that a peer agent is an expert who knows the ego agent’s true intent. (3) We demonstrate how N-PACE, along with access to the learning dynamics of the peer agent, can be leveraged for intent communication and signaling to facilitate mutual learning and adaptation. The closest works to ours are [ 5 , 15 , 16 ] , where [ 5 ] uses iLQ for solving complete information multi-agent interactions. In [ 15 ] , the iLQ approach is employed in a general-sum game where an “expert” demonstrates its intent to a learner agent knowing their learning dynamics, but it does not consider the situation when two learning agents (no experts) try to coordinate which is a more practical setting in multi-agent interactions. Finally, while [ 16 ] studies the problem of mutual learning and adaptation, its problem setup is limited to a single-step optimization in the cost function. This work extends it to a finite-horizon optimization problem which is more common in multi-agent interactions. II Related Work Solving general-sum games. Multi-agent interactions have been explored using various frameworks, including multi-agent reinforcement learning [ 17 , 18 ] , adaptive control [ 19 ] , and game theory [ 20 , 21 ] . A large portion of these interactions can be modeled as general-sum dynamic games [ 22 , 23 ] . Traditional approaches for solving such games rely on adaptive dynamic programming to compute feedback Nash equilibria (FBNE) [ 24 ] . However, these methods are often hindered by the curse of dimensionality [ 7 ] , making them impractical for high-dimensional problems and real-time implementation in robotics. To overcome these limitations, physics informed neural networks (PINN) have been used to solve the (HJB) equation [ 23 ] more efficiently. Some recent work has focused on iterative linear-quadratic (iLQ) approximations, which offer a more computationally efficient strategy for for approximating the multi-agent interactions [ 5 ] solution by finding the local Nash equilibrium of such games. In this paper, we employ iLQR for both the learning and control phases to find the approximated Nash equilibrium policies. Incomplete information general-sum games. Solving incomplete information general-sum games is significantly more challenging than their complete-information counterparts [ 25 , 26 ] . These games often require simplifications, such as modeling one agent as a learner while assuming others are fully informed [ 14 , 27 ] , effectively simplifying them as optimal or adaptive control problems [ 4 ] . Other approaches discretize the intent parameter and rely on offline datasets for intent inference and belief updates [ 28 , 3 ] . Our work is motivated by the lack of studies on incomplete information dynamic games where all agents are learners [ 29 ] . We highlight the potential issues caused by these simplifications, such as assuming other agents are fully informed. Learning from a learner. Online inference of an expert agent’s objective function has been extensively studied through inverse optimal control and inverse reinforcement learning [ 30 ,