Adaptively Coordinating with Novel Partners via Learned Latent Strategies
Benjamin Li, Shuyang Shi, Lucia Romero, Huao Li, Yaqi Xie, Woojun Kim, Stefanos Nikolaidis, Michael Lewis, Katia Sycara, Simon Stepputtis, [1ex] Carnegie Mellon University · 2025
Read Adaptively Coordinating with Novel Partners via Learned Latent Strategies by Benjamin Li, Shuyang Shi, Lucia Romero, Huao Li, Yaqi Xie, Woojun Kim, Stefanos Nikolaidis, Michael Lewis, Katia Sycara, Simon Stepputtis, [1ex] Carnegie Mellon University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12754.txt.
Introduction
Benjamin Li and nine co-authors build an agent that can team up with partners it has never met. In human-agent teams, people bring their own habits, and those habits can change mid-task, so a fixed partner model fails. The authors' framework first learns a latent space of strategies from agent trajectories using a variational autoencoder, then clusters that space into recognizable strategy types. A cooperator agent trains against partners of each type.
When a new partner arrives, a fixed-share regret algorithm updates the agent's guess of the partner's type during play. Tests used a modified Overcooked game, a two-player cooking task with time pressure and many possible strategies, plus an online study with human players. The agent beat existing baselines when paired with novel human and agent teammates. The work targets real teamwork settings where reading a partner quickly is the whole job.
Catalog record
| CREATOR | Benjamin Li, Shuyang Shi, Lucia Romero, Huao Li, Yaqi Xie, Woojun Kim, Stefanos Nikolaidis, Michael Lewis, Katia Sycara, Simon Stepputtis, [1ex] Carnegie Mellon University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-12754
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adaptively Coordinating with Novel Partners via Learned Latent Strategies Benjamin Li, Shuyang Shi, Lucia Romero, Huao Li, Yaqi Xie, Woojun Kim, Stefanos Nikolaidis, Michael Lewis, Katia Sycara, Simon Stepputtis, [1ex] Carnegie Mellon University Abstract Adaptation is the cornerstone of effective collaboration among heterogeneous team members. In human-agent teams, artificial agents need to adapt to their human partners in real time, as individuals often have unique preferences and policies that may change dynamically throughout interactions. This becomes particularly challenging in tasks with time pressure and complex strategic spaces, where identifying partner behaviors and selecting suitable responses is difficult. In this work, we introduce a strategy-conditioned cooperator framework that learns to represent, categorize, and adapt to a broad range of potential partner strategies in real-time. Our approach encodes strategies with a variational autoencoder to learn a latent strategy space from agent trajectory data, identifies distinct strategy types through clustering, and trains a cooperator agent conditioned on these clusters by generating partners of each strategy type. For online adaptation to novel partners, we leverage a fixed-share regret minimization algorithm that dynamically infers and adjusts the partner’s strategy estimation during interaction. We evaluate our method in a modified version of the Overcooked domain, a complex collaborative cooking environment that requires effective coordination among two players with a diverse potential strategy space. Through these experiments and an online user study, we demonstrate that our proposed agent achieves state of the art performance compared to existing baselines when paired with novel human, and agent teammates. 1 Introduction As an increasing number of AI agents and robots enter our daily lives, it is becoming increasingly important to explore methods of effective collaboration between agent and human. When an agent attempts to collaborate with an unknown partner (whether human or artificial), the challenge can typically be divided into two steps: (1) accurately predicting the partner’s behavior, and (2) choosing actions that advance the team toward its common goal. Our approach focuses on understanding and responding to behavioral patterns that distinguish different types of collaborators. Equipping an agent with the ability to identify partner type can enable it to exploit knowledge about that type when choosing its own actions, enabling adaptation for coherent and fluid collaboration. This type of adaptation is essential, as misaligned actions between teammates inevitably decrease overall performance. The problem of adapting to a previously unknown human is an instance of the more general problem of ad hoc teamwork, cooperating with any previously unknown teammate 38 . The difficulties of the problem arise if the agent begins without prior knowledge of its teammate’s behavioral patterns, yet must select its own actions to match the teammate’s intended strategy for completing the collective goal. This presents significant challenges given the multiplicity of possible strategies for any collaborative task. The difficulty intensifies in human-agent partnerships, where humans typically employ diverse, non-stationary, and often suboptimal approaches that can shift rapidly during interaction. Consequently, effective collaborative agents must be capable of interpreting their partners’ intentions and behaviors in real-time while demonstrating rapid adaptive responses. In coordinated team settings, roles and specializations can be learned by agents in a centralized manner [ 23 ; 43 ] . However, human-agent teams are often decentralized , in that roles and strategies are often implicit and need to be inferred [ 24 ] . To account for the diverse set of teammates an agent may encounter at test-time, an effective cooperator agent in this setting must be able to successfully learn and play the best response to each potential teammate’s strategy. A popular class of methods that seeks to address this is population-based training, in which a population of agent policies is generated, from which agents are drawn to serve as teammates to train a cooperator [ 18 ; 41 ] . At their core, these methods seek to construct agent populations with sufficient behavioral diversity to span the distribution of strategies exhibited by human players, often aided by a secondary objective during population training that enforces diversity in the set of training partners [ 4 ; 25 ; 34 ; 39 ; 51 ] . To further increase the strategic diversity of training partners, these agent populations can be used to train a generative model, which enables sampling from a continuous latent strategy distribution to simulate a much wider range of novel teammates [ 22 ] . Complementary to population-based approaches, agent modeling methods focus on online adaptation by inferring teammate behaviors and conditioning agent actions on these inferred strategies [ 29 ; 46 ; 50 ; 52 ] . However, these approaches often rely on a limited or hand-defined collection of strategy types to operate over, or lack the diverse synthetic training data provided by population-based methods. We argue that unifying these two paradigms can enable more effective team collaboration: leveraging diverse population data to learn a latent strategy space from which unique partners can be generated during training, while explicitly reasoning over teammate types at test-time to enable rapid online adaptation and best-response action selection. In this work, we introduce T eam A daptation via L at E nt N o-regre T S trategies ( TALENTS ) (see Fig. 1 ), a novel method for zero-shot coordination that models partner behaviors in a semantic strategy space. At the core of our TALENTS agent is a latent behavior embedding created by a variational autoencoder. This autoencoder is trained from offline trajectory data in the respective environment, collected from a population of baseline agents. With this dataset, we train a variational auto-encoder (VAE) with the goal of predicting each agent’s next action and utilize its latent space as behavior representation. Next, we determine an appropriate number and location of behavior clusters utilizing K-Means with silhouette analysis [ 32 ] . At test time, the type of a new partner, with respect to the behavior clusters, is determined by assessing each latent cluster mean’s decoded prediction accuracy with respect to the partner’s actual action (observed at the next timestep). Our agent’s belief of its partner is updated according to this per-step loss, and determines the most likely partner type which is then used as a conditioning term for its own action. Figure 1 : Overview of TALENTS : Provided an observation of a teammate, the VAE’s latent strategy clusters are queried to generate action predictions. At subsequent timesteps, the teammate’s actual actions are compared to these predictions, and the belief over the teammate’s latent strategy is progressively updated. We evaluate our agent in a modified Overcooked environment that extends prior work [ 2 ; 10 ; 45 ] with additional temporal pressure through order timers and bonus rewards for fast deliveries. This setting requires the coordinated use of three cooking stations to prepare two distinct recipes, significantly increasing the demands for effective teamwork and strategic coordination. We conduct experiments in both agent-agent and human-agent zero-shot coordination settings. In agent-agent evaluations, our method demonstrates overall superior performance compared to existing baselines. Furthermore, we deploy these same agents in human studies, demonstrating their ability to collaborate effectively with diverse unseen human partners. In summary, the work introduces the following contributions: • We introduce a novel method for learning strategy-specific best responses for human-agent collaboration using a generative