Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration

Benjamin Li 1 , Shuyang Shi 1 , Lucia Romero 2 , Huao Li 2 , Yaqi Xie 1 , Woojun Kim 1 , Stefanos Nikolaidis 3 , Michael Lewis 2 , Katia Sycara 1 , and Simon Stepputtis 1 4 · 2025

Read Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration by Benjamin Li 1 , Shuyang Shi 1 , Lucia Romero 2 , Huao Li 2 , Yaqi Xie 1 , Woojun Kim 1 , Stefanos Nikolaidis 3 , Michael Lewis 2 , Katia Sycara 1 , and Simon Stepputtis 1 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05244.txt.

Introduction

Benjamin Li and co-authors present TALENTS, a framework that teaches software agents to adapt to new human partners, in this 2025 paper. Teamwork demands reading a partner's strategy on the fly, which is hard when time is short and the strategy space is wide. Their testbed is a customized version of the cooking game Overcooked, where coordination decides success.

The system first learns a hidden strategy space from recorded play, using a variational autoencoder, then clusters that space into strategy types. A cooperator agent is trained against stand-ins for each type. When a new partner appears, a regret-based rule updates the guess about which strategy the partner is using, so the agent adjusts as the game unfolds. In an online study with real people, the agent outperformed current baselines when paired with unfamiliar human partners.

Catalog record

CREATORBenjamin Li 1 , Shuyang Shi 1 , Lucia Romero 2 , Huao Li 2 , Yaqi Xie 1 , Woojun Kim 1 , Stefanos Nikolaidis 3 , Michael Lewis 2 , Katia Sycara 1 , and Simon Stepputtis 1 4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-05244

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaboration Benjamin Li 1 , Shuyang Shi 1 , Lucia Romero 2 , Huao Li 2 , Yaqi Xie 1 , Woojun Kim 1 , Stefanos Nikolaidis 3 , Michael Lewis 2 , Katia Sycara 1 , and Simon Stepputtis 1 4 Abstract In collaborative tasks, being able to adapt to your teammates is a necessary requirement for success. When teammates are heterogeneous, such as in human-agent teams, agents need to be able to observe, recognize, and adapt to their human partners in real time. This becomes particularly challenging in tasks with time pressure and complex strategic spaces where the dynamics can change rapidly. In this work, we introduce TALENTS, a strategy-conditioned cooperator framework that learns to represent, categorize, and adapt to a range of partner strategies, enabling ad-hoc teamwork. Our approach utilizes a variational autoencoder to learn a latent strategy space from trajectory data. This latent space represents the underlying strategies that agents employ. Subsequently, the system identifies different types of strategy by clustering the data. Finally, a cooperator agent is trained to generate partners for each type of strategy, conditioned on these clusters. In order to adapt to previously unseen partners, we leverage a fixed-share regret minimization algorithm that infers and adjusts the estimated partner strategy dynamically. We assess our approach in a customized version of the Overcooked environment, posing a challenging cooperative cooking task that demands strong coordination across a wide range of possible strategies. Using an online user study, we show that our agent outperforms current baselines when working with unfamiliar human partners. I Introduction As AI agents and robots become increasingly integrated into daily life, the development of methods for effective human-agent collaboration is more critical than ever. In ad hoc teamwork settings, where agents must cooperate without prior knowledge of the partner, success depends on accurately predicting partner behavior and selecting actions that maximize joint performance [ 33 ] . To be effective, collaborative agents must interpret the behaviors of their partners in real time and respond quickly and effectively to account for the diverse, non-stationary, and often suboptimal strategies employed by human partners. To this end, we introduce T eam A daptation via L at E nt N o-regre T S trategies ( TALENTS ) (see Fig. 1 ), a novel method for zero-shot coordination that models partner behaviors in a semantic strategy space. Our agent utilizes a latent behavior representation learned using a variational autoencoder (VAE). The VAE is trained offline on trajectory data collected from a diverse population of baseline agents, trained with various reward-shaped targets, with the objective of predicting each agent’s next high-level action (e.g. “pick up a plate”). Subsequently, the latent space is utilized to capture meaningful behavioral patterns through K-Means clustering, using silhouette analysis [ 29 ] to determine the optimal number and placement of such clusters. The TALENTS cooperator training process leverages the VAE’s generative capability to produce partner agents by sampling from the clusters, while conditioning the cooperator on those clusters to develop strategy-specific responses. At test time, the agent infers the type of a new partner by comparing a sampled action from each behavior cluster with the partner’s actually observed action, utilizing the cluster with the best-matching action as likely partner type. TALENTS continuously updates its belief and conditions its own actions on the inferred partner type to enable more effective coordination, thereby behaving like an optimal response to the estimated partner. Fig. 1 : The TALENTS framework: Given an unknown teammate’s observation, strategy clusters are used to generate action predictions. These predictions are then used to infer teammate type and corresponding best responses using a tracking-regret minimization algorithm. To test our agent, we utilize a modified Overcooked setting with a set of human subjects in an ad-hoc teamwork task. In contrast to prior Overcooked environments, our setting introduces additional temporal pressure through timers on orders, as well as bonus rewards for fast deliveries. Moreover, it requires the utilization of three cooking stations for two distinct recipes, significantly increasing the need for effective teamwork. In summary, we present a novel approach for strategy-conditioned human-agent collaboration, enabling our agent to infer teammate types and adapt its behavior in real time, achieving state-of-the-art zero-shot performance in a challenging Overcooked evaluation with previously unseen human partners. Furthermore, we demonstrate that human partners prefer to interact with our agent based on the perceived fluency and trustworthiness of their teamwork, further underscoring the effectiveness of our adaptive agent. II Related Work Strategy Inference The ability to infer a teammate’s strategy during collaboration enables agents to coordinate effectively. Methods for strategy inference include learning latent embeddings from trajectory datasets [ 24 , 38 , 37 , 14 ] , estimating policy updates [ 6 ] , and Bayesian inference [ 2 , 7 , 40 ] . Other approaches use triplet losses for distinguishability [ 9 ] , action prediction [ 26 , 28 ] , or maintaining strategy libraries [ 19 , 11 ] . Theory of Mind modeling is also often used as a technique for predicting teammate intentions and beliefs [ 20 , 27 , 25 , 18 ] . TALENTS learns latent representations of teammate strategies using a variational autoencoder, unsupervised clustering to identify similar characteristics, then performs partner inference via tracking-regret minimization at test-time settings to adapt to teammates. Zero-Shot Coordination The central aim of Zero-Shot Coordination (ZSC) is to develop agents that are able to collaborate with partners that were unseen to the agent during train time [ 15 ] . Two common methods to achieve this are Self-play (SP) [ 30 , 12 ] and Population-Based Training (PBT) [ 16 ] . SP methods have shown remarkable performance in zero-sum environments [ 32 , 35 ] , but are unable to achieve comparable performance in common-payoff cooperative settings since self-play agents tend to develop rigid behavior patterns [ 1 ] . PBT encourages less rigid convention formation by optimizing over diverse partner sets [ 22 , 34 , 3 , 39 ] , enabling robustness to a greater variety of partner strategies [ 36 ] . GAMMA [ 21 ] learns generative models over policy populations, encoding trajectories into latent representations. TALENTS similarly trains generative models but utilizes a sequential architecture to learn long-term strategy representations and performs controlled sampling over these strategy clusters. III Team Adaptation via No-Regret Strategies In the following, we introduce TALENTS, a method for training a cooperator agent on a diverse set of strategies and behaviors, enabling it to adapt to unseen teammates. Given offline trajectories exhibiting various strategies (see Sec. III-A ), we use a variational autoencoder (VAE) [ 17 ] to learn a latent space of strategy features, then use unsupervised clustering to segment the space into discrete clusters representing unique high-level strategies. Each cluster corresponds to a hidden parameter in a HiP-MDP [ 5 ] formulation, which we can optimize over by training a best response agent (see Sec. III-B ). This is done by generating strategic partners through sampling cluster means and conditioning the cooperator on these means for strategy-dependent responses. At test time, we use the fixed share algorithm to select appropriate latent means, enabling

Read the complete text