Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling
Eilam Shapira, Moshe Tennenholtz, Roi Reichart · 2026
Read Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling by Eilam Shapira, Moshe Tennenholtz, Roi Reichart (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12411.txt.
Introduction
Shapira, Tennenholtz and Reichart ask whether one AI agent can predict a stranger's next move in bargaining after only a few rounds. Each decision becomes a table row joining game state, offer history and dialogue, while a handful of the opponent's past games serve as labelled examples in the prompt. Their model rests on a tabular foundation model plus a frozen small language model that reads the decision-time state as an observer. Tests in controlled negotiation games show the approach can forecast unfamiliar counterparts from limited interaction.
Catalog record
| CREATOR | Eilam Shapira, Moshe Tennenholtz, Roi Reichart |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64182 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12411
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling Eilam Shapira, Moshe Tennenholtz, Roi Reichart Abstract AI agents increasingly negotiate and transact in natural language with unfamiliar counterparts: a buyer bot facing an unknown seller, or a procurement assistant negotiating with a supplier. In such interactions, the counterpart’s underlying LLM, prompts, control logic, and rule-based fallbacks are hidden, while each decision can have monetary consequences. We ask whether an agent can predict an unfamiliar counterpart’s next decision from only a few prior interactions. To avoid real-world logging confounds, we study this problem in controlled bargaining and negotiation games, formulating it as target-adaptive text-tabular prediction: each decision point is a table row combining structured game state, offer history, and dialogue, while K K previous games of the same target agent, i.e., the counterpart being modeled, are provided in the prompt as labeled adaptation examples. Our model is built on a tabular foundation model that represents rows using game-state features and LLM-based text representations, and adds LLM-as-Observer as an additional representation: a small frozen LLM reads the public decision-time state and dialogue; its answer is discarded, and its hidden state becomes a decision-oriented feature, making the LLM an encoder rather than a direct few-shot predictor. Training on 13 frontier-LLM agents and testing on 91 held-out scaffolded agents, the full model outperforms direct LLM-as-Predictor prompting and game+text features baselines. Within this tabular model, Observer features contribute beyond the other feature schemes: at K = 16 K=16 , they improve response-prediction AUC by about 4 points across both tasks and reduce bargaining offer-prediction error by 14%. These results show that formulating counterpart prediction as a target-adaptive text-tabular task enables effective adaptation, and that hidden LLM representations expose decision-relevant signals that direct prompting does not reliably surface. 1 1 1 Code and the 91 91 -agent dataset of 4,921 4{,}921 bargaining and negotiation games will be released upon acceptance. 1 Introduction AI agents increasingly negotiate and transact in natural language with unfamiliar counterparts: a buyer bot facing an unknown seller, or a procurement assistant negotiating with a supplier. In such interactions, the counterpart’s underlying LLM, prompts, control logic, and rule-based fallbacks are hidden, while each decision can have monetary consequences. We ask whether an agent can predict an unfamiliar counterpart’s next decision from only a few prior interactions. Real marketplace logs would be the most direct testbed, but they are rarely public and typically do not support systematic comparison across many agents under matched strategic conditions with known payoffs and ground-truth decisions. We therefore study the problem in controlled bargaining and negotiation games. These games preserve key elements of language-mediated commerce: multi-turn offers, accept/reject decisions, private valuations, monetary payoffs, and free-text dialogue. They also let us vary horizons, valuations, and information regimes while observing the decisions agents actually make. We call the unfamiliar counterpart being modeled the target agent . For each target, the predictor is given K complete prior games played by that same agent, which serve as labeled examples of the target’s behavior. At test time, the predictor receives a new decision point: the public game state, the offer history, and the dialogue so far. It must predict the target’s next move. We study two complementary tasks, illustrated in Figure 1 : response prediction , a binary classification task asking whether the target accepts the current offer, and proposal prediction , a regression task asking what offer the target will make next. Figure 1 : Alice (seller) and Bob (buyer) negotiate via free-text offers. Following Bob’s $5,000 round-4 offer, Alice’s next move is the prediction target. (a) Response prediction (classification): will she accept? (b) Proposal prediction (regression): if she rejects, what will she propose? We formulate this as target-adaptive text-tabular prediction . Each decision point is represented as a table row combining structured game variables, offer history, and dialogue-derived text features. A tabular foundation model conditions on labeled rows from a source population of previously observed agents together with the K labeled games of the current target agent. This allows the predictor to combine population-level regularities with target-specific evidence, adapting to a new counterpart without observing its prompt, code, or control logic. Our model uses three complementary feature blocks. The first contains game-state features, such as public configuration variables, round number, current offer, and previous offers. The second contains generic text representations of the dialogue. The third is our new decision-oriented representation, LLM-as-Observer : a small frozen LLM reads the public decision-time state and dialogue, its direct answer is discarded, and its hidden state is used as an additional feature for the tabular predictor. Thus, the LLM is used as an encoder rather than as the final few-shot predictor. This design contrasts with a natural alternative, LLM-as-Predictor : prompting a large frontier LLM with the current game and the target’s K prior games, and asking it to predict the next decision directly. Direct prompting can read the dialogue and reason over examples in context, but it must commit to an answer and cannot easily combine the target’s few games with a large labeled source population. In our formulation, the LLM contributes a reusable representation, while adaptation is performed by the tabular learner over source and target rows. For the source population, we use the 13-agent round-robin tournament released as part of GLEE [ 59 ] , where frontier LLMs 2 2 2 frontier LLMs: state-of-the-art API Large Language Models from six providers play under identical prompts, varying only in the underlying LLM. For the held-out target population, we introduce a 91-agent university-hackathon dataset: student-built agents that share one underlying LLM but differ in prompting, control logic, and rule-based fallbacks. This split tests whether predictors learned from one axis of agent variation transfer to newly encountered engineered agents whose heterogeneity comes from scaffolding. The full target-adaptive text-tabular model, trained on 13 frontier-LLM agents and tested on 91 held-out scaffolded agents, outperforms direct LLM-as-Predictor prompting and game+text features baselines. Within the tabular model, Observer features add complementary signal beyond structured game features and generic dialogue representations. At K=16 , they improve response-prediction AUC by about four percentage points across both game families and reduce bargaining offer-prediction error by 14\% . The gain is not mainly in the Observer’s committed answer: hidden states provide substantially more value than its direct output, suggesting that frozen LLM representations expose decision-relevant information that direct prompting does not reliably surface. Contributions. First, we formulate few-shot prediction of unfamiliar language-based agents as a target-adaptive text-tabular task, where K prior games of the target agent provide labeled adaptation examples. Second, we build a prediction model that combines game-state features, dialogue representations, and a new decision-oriented feature block, LLM-as-Observer. Third, we introduce a 91-agent hackathon dataset and a cross-population transfer evaluation from frontier-LLM agents to scaffolded agents, showing that the full model outperforms direct LLM-as-Predictor prompting and game+text features baselines, and that Observer hidden states add complementary decision-relevant signal. 2 Related work Multi-agent applications and the role of language. The applications motivating this paper sit in language-mediated commerce: consumer-to-consumer marketplaces [ 29 , 75 ] , residential real-estate transactions [ 30 ] , tourism and travel-package negotiations [ 52 ] , multi-stakeholder contract deliberations [ 1 ] , and the broader