A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data
Leo Benac, Jonas Raedler, Zilin Ma, Finale Doshi-Velez · 2026
Read A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data by Leo Benac, Jonas Raedler, Zilin Ma, Finale Doshi-Velez (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14066.txt.
Introduction
Benac and co-authors build a benchmark for multi-party negotiation games drawn from real negotiation data. Real talks unfold as chains of binding, action-level pledges, not one final deal. Their setup pairs a configurable game generator with cases grounded in documents from a climate negotiation exercise. Several baseline solvers are included for comparison. No solver wins across all game types, since results hinge on each game's structure. The authors call for methods that value partial pledges more robustly.
Catalog record
| CREATOR | Leo Benac, Jonas Raedler, Zilin Ma, Finale Doshi-Velez |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66713 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14066
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data Leo Benac, Jonas Raedler, Zilin Ma, Finale Doshi-Velez Abstract Many real-world multi-party negotiations unfold as sequences of binding, action-level commitments rather than a single final outcome. We introduce a benchmark for this under-studied regime featuring a configurable game generator that sweeps key structural properties such as incentive alignment, goal complexity, and payoff distribution. To evaluate decision-making, we test three value-function approximations—myopic reward, an optimistic upper bound, and a pessimistic lower bound—that act as biased lenses on deal evaluation. Through exact evaluation on small games and comparative evaluation on large, document-grounded instances derived from the Harvard Negotiation Challenge, we map the strategic regimes where each approximation succeeds or fails. We observe that different game structures demand different valuation strategies, motivating agents that learn robust state values and plan effectively over long horizons under binding commitments and terminal only rewards. Code and data for the benchmark are available at: https://github.com/dtak/negotiation_benchmark_public . 1 Introduction Computational models of negotiation have long been studied in game theory and multi-agent reinforcement learning ( Rubinstein 1982 ; Baarslag et al. 2010 ; Abdelnabi et al. 2024 ; Mak et al. 2023 ; Meta Fundamental AI Research Diplomacy Team et al.(2022)Meta Fundamental AI Research Diplomacy Team (FAIR), Bakhtin, Brown, Dinan, Farina, Flaherty, Fried, Goff, Gray, Hu, Jacob, Komeili, Konath, Kwon, Lerer, Lewis, Miller, Mitts, Renduchintala, Roller, Rowe, Shi, Spisak, Wei, Wu, Zhang, and Zijlstra FAIR ) , with the promise of helping bounded decision-makers identify effective strategies in mixed-motive settings. Much of this work operationalizes negotiation as bargaining over a specific outcome, and offers are typically non-binding until acceptance ( Lewis et al. 2017 ; Baarslag et al. 2010 ) . However, many high-stakes, real-world multi-party negotiations cannot be reduced to a single agreement over a final outcome. In our collaborations with professional frontline negotiators, we find that agreements are reached through sequences of bilateral and small-group deals in which parties make binding commitments over concrete actions. Each deal constrains future options, reshaping future bargaining opportunities. Here, the strategy includes not just on the final agreement, but partner order, commitment sequencing, and how intermediate commitments interact. While existing MARL algorithms could be applied to these scenarios, the regime is under-benchmarked, and it remains unclear when current methods can provide reliable guidance (e.g., which parties to engage first). Drawing from our experience with real frontline negotiators, we introduce a benchmark for this underrepresented regime: multi-party, sequential, action-level negotiation games with binding commitments and terminal-only rewards. Specifically, our contributions are: Contribution 1: A controllable game generator. We provide a configurable generator that produces diverse instances of sequential action-level negotiation games with binding commitments. The generator supports controlled sweeps over structural properties that shape strategic difficulty, including: (i) scale (number of players, goals, and commitments); (ii) incentive alignment (more cooperative vs. more adversarial preference structure); (iii) payoff structure (balanced vs. positive-/negative-dominated utilities); (iv) goal complexity (how many commitments goals require); (v) goal non-linearity (the fraction of all-or-nothing vs. linear goals); and (vi) preference structure across goals (controlled via latent-factor dimensionality that sets how correlated utilities are across issues). Together, these controls make it possible to systematically construct and study qualitatively different negotiation landscapes – from incremental, locally-improvable regimes to settings with complementarity, delayed credit assignment, and traps. Contribution 2: A reference gameplay procedure and evaluation suite. We provide a simple reference decision procedure – turn-taking with partner selection, proposal construction, and acceptance – that instantiates one natural way to interact with our games. We use this fixed procedure to isolate instance difficulty and compare different value-function approximations under controlled conditions. We support exact evaluation on small instances via dynamic programming over the induced negotiation tree, and comparative evaluation on large instances where the exact solution is intractable. Importantly, the benchmark itself is protocol-agnostic: the same game instances can be paired with alternative protocols, information structures, or action spaces. Contribution 3: Exposing challenging regimes via “value-function lenses.” A central challenge in sequential action-level negotiation is estimating the downstream value of partial commitments. We study the benchmark through a small set of simple value-function approximations, each encoding a different assumption about how the remaining (uncommitted) part of the game may unfold. These approximations act as intentionally biased lenses: when instantiated within a simple reference gameplay procedure (used for controlled comparison), they induce distinct strategic behaviors (e.g., avoidance of negative exposure versus willingness to accept short-term sacrifices to unlock future gains). We use large-scale parameter sweeps to map where each lens succeeds or fails, showing that no single approximation dominates. The resulting regime-level characterization highlights which structural properties (e.g., payoff structure, non-linear goal ratio, and incentive alignment) drive negotiation difficulty, providing concrete targets for future algorithm design. Contribution 4: Realistic large instances from negotiation settings. We complement synthetic instances with large negotiation games derived from converting position papers from the Harvard Negotiation Challenge climate exercise into machine-readable game instances. This bridges controlled synthetic analysis with realistic, document-grounded scenarios, and allows us to test whether qualitative patterns discovered in regime sweeps transfer to large, structured negotiation problems. 2 Related Works Negotiations span a wide range of settings, some of which have well-established benchmarks and some that do not. We organize this space around five factors that shape strategic behavior, helping us surface both well-studied settings and under-benchmarked regimes: • Object of negotiation (outcome vs actions/commitments): Do agents negotiate directly over a final outcome (e.g., a complete allocation or agreement), or over individual actions/commitments that accumulate over time and jointly determine the eventual outcome? • Number of parties (bilateral vs multilateral): Bilateral negotiation vs. multilateral settings, where coalition dynamics and strategic externalities can arise. • Temporal structure (simultaneous vs sequential): Simultaneous choice vs. sequential offer/response dynamics (often with horizons or deadlines). • Information structure (perfect vs imperfect): Common knowledge utilities and state vs. imperfect information, often with private values or hidden constraints. • Commitment structure (non-binding vs. binding): Non-binding proposals until agreement vs. binding, persistent commitments that directly constrain future feasible decisions. Benchmarks for Outcome-level Bargaining Games. A large fraction of negotiation research evaluates agents in outcome-level bargaining settings: typically bilateral (though sometimes multi-party) interactions with private preferences , where agents exchange proposals – most commonly via sequential alternating offers – over a final allocation (e.g., a price or division of items), and proposals remain non-binding until acceptance . Rubinstein 1982 ’s alternating-offers model formalized this protocol and served as a reference point for many subsequent benchmark designs. This setting has motivated several widely used benchmarks. The Automated Negotiating Agents Competition (ANAC) established a standard suite for bilateral multi-issue negotiation under private preference profiles, which primarily uses this alternating-offers model ( Baarslag et al. 2010 ) . Outcome-level bargaining has also been instantiated in language-based benchmarks: Deal or No Deal operationalizes bargaining over item divisions with private values through dialogue ( Lewis et al. 2017 ) , and the “CraigslistBargain” benchmark enables training and evaluation for buyer-seller price negotiation with alternating offers ( He et al. 2018 ) . More recent work extends outcome-level bargaining to multilateral settings with LLM agents ( Abdelnabi et al. 2024 ) . While these benchmarks are well-suited for bargaining over a final allocation, they generally do not model binding intermediate