ZeroSumEval : Scaling LLM Evaluation with Inter-Model Competition

Haidar Khan, Hisham A. Alyahya, Yazeed Alnumay, M Saiful Bari, Bülent Yener · 2025

Read ZeroSumEval : Scaling LLM Evaluation with Inter-Model Competition by Haidar Khan, Hisham A. Alyahya, Yazeed Alnumay, M Saiful Bari, Bülent Yener (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12562.txt.

Introduction

Haidar Khan and co-authors judge language models by making them compete in games in a paper from two thousand twenty-five. Static test sets grow stale, leak into training data, and cost much to grade. A contest format keeps the test fresh because rivals set the challenge.

Their suite, ZeroSumEval, spans seven games: a coding jailbreak test, chess, dice and poker, a math quiz, and persuasion duels. Over seven thousand matches across thirteen models showed a split. Top models from two leading families handled common games and questions well. They faltered when asked to invent hard new questions or to jailbreak each other. The authors release the code in public.

Catalog record

CREATORHaidar Khan, Hisham A. Alyahya, Yazeed Alnumay, M Saiful Bari, Bülent Yener
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12562

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ZeroSumEval : Scaling LLM Evaluation with Inter-Model Competition Haidar Khan, Hisham A. Alyahya, Yazeed Alnumay, M Saiful Bari, Bülent Yener Abstract Evaluating the capabilities of Large Language Models (LLMs) has traditionally relied on static benchmark datasets, human assessments, or model-based evaluations - methods that often suffer from overfitting, high costs, and biases. ZeroSumEval is a novel competition-based evaluation protocol that leverages zero-sum games to assess LLMs with dynamic benchmarks that resist saturation. ZeroSumEval encompasses a diverse suite of games, including security challenges (PyJail), classic games (Chess, Liar’s Dice, Poker), knowledge tests (MathQuiz), and persuasion challenges (Gandalf, Debate). These games are designed to evaluate a range of AI capabilities such as strategic reasoning, planning, knowledge application, and creativity. Building upon recent studies that highlight the effectiveness of game-based evaluations for LLMs, ZeroSumEval enhances these approaches by providing a standardized and extensible framework. To demonstrate this, we conduct extensive experiments with >7000 simulations across 7 games and 13 models. Our results show that while frontier models from the GPT and Claude families can play common games and answer questions, they struggle to play games that require creating novel and challenging questions. We also observe that models cannot reliably jailbreak each other and fail generally at tasks requiring creativity. We release our code at https://github.com/facebookresearch/ZeroSumEval . 1 Introduction Large Language Models (LLMs) are being developed at an unprecedented pace ( Zhao et al., 2024 ) , requiring significant investment for their training and refinement ( Kevin Lee, 2024 ; Miller, 2022 ; Kimball, 2024 ) . As the performance and complexity of these models continue to grow ( Chen et al., 2024b ) , selecting the most appropriate model for a specific application has become an increasingly challenging and costly decision ( Kaplan et al., 2020 ; Hoffmann et al., 2022 ) . Benchmarking emerges as critical in this context ( Laskar et al., 2023 ; Qin et al., 2023 ) , providing standardized metrics and evaluations to guide these choices. Current benchmarking practices face several significant issues. Many benchmarks suffer from data contamination ( Yang et al., 2023 ) , where models inadvertently train on portions of the test data ( Dubey et al., 2024 ; Groeneveld et al., 2024 ) , leading to inflated performance metrics. Sensitivity to prompt variations ( Alzahrani et al., 2024 ) and a lack of diversity in evaluation tasks ( Laskar et al., 2024 ) further undermine the reliability and robustness of these benchmarks. Additionally, the high cost and effort required to develop new benchmarks often result in outdated evaluation methods that do not keep pace with the rapid development of LLMs ( Kiela et al., 2021 ; Vu et al., 2023 ) . Figure 1 : Cumulative ratings of 13 models on ZeroSumEval . The top performing models ( gpt-4o and claude-3.7-sonnet ) show mostly on-par performance across ZeroSumEval games. Thinking model quality varies between model families (e.g. claude-3.7-sonnet-thinking vs deepseek-r1 ). Surprisingly, o3-mini-high performs worst amongst this cohort of models. An observed disparity exists between the computational resources measured in floating-point operations per second, or FLOPs used to train LLMs and those allocated for their evaluation. Training these models involves massive computational efforts ( Hoffmann et al., 2022 ) , yet the evaluation phase typically utilizes a negligible fraction of this capacity ( Laskar et al., 2024 ) . Scaling up evaluation by increasing the number of evaluation tokens is essential for a more thorough understanding of model capabilities. Traditionally, this scaling involves incorporating human-crafted data ( Holland et al., 2018 ) , which is resource-intensive ( Hutchinson et al., 2021 ) and may not adequately capture the complexities of language ( Mehrabi et al., 2021 ) and reasoning required to challenge advanced LLMs ( Gudibande et al., 2023 ) . Previous work has proposed the use of games as benchmarks ( Topsakal et al., 2024 ) , offering a promising avenue for evaluating complex reasoning ( Wong et al., 2023 ) and decision-making abilities of LLMs ( Warstadt et al., 2023 ; Park et al., 2023 ; Wang et al., 2023 ) . Games provide interactive and dynamic environments that can test models beyond static datasets. However, existing game-based benchmarks are often (i) inflexible and limited in scope, (ii) not easily extendable, (iii) restricted in their effectiveness for comprehensive model evaluation, and (iv) depend on predefined prompts. Scaling evaluation is fundamental not only for assessing performance but also for uncovering hidden dynamics within LLMs, such as potential backdoors or biases ( Schuster et al., 2020 ) , and for evaluating their emerging reasoning capabilities ( Brown et al., 2020 ; Sanh et al., 2022 ; Wei et al., 2023b ; Wei et al., 2023a ) . Implementing environments for simulations or games offers a scalable solution to these challenges ( OpenAI et al., 2019 ; OpenAI, 2019 ; Silver et al., 2016 ; Silver et al., 2017 ; Zheng et al., 2021 ) . Existing evaluation protocols possess several key issues: • Limited Diversity : Traditional evaluation methods rely on static datasets, which are inherently limited by their dependency on human curation and annotation. This makes it challenging to continuously introduce new, diverse test data. • Crowd and Annotator Bias : LLM evaluations conducted by large crowds often tend to be susceptible to social hacking, and it can depend on geographic, temporal, and narrative factors ( Gururangan et al., 2018 ) . Controlled and interpretable environments can mitigate these biases by providing consistent, objective evaluation criteria. • Sensitivity to Prompts : Previous work ( Zheng et al., 2024 ; Pezeshkpour & Hruschka, 2023 ; Lu et al., 2022 ; Alzahrani et al., 2024 ; Wang et al., 2024a ) has shown that models are sensitive to benchmark formats. These prompt modifications are shown to result in substantially different relative performance between models ( Alzahrani et al., 2024 ) . • Saturation : With the rapid improvement of LLMs, evaluation benchmarks quickly become obsolete and saturated, with frontier models achieving almost perfect scores, which necessitates the development of new benchmarks. For instance, GSM8K ( Cobbe et al., 2021 ) tests models on grade school-level math, and most state-of-the-art models achieve scores above 90% ( Dubey et al., 2024 ; Anthropic, 2024 ) . Thus, the more difficult MATH ( Hendrycks et al., 2021b ) dataset, which consists of math competition questions, was developed and is now commonly used 1 1 1 HuggingFace’s Open LLM Leaderboard ( Beeching et al., 2023 ; Fourrier et al., 2024 ) migrated from GSM8K in v1 to MATH in v2. . A similar trend is observed in academic examination benchmarks with the migration from MMLU ( Hendrycks et al., 2020 ) to MMLU-Pro ( Wang et al., 2024b ) and GPQA. ( Rein et al., 2023 ) 2 2 2 Similar to 1 , the leaderboard transitioned from MMLU in v1 to MMLU-Pro and GPQA in v2. . To address these challenges, we introduce ZeroSumEval , an extensible open-source framework designed to evaluate LLMs through the simulation of multi-player zero-sum games. The evaluations in the framework naturally scale and resist saturation as model capabilities improve. In this work, we show that ZeroSumEval possesses the following properties: • Scalable Evaluation

Read the complete text