Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation
D. Sculley, Will Cukierski, Phil Culliton, Sohier Dane, Maggie Demkin, Ryan Holbrook, Addison Howard, Paul Mooney, Walter Reade, Megan Risdal, Nate Keating · 2025
Read Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation by D. Sculley, Will Cukierski, Phil Culliton, Sohier Dane, Maggie Demkin, Ryan Holbrook, Addison Howard, Paul Mooney, Walter Reade, Megan Risdal, Nate Keating (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00612.txt.
Introduction
Authors writing as D. Sculley and co-authors argue a position in a 2025 paper: artificial intelligence competitions should be the gold standard for judging generative models. Standard benchmarks falter because these models take almost any input, produce almost any output, and lack one right answer to check against.
The deepest problem, they argue, is leakage: test material seeping into training data quietly inflates scores. Competition organizers have spent years building defenses against cheating and leakage, which makes their results unusually trustworthy. The authors urge the field to harvest competition outcomes as rigorous evidence rather than treating them as side events.
Catalog record
| CREATOR | D. Sculley, Will Cukierski, Phil Culliton, Sohier Dane, Maggie Demkin, Ryan Holbrook, Addison Howard, Paul Mooney, Walter Reade, Megan Risdal, Nate Keating |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00612
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation D. Sculley, Will Cukierski, Phil Culliton, Sohier Dane, Maggie Demkin, Ryan Holbrook, Addison Howard, Paul Mooney, Walter Reade, Megan Risdal, Nate Keating Abstract In this position paper, we observe that empirical evaluation in Generative AI is at a crisis point since traditional ML evaluation and benchmarking strategies are insufficient to meet the needs of evaluating modern GenAI models and systems. There are many reasons for this, including the fact that these models typically have nearly unbounded input and output spaces, typically do not have a well defined ground truth target, and typically exhibit strong feedback loops and prediction dependence based on context of previous model outputs. On top of these critical issues, we argue that the problems of leakage and contamination are in fact the most important and difficult issues to address for GenAI evaluations. Interestingly, the field of AI Competitions has developed effective measures and practices to combat leakage for the purpose of counteracting cheating by bad actors within a competition setting. This makes AI Competitions an especially valuable (but under-utilized) resource. It is now time for the field to view AI Competitions as the gold standard for empirical rigor in GenAI evaluation, and to harness and harvest their results with according value. 1 Introduction As Generative AI (GenAI) models such as Large Language Models (LLMs) become ever more important to the field and to the world, it has become clear that performing empirical evaluation of these models and methods is extremely difficult to do in a rigorous and comprehensive way. This difficulty is of course not due to lack of effort or expertise by researchers. Indeed, enormous effort and resources have been poured into creating myriad benchmarks and test cases ( Chiang et al. 2024b ; Fourrier et al. 2024 ; Hendrycks et al. 2021 ; Cobbe et al. 2021 ; Zellers et al. 2019 ; Chen et al. 2021b ) . However, even accounting for these many important efforts and achievements, our position is that the current state of evaluation is insufficient to meet the needs of this moment in GenAI for the field and for the world. In our view, the root cause of this insufficiency is that the evaluation needs of GenAI models fundamentally break the paradigm of traditional benchmarking that served the field of machine learning (ML) so well during decades of remarkable progress. This breakage goes beyond the familiar difficulty of defining what, exactly, is in the training data for an LLM. In our view, we need a broader conception of generalization for GenAI that moves beyond the idea of generalizing to new independently drawn examples from a stationary distribution, and instead refers to performing well on tasks that are entirely novel from a model’s perspective. This higher bar is rooted in commonsense standards for human intelligence ( Chollet 2019 ; Dennett 1991 ) , but has far reaching consequences, most notably that it implies the problems of data leakage and contamination in evaluation are the most pressing concerns. Together, these factors imply that rigorous and robust evaluation of GenAI models requires a steady source of novel tasks structured to avoid leakage, contamination, and other forms of inadvertent “cheating”. Fortunately, AI Competitions—such as those hosted on platforms like Kaggle and others—act as a solution to GenAI evaluation challenges by providing a continual source of new tasks for evaluation and significant structures to avoid leakage and related issues. We define an AI Competition as a problem or task with an objective evaluation function for ranking solutions or models in which multiple parallel attempts are made during a time-bound period by independent teams. 1.1 Summarizing Our Position Our position can be summarized by the following points: • Traditional paradigms for ML evaluation are ill-equipped to meet the demands of GenAI Evaluation. • Leakage should be viewed by the field as the most important pitfall to avoid in evaluations. • GenAI evaluations should be considered leaked the moment test data has been shared online or sent over the wire to models. • If we have to choose between reproducibility and robustness in GenAI evaluations, we should choose to prioritize robustness. • We should replace the notion of reproducible static benchmarks with repeatable processes and procedures. • The field should use established AI Competitions platforms as a renewable stream of novel evaluation tasks. • The standards and practices developed that help AI Competitions guard against cheating should be viewed by the field as the gold standard for empirical rigor in evaluation. • Meta-analyses across evaluations should be valued as highly in the field of AI as they are in fields such as medicine. 1.2 Structure of This Paper In the remainder of this paper, we will first review the most typical structure and assumptions in traditional ML evaluation and discuss why they are insufficient for GenAI evaluations. We will examine the nature of generalization for GenAI, how this leads to specific concerns around leakage, and additionally show how goals of reproducibility and robustness in evaluation may be fundamentally at odds. We will then show how difficult the problem of leakage is even for traditional ML evaluations with some brief case studies, and look at current GenAI benchmarks that are aiming to overcome leakage and contamination. We finish with an examination of the ways that AI Competitions address these issues, discuss our recommendations, and examine alternate viewpoints. Our goal is to provide convincing support of the view that AI Competitions do indeed provide a gold standard for empirical rigor in evaluating GenAI models, and that the field should place accordingly high value and attention on their results. Figure 1: IID Evaluations vs. Novelty-Centric Evaluations. In the IID evaluation, left, both training and test data are drawn from the same distribution, resulting in significant overlap in examples in each set. In the novelty-centric version, right, no test example is allowed to be too similar to any given training example. We argue that the latter conceptualization more closely mirrors desired behavior for GenAI evaluations, where generalization is expected to connote the ability to respond well on totally novel inputs. 2 Background: Revisiting Benchmarking Traditional ML benchmarking has been founded on the idea of a test-train split , in which an evaluation is structured by training a model from scratch on a given portion of training data and then evaluating that trained model on a holdout set of test data ( Mitchell 1997 ) . This conceptual structure is so fundamental to modern ML practice that it may sometimes be taken for granted. So let us take a moment to examine this basic structure and its implications. In classical supervised ML, the most common traditional setup is to evaluate a model f({\bf x})\rightarrow y , with {\bf x}\in\Re^{d} as feature vectors in some d dimensional feature space and y\in Y as a space of possible labels, such as \{0,1\} for binary classification or (0,1) for regression on probabilities. Labeled examples ({\bf x},y) are assumed to have come from some distribution D . The training set D_{train} and test set D_{test} are each independently and identically drawn (IID) from D , with only the examples in the training set