Position: Science of AI Evaluation Requires Item-level Benchmark Data
Han Jiang, Susu Zhang, Xiaoyuan Yi, Xing Xie, Ziang Xiao* · 2026
Read Position: Science of AI Evaluation Requires Item-level Benchmark Data by Han Jiang, Susu Zhang, Xiaoyuan Yi, Xing Xie, Ziang Xiao* (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03244.txt.
Introduction
Jiang and colleagues argue that AI testing needs item-level data as standard practice, not just overall scores. Without item text, test setup, answers, and score stats, validity claims cannot be checked, and inflated claims about skill go uncaught. They build OpenEval, a store of more than 10 million answers across 160,000 items, and show item data can expose weak items and skewed constructs.
Catalog record
| CREATOR | Han Jiang, Susu Zhang, Xiaoyuan Yi, Xing Xie, Ziang Xiao* |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38171 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.03244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Position: Science of AI Evaluation Requires Item-level Benchmark Data Han Jiang, Susu Zhang, Xiaoyuan Yi, Xing Xie, Ziang Xiao* Abstract AI evaluations have become the primary evidence for deploying generative AI systems across high-stakes domains. However, current evaluation paradigms often exhibit systemic validity failures. These issues, ranging from unjustified design choices to misaligned metrics, remain intractable without a principled framework for gathering validity evidence and conducting granular diagnostic analysis. In this position paper, we argue that item-level AI benchmark data is essential for establishing a rigorous science of AI evaluation. Item-level analysis enables fine-grained diagnostics and principled validation of benchmarks. We substantiate this position by dissecting current validity failures and revisiting evaluation paradigms across computer science and psychometrics. Through illustrative analyses of item properties and latent constructs, we demonstrate the unique insights afforded by item-level data. To catalyze community-wide adoption, we introduce OpenEval , a growing repository of item-level benchmark data designed supporting evidence-centered AI evaluation. 1 Introduction Generative AI has drastically expanded automation, reshaping the social and economic fabric ( 83 ) . As these models move into high-stakes deployments, the risk of unpredictable behavior necessitates rigorous oversight and regulation. Consequently, AI evaluation, which is currently dominated by benchmarking practices ( 18 ) , is essential for understanding model capabilities, informing AI policy, and guiding responsible deployment across domains. Benchmarks have a long history as standardized tests for comparing systems, such as computing power (e.g., 16 ) and component-level efficiency (e.g., 67 ). Contemporary AI benchmarks primarily rely on curated datasets designed to operationalize specific capabilities, including reasoning (e.g., 68 ), tool use (e.g., 30 ), social skills (e.g., 11 ), and domain knowledge (e.g., 55 ). Characterized by capability-to-task breakdowns, task-specific metrics, and score aggregation ( 46 ) , this paradigm provides a standardized, empirical basis for comparing AI system performance. However, providing a defensible picture of model capabilities is increasingly difficult as the validity of AI benchmarks called into question ( 57 ) . A central limitation is that critical design choices — including capability definitions, content curation, and metric selection — often lack transparency or formal justification ( 46 ) . This opacity undermines the validity evidence ( 7 ; 46 ) needed to support the interpretations of results, making it unclear whether benchmarks genuinely measure their intended constructs, despite explicit metadata or task descriptions ( 1 ) . Furthermore, when the motivations for specific design paths are not clearly articulated, benchmarks become prone to redundancy ( 56 ; 53 ) . Compounding these internal design limitations is the rapid evolution and opacity of modern AI systems, which continue to impose new pressures on benchmarking practices. The speed of AI development manifests as benchmark saturation ( 53 ) , rapidly outdated content ( 33 ) , and widespread data contamination ( 76 ) , rendering the resulting scores uninformative or misleading for deployment decisions ( 24 ) . Further, the proliferation of model capabilities and task settings has created a widening socio-technical gap between technical solutions and real-world requirements ( 69 ; 42 ) . This lack of clear rationale also leads to redundancy, as new benchmarks often duplicate existing efforts without adding unique evaluative insights ( 6 ) . Crucially, many validity issues are not diagnosable from benchmark analyses with the model scores at the benchmark-level. Benchmark-level scores do not provide a sufficient evidentiary basis for evaluating item quality, construct coverage, or potential confounders. Foundational questions—including whether items effectively differentiate model capabilities, how construct-irrelevant nuisance factors drive performance, or whether gains reflect genuine reasoning rather than artifacts—are inherently item-level inquiries. Without item-level response data, our field lacks the empirical evidence required to evaluate and curate effective benchmarks. We therefore argue that broader access to and analysis of item-level AI benchmark data are essential for establishing a more scientifically grounded, evidence-centered approach to AI evaluation. Item-level assessment data have long been significant to test development and validation across education, psychology, and other social sciences disciplines concerned with measuring human abilities. Adapting these established practices to AI evaluation, through a community-wide push for broader data access, could transform fragmented results into cumulative empirical evidence. Leveraging the rich information from item content, score statistics, and per-item model responses, item-level analyses complement existing paradigms to enable more rigorous benchmark validation, granular diagnostics of benchmark characteristics, and more informed benchmark design. In this position paper, we analyze validity failures in AI benchmarking and link them directly to the current neglect of item-level data in Sec. 2 . By contrasting paradigms in computer science and psychometrics, we motivate an item-level shift (Sec. 3 ) and formalize our position (Sec. 4 ). In Sec. 5 , we present illustrative item-level analyses on selected AI benchmark datasets to highlight the unique insights afforded by item-level data. Finally, we introduce a growing, large-scale item-level benchmark data repository, OpenEval , in Sec. 6 to encourage further community effort and action, and discuss broader opportunities beyond benchmarking enabled by item-level benchmark data in Sec. 7 . 2 Validity Issues in AI Benchmarking Despite their pivotal role in AI evaluation, current AI benchmarks face methodological and practical validity challenges that stem from a neglect of item-level data. 2.1 Validity Problems in Benchmark Design The term validity , specifically construct validity , was formalized by 13 in modern psychometrics and is directly applicable to AI evaluation ( 75 ) . A construct refers to a theoretical, unobservable attribute or trait (e.g., a perceived AI capability) that a test is intended to measure. Validity reflects how well an assessment measures the intended underlying outcome ( 63 ) and is critical for ensuring benchmark quality. As noted by 72 , construct validity is a central concept in psychology, particularly for informing the design of psychological measures. In AI evaluation, however, it has received limited attention, despite benchmark creation involving numerous design decisions, including target user selection, construct-to-task operationalization, and scoring metrics. These decisions are often oversimplified due to a dominant focus on benchmark-level results , leaving only standard or default settings (e.g., 41 ; 70 ) and insufficient evidence for validity justification ( 46 ) . These hidden ambiguities have resulted in a lack of a common language for communicating construct validity within the AI community. Although there are several studies exploring validity-relevant properties (e.g., 79 ; 74 ), research explicitly assessing the construct validity of AI benchmarks remains limited. Existing analyses suggest that many benchmarks’ construct validity has yet to improve and that detailed validity evidence should be extracted not only from benchmark-level information (e.g., 7 ; 4 ; 61 ), highlighting the need to examine item-level data. 2.2 Validity Degradation over the Benchmark Lifecycle Once released, a benchmark inevitably experiences validity degradation over its lifecycle. Many popular static benchmarks are approaching their sunset stage, where results and conclusion derived from them are no longer informative or reliable ( 14 ; 34 ) . First, as AI systems and real-world knowledge evolve, some benchmarks have gradually saturated, with most items becoming too easy to distinguish between current models ( 53 ; 15 ) . For instance, RealToxicityPrompts ( 22 ) is one of the most widely used AI toxicity benchmarks. Although by 2023 it could no longer differentiate between