How Much Do Large Language Model Cheat on Evaluation? Benchmarking Overestimation under the One-Time-Pad-Based Framework

Zi Liang, Liantong Yu, Shiyu Zhang, Qingqing Ye, Haibo Hu · 2025

Read How Much Do Large Language Model Cheat on Evaluation? Benchmarking Overestimation under the One-Time-Pad-Based Framework by Zi Liang, Liantong Yu, Shiyu Zhang, Qingqing Ye, Haibo Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19219.txt.

Introduction

Zi Liang and co-authors measure how much chat model scores are puffed up by leaked test sets, in a study from 2025. When test items slip into training data, a model can look keen for the wrong reason. Past fixes hide tests or use human judges, but they lose speed, trust, or open checks.

Their frame, ArxivRoll, borrows the one-time pad from code work: each test is used once, then thrown away. A tool builds fresh private tests from new arXiv papers each half year. A score then shows how much of the public score may be taint or bias. Wide tests back the quality of the fresh sets. The team shares code in public so other labs can run the same check.

Catalog record

CREATORZi Liang, Liantong Yu, Shiyu Zhang, Qingqing Ye, Haibo Hu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-19219

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How Much Do Large Language Model Cheat on Evaluation? Benchmarking Overestimation under the One-Time-Pad-Based Framework Zi Liang, Liantong Yu, Shiyu Zhang, Qingqing Ye, Haibo Hu Abstract Overestimation in evaluating large language models (LLMs) has become an increasing concern. Due to the contamination of public benchmarks or imbalanced model training, LLMs may achieve unreal evaluation results on public benchmarks, either intentionally or unintentionally, which leads to unfair comparisons among LLMs and undermines their realistic capability assessments. Existing benchmarks attempt to address these issues by keeping test cases permanently secret, mitigating contamination through human evaluation, or repeatedly collecting and constructing new samples. However, these approaches fail to ensure reproducibility, transparency, and high efficiency simultaneously. Moreover, the extent of overestimation in current LLMs remains unquantified. To address these issues, we propose ArxivRoll, a dynamic evaluation framework inspired by one-time pad encryption in cryptography. ArxivRoll comprises two key components: i) SCP (Sequencing, Cloze, and Prediction) , an automated generator for private test cases, and ii) Rugged Scores (RS) , metrics that measure the proportion of public benchmark contamination and training bias. Leveraging SCP, ArxivRoll constructs a new benchmark every six months using recent articles from ArXiv and employs them for one-time evaluations of LLM performance. Extensive experiments demonstrate the high quality of our benchmark, and we provide a systematic evaluation of current LLMs. The source code is available at https://github.com/liangzid/ArxivRoll/ . 1 Introduction With the rapid development of large language models (LLMs), their evaluation has attracted growing attention. Numerous challenging and widely recognized benchmarks Hendrycks et al. 2021 ; Cobbe et al. 2021 ; White et al. 2024 ; Chiang et al. 2024 ; Jimenez et al. 2024 ; Team 2025 have been introduced to assess the knowledge and reasoning capabilities of these models. As a result, these evaluations have become the primary, and often the only, standard for comparing the performance of large language models. Despite their effectiveness, recent research Wu et al. 2024 ; Dong et al. 2024 ; Jiang et al. 2024 increasingly highlights the shortcomings of current evaluation mechanisms, arguing that the capabilities of LLMs are often universally overestimated . This occurs mainly due to evaluation leakage, where test samples, benchmark details or formatting information can be exploited to game the benchmark. Consequently, it may inflate the perceived performance of a model, resulting in unreliable evaluations and unfair comparisons among LLMs. Malicious developers could further fool benchmarks by incorporating test samples or benchmark-specific information during training or fine-tuning. For instance, a previous study Yang et al. 2023 demonstrated that a 13-billion-parameter Llama model can easily achieve results comparable to GPT-4 on benchmarks like MMLU Hendrycks et al. 2021 through post-processing-based fine-tuning. Additionally, popular open-source LLMs such as Llama-4 and Qwen-2.5 have been reported to experience test-data-contaminated training. Such intentional or unintentional cheating behaviors distort the true capabilities of LLMs, misleading subsequent training procedures and corresponding discoveries Wu et al. 2025 . Specifically, there are two main types of abuse involving evaluation benchmarks. The first is data contamination Palavalli, Bertsch, and Gormley 2024 ; Li et al. 2024c ; Dong et al. 2024 ; Xu et al. 2024 ; Jiang et al. 2024 , where test cases from the benchmarks are included in the training set of large language models, enabling them to become familiar with or even memorize these samples, resulting in artificially improved performance. The second is biased overtraining , where models are claimed to be “comprehensive” but actually prioritize improving their performance in the evaluated domain at the expense of undertraining in other areas. Both scenarios significantly undermine the effectiveness, fairness, and reliability of evaluation results. Unfortunately, existing benchmarks designed to mitigate cheating behaviors have notable limitations. Private benchmarks maintained by trusted third-party platforms, such as SEAL 1 1 1 https://scale.com/leaderboard , and Arena-like benchmarks Chiang et al. 2024 ; Huang et al. 2024 ; Li et al. 2024b ; Li et al. 2024a , such as Chatbot Arena Chiang et al. 2024 , lack transparency and reproducibility in their evaluation processes. Symbolic formatting benchmarks for specific domains Zhu et al. 2024a ; Zhang, Chen, and Yang 2024 ; Zhu et al. 2024b , such as GSM-Symbolic Mirzadeh et al. 2024 and LiveBench White et al. 2024 , are restricted to narrow fields and therefore fail to provide a comprehensive evaluation of LLMs. Furthermore, the above benchmarks primarily focus on assessing the realistic abilities of LLMs, without offering a clear quantification of the extent of overestimation. As a result, a stable, transparent, reproducible, and human-effort-free framework and benchmark for evaluating LLMs has yet to be developed. To address these issues, we propose ArxivRoll, a robust and dynamic framework designed to evaluate both the realistic performance and the overestimation of large language models. ArxivRoll consists of two key components: 1) SCP (Sequencing, Cloze, and Prediction), a novel method that automatically generates test cases from newly published articles on ArXiv to construct private benchmarks; and 2) Rugged Scores (RS), indicators that quantify the performance difference between public and private benchmarks, providing a clear measure of overestimation. Inspired by the security guarantee of One-Time Pad Miller 1882 ; Shannon 1949 in cryptography, which uses a unique secret key for each use, ArxivRoll divides benchmarks into public benchmarks (existing ones) and private ArxivRollBenches (generated by SCP), and regard the private benchmarks as the one-time-used secrets to mitigate the overestimation. After evaluation, the private benchmarks are publicly released to ensure reproducibility of evaluation but are marked as expired to prevent future use or reference. Extensive meta-evaluations on ArxivRollBench demonstrate that SCP consistently produces high-quality test samples. Besides, the private benchmarks exhibit a strong correlation with existing private yet non-transparent benchmarks, confirming their reliability and relevance. Our contributions are summarized as follows: • We devise a novel private benchmark construction strategy, SCP (Sequencing, Cloze, and Prediction) based on Arxiv, which automatically generates high-quality, challenging, and fresh test cases tailored for assessing the capabilities of LLMs. Extensive experiments have proved the high quality of our generated private benchmarks. • We design rugged scores (RS) to quantify the proportion of cheating behavior in a given LLM when tasked with specific challenges. To the best of our knowledge, this is the first study to measure the proportion of overestimation and the biased overtraining. • Leveraging RS and SCP, we present a novel and one-time-pad-based evaluation framework, namely ArxivRoll. This framework not only evaluates the performance of current LLMs but also considers their overestimation situations. Through ArxivRoll, we conduct a systematic evaluation and establish a leaderboard 2 2 2 https://arxivroll.moreoverai.com for popular LLMs, providing a comprehensive evaluation of their capabilities. Figure 1: Framework of ArxivRoll , which categorizes benchmarks into two distinct groups: public benchmarks and private benchmarks (i.e., ArxivRollBench). These benchmarks are utilized to estimate both the overestimation proportion and performance of large language models (LLMs). Notably, ArxivRoll represents a dynamic benchmarking system, where private benchmarks are utilized exclusively once and then expire for subsequent evaluation stages, ensuring freshness and reliability in each assessment. 2 ArxivRoll Figure 2: An illustrative example of symbolic formatting for test samples , encompassing three formats: sequencing, cloze, and prediction (SCP). We have reformatted SCP into a four-candidate selection task, as detailed in Figure 7 . In this section, we will introduce

Read the complete text