Beyond the Singular: Revealing the Value of Multiple Generations in Benchmark Evaluation
Wenbo Zhang, Hengrui Cai, Wenyu Chen · 2025
Read Beyond the Singular: Revealing the Value of Multiple Generations in Benchmark Evaluation by Wenbo Zhang, Hengrui Cai, Wenyu Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08943.txt.
Introduction
Wenbo Zhang, Hengrui Cai, and Wenyu Chen argued in a 2025 paper that language model benchmarks should sample many answers, not one. Models are random generators, yet standard scoring uses fixed settings or a single draw, leaving sampling luck inside every published number. Their hierarchical statistical model captures both the benchmark's traits and the model's randomness, giving a truer picture of the test itself.
Multiple generations sharpen score estimates and cut their variance. The samples also yield a per-prompt difficulty score: the share of correct answers across draws. Plotted on a data map with meaning, these scores expose bad or ambiguous items during benchmark building. The authors frame repeated sampling as cheap insurance for the field's main measuring stick.
Catalog record
| CREATOR | Wenbo Zhang, Hengrui Cai, Wenyu Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-08943
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond the Singular: Revealing the Value of Multiple Generations in Benchmark Evaluation Wenbo Zhang, Hengrui Cai, Wenyu Chen Abstract Large language models (LLMs) have demonstrated significant utility in real-world applications, exhibiting impressive capabilities in natural language processing and understanding. Benchmark evaluations are crucial for assessing the capabilities of LLMs as they can provide a comprehensive assessment of their strengths and weaknesses. However, current evaluation methods often overlook the inherent randomness of LLMs by employing deterministic generation strategies or relying on a single random sample, resulting in unaccounted sampling variance and unreliable benchmark score estimates. In this paper, we propose a hierarchical statistical model that provides a more comprehensive representation of the benchmarking process by incorporating both benchmark characteristics and LLM randomness. We show that leveraging multiple generations improves the accuracy of estimating the benchmark score and reduces variance. Multiple generations also allow us to define ℙ ( correct ) \mathbb{P}\left(\text{correct}\right) , a prompt-level difficulty score based on correct ratios, providing fine-grained insights into individual prompts. Additionally, we create a data map that visualizes difficulty and semantics of prompts, enabling error detection and quality control in benchmark construction. † † footnotetext: * Work done during internship at Meta † † footnotetext: {\dagger} Co-correspondence 1 Introduction In recent years, advanced large language models have demonstrated remarkable versatility across a wide range of tasks and domains, with their development continuing to accelerate. To effectively track their progress, numerous generative benchmark datasets have been curated to assess both their general and specialized capabilities. There are two primary ways for generating responses from large language models (LLMs): greedy decoding and random sampling ( Holtzman et al., 2019 ) . Greedy decoding selects the next token with the highest probability, resulting in a deterministic output. In contrast, random sampling, such as nucleus sampling ( Holtzman et al., 2019 ) , incorporates randomness during decoding by sampling a token at each step based on a probability distribution. This approach leads to non-deterministic output. Current LLM benchmarks typically employ one of these methods; for instance, LiveBench ( White et al., 2024 ) WildBench ( Lin et al., 2024 ) and OpenLLM leaderboard ( Beeching et al., 2023 ) use greedy decoding, while TrustLLM ( Huang et al., 2024 ) , MT Bench ( Zheng et al., 2023 ) and Alpaca Eval ( Li et al., 2023 ) employ a non-deterministic sampling configuration. During evaluations, LLMs generate a single response for each prompt in the benchmark, and the correctness of these responses is determined by comparing them to the ground truth answers. The final benchmark score is then calculated as the average of these individual scores. However, this presents challenges within the current generative-evaluation paradigm. Firstly, deterministic generation does not align with the real-world application of LLMs, where randomness is inherent. This misalignment can lead to biased estimations of LLM performance. Even with random generation, relying on a single generation can result in significant variance in benchmark scores, particularly when the sample size is small. Furthermore, a single generation is not sufficiently informative for individual prompts, as it cannot address prompt-level questions such as, "Which question is more challenging?" This limitation creates obstacles to understanding the overall composition of the benchmark data. In this paper, we regard the benchmark as an estimation problem characterized by a statistical model and highlight the significance of incorporating multiple random generations in a principled way. We theoretically demonstrate that increasing the number of generations decreases the variance in benchmark score estimation. Moreover, by leveraging multiple samples, we introduce a fine-grained difficulty metric, \mathbb{P}\left(\text{correct}\right) , derived from the inherent latent parameters of our statistical model, to quantify the difficulty of individual prompts. This enables comparisons across different prompts. Additionally, we demonstrate that mislabeled or ambiguous prompts can be effectively detected using multiple generations, highlighting its potential as a tool in benchmark construction. 2 Benchmarking Procedure is a Hierarchical Model In this section, we show that the benchmark is an estimation problem. Without loss of generality, we consider random sampling as the generation strategy where each token is randomly sampled from a token distribution conditional on previously generated tokens. We also assume the correctness of generations can be obtained using a judgment function, which can be accomplished either by comparing the response with ground truth or by determining whether it passes unit tests. Given an LLM parameterized by parameters \theta , including both model parameters and sampling parameters, for example temperature T and top P , etc.), and a benchmark dataset \mathcal{D}=\{x_{i}\}_{i=1}^{n} , we can define difficulty of the i -th prompt with respect to the LLM as a random variable drawn from the unknown benchmark difficulty distribution \mathbb{P}(\mu,\sigma;\theta) , with mean \mu and standard deviation \sigma . Without loss of generality, with k generations per prompt, we can then regard the benchmarking procedure as a hierarchical model as follows: \begin{gathered}p_{i}\sim\mathbb{P}(\mu,\sigma;\theta)\quad\text{for }i=1,\cdots,n,\\ y_{i,j}\sim\text{Bernoulli}(p_{i})\quad\text{for }j=1,\cdots,k,\end{gathered} (1) where prompt difficulty p_{i} is sampled from \mathbb{P}(\mu,\sigma;\theta) and p_{i} represents the probability that the LLM can correctly answer the i -th prompt., i.e., \mathbb{P}\left(\text{A generated answer to $i$-th prompt is correct}\right)=p_{i} . This represents a latent difficulty of prompts, We denote the he k -th generation of the i- th prompt as z_{i,j} and then y_{i,j} is the correctness indicator for it, where y_{i,j}=1 if it’s correct otherwise y_{i,k}=0 . Here both benchmark distribution \mathbb{P}(\mu,\sigma;\mathcal{D}) and p_{i} are unknown needs to be estimated with \{y_{i,j}\}_{j=1}^{k} for i=1,\cdots,n . To estimate p_{i} and \mu , we can use a straight forward method of moment estimators \hat{p}_{i}=\frac{\sum_{j=1}^{k}y_{i,j}}{k} , \hat{\mu}=\frac{\sum_{i=1}^{n}\hat{p}_{i}}{n}=\frac{\sum_{i=1}^{n}\sum_{j=1}^{k}y_{i,j}}{nk} . We observe that a widely used item response theory ( Polo et al., 2024 ; Madaan et al., 2024 ; Ding et al., 2024 ) , employed to model the difficulty of prompts, represents a specific parametrization of \mathbb{P}(\mu,\sigma;\mathcal{D}) . Further elaboration on this can be found in Appendix A . Note that, when k=1 , the benchmark score computed based on a single random generation is an estimation of \mu , which only utilizes a single generation which leads to a large variance. We can show this by explicitly calculating the variance of our estimators. Lemma 2.1 . Given the hierarchical model in ( 1 ) and the moment estimators \hat{\mu}=\frac{\sum_{i=1}^{n}\sum_{j=1}^{k}y_{i,j}}{nk}. Then \hat{\mu} is an unbiased estimator for \mu and its variance equals: \text{Var}(\hat{\mu})=\underbrace{\frac{1}{nk}\left(\mu-\mu^{2}-\sigma^{2}\right)}_{\text{Withth-prompt Variance}}+\underbrace{\frac{1}{n}\sigma^{2}}_{\text{Between-prompt Variance}}. (2) Here, \text{Var}(\hat{\mu}) can be decomposed into within-prompt variance and between-prompt variance. Both terms decrease as the number of benchmark data n increases. However, since benchmark data is typically fixed, we analyze the influence of sampling in terms of k . Within-prompt variance captures the randomness in sampling y_{ij} conditional on the i- th prompt, and it can be effectively reduced by increasing the number of samples k , converging to 0 as k\to\infty . The between-prompt variance term, on the other hand, captures the variability of prompt difficulty p_{i} across groups, reflecting the randomness of difficulty distribution \mathbb{P}(\mu,\sigma;\theta) , and thus remains unaffected by k . We can further plug in sample variance \hat{\sigma}^{2}=\frac{1}{n-1}\sum_{i=1}^{n}(\hat{p}_{i}-\frac{\sum_{i=1}^{n}\hat{p}_{i}}{n})^{2} and \hat{\mu} into ( 2 ) to get \widehat{\text{Var}(\hat{\mu})} . Finally, based on the central limit theorem, a 95\% confidence interval is: \hat{\mu}\pm 1.96\sqrt{\widehat{\text{Var}(\hat{\mu})}} . (a) MMLU-Pro (b) GSM8K (c) IFEval (d) MuSR Figure 1: Distribution of \mathbb{P}\left(\text{correct}\right)