Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarking Polina Gordienko, Christoph Jansen, Julian Rodemann, Georg Schollmeyer Abstract Modern benchmarks such as HELM MMLU account for multiple metrics like accuracy, robustness and efficiency. When trying to turn these metrics into a single ranking, natural aggregation procedures can become incoherent or unstable to changes in the model set. We formalize this aggregation as a social choice problem where each metric induces a preference ranking over models on each dataset, and a benchmark operator aggregates these votes across metrics. While prior work has focused on Arrow’s impossibility result, we argue that the impossibility often originates from pathological examples and identify sufficient conditions under which these disappear, and meaningful multi-criteria benchmarking becomes possible. In particular, we deal with three restrictions on the combinations of rankings and prove that on single-peaked, group-separable and distance-restricted preferences, the benchmark operator allows for the construction of well-behaved rankings of the involved models. Empirically, we investigate several modern benchmark suites like HELM MMLU and verify which structural conditions are fulfilled on which benchmark problems. 1 1 1 The code for reproducing all experiments is available at https://github.com/polinamgordienko-glitch/Beyond-Arrow-From-Impossibility-to-Possibilities-in-Multi-Criteria-Benchmarking . 1 Introduction Benchmarks are fundamental to progress in modern machine learning, for they provide standardized frameworks for model evaluation and comparison ( 28 ) . Within the past decades, evaluation methods in the field have expanded from single-task benchmarks such as leaderboards based on the ImageNet dataset to multi-task suites including GLUE ( 58 ) , SuperGLUE ( 57 ) , MMLU ( 29 ) and BigBench ( 55 ) . More recently, holistic evaluation frameworks like HELM ( 43 ) have been introduced that combine hundreds of tasks to assess language models across a broad range of scenarios and along different metrics that go beyond accuracy. Yet, as these benchmark suites have grown more complex, a central challenge has emerged: how should we aggregate performance across multiple evaluation criteria, each measured over several datasets, into one meaningful ranking of models? The prevailing approach in benchmarking is based on the “unspoken utilitarian principles” ( 52 ) , ranking systems by the arithmetic mean of their scores across task-specific metrics ( 58 ; 57 ; 29 ) . Nonetheless, this aggregation procedure has been increasingly viewed as inadequate, for instance, due to its sole focus on the value of model predictions and disregard for the cost (e.g., model size, training time) of those predictions ( 26 ) . Moreover, mean aggregation can be easily dominated by performance on a few outlier tasks ( 1 ) . In general, averaging methods can mislead when samples vary in difficulty, resulting in inflated model performance and unreliable rankings ( 47 ) . The problem is amplified when benchmarks include ordinal criteria such as interpretability, since mean aggregation presupposes a meaningful scale of differences that such metrics do not provide. Further challenges such as task selection bias ( 21 ) , saturation of benchmarks over time ( 40 ) as well as limitations of dynamic benchmarks ( 54 ) have been identified. However, it has also been argued that a focus on failures merely incentivizes the development of models that sidestep those mistakes by shifting errors elsewhere, creating only the appearance of progress ( 15 ) . Instead of focusing on limitations, our work identifies conditions under which meaningful multi-criteria benchmarking becomes possible. Rather than asking “what goes wrong?” , we ask “under what structural conditions can benchmarking succeed?” . Social choice theory – the “study of collective decision procedures and mechanisms” ( 44 ) – offers a precise vocabulary to assess aggregation processes. While Arrow’s theorem ( 3 ) establishes that no aggregation function can satisfy all desirable properties simultaneously over the universal domain (i.e., the function works for every possible pattern of preferences), the social choice literature has long recognized that domain restrictions – natural constraints on the inputs of the aggregation rule – can restore the possibility of meaningful aggregation ( 9 ; 31 ; 53 ; 23 ; 50 ) . In the context of benchmarking, this raises a fundamental question that has so far received surprisingly little attention: Do multi-criteria benchmarks naturally exhibit structure in the domain that restores possibility? In contrast to prior work focusing on impossibility ( 59 ) , we deal with sufficient conditions under which meaningful multi-criteria benchmarking becomes possible. We formalize multi-criteria benchmarking as a preference aggregation problem, where metrics such as accuracy, fairness, efficiency act as voters that form preferences on models. Given a fixed benchmark suite, we identify three domain restrictions – single-peakedness , group separability and distance-restrictedness – and analyze the properties of benchmarks when the preferences induced by metrics satisfy such structure. Intuitively, single-peakedness means that there exists a one-dimensional ordering of models such that each metric has a single “sweet spot”, and moving away from it along the ordering makes models consistently less preferred by that metric. We then verify this structure empirically on several modern benchmark suites, in particular HELM MMLU. Figure 1 : We fix seven language models and a set of accuracy and efficiency metrics from HELM. For each MMLU subject dataset, each metric induces a ranking of models; we study when these rankings are consistent with a single shared ordering of models (x-axis) so that each metric has one “sweet spot” (one peak ) along that ordering. The subject Business Ethics (left) satisfies this structure; Abstract Algebra (right) does not. 2 Related Work A growing body of literature has turned to social choice theory to formalize the aggregation problem in benchmarks. Early contributions include 27 who frame benchmarking as consensus over dataset-level preference relations as well as 46 who study benchmarking of optimization algorithms as consensus-ranking problem over many test functions. 18 propose an aggregation method based on Kemeny consensus, while 30 develop a ranking approach using the Borda count for managing missing scores in benchmarks. 52 introduce VOTE’N’RANK, a framework applying eight social choice procedures onto multi-task benchmarks. The study emphasizes that classical voting rules (e.g., Borda count, Minimax and Condorcet) can give more robust and interpretable rankings than simple mean aggregation. 59 use Arrow’s theorem to analyze the aggregation problem in multi-task benchmarks, highlighting a trade-off between diversity and stability. 41 employ principles from social choice and game theory for building a framework to evaluate general agents. Complementary work ( 22 ; 7 ; 34 ; 35 ; 45 ) proposes statistically sound methods for aggregating performance across multiple tasks. 3 Motivation We deal with multi-criteria benchmarking that aims to evaluate and compare models along different criteria/metrics (e.g. accuracy, robustness, efficiency), each measured over multiple datasets. The central challenge is how to obtain a meaningful, overall ranking of models from different metrics. But what does “meaningful” mean? We argue that two intuitive properties are coherence and stability . Coherence. One plausible constraint on meaningful benchmarks is that the overall ranking of models should not contradict itself. For instance, when we consider evaluation criteria across qualitatively different dimensions, aggregation can become inherently controversial. For three models A,B,C , it may happen that metrics collectively prefer A to B , B to C , and yet C to A . Even if each metric induces a sound ordering of models, there does not necessarily exist a coherent aggregated ranking over all metrics. We illustrate this tension on the MMLU benchmark as implemented in HELM v1.0.0, which we refer to throughout the paper as our running example. The benchmark evaluates 23 language models across 57 subject datasets and records multiple metrics per subject ( 29 ; 43 ) . Each metric generates a ranking of models for each subject. A natural democratic aggregation method that treats all metrics equally is pairwise majority comparison. We consider each metric as one vote and declare that A beats B if more metrics prefer A to B than prefer B to A . Table 1 illustrates that these pairwise comparisons may result in cyclic majority rankings such as GPT-4 \succ Qwen1.5 \succ GPT-3.5 \succ GPT-4 (where \succ is short for “is preferred to”). This paradoxical behavior means that the majority ranking is not well-behaved; even this basic aggregation method fails at delivering a coherent ranking of models across three metrics. Since the pairwise majority relation is cyclic on a subset of models, there is no transitive ranking over the full model set that agrees with all majority comparisons at once. This phenomenon is not rare. Given a small subset of accuracy and efficiency metrics in HELM, we find such cycles in most MMLU subjects (see Appendix A.6 ). Table 1 : For the MMLU subject Formal Logic , consider three metrics and three representative language models. Columns list the three models from best to worst under that metric, with corresponding values in parentheses; for Accuracy higher means better, for Inference Time and Output Length lower means better. Pairwise majority comparison across the metrics yields a cyclic ordering of models: GPT-4 \succ Qwen1.5 \succ GPT-3.5 \succ GPT-4. The exact model identifiers can be found in Appendix A.6 . Accuracy Inference Time (s) Output Length (bytes) GPT-4 (0.65) Qwen1.5 (0.32) GPT-3.5 (1.00) Qwen1.5 (0.49) GPT-3.5 (0.41) GPT-4 (1.17) GPT-3.5 (0.40) GPT-4 (0.49) Qwen1.5 (2.00) Table 2 : Aggregated rankings of models across three metrics Accuracy , Inference Time and Output Length for the MMLU subject High School World History . We rank models by average rank across the three metrics. The left column shows the overall ranking of 15 models; the right column presents the ranking after Llama-2 has been added to the model set. Several positions shift. Notably, the relative order of Claude-3-Opus and GPT-3.5 flips: Claude-3-Opus \succ GPT-3.5 before adding Llama-2 and GPT-3.5 \succ Claude-3-Opus after adding Llama-2. Before After 1. GPT-4-0613 1. GPT-4-0613 2. GPT-4-1106 2. GPT-4-1106 3. Claude-3-Opus 3. Qwen1.5 4. Qwen1.5 4. GPT-3.5 5. GPT-3.5 5. Claude-3-Opus ⋮ ⋮ 15. Google-Text-Bison 15. Google-Text-Bison 16. Llama-2 Stability. We can avoid cycles in benchmark rankings by using another aggregation method, for instance, any rule based on combining ranks of models across different datasets/tasks or metrics. One example is the winning rate of models, defined as the fraction of head-to-head comparisons across datasets where a model is better on that metric, which is widely used in benchmarks ( 43 ; 42 ; 59 ) . Another popular example is Borda score ( 19 ) , which is being increasingly used in benchmarks such as MTEB ( 17 ) . Then, however, we face another problem: benchmarks may become unstable to irrelevant changes in the model set. Intuitively, stability means that adding or removing a model C should not flip the ranking between two existing models A and B . Table 2 demonstrates the violation of stability on one dataset in the HELM MMLU benchmark. We consider 15 models with the highest accuracy on this subject among those with complete measurements for the three metrics, and then add one additional model. We compute the overall rankings of models before and after the change across metrics by averaging the metrics’ ranks of models (Borda score over ranks). Note that the new model, Llama-2, is worse on Accuracy compared to all existing 15 models, yet relatively efficient with an Inference Time faster than Claude-3-Opus but slower than GPT-3.5. Crucially, the addition of Llama-2 changes the rankings of models near the top of the leaderboard. We find such flips in 44 out of 57 MMLU subjects (see Appendix A.6 ). By no means do these failures imply the impossibility of meaningful multi-criteria benchmarking. Rather, these examples tell us what must be true of the rankings induced by each metric if coherent and stable benchmarking is to be possible. For the aforementioned pathologies arise only in the unstructured combinations of rankings. In this paper, we show meaningful aggregation can be achieved. We formalize each metric’s ranking as a vote and analyze the structure in those votes across metrics and datasets that enables fulfillment of coherence, stability and further desirable properties from social choice theory that we explore in the next section. 4 Benchmarking as a Social Choice Problem Social choice theory studies the aggregation of individual inputs (e.g. preferences, judgements, probabilities) into a collective output. In particular, it investigates axiomatic properties that such collective decision procedures can or cannot satisfy. In this section, we formalize multi-criteria benchmarking as a preference aggregation problem. 4.1 Notation Multi-criteria benchmarking. Let \mathcal{D} be some fixed universe of instances/datasets and let \mathcal{A} be some fixed and finite set of algorithms/models with k:=|\mathcal{A}|\in\mathbb{N} . Let (\phi_{i})_{i\in[n]} denote a family of metrics/criteria \phi_{i}:\mathcal{A}\times\mathcal{D}\to[0,1], for some n\in\mathbb{N} , where [n]:=\{1,\dots,n\} . Moreover, let \Phi=(\phi_{1},\dots,\phi_{n}):\mathcal{A}\times\mathcal{D}\to[0,1]^{n} be a multivariate metric. 2 2 2 Note that our framework is more general than a mere multi-criteria benchmarking problem; it is generic, in the sense, that any problem that produces evaluations over a finite set of alternatives can be cast in terms of \mathcal{D} , \mathcal{A} and \Phi . For instance, one may treat datasets or tasks as voters (model comparison across datasets; 59 ) or consider test functions as voters in optimizer benchmarking ( 51 ) . The goal of multi-criteria benchmarking can be stated as follows: We search for a model from \mathcal{A} that best balances the metrics \phi_{1},\dots,\phi_{n} across the various datasets D\in\mathcal{D} . In order to identify the best algorithm, the entire information contained in \mathcal{A} , \mathcal{D} and \Phi should be exploited. Preference aggregation problem. Let pref (\mathcal{A}) be the set of all complete and transitive preference relations on \mathcal{A} . Each fixed metric i\in[n] and dataset D\in\mathcal{D} induce a preference relation R_{D,i}\in pref (\mathcal{A}) , defined by setting A\succeq_{R_{D,i}}A^{{}^{\prime}}:\Leftrightarrow\phi_{i}(A,D)\geq\phi_{i}(A^{{}^{\prime}},D). Hence, R_{D,i} corresponds to the ranking of models in \mathcal{A} induced by the scores of the i th performance metric, evaluated on instance D . We denote by P_{D,i} the strict part of R_{D,i} defined by A\succ_{P_{D,i}}A^{{}^{\prime}}:\Leftrightarrow\phi_{i}(A,D)>\phi_{i}(A^{{}^{\prime}},D) . Since metric scores can tie, R_{D,i} may include indifference between models; we use P_{D,i} whenever strict comparisons are necessary. We obtain P_{D,i} from R_{D,i} by breaking ties according to a fixed deterministic rule. Furthermore, let prof (\mathcal{A},\mathcal{D},\Phi) denote the set of preference profiles induced by \mathcal{D} , i.e., the set \text{prof}(\mathcal{A},\mathcal{D},\Phi)=\{R_{D}:=(R_{D,1},\dots,R_{D,n})|D\in\mathcal{D}\}. Each profile R_{D} arises by collecting the preference rankings induced by metrics \phi_{1},\dots,\phi_{n} evaluated on instance D in an n -tuple. Let B:\text{prof}(\mathcal{A},\mathcal{D},\Phi)\to pref (\mathcal{A}) be a benchmark operator – a map returning a preference order among the algorithms of interest for every profile of such rankings that is inserted to it. Importantly, the domain of the operator B depends on each of the objects \mathcal{A},\mathcal{D},\Phi . In HELM MMLU, \mathcal{D} corresponds to the set of 57 MMLU subject datasets, while each D\in\mathcal{D} is a single subject (e.g. Formal Logic ). The set of evaluated language models is denoted by \mathcal{A} , while \Phi is the collection of all reported HELM metrics, with each \phi_{i} corresponding to the metric i . The relation R_{\textit{Formal Logic, Inference Time}} denotes the ranking of models according to metric Inference Time on subject Formal Logic , while the profile R_{\textit{Formal Logic}} includes the rankings induced by all metrics on this subject. The benchmark’s overall ranking of models on that subject is given by B(R_{\textit{Formal Logic}}) . There are many kinds of benchmark operators, including pairwise majority and average rank mentioned in Section 3 . In general, pairwise majority aggregation is the most reasonable local aggregation rule, though its downside – the possibility of cyclic majority rankings – is well-known. Our contribution is to show that there are many realistic benchmarking scenarios where pairwise majority not only produces coherent (acyclic) overall ranking of models but also satisfies further desirable properties that we introduce in the next section. For this reason, the remainder of the paper studies the benchmark operator B_{M} that aggregates metric rankings using the pairwise majority. 4 4 4 In benchmarking, other aggregation rules, in particular those based on combining ranks of models across tasks (e.g. Borda count) are common. However, these procedures face different problems, since they are unstable to changes in model set (e.g., see Table 2 ) and violate Arrow’s axiom of independence of irrelevant alternatives ( 8 ) . For any A,A^{{}^{\prime}}\in\mathcal{A} and any D\in\mathcal{D} : \displaystyle M_{D}(A,A^{{}^{\prime}}):=|\{i\in[n]:A\succeq_{R_{D,i}}A^{{}^{\prime}}\}|, \displaystyle A\succeq_{M}^{D}A^{{}^{\prime}}\Leftrightarrow M_{D}(A,A^{{}^{\prime}})\geq M_{D}(A^{{}^{\prime}},A). 4.2 Social Choice: From Impossibility to Possibility Social choice is well-known for its impossibility results, most prominently Arrow’s theorem ( 3 ) . However, the underlying problem behind impossibilities in voting was recognized much earlier, by Condorcet ( 20 ) . He formalized a situation where three voters rank the alternatives x , y , and z from the most-preferred to the least-preferred and aggregate their preferences by simple majority voting on each pair of alternatives. Even though each voter’s preference is rational, majority prefers x over y , y over z , and z over x , yielding a Condorcet cycle and an irrational collective preference ranking. The cyclic behavior of the pairwise majority relation in Table 1 corresponds precisely to the Condorcet paradox. Arrow’s theorem. Arrow’s impossibility theorem ( 3 ) generalized Condorcet’s paradox by showing that, given a set of individual preferences over three or more alternatives, there exists no aggregation procedure that satisfies a few quite reasonable assumptions concerning the autonomy of voters and the rationality of their preferences ( 48 ) . The conditions Arrow imposes on aggregation rules, often called axioms , include Non-Dictatorship , Independence of Irrelevant Alternatives , Weak Pareto , Social Ordering and Universal Domain . Theorem 4.1 . ( 3 ) Suppose k\geq 3 and n\geq 2 . Let F be an operator that maps each profile from ( pref (\mathcal{A}))^{n} ( Universal Domain ) to a preference relation in pref (\mathcal{A}) ( Social Ordering ). For a profile R=(R_{1},\dots,R_{n})\in( pref (\mathcal{A}))^{n} , let P_{i} be the strict part of R_{i} and P the strict part of F(R) . Then F cannot satisfy the following conditions simultaneously: • Non-Dictatorship: There is no i\in[n] such that for all profiles R and for all A,A^{{}^{\prime}}\in\mathcal{A} , if A\succ_{P_{i}}A^{{}^{\prime}} , then A\succ_{P}A^{{}^{\prime}} . • Weak Pareto: For all profiles R and for all A,A^{{}^{\prime}}\in\mathcal{A} , if \ \forall i\in[n]\ \ A\succ_{P_{i}}A^{{}^{\prime}} , then A\succ_{P}A^{{}^{\prime}} . • Independence of Irrelevant Alternatives (IIA): For all R , R^{{}^{\prime}}\in( pref (\mathcal{A}))^{n} and for all A,A^{{}^{\prime}}\in\mathcal{A} , if \ \forall i\in[n]\ \ A\succeq_{P_{i}}A^{{}^{\prime}}\Leftrightarrow A\succeq_{P^{{}^{\prime}}_{i}}A^{{}^{\prime}} , then A\succeq_{P}A^{{}^{\prime}}\Leftrightarrow A\succeq_{P^{{}^{\prime}}}A^{{}^{\prime}} . Note that the operator F generally differs from the operator B as defined earlier, since in most cases the benchmark suite B does not induce the full domain of preference profiles ( pref (\mathcal{A}))^{n} . This is why Arrow’s impossibility theorem cannot be explicitly applied to the benchmark operator B . Social Ordering requires the benchmark to output a coherent ranking, i.e. a complete and transitive binary relation on \mathcal{A} . Some natural aggregation rules, in particular pairwise majority, can fail this requirement: when a Condorcet cycle is present, the collective relation is intransitive. Universal Domain demands that the aggregation rule admits any logically possible combination of metric rankings over \mathcal{A} . This implies that we are not allowed to make any assumptions about metrics’ behavior or relationships between metrics in advance. In benchmarking, however, Universal Domain is not a reasonable assumption. For metrics do exhibit systematic behavior, and trade-offs across families of metrics are well-known (e.g., 36 ; 2 ). Weak Pareto asks that if every metric ranks model A over model A^{{}^{\prime}} , then the collective ranking across metrics must prefer A to A^{{}^{\prime}} . Non-Dictatorship rules out any single metric that always determines the overall preferential ranking across all metrics. Finally, the theorem’s most demanding condition of IIA states that the social comparison of models A and A^{{}^{\prime}} depends only on how each metric compares the two models and not on how any metric ranks other models. Restricted preference domains. A longstanding literature in social choice theory ( 9 ; 53 ; 32 ; 23 ; 24 ) investigates possibilities , under which majority cycles can be avoided. The idea is to relax or violate one of Arrow’s axioms in a natural, plausible manner under which the remaining conditions can become jointly satisfiable. One notable “escape route” from impossibility is to violate Universal Domain : we restrict attention to preference profiles with certain structure rather than allowing all logically possible profiles of complete and transitive preferences. It has been established that if a voter’s preferences fall into a suitably restricted domain, the possibility of meaningful aggregation can be restored. The well-behaved restricted domains are called Condorcet domains and prominent examples include domains of single-peaked, group-separable and value-restricted preferences ( 50 ) . Sen’s theorem ( 53 ) provides a general sufficient condition under which the pairwise majority relation yields a transitive ranking. While Sen’s condition is very general, it is comparatively technical ( 25 ) and not particularly interpretable in the benchmarking context. Therefore, we focus on restricted domains with clearer meaning and investigate domains of single-peaked, group-separable and distance-restricted preferences in Section 5 . 4.3 Aggregation Across Datasets Our formalization of benchmarking tasks using operators is fundamentally based on a social choice perspective: The operator assigns exactly one aggregate (or consensus) relation on the models to each preference profile generated by the various metrics on a specific dataset. In particular, the relations obtained generally vary depending on the dataset . However, in many applied benchmark studies, this dependence on the instance under consideration is undesirable. Instead, the benchmark suite \mathcal{D} should be used to extract a ranking of the models per se , which carefully weighs the information contained in the suite. So how can we go from a benchmark operator to a ranking of the models over all datasets? We consider the family of orders generated by a benchmark operator B , given by \text{ord}(B,\mathcal{D}):=\bigl(B(R_{D})\bigr)_{D\in\mathcal{D}}, which is an ordered list of all rankings of the models in \mathcal{A} that occur across the different datasets in \mathcal{D} , if performance is operationalized by the multi-dimensional metric \Phi . If we now want to choose one ranking from this list, we should, if possible, choose one that best represents the family. But how can we determine a representative element, such as an abstraction of the classical median, from a list of relations? Generally, there are many possibilities, here. Under a statistical perspective one can treat every dataset D and its associated ranking B(R_{D}) as a data point of some population (i.e., of \mathcal{D} ) of interest. Under this understanding, a modern and natural approach is based on the theory of depth functions ( 60 ) and its generalizations to non-standard data spaces, such as relations in our case. For choosing the most central dataset/ranking from a suite, there are different depth functions available, for example the ufg depth ( 11 ; 12 ) or a generalization of Tukey depth ( 56 ) , firstly used in 33 under the name commonality sharing rule and analyzed in 13 . In this paper, we use the generalized Tukey depth where the objects of interest are rankings. The idea is to choose from the population \text{ord}(B,\mathcal{D}) of rankings the so-called commonality sharing ranking(s) – the ranking(s) that share(s) with every subpopulation of minimum-size k the commonalities of the subpopulation, where k is chosen as small as possible. Here, commonalities are all pairs (A,A^{{}^{\prime}}) which are ranked identically by all members of the subpopulation. It is important to note that this only works for benchmark operators that produce exclusively acyclic relations as aggregates. Otherwise, the commonality sharing rule is not well defined 5 5 5 The generality of the commonality sharing rule (and other methods like the ufg depth) would in principle allow to adopt the aggregation ideas to acyclic relations. However, the aggregated result would then of course be generally only a cyclic relation and this is of course not satisfying. . Nonetheless, if it is ensured that the operator generates only preference relations (e.g., via suitable domain restrictions), the most central relation in the ordered list \text{ord}(B,\mathcal{D}) according to the generalized Tukey depth can be selected as the aggregate. In this sense, our enquiry of finding meaningful benchmark operators also plays a central role in the systematic identification of a suitable ranking of the models in \mathcal{A} per se. 5 Restricted Preference Domains in Benchmarking 5.1 Single-Peaked Preferences In this section, we investigate domain restrictions in the context of multi-criteria benchmarking. Given fixed objects \mathcal{A} , \mathcal{D} and \Phi , we examine the structure in the set of preference profiles \text{prof}(\mathcal{A},\mathcal{D},\Phi) . We consider three types of domain restrictions: single-peakedness, group separability and distance-restrictedness. Conditional on benchmark suites with this specific structure, we show that the operator B_{M} satisfies Arrow’s axioms of Non-Dictatorship , IIA , Weak Pareto and Social Ordering . We pair each domain restriction with an empirical check, testing our assumptions on the structure of the fixed benchmark suite in Section 6 . The first and most famous restricted preference domain was established by 9 and independently discovered by 4 . The key idea is that alternatives can be embedded on a one-dimensional axis such that each voter has a single most-preferred point (a peak ) on that axis, and the voter’s preferences decline when moving away from that point on the axis. On such domain of single-peaked preferences, Condorcet cycles are impossible. Interpretation in benchmarking. Single-peakedness means that there exists a way to arrange models along some spectrum so that each metric’s ranking of models has a single peak on this spectrum. This can be viewed as the “sweet spot” of each metric where the most preferred models for this metric lie, while performance decreases when moving further away from that region on the spectrum. In benchmark practice, this one-dimensional axis can be observed in several contexts, reflecting different trade-offs of model design. For instance, imagine ordering algorithms by the number of parameters, and then examining different metrics’ rankings along this ordering. We would expect that interpretability would be highest for simpler models and decrease as the number of parameters goes up. Accuracy would be low for very simple models and peak for models with a moderate number of parameters, and then possibly decrease due to overfitting. Efficiency would prefer very simple models and drop as we move to models with larger number of parameters. Our definition of single-peaked preference profiles follows 6 . Assume S\subseteq\mathcal{A} (with |S|\geq 3 ). Let \mathcal{L} be the set of all strict linear orders over S and set L\in\mathcal{L} . For any two models A,A^{{}^{\prime}}\in\mathcal{A} , we write A\succ_{L}A^{{}^{\prime}} if A precedes A^{{}^{\prime}} in the ordering L . Definition 5.1 . For any set S\subseteq\mathcal{A} let w(S,P_{D,i}) denote the least preferred model in S according to the strict preference relation P_{D,i} and let b(S,P_{D,i}) be the most preferred model in S according to P_{D,i} . Let L be an admissible orientation of S with respect to P_{D,i} if for any three models A,A^{{}^{\prime}},A^{{}^{\prime\prime}}\in S with A^{{}^{\prime}}=b(S,P_{D,i}) being the most preferred model in S according to metric i such that A^{{}^{\prime}}\succ_{L}A\succ_{L}A^{{}^{\prime\prime}} or A^{{}^{\prime\prime}}\succ_{L}A\succ_{L}A^{{}^{\prime}} , we have \forall A,A^{{}^{\prime\prime}}\in S , A\succ_{P_{D,i}}A^{{}^{\prime\prime}} . Denote the set of all admissible orientations of S\subseteq\mathcal{A} with respect to P_{D,i} by \mathcal{L}_{S}(P_{D,i}) and set \mathcal{L}_{S}(R_{D})=\bigcap_{i\in[n]}\mathcal{L}_{S}(P_{D,i}) . A profile R_{D} is called single-peaked if there exists an ordering L over the set of algorithms \mathcal{A} with L\in\mathcal{L}_{\mathcal{A}}(R_{D}) such that \mathcal{L}_{\mathcal{A}}(R_{D})\neq\varnothing . Theorem 5.2 . Fix a dataset D\in\mathcal{D} and a finite set of algorithms \mathcal{A} . Let R_{D} be a single-peaked profile induced by D\in\mathcal{D} . Then the weak pairwise majority relation \succeq_{M}^{D} satisfies Non-Dictatorship, Weak Pareto, IIA and yields a complete and transitive order on \mathcal{A} . 6 6 6 The proof of this and the following theorems can be found in Appendix A.1 - A.3 . 5.2 Group-Separable Preferences 31 ; 32 has introduced the class of group-separable domains which are sufficient for a transitive social ranking as well as the fulfillment of all Arrow’s axioms except for Universal Domain , provided that the number of voters is odd. In social choice theory, group separability property means that every subset of alternatives can be partitioned into two blocks such that all voters rank every element of one block above every element of the other. Interpretation in benchmarking. In a group-separable profile, for any subset of models, there exists at least one group of models that every metric regards as entirely above (or entirely below) the rest. This is very plausible in cases where the metrics measure the same latent construct, for instance, when using several accuracy metrics. Even if the metrics are not identical, they typically agree that some models are better than the others. But the condition can also hold in multi-objective comparisons, such as combining both accuracy and efficiency metrics in HELM MMLU. This is because this benchmark contains models that are consistently dominated on the chosen metric set, e.g. models that tend to be less accurate and less efficient than several others such as Mistral-7b. Whenever such dominated models are present, the metrics agree on a separation that places them entirely below the rest (and similarly, if a subset contains models that are both accurate and efficient, they can form a block that all metrics place above the rest). We define group-separability following 6 . Definition 5.3 . Assume S\subseteq\mathcal{A} (with |S|\geq 3 ). Let a non-empty set E\subset S be a separation with respect to P_{D,i} if either \forall A\in E,\forall A^{{}^{\prime}}\in S\setminus E:A\succ_{P_{D,i}}A^{{}^{\prime}} or \forall A\in E,\forall A^{{}^{\prime}}\in S\setminus E:A^{{}^{\prime}}\succ_{P_{D,i}}A . Let \mathcal{S}_{P_{D,i}} be the set of all separations of S with respect to P_{D,i} and \mathcal{S}_{R_{D}}=\bigcap_{i\in[n]}\mathcal{S}_{P_{D,i}} . A profile R_{D} is called group-separable if for any S\subseteq\mathcal{A} , \mathcal{S}_{R_{D}}\neq\varnothing . Theorem 5.4 . Fix a dataset D\in\mathcal{D} and a finite set of algorithms \mathcal{A} . Let R_{D} be a group-separable profile induced by D\in\mathcal{D} . Assume n is odd (with n\geq 3 ). Then the strict pairwise majority relation \succ_{M}^{D} satisfies Non-Dictatorship, Weak Pareto, IIA and yields a complete and transitive order on \mathcal{A} . 5.3 Distance-Restricted Preferences So far we have investigated two different origins of structure in benchmarks: structure from an axis over models resulting in each metric’s “sweet spot” and structure from clustering of models with respect to their performance on different metrics. Now we deal with structure emerging from a situation of near-consensus among metrics. This is particularly valuable in applications when benchmarking aims at comparing models in terms of a single evaluation criterion (e.g., robust accuracy as used in 35 ) which, however, is too complex to be expressed in terms of a single metric so that it is being approximated by a set of metrics. Definition 5.5 . A profile R_{D} is called distance-restricted to degree p\in\mathbb{N} , if for all i,j\in[n] we have that d_{S}(P_{D,i},P_{D,j})\leq p , where d_{S}(P_{D,i},P_{D,j}) is defined as \left|\left\{(A,A^{{}^{\prime}})\mid A\succ_{P_{D,i}}A^{{}^{\prime}}\text{ and }A^{{}^{\prime}}\succ_{P_{D,j}}A\right\}\right|. The expression d_{S}(P_{D,i},P_{D,j}) is the swap distance counting the number of disagreeing pairs between P_{D,i} and P_{D,j} . The distance d_{S} coincides with the Kendall Tau distance between strict rankings ( 39 ) . In social choice literature, it is also known as the Kemeny distance between two preference orders, measuring the number of pairwise comparisons on which two orders disagree ( 38 ; 37 ; 5 ; 14 ; 16 ) . Interpretation in benchmarking. A p -distance-restricted profile means that any two metrics disagree on at most p pairwise comparison of models. For instance, all three metrics exact_match , prefix_exact_match and quasi_exact_match from HELM measure the fraction of instances where the correct answer matches the model’s prediction. While exact_match is very strict, prefix_exact_match and quasi_exact_match allow for additional tokens in the answer (e.g., explanation, punctuation differences). In situations when metrics assess the same latent construct, we expect a near-consensus across those metrics. They do not disagree at multiple pairwise comparison of models on a fixed dataset at once. This results in almost identical rankings of models, with only a small number of pairwise disagreements, and thus tiny swap distance. In the case of p=1 where any two metrics disagree on at most one pair of models, Condorcet cycles cannot appear and we get the following result. Theorem 5.6 . Fix a dataset D\in\mathcal{D} and a finite set of algorithms \mathcal{A} . Assume the induced profile R_{D} is distance-restricted to degree 1 . Then, the weak pairwise majority relation \succeq_{M}^{D} is transitive and complete relation on \mathcal{A} . It satisfies Non-Dictatorship, Weak Pareto and IIA. 6 Experiments 6.1 Experimental Setup For the verification of restricted preference domains we use HELM MMLU benchmark (v1.0.0) which includes 23 language models evaluated on 57 MMLU subjects. Each subject is a single dataset reporting multiple evaluation metrics per model. For each run directory we identify the MMLU subject as well as the evaluated model and extract the subject-specific mean metric values reported by HELM. When multiple runs exist for the same model-subject pair under different evaluation settings, we keep the most frequent setting and average across repeats. We run our tests on four sets of models and two sets of metrics. \mathcal{A}_{1} includes mostly high-performing models (w.r.t. accuracy); \mathcal{A}_{2} is an intermediate set consisting of high-performing models and mid-range open models; \mathcal{A}_{3}\subset\mathcal{A}_{2} represents a mixed set combining one high-performing model with three smaller open models; \mathcal{A}_{4}\subset\mathcal{A}_{1} contains five high-performing models (see Appendix A.4 for the full list of models). The set \Phi_{acc} consists of three closely related accuracy metrics exact_match , prefix_exact_match , quasi_exact_match . \Phi_{mix} includes two accuracy metrics exact_match , quasi_exact_match and one efficiency metric inference_runtime . Metrics are considered as voters, while models act as alternatives. For each subject and each metric we sort the fixed models by their scores to obtain a preference ranking. If two models have the same score, we apply a deterministic rule to resolve the conflict, sorting the models alphabetically by their model identifier. For each MMLU subject we get a profile of preference rankings over the models. For single-peakedness, we run three tests. First, we focus on (\mathcal{A}_{1},\Phi_{acc}) . For each subject we enumerate all possible permutations of the models that correspond to potential axes and check whether each metric’s ranking is single-peaked with respect to that axis. In the second and third test, we use (\mathcal{A}_{3},\Phi_{mix}) and (\mathcal{A}_{2},\Phi_{mix}) correspondingly, and then proceed analogously as above. Using the profiles of strict metrics’ rankings, we test for group separability. Within any set S of models, we repeatedly search for a nontrivial subset E\subset S that all metrics rank entirely above or entirely below S\setminus E . If such a separation exists, we split S into E and S\setminus E and continue on each part until only sets of fewer than three models remain. We examine (\mathcal{A}_{2},\Phi_{acc}) , (\mathcal{A}_{3},\Phi_{mix}) and (\mathcal{A}_{1},\Phi_{mix}) . To test for distance-restrictedness, we compute the swap distance d_{S} between every pair of metrics’ rankings for each dataset. A subject is then declared distance-restricted to degree 1 if the maximum swap distance is at most 1. Here, we run our experiments on (\mathcal{A}_{4},\Phi_{acc}) , (\mathcal{A}_{3},\Phi_{acc}) and (\mathcal{A}_{2},\Phi_{mix}) . Further experiments on benchmark suites such as PMLB and OpenML are reported in Appendix A.5 . For domains which satisfy our structural assumptions, we also compute the overall ranking across all datasets by using the commonality sharing aggregation rule described in Section 4.3 . 6.2 Results In our first test for single-peakedness with (\mathcal{A}_{1},\Phi_{acc}) , we find that the induced profiles are single-peaked on all 57 subjects. This is consistent with the interpretation that the three accuracy metrics measure a shared latent construct, resulting in a restricted preference domain. In the second experiment, for (\mathcal{A}_{3},\Phi_{mix}) , we find that single-peakedness is also satisfied across all 57 datasets. The finding suggests a trade-off between accuracy and inference time: GPT-4-0613 is highly accurate, while other less accurate models such as Llama-2 are faster than GPT-4-0613. For (\mathcal{A}_{2},\Phi_{mix}) single-peakedness holds only for three out of 57 datasets. Compared to the second test, we have now added more high-performing models such as Claude-3-Opus and Claude-3-Sonnet, which erases the single-peaked structure across the preferences induced by the multi-objective metric set \Phi_{mix} . For the first two tests, which satisfy single-peakedness, we aggregate the pairwise majority relations of all datasets into one relation that represents the whole benchmark suite. Figure 2 shows the most representative (in terms of data depth: the deepest) majority relation for the first two experiments. The best performing models according to the most representative ranking are Claude-3-Opus for the first, and GPT-4-0613 for the second experiment. For other tests concerning group separability and distance-restrictedness, the results and the corresponding aggregated rankings can be found in Appendix A.4 . Figure 2 : Aggregated ranking across all MMLU subjects according to the commonality sharing rule for two domains of single-peaked preferences (left: (\mathcal{A}_{1},\Phi_{acc}) , right: (\mathcal{A}_{3},\Phi_{mix}) ). 7 Discussion What conditions must multi-criteria benchmarks satisfy to produce rankings that are both coherent and stable? Our starting point was that the pathologies in Section 3 appear only in the unstructured combinations of rankings. The results of our study on single-peaked, group-separable and distance-restricted preferences on HELM MMLU suggest that meaningful aggregation is possible, recovering Arrow’s axioms under these structural conditions. In particular, we find that whether the induced domain of profiles satisfies one of our sufficient structural assumptions strongly depends on the choice of models and metrics. When metrics aim to measure one underlying capability, comparing strong models such as Claude-3-Opus and GPT-4-0613 with each other may be adequate. Yet, when metrics encode competing objectives, focusing only on highly accurate models can make the domain less structured. In these cases, the model set should be chosen to represent the trade-off. Our study is descriptive: we verify domain restrictions on particular benchmarks under different choices of metrics and models. A natural next step is to examine restricted preference domains on other benchmarks and add inferential guarantees for our results. Another direction is to extend the social choice lens to strategic behavior: restricted preference domains such as single-peakedness can enable strategy-proof aggregation ( 49 ) . This suggests studying when benchmark aggregation is robust to strategic behavior (e.g., by optimization against the benchmark) and how structure in the rankings can be used to design evaluation methods whose conclusions remain stable under such incentives. Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here. Appendix A Appendix A.1 Single-Peaked Preferences Fix a dataset D , a set of models \mathcal{A} , and a multivariate performance measure \Phi=(\phi_{1},\dots,\phi_{n}) . After applying a fixed deterministic tie-breaking rule, each metric i induces a strict linear order P_{i} on \mathcal{A} . Lemma A.1 . ( 9 ) Let R_{D} be a single-peaked profile. Then the pairwise majority relation \succeq_{M}^{D} is transitive. Proof. Since R_{D} is single-peaked, there exists an ordering L over \mathcal{A} with L\in\mathcal{L}_{\mathcal{A}}(R_{D}) . By 9 the pairwise majority is transitive for profiles that are single-peaked with respect to a common axis. Hence, \succeq_{M}^{D} is transitive. ∎ Theorem A.2 . Let R_{D} be a single-peaked profile induced by dataset D\in\mathcal{D} . Then the pairwise majority relation \succeq_{M}^{D} satisfies Non-Dictatorship, Weak Pareto, IIA and yields a complete and transitive relation on \mathcal{A} . Proof. The pairwise majority relation \succeq_{M}^{D} is complete by definition. By Lemma A.1 it is transitive. Furthermore, the relation \succeq_{M}^{D} satisfies IIA by construction. Weak Pareto holds because if every metric prefers model A to model A^{{}^{\prime}} , then the pairwise majority count for A over A^{{}^{\prime}} is n , and A\succ_{M}^{D}A^{{}^{\prime}} . For Non-Dictatorship consider the following. Fix a metric i . Since R_{D} is single-peaked, there exists an ordering L over \mathcal{A} with L\in\mathcal{L}_{\mathcal{A}}(R_{D}) . Assume that three models A,A^{{}^{\prime}},A^{{}^{\prime\prime}} are ordered on L such that A\succ_{L}A^{{}^{\prime}}\succ_{L}A^{{}^{\prime\prime}} . Further assume that a metric i has its peak at A so that A\succ_{P_{D,i}}A^{{}^{\prime}}\succ_{P_{D,i}}A^{{}^{\prime\prime}} and every other metric j\neq i has its peak at A^{{}^{\prime\prime}} so that A^{{}^{\prime\prime}}\succ_{P_{D,j}}A^{{}^{\prime}}\succ_{P_{D,j}}A . Then, pairwise majority over all metrics does not rank A over A^{{}^{\prime\prime}} . It follows that metric i is not a dictator. Since i was arbitrarily chosen, no dictator exists. ∎ A.2 Group-Separable Preferences Fix a dataset D , a set of models \mathcal{A} , and a multivariate performance measure \Phi=(\phi_{1},\dots,\phi_{n}) . After applying a fixed deterministic tie-breaking rule, each metric i induces a strict linear order P_{i} on \mathcal{A} . Lemma A.3 . ( 31 ; 32 ) Let R_{D} be a group-separable profile. Assume n is odd (with n\geq 3) . Then the strict pairwise majority relation \succ_{M}^{D} is transitive. Proof. By 6 , the profile R_{D} is medium-restricted, since this condition is implied by group separability. By 31 ; 32 , the pairwise majority relation is transitive for a medium-restricted profile, given an odd number of metrics under consideration. Hence, \succ_{M}^{D} is transitive. ∎ Theorem A.4 . Let R_{D} be a group-separable profile induced by dataset D\in\mathcal{D} . Assume n is odd (with n\geq 3 ). Then the pairwise majority relation \succ_{M}^{D} satisfies Non-Dictatorship, Weak Pareto, IIA and yields a complete and transitive order on \mathcal{A} . Proof. Fix A\neq A^{{}^{\prime}} . Since each metric induces a strict linear order P_{i} , for every i\in[n] , exactly one of A\succ_{P_{i}}A^{{}^{\prime}} and A^{{}^{\prime}}\succ_{P_{i}}A holds. Hence, the two majority counts sum up to n . Since n is odd, either A\succ_{M}^{D}A^{{}^{\prime}} or A^{{}^{\prime}}\succ_{M}^{D}A holds. It follows that the pairwise majority relation \succ_{M}^{D} is complete. By Lemma A.3 it is transitive. The relation \succ_{M}^{D} fulfills the condition IIA by construction. The condition Weak Pareto is satisfied because if every metric prefers model A to model A^{{}^{\prime}} , then the pairwise majority count for A over A^{{}^{\prime}} is exactly n , and, thus, A\succ_{M}^{D}A^{{}^{\prime}} . For Non-Dictatorship consider the following. Fix a metric i . Consider three models A,A^{{}^{\prime}},A^{{}^{\prime\prime}}\in\mathcal{A} . Assume that metric i ranks A\succ_{P_{D,i}}A^{{}^{\prime}}\succ_{P_{D,i}}A^{{}^{\prime\prime}} and every other metric j\neq i ranks A^{{}^{\prime}}\succ_{P_{D,j}}A\succ_{P_{D,j}}A^{{}^{\prime\prime}} . Since A^{{}^{\prime\prime}} is ranked consistently at the bottom by every metric, \{A^{{}^{\prime\prime}}\} is a separation of \{A,A^{{}^{\prime}},A^{{}^{\prime\prime}}\} with respect to P_{D,l} for all l\in[n] . Consider all other models Z\in\mathcal{A}\setminus\{A,A^{{}^{\prime}},A^{{}^{\prime\prime}}\} . Fix Z=\{Z_{1},\dots Z_{q}\} and assume Z_{1}\succ_{P_{D,l}}\dots\succ_{P_{D,l}}Z_{q} for all l\in[n] , with all Z ranked above A,A^{{}^{\prime}},A^{{}^{\prime\prime}} for every l . Thus, the resulting profile is group-separable by definition. The pairwise majority over all metrics ranks A^{{}^{\prime}} over A . Therefore, metric i is not a dictator. Since i was arbitrarily chosen, no dictator exists. ∎ A.3 Distance-Restricted Preferences Fix a dataset D , a set of models \mathcal{A} , and a multivariate performance measure \Phi=(\phi_{1},\dots,\phi_{n}) . Assume n\geq 3 . After applying a fixed deterministic tie-breaking rule, each metric i induces a strict linear order P_{i} over the finite set of models \mathcal{A} . Let P_{i},P_{j} be such strict linear orders on \mathcal{A} with P_{i}:=P_{D,i} for all i . Let d_{S}(P_{i},P_{j}) be the number of pairwise comparisons between models in \mathcal{A} on which P_{i} and P_{j} disagree. Assume the induced profile R_{D} is 1 -distance-restricted so that d_{S}(P_{i},P_{j})\leq 1 for all i,j . Lemma A.5 . If d_{S}(P_{i},P_{j})\leq 1 , then either P_{i}=P_{j} or P_{j} is obtained from P_{i} by swapping exactly two adjacent models in P_{i} . Proof. Assume d_{S}(P_{i},P_{j})\leq 1 and P_{i}\neq P_{j} . Then there exists a pair of models A,A^{{}^{\prime}}\in\mathcal{A} such that P_{i} ranks A above A^{{}^{\prime}} and P_{j} ranks A^{{}^{\prime}} above A . If models A and A^{{}^{\prime}} are not adjacent in the order P_{i} , then there must exist at least one model A^{{}^{\prime\prime}} that lies between A and A^{{}^{\prime}} in P_{i} . For P_{j} to rank A^{{}^{\prime}} above A , A^{{}^{\prime}} must move past A^{{}^{\prime\prime}} or A must move past A^{{}^{\prime\prime}} . This, however, changes at least one additional pairwise comparison relative to P_{i} . We get two pairs on which P_{i} and P_{j} disagree which contradicts d_{S}(P_{i},P_{j})\leq 1 . Therefore, A and A^{{}^{\prime}} are adjacent in P_{i} . Since there is only one disagreement, P_{j} is obtained from P_{i} by swapping exactly that pair of models in P_{i} . ∎ Lemma A.6 . Fix an order P:=P_{1} such that d_{S}(P_{i},P_{1})\leq 1 for all i . There exists a pair A,A^{{}^{\prime}}\in\mathcal{A} such that for all i either P_{i}=P_{1} or P_{i} is obtained from P_{1} by swapping the pair A,A^{{}^{\prime}} . Proof. By d_{S}(P_{i},P_{1})\leq 1 and Lemma A.5 , we must have either P_{i}=P_{1} or P_{i} is obtained from P_{1} by swapping exactly one adjacent pair of models in P_{1} . Assume for contradiction that there exist metrics i\neq j such that P_{i} is obtained from P_{1} by swapping one adjacent pair A,A^{{}^{\prime}} , and that P_{j} is obtained from P_{1} by swapping a different pair of models Q,Q^{{}^{\prime}} . Then, P_{i} and P_{j} disagree on the comparison between A and A^{{}^{\prime}} and, additionally, on the comparison between Q and Q^{{}^{\prime}} . Thus, there are two pairs of models on which P_{i} and P_{j} disagree, which contradicts the assumption d_{S}(P_{i},P_{j})\leq 1 for all i,j . Therefore, all orders other than P_{1} must swap the same adjacent pair of models A,A^{{}^{\prime}} . ∎ Lemma A.7 . The pairwise majority relation \succeq_{M}^{D} is transitive. Proof. By Lemma A.6 , for all i , P_{i} coincides with P_{1} on every pairwise comparison except possibly the single pair of models A,A^{{}^{\prime}} . It follows that for any pair of models Z,Z^{{}^{\prime}}\in\mathcal{A} different from A,A^{{}^{\prime}} , every metric agrees on the ranking of Z compared to Z^{{}^{\prime}} . Thus, majority agrees on the ranking of Z compared to Z^{{}^{\prime}} as well and ranks both models exactly the same way as in the order P_{1} . Therefore, the only pair on which the majority outcome might differ from P_{1} is A,A^{{}^{\prime}} . The majority can either rank A above A^{{}^{\prime}} or A^{{}^{\prime}} above A or tie them. In all cases, for every third model Z , the comparisons between Z and A and between Z and A^{{}^{\prime}} are exactly as in P_{1} . Since at most one pairwise comparison can differ from P_{1} , a cycle involving three distinct models cannot exist. Hence, the pairwise majority relation \succeq_{M}^{D} is transitive. ∎ Theorem A.8 . Fix a dataset D\in\mathcal{D} and a finite set of algorithms \mathcal{A} . Assume the induced profile R_{D} is distance-restricted to degree 1 . Then, the pairwise majority relation \succeq_{M}^{D} is transitive and complete relation on \mathcal{A} . It satisfies Non-Dictatorship, Weak Pareto and IIA. Proof. The relation \succeq_{M}^{D} is complete by definition. By Lemma A.7 , \succeq_{M}^{D} is transitive. The relation \succeq_{M}^{D} fulfills IIA by construction. Weak Pareto holds because if all metrics strictly prefer model A to model A^{{}^{\prime}} , then the pairwise majority count for A over A^{{}^{\prime}} is exactly n and A\succ_{M}^{D}A^{{}^{\prime}} . For Non-Dictatorship consider the following. Fix a metric i . Consider the fixed order P_{1} . Let A,A^{{}^{\prime}} be the pair of models from Lemma A.6 such that the only possible deviation from P_{1} is swapping A and A^{{}^{\prime}} . Consider a preference profile where metric i ranks A over A^{{}^{\prime}} and every other metric j\neq i ranks A^{{}^{\prime}} over A . Since the orders differ on at most one swapped pair, the profile is distance-restricted to degree 1 . The pairwise majority over all metrics does not rank A over A^{{}^{\prime}} : metric i ranks A over A^{{}^{\prime}} but the collective preference over all metrics is A^{{}^{\prime}} over A . Hence, metric i is not a dictator. Since i was arbitrarily chosen, there is no dictator. ∎ A.4 Experiments on Restricted Preference Domains: HELM MMLU Model sets. The models included in the sets \mathcal{A}_{1} , \mathcal{A}_{2} , \mathcal{A}_{3} , \mathcal{A}_{4} from the Section 6 are as follows. Model set \mathcal{A}_{1}=\{ GPT-4-0613,GPT-4-1106-Preview, Claude-3-Opus-20240229, Claude-3-Sonnet-20240229, Qwen1.5-72b, Llama-2-70b, Claude-3-Haiku-20240307, Yi-34b, Yi-6b \} . Model set \mathcal{A}_{2}=\{ GPT-4-0613,GPT-4-1106-Preview, Claude-3-Opus-20240229, Claude-3-Sonnet-20240229, Qwen1.5-72b, Llama-2-70b, Mistral-7b-v0.1 \} . Model set \mathcal{A}_{3}=\{ GPT-4-0613, Qwen1.5-72b, Llama-2-70b, Mistral-7b-v0.1 \} . Model set \mathcal{A}_{4}=\{ GPT-4-0613,GPT-4-1106-Preview, Claude-3-Opus-20240229, Claude-3-Sonnet-20240229, Qwen1.5-72b \} . Group-separable preferences. For (\mathcal{A}_{2},\Phi_{acc}) and (\mathcal{A}_{3},\Phi_{mix}) , we find group separability fulfilled across all 57 MMLU subjects. In the third test, for (\mathcal{A}_{1},\Phi_{mix}) , we observe group separability only in six out of 57 datasets. Distance-restricted preferences. For (\mathcal{A}_{4},\Phi_{acc}) and (\mathcal{A}_{3},\Phi_{acc}) , we find that all 57 MMLU subjects are distance-restricted to degree 1. On the contrary, for (\mathcal{A}_{2},\Phi_{mix}) , none of the 57 subjects is distance-restricted to degree 1. The addition of the efficiency metric causes rankings to disagree on several pairwise comparisons of models, resulting in larger swap distance. The results of all tests on single-peakedness, group separability and distance-restrictedness remain (nearly 7 7 7 The only test whose outcome changes after switching to another tie-breaking rule is the group separability experiment for (\mathcal{A}_{1},\Phi_{mix}) : Under alphabetical tie-breaking, six out of 57 datasets are group-separable; under the reverse alphabetical tie-breaking, five out of 57 datasets are group-separable. ) the same after switching to the reverse alphabetical tie-breaking rule. Aggregation across datasets. For the experiments on group separability and distance restrictedness, Figure 3 shows for all settings, for which the suite constitutes a Condorcet domain, the commonality sharing aggregate of all rankings in the suite. The commonality sharing ranking can be understood as a representative ranking in the whole benchmark suite. In statistical terms, it is some kind of a generalization of the concept of a median to the case of data sets, where each data point is a ranking. The computation of the deepest relation is done by calculating the Tukey depth of all rankings. Concretely, we use the R package ddandrda ( 10 ) . Figure 3 : Aggregated ranking across all 57 datasets of HELM MMLU according to the commonality sharing rule (cf., Section 4.3 ) for the experiments for group separability (left, (\mathcal{A}_{2},\Phi_{acc}) and (\mathcal{A}_{3},\Phi_{mix}) ) and for distance-restrictedness (right, (\mathcal{A}_{4},\Phi_{acc}) and (\mathcal{A}_{3},\Phi_{acc}) ). A.5 Experiments on Restricted Preference Domains: PMLB and OpenML For our verifications of restricted preference domains on PMLB and OpenML benchmarks we use the final results from 35 and then proceed exactly as in the tests on HELM MMLU. PMLB. The benchmark suite includes six classifiers (cre, J48, glmnet, knn, ranger, svmRadial) evaluated on three metrics (predictive accuracy, robustness to feature perturbation and robustness to label perturbation) over 63 datasets. We find that seven out of 63 datasets are single-peaked. For group separability, we observe that 11 out of 63 datasets are group-separable. The results remain very similar after switching to the reverse alphabetical tie-breaking. Since the metrics measure related aspects of one latent concept (robust accuracy), it is plausible that we observe some single-peaked and group-separable datasets; however, the metrics are still different enough so that there is non-trivial disagrement between rankings which leads to multiple violations of our structural assumptions. OpenML. The suite contains seven classifiers (classif.glmnet, classif.kknn, classif.multinom, classif.ranger, classif.rpart, classif.svm, classif.xgboost) evaluated on 80 datasets and three metrics (predictive accuracy, computation time on training data and computation time on test data). We find that none of 80 datasets is single-peaked, and four out of 80 datasets are group-separable. The results are unchanged under the reverse alphabetical tie-breaking. The combination of accuracy with training and test runtime induces a strong trade-off, meaning that different metrics disagree on many pairwise comparisons, which implies less structured preference profiles. A.6 Experiments: Coherence and Stability Condorcet Cycles. We preprocess the HELM MMLU data exactly as in Section 6.1 . We align metrics so that larger values mean better performance by flipping the sign for “lower-is-better” metrics such as Inference time and Output length . Then, for each subject, we search over all triples of metrics in the fixed set \{ exact_match , inference_runtime , num_bytes , logprob , perplexity , num_output_tokens \} . Within a metric triple, we restrict to models with complete observations and define pairwise majority comparisons. For each pair of models, each metric “votes” only when the two aligned scores differ by more than a small tolerance; ties and near-tie situations are treated as abstentions. Then, we search for cycles i.e., triples of models A,A^{{}^{\prime}},A^{{}^{\prime\prime}} such that A is preferred to A^{{}^{\prime}} , A^{{}^{\prime}} is preferred to A^{{}^{\prime\prime}} , and A^{{}^{\prime\prime}} is preferred to A . A subject is counted as cyclic if at least one metric triple yields such a cycle. For each subject, we keep the most robust witnessed cycle and assign it a buffer. For each of the three pairwise majority wins in the cycle, we take the smallest score gap among the metrics supporting that win so that the cycle’s buffer is the minimum of these three values. Larger buffers therefore correspond to cycles that persist under small perturbations of the metric values. Under this procedure, we find at least one Condorcet cycle in 51 of 57 MMLU subjects. This suggests that cyclic preferences are widespread once accuracy and efficiency metrics are combined. The full model names for the cycle presented in Table 1 are GPT-3.5-Turbo-0613, GPT-4-1106-Preview and Qwen1.5-14b. Note that in the experiments in Section 6.1 (also covered in Appendix A.4 ) the model name Qwen1.5 is used as the abbreviation for the model Qwen1.5-72b. Instability to changes in the model set. For each MMLU subject, we fix the metric set \{ exact_match , inference_runtime , num_bytes \} (flipping the sign for “lower-is-better” metrics as above). For each subject, we restrict to models with complete observations and focus only on the top 15 models by the metric exact_match . For each subject, we compute the overall ranking by averaging the metrics’ ranks, then add one additional model and recompute the ranking. We record whether any pairwise order among the original 15 models flips under this change. We find such flips in 44 out of 57 MMLU subjects. The full model names for all models mentioned in Table 2 are as follows: GPT-4-0613, GPT-4-1106-Preview, Claude-3-Opus-20240229, Qwen1.5-72b, GPT-3.5-Turbo-0613, Google-Text-Bison@001 and Llama-2-13b. Note that in the experiments in Section 6.1 (also covered in Appendix A.4 ) the model name Llama-2 is used as the abbreviation for the model Llama-2-70b.