Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarking

Polina Gordienko, Christoph Jansen, Julian Rodemann, Georg Schollmeyer · 2026

Read Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarking by Polina Gordienko, Christoph Jansen, Julian Rodemann, Georg Schollmeyer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07593.txt.

Description

Benchmarks that rank models across accuracy, robustness, and efficiency often produce incoherent or unstable rankings when the model set changes. Polina Gordienko and three co-authors formalize multi-criteria benchmarking as a vote problem: each metric ranks models, and an operator aggregates the votes.

Rather than stopping at Arrow's impossibility result, they name sufficient settings under which it fades. On single-peaked, group-separable, and distance-restricted preference combinations, it supports well-behaved pooling. Evaluation teams get a constructive answer to a foundational worry, with settings under which multi-metric rankings can be trusted. The checks cover several modern suites, including HELM and MMLU.

Catalog record

CREATORPolina Gordienko, Christoph Jansen, Julian Rodemann, Georg Schollmeyer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07593

Browse the full library · About the Tower of Babel