How should AI Safety Benchmarks Benchmark Safety?

Cheng Yu, Severin Engelmann, Ruoxuan Cao, Dalia Ali, Orestis Papakyriakopoulos · 2026

Read How should AI Safety Benchmarks Benchmark Safety? by Cheng Yu, Severin Engelmann, Ruoxuan Cao, Dalia Ali, Orestis Papakyriakopoulos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23112.txt.

Description

Cheng Yu and four co-authors ask how AI safety benchmarks should themselves be judged. Safety benchmarks are pivotal checks on advanced AI systems. Yet the authors find deep technical, epistemic, and social shortcomings across the field.

Their review covers more than two hundred safety benchmarks and maps the common failures. They draw on engineering science and long-standing theories of risk and safety for guidance. If the measuring tools are flawed, safety claims built on them cannot be trusted. Engineering sciences and established risk theory supply the standards the review uses to judge the benchmark field. Shortcomings are grouped as technical, epistemic, and sociotechnical, a map meant to guide repairs.

Catalog record

CREATORCheng Yu, Severin Engelmann, Ruoxuan Cao, Dalia Ali, Orestis Papakyriakopoulos
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-23112

Browse the full library · About the Tower of Babel