BenchScope: How Many Independent Signals Does Your Benchmark Provide?
Tommy Sha *, Stella Zhao * · 2026
Read BenchScope: How Many Independent Signals Does Your Benchmark Provide? by Tommy Sha *, Stella Zhao * (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29357.txt.
Description
Sha and Zhao ask how many independent signals a benchmark really gives. Score suites often report many numbers that repeat the same information. They define Effective Dimensionality from the spectrum of benchmark scores. It acts as a fast upper-bound check on measurement breadth. They applied it to 22 benchmarks and over 8,400 model runs. The six-score Open LLM Leaderboard behaved like about two axes. BBH and MMLU-Pro carried near-identical signal. The authors release an atlas and a four-step workflow.
Catalog record
| CREATOR | Tommy Sha *, Stella Zhao * |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80808 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.29357