DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models
Utkarsh Tiwari*Aryan Seth*Adi MukherjeeKaavya MerKavish, Dhruv Kumar Birla Institute of Technology and Science, Pilani · 2025
Read DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models by Utkarsh Tiwari*Aryan Seth*Adi MukherjeeKaavya MerKavish, Dhruv Kumar Birla Institute of Technology and Science, Pilani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-06279.txt.
Description
Utkarsh Tiwari and five co-authors build a debate-based benchmark for language models in a 2025 paper. Judging a full competitive debate demands long memory and structured reasoning: eight seven-minute speeches, rules to learn, and scores that must match expert adjudicators. That is exactly the kind of task that strains today's models.
The data set collects 256 speeches from thirty-two British Parliamentary debates at top tournaments, each debate over an hour and about thirty-two thousand tokens, with official speech scores and rankings attached. Models must learn the format from context, follow every argument, and produce final standings. Leading models, including top reasoning systems, struggled in early tests. The authors offer the benchmark as a hard target for long-context reasoning research. Debate, they note, compresses argument, rebuttal, and judgment into a single measurable skill.
Catalog record
| CREATOR | Utkarsh Tiwari*Aryan Seth*Adi MukherjeeKaavya MerKavish, Dhruv Kumar Birla Institute of Technology and Science, Pilani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-06279