DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models

Utkarsh Tiwari*Aryan Seth*Adi MukherjeeKaavya MerKavish, Dhruv Kumar Birla Institute of Technology and Science, Pilani · 2025

Read DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models by Utkarsh Tiwari*Aryan Seth*Adi MukherjeeKaavya MerKavish, Dhruv Kumar Birla Institute of Technology and Science, Pilani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-06279.txt.

Description

Utkarsh Tiwari and five co-authors build a debate-based benchmark for language models in a 2025 paper. Judging a full competitive debate demands long memory and structured reasoning: eight seven-minute speeches, rules to learn, and scores that must match expert adjudicators. That is exactly the kind of task that strains today's models.

The data set collects 256 speeches from thirty-two British Parliamentary debates at top tournaments, each debate over an hour and about thirty-two thousand tokens, with official speech scores and rankings attached. Models must learn the format from context, follow every argument, and produce final standings. Leading models, including top reasoning systems, struggled in early tests. The authors offer the benchmark as a hard target for long-context reasoning research. Debate, they note, compresses argument, rebuttal, and judgment into a single measurable skill.

Catalog record

CREATORUtkarsh Tiwari*Aryan Seth*Adi MukherjeeKaavya MerKavish, Dhruv Kumar Birla Institute of Technology and Science, Pilani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-06279

Browse the full library · About the Tower of Babel