CoverageBench: Evaluating Information Coverage across Tasks and Domains DOI: 10.1145/1122445.1122456 Conference: SIGIR ’26: International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Naarm, Australia SIGIR ’26: ACM SIGIR Conference on Research and Development in Information Retrieval, July 20–24, 2026, Naarm, Australia CCS: Information systems Information retrieval diversity CCS: Information systems Summarization CCS: Computing methodologies Natural language generation
Saron Samuel, Andrew Yates, Dawn Lawrie, Ian Soboroff, Trevor Adriaanse, Benjamin Van Durme, Eugene Yang · 2026
Read CoverageBench: Evaluating Information Coverage across Tasks and Domains DOI: 10.1145/1122445.1122456 Conference: SIGIR ’26: International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Naarm, Australia SIGIR ’26: ACM SIGIR Conference on Research and Development in Information Retrieval, July 20–24, 2026, Naarm, Australia CCS: Information systems Information retrieval diversity CCS: Information systems Summarization CCS: Computing methodologies Natural language generation by Saron Samuel, Andrew Yates, Dawn Lawrie, Ian Soboroff, Trevor Adriaanse, Benjamin Van Durme, Eugene Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20034.txt.
Description
Samuel and co-authors build CoverageBench, a test suite for measuring information coverage in search. Classic metrics reward finding more relevant documents, but they ignore overlap between documents. High recall alone does not prove the results cover the topic's full range. This gap matters when search feeds generative models in RAG systems. CoverageBench reworks existing collections into a unified testbed. It spans many genres and task types. Topics come with nuggets that mark distinct pieces of needed information. Researchers can now score coverage directly, not just relevance.
Catalog record
| CREATOR | Saron Samuel, Andrew Yates, Dawn Lawrie, Ian Soboroff, Trevor Adriaanse, Benjamin Van Durme, Eugene Yang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34380 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20034