Benchmarking Information Retrieval Models on Complex Retrieval Tasks Conference: ; ;

Julian Killingback, Hamed Zamani · 2025

Read Benchmarking Information Retrieval Models on Complex Retrieval Tasks Conference: ; ; by Julian Killingback, Hamed Zamani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07253.txt.

Description

Julian Killingback and 1 co-author argue in 2025 that search models need harder tests. Retrieval models find documents that match a query, and most benchmarks still use short, single-topic queries. Real users, shaped by chat-style AI tools, now ask complex questions with several parts, limits, and conditions at once. Few public test sets measure how retrieval handles such requests in realistic settings.

To close the gap, the authors build a diverse set of complex retrieval tasks and run a representative group of leading retrieval models on them. The benchmark is meant to show the true capability of each model, not just its skill on easy queries. Better measurement matters because retrieval quietly underpins search engines, research tools, and AI assistants. The authors hope the new tasks spur a generation of retrieval models built for real, messy information needs.

Catalog record

CREATORJulian Killingback, Hamed Zamani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE153 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07253

Browse the full library · About the Tower of Babel