Building Russian Benchmark for Evaluation of Information Retrieval Models
2025
Read Building Russian Benchmark for Evaluation of Information Retrieval Models (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12879.txt.
Description
Grigory Kovalev and co-authors present RusBEIR, a benchmark for Russian information retrieval, in this 2025 paper. Retrieval models are usually tested in English. Russian is morphologically rich, which changes how word based search behaves. A shared test set is needed for fair comparison.
RusBEIR gathers 17 datasets from varied domains. Some were adapted, some translated, and some built new for this benchmark. Word based and neural models were compared in a zero shot setting. Careful preprocessing mattered greatly for word based models. A classic method, BM25, stayed a strong baseline on full documents. Neural models won on most sets but struggled with long documents due to input limits. The framework is open source.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-12879