IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval DOI: XXXXXXX.XXXXXXX Conference: SIGIR 2026 Submission; July 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Information retrieval CCS: Information systems Music retrieva
Benjamin Clavié, Atoof Shakir, Jonah Turner, Sean Lee, Aamir Shakir, Makoto P. Kato · 2026
Read IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval DOI: XXXXXXX.XXXXXXX Conference: SIGIR 2026 Submission; July 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Information retrieval CCS: Information systems Music retrieva by Benjamin Clavié, Atoof Shakir, Jonah Turner, Sean Lee, Aamir Shakir, Makoto P. Kato (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11941.txt.
Description
Music retrieval has grown strong, yet it lacks high-quality benchmarks for fair comparison. Benjamin Clavié and five co-authors present IncompeBench, a carefully labeled benchmark of more than 1,500 permissively licensed, high-quality music snippets, 500 diverse queries, and over 125,000 individual relevance judgments. A multi-stage pipeline produced strong agreement between human raters and machine data.
The benchmark targets a gap: existing tests often lean on narrow catalogs or noisy labels, making it hard to tell which systems truly retrieve well. By pairing cleared snippets with many queries and judgments, it supports fine-grained evaluation of music search systems. The datasets and prompts are publicly released, giving researchers a shared, reusable testbed for the next round of music search work.
Catalog record
| CREATOR | Benjamin Clavié, Atoof Shakir, Jonah Turner, Sean Lee, Aamir Shakir, Makoto P. Kato |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11941