AudioRAG: A Challenging Benchmark for Audio Reasoning and Information Retrieval
Jingru Lin, Chen Zhang, Tianrui Wang, Haizhou Li · 2026
Read AudioRAG: A Challenging Benchmark for Audio Reasoning and Information Retrieval by Jingru Lin, Chen Zhang, Tianrui Wang, Haizhou Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10656.txt.
Description
Jingru Lin and three co-authors benchmark audio thinking that needs outside information in a 2026 paper. Audio-language models perform well on sound, speech, and music tasks, but current benchmarks test only internal knowledge. Real questions often demand retrieval: naming a sound may require looking something up.
Their benchmark, AudioRAG, mixes generated and manually curated question-answer pairs in realistic web settings, testing thinking augmented by retrieval. Tests show even leading audio-language models struggle with these questions. The authors also propose an agentic pipeline combining audio thinking with retrieval-augmented making as a stronger baseline. The benchmark sets a harder, more realistic bar for audio intelligence.
Catalog record
| CREATOR | Jingru Lin, Chen Zhang, Tianrui Wang, Haizhou Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-10656