AudioRAG: A Challenging Benchmark for Audio Reasoning and Information Retrieval

Jingru Lin, Chen Zhang, Tianrui Wang, Haizhou Li · 2026

Read AudioRAG: A Challenging Benchmark for Audio Reasoning and Information Retrieval by Jingru Lin, Chen Zhang, Tianrui Wang, Haizhou Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10656.txt.

Description

Jingru Lin and three co-authors benchmark audio thinking that needs outside information in a 2026 paper. Audio-language models perform well on sound, speech, and music tasks, but current benchmarks test only internal knowledge. Real questions often demand retrieval: naming a sound may require looking something up.

Their benchmark, AudioRAG, mixes generated and manually curated question-answer pairs in realistic web settings, testing thinking augmented by retrieval. Tests show even leading audio-language models struggle with these questions. The authors also propose an agentic pipeline combining audio thinking with retrieval-augmented making as a stronger baseline. The benchmark sets a harder, more realistic bar for audio intelligence.

Catalog record

CREATORJingru Lin, Chen Zhang, Tianrui Wang, Haizhou Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-10656

Browse the full library · About the Tower of Babel