Scaling Audio–Text Retrieval with Multimodal Large Language Models
Jilan Xu Carl Thomé Danijela Horak Weidi Xie Andrew Zisserman, [0.6em] Visual Geometry Group, University of Oxford Epidemic Sound · 2026
Read Scaling Audio–Text Retrieval with Multimodal Large Language Models by Jilan Xu Carl Thomé Danijela Horak Weidi Xie Andrew Zisserman, [0.6em] Visual Geometry Group, University of Oxford Epidemic Sound (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18010.txt.
Description
Jilan Xu and colleagues re-purpose multimodal large language models for audio-text retrieval, the task of bridging acoustic signals and natural language. Contrastive dual-encoder systems like CLAP show promise, but small-scale encoders cap their power. Their text encoders especially struggle with complex queries that need reasoning or world knowledge.
The authors propose AuroLA, a contrastive language-audio pretrained model that uses a multimodal large language model as a single backbone for retrieval. A scalable data pipeline curates diverse audio from many sources and generates multi-granular captions, from long descriptions to structured tags, through automated annotation. The model is adapted by prompting it to summarize the audio or text input, with the hidden state of a special token serving as the embedding. In experiments, AuroLA consistently beat leading dual-encoder models, including the recent PE-AV, supporting the unified-backbone approach.
Catalog record
| CREATOR | Jilan Xu Carl Thomé Danijela Horak Weidi Xie Andrew Zisserman, [0.6em] Visual Geometry Group, University of Oxford Epidemic Sound |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-18010