BASS A I : Benchmarking Audio LMs for Musical Structure and Semantic Reasoning

Min Jang, Orevaoghene Ahia, Nazif Tamer, Sachin Kumar, Yulia Tsvetkov, Noah A. Smith · 2026

Read BASS A I : Benchmarking Audio LMs for Musical Structure and Semantic Reasoning by Min Jang, Orevaoghene Ahia, Nazif Tamer, Sachin Kumar, Yulia Tsvetkov, Noah A. Smith (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04085.txt.

Description

Can an audio model reason about music, not just hear it? Min Jang and 5 co-authors built BASS, a benchmark of over twenty-six hundred questions across nearly two thousand songs. Their 2026 paper tests audio language models on structure, lyrics, musicological analysis, and artist collaboration.

Fourteen leading models were evaluated. Lyric transcription came easiest. Structural segmentation and collaboration questions, which demand higher-level reasoning, defeated even the best systems. Models lean on language priors but stumble over musical form and vocal detail. BASS gives the field a public yardstick for a skill current models mostly fake.

Catalog record

CREATORMin Jang, Orevaoghene Ahia, Nazif Tamer, Sachin Kumar, Yulia Tsvetkov, Noah A. Smith
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-04085

Browse the full library · About the Tower of Babel