Dave : D iagnostic benchmark for A udio V isual E valuation

Gorjan Radevski, Teodora Popordanoska, Matthew B. Blaschko, Tinne Tuytelaars · 2025

Read Dave : D iagnostic benchmark for A udio V isual E valuation by Gorjan Radevski, Teodora Popordanoska, Matthew B. Blaschko, Tinne Tuytelaars (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-09321.txt.

Description

Models that answer questions about video must truly use both the picture and the sound. Yet many benchmarks let them pass on vision alone, and their single scores hide where the failure lies. Gorjan Radevski and co-authors built DAVE, a diagnostic benchmark for audio visual evaluation. Its questions are constructed so that both channels are needed to answer correctly. The scoring is split into small, atomic categories, so a model can be shown to fail at hearing, at seeing, or at aligning the two.

Testing leading models on DAVE exposed specific failure modes that ordinary benchmarks blur. The authors offer the dataset and framework as a standard diagnostic tool, so developers can aim repairs at the true weak point instead of chasing a single aggregate number. The benchmark is public, with data and code released for other teams to use.

Catalog record

CREATORGorjan Radevski, Teodora Popordanoska, Matthew B. Blaschko, Tinne Tuytelaars
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-09321

Browse the full library · About the Tower of Babel