Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
Fabian David Schmidt, Ivan Vulić, Goran Glavaš, David Ifeoluwa Adelani · 2025
Read Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding by Fabian David Schmidt, Ivan Vulić, Goran Glavaš, David Ifeoluwa Adelani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06117.txt.
Description
Fabian David Schmidt and co-authors build a speech understanding test that spans more than a hundred languages, in a 2025 paper. Half the world's languages lack writing, so speech systems must grasp meaning straight from sound. Evaluation, though, has stuck to shallow tasks in few languages.
Their benchmark, Fleurs-SLU, holds 692 hours of speech for topic sorting in 102 languages. A listening quiz adds 944 hours across 92 languages. The team graded direct speech models, transcribe-then-read pipelines, and speech-aware language models. Pipelines proved sturdiest overall, while strong speech encoders rivaled them at topic sorting. Closed speech models matched or beat the pipelines. Scores tracked how reliable speech recognition is per language.
Catalog record
| CREATOR | Fabian David Schmidt, Ivan Vulić, Goran Glavaš, David Ifeoluwa Adelani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-06117