VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models
Wenqian Cui, Xiaoqi Jiao, Ziqiao Meng, Irwin King · 2025
Read VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models by Wenqian Cui, Xiaoqi Jiao, Ziqiao Meng, Irwin King (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04962.txt.
Description
Wenqian Cui, Xiaoqi Jiao, Ziqiao Meng, and Irwin King introduce VoxEval in a 2025 arXiv paper. VoxEval is a benchmark that tests spoken language models on knowledge questions through speech alone. Older question-answering tests cannot judge models that take in and reply with speech.
The benchmark keeps both input and output in speech form. It varies audio conditions to test robustness, and it even poses math reasoning tasks in spoken form. Tests show current spoken models struggle with the benchmark. Scores shift with audio quality, and reasoning is a clear weak point. The authors offer the set to guide work on stronger spoken models.
Catalog record
| CREATOR | Wenqian Cui, Xiaoqi Jiao, Ziqiao Meng, Irwin King |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-04962