ProfASR-Bench: A Professional-talk ASR Dataset for High-Stakes Applications Exposing the Context-Utilization Gap
Deepak Babu Piskala · 2025
Read ProfASR-Bench: A Professional-talk ASR Dataset for High-Stakes Applications Exposing the Context-Utilization Gap by Deepak Babu Piskala (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23686.txt.
Description
Deepak Babu Piskala releases ProfASR-Bench, a benchmark for speech recognition in professional settings where mistakes carry real costs. In finance, medicine, law, and technology, speech is dense with specialized terms and formal registers, and an error on a critical name or number is far worse than an average word error. Existing benchmarks underplay those pressures. The arXiv record's title and framing govern over the catalog entry.
Each test item pairs a natural-language context prompt, a domain cue or speaker profile, with an entity-rich utterance to transcribe. Scoring covers ordinary word error plus entity-aware measures, reported by accent and gender slices. Testing Whisper and Qwen-Omni families under several context conditions, Piskala finds a context-utilization gap: even perfect oracle context barely improves recognition, meaning today's promptable systems underuse side information they are given.
Catalog record
| CREATOR | Deepak Babu Piskala |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-23686