Quantizing Whisper-small: How design choices affect ASR performance
Arthur Söhler, Julian Irigoyen, Andreas Søeborg Kirkedal · 2025
Read Quantizing Whisper-small: How design choices affect ASR performance by Arthur Söhler, Julian Irigoyen, Andreas Søeborg Kirkedal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08093.txt.
Description
Arthur Sohler, Julian Irigoyen, and Andreas Kirkedal compare ways to quantize Whisper small for speech recognition on small devices. Large models are accurate but demand too much computation for edge hardware. Post training quantization lowers precision after training, and many design choices compete across four software libraries.
Experiments on LibriSpeech clean and other sets separate the effects of scheme, method, granularity, and bit width. Dynamic eight bit quantization with Quanto gave the best trade, shrinking the model by over half while improving word error rate. Aggressive formats compressed further but lost accuracy, especially in noisy conditions.
Catalog record
| CREATOR | Arthur Söhler, Julian Irigoyen, Andreas Søeborg Kirkedal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-08093