Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR
Julian Irigoyen, Arthur Söhler, Andreas Søeborg Kirkedal · 2025
Read Pruning as Regularization: Sensitivity-Aware One-Shot Pruning in ASR by Julian Irigoyen, Arthur Söhler, Andreas Søeborg Kirkedal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08092.txt.
Description
Julian Irigoyen, Arthur Sohler, and Andreas Kirkedal argue that pruning a speech recognition model can act as regularization, not just compression. Removing weights usually aims to shrink a network, yet chosen well it may also improve generalization. Using Whisper small, they combine gradient and Fisher sensitivity checks with targeted pruning of single components.
Some parts prove fragile, especially decoder feed forward layers, while decoder attention and late encoder layers hold spare capacity. Pruning half of those safer parts, with no fine tuning, cut word error rates on LibriSpeech and gains held on other sets. At forty percent sparsity their approach preserves accuracy where global pruning fails badly.
Catalog record
| CREATOR | Julian Irigoyen, Arthur Söhler, Andreas Søeborg Kirkedal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-08092