Selective Attention Merging for low resource tasks: A case study of Child ASR
Natarajan Balaji Shankar, Zilai Wang, Eray Eren, Abeer Alwan · 2025
Read Selective Attention Merging for low resource tasks: A case study of Child ASR by Natarajan Balaji Shankar, Zilai Wang, Eray Eren, Abeer Alwan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08468.txt.
Description
Natarajan Balaji Shankar, Zilai Wang, Eray Eren, and Abeer Alwan merge speech models with care to sharpen child speech recognition, in work from twenty twenty-five. Big speech models shine on adult speech but falter on children's voices, since child data is scarce. Merging models trained on richer speech could fill the gap.
Their method, Selective Attention Merge, blends only the attention parts of task vectors from donor models. On a standard child speech database, word error rates fell by up to fourteen percent relative. The method beat older merging tricks and data augmentation alone. Combined with augmentation, it set a new best score for a small Whisper model on that database.
Catalog record
| CREATOR | Natarajan Balaji Shankar, Zilai Wang, Eray Eren, Abeer Alwan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-08468