Selective Attention Merging for low resource tasks: A case study of Child ASR

Natarajan Balaji Shankar, Zilai Wang, Eray Eren, Abeer Alwan · 2025

Read Selective Attention Merging for low resource tasks: A case study of Child ASR by Natarajan Balaji Shankar, Zilai Wang, Eray Eren, Abeer Alwan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08468.txt.

Description

Natarajan Balaji Shankar, Zilai Wang, Eray Eren, and Abeer Alwan merge speech models with care to sharpen child speech recognition, in work from twenty twenty-five. Big speech models shine on adult speech but falter on children's voices, since child data is scarce. Merging models trained on richer speech could fill the gap.

Their method, Selective Attention Merge, blends only the attention parts of task vectors from donor models. On a standard child speech database, word error rates fell by up to fourteen percent relative. The method beat older merging tricks and data augmentation alone. Combined with augmentation, it set a new best score for a small Whisper model on that database.

Catalog record

CREATORNatarajan Balaji Shankar, Zilai Wang, Eray Eren, Abeer Alwan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-08468

Browse the full library · About the Tower of Babel