Continuous Learning for Children’s ASR: Overcoming Catastrophic Forgetting with Elastic Weight Consolidation and Synaptic Intelligence

Edem, Vishwanath, Tomi, Ville · 2025

Read Continuous Learning for Children’s ASR: Overcoming Catastrophic Forgetting with Elastic Weight Consolidation and Synaptic Intelligence by Edem, Vishwanath, Tomi, Ville (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20216.txt.

Introduction

Edem Ahadzi and co-authors keep child speech models from forgetting in this 2025 paper. Speech tools for children must learn from data that arrives in streams, both for apps and for privacy. Plain fine-tuning, though, erases old skills as new voices arrive.

Two memory-guarding methods were tested on a child speech corpus under a streaming plan. One freezes weights that mattered for past tasks, while the other scores each weight's past use. Both cut word error rates versus plain fine-tuning by several percent. The authors call this the first study of streaming learning for child speech tools.

Catalog record

CREATOREdem, Vishwanath, Tomi, Ville
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-20216

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Continuous Learning for Children’s ASR: Overcoming Catastrophic Forgetting with Elastic Weight Consolidation and Synaptic Intelligence Edem, Vishwanath, Tomi, Ville Abstract In this work, we present the first study addressing automatic speech recognition (ASR) for children in an online learning setting. This is particularly important for both child-centric applications and the privacy protection of minors, where training models with sequentially arriving data is critical. The conventional approach of model fine-tuning often suffers from catastrophic forgetting. To tackle this issue, we explore two established techniques: elastic weight consolidation (EWC) and synaptic intelligence (SI). Using a custom protocol on the MyST corpus, tailored to the online learning setting, we achieve relative word error rate (WER) reductions of 5.21 % with EWC and 4.36 % with SI, compared to the fine-tuning baseline. Ahadzi Pratap Singh Kinnunen Hautamaki University of Eastern FinlandFinland keywords automatic speech recognition, children speech recognition, continual learning, elastic weight consolidation, synaptic intelligence † † email: [email protected], [email protected], [email protected], [email protected] 1 Introduction Automatic speech recognition (ASR) is a mature technology that achieves high accuracy on varied and realistic speech data produced by adult speakers [ 1 , 2 ] . Unfortunately, the overfocus on adult speech has hindered the development of ASR systems for children. While essential in applications such as educational tools, communication aids, and interactive learning environments, children ASR performance remains low [ 3 , 4 ] . Children’s speech poses unique challenges due to rapid developmental changes and distinct acoustic and linguistic characteristics [ 4 , 5 , 6 ] . Anatomically, children’s vocal tracts are shorter compared to adults, resulting in higher pitch and different formant structures [ 7 , 8 ] . Linguistically, underdeveloped pronunciation skills lead to inconsistent articulation [ 9 ] . Although the usual approach in large-scale machine learning tasks, including ASR, is to boost performance by increasing the quantity and diversity of training data, data resources available for children remain relatively scarce [ 10 ] . The above challenges are further exacerbated by ASR training methods, which typically rely on static , non-evolving datasets where models are exposed to a fixed corpus all at once [ 11 ] . This is problematic for any application where data arrives sequentially, with its underlying properties drifting over time. In addition, specifically relevant to children, strict privacy regulations such as the General Data Protection Regulation (GDPR) in the European Union (Article 5(1)(e)) and the Children’s Online Privacy Protection Act (COPPA) in the United States (16 CFR §312.10) mandate deletion of minors’ data once the immediate processing purpose has been met. These requirements make it impractical to store and revisit earlier datasets to be used for subsequent training runs. These constraints motivate the need for approaches that can efficiently train models on new data as it becomes available. The common approach is to fine-tune or re-train the ASR model as soon as new data is available. While this approach readily adapts to the latest information, especially retraining is computationally demanding. But even more critically, both retraining and finetuning lead easily to so-called catastrophic forgetting [ 12 , 13 , 14 ] , a phenomenon where the model focus on the new data, but loses the previously acquired knowledge [ 15 ] . An ASR system that performed well earlier is at risk of being ”downgraded” due to its over-emphasis on the most recent data. There are various approaches for mitigating catastrophic forgetting. These include data-based methods (which store or generate past examples to replay and preserve knowledge) [ 16 , 17 , 18 , 19 , 20 ] , parameter-regularization approaches (which penalize large changes in model parameters critical for previous tasks) [ 12 , 21 , 14 , 22 , 23 ] , and dynamic architecture techniques (which expand or reorganize model components to accommodate new tasks) [ 24 , 25 , 26 ] . We focus on two methods, elastic weight consolidation (EWC) [ 12 ] and synaptic intelligence (SI) [ 14 ] . Both are computationally efficient, robust, and theoretically grounded (see Section 2 ) approaches for balancing the retention of past knowledge with the acquisition of new information. EWC has been particularly successful in alleviating catastrophic forgetting in ASR. Prior studies have applied it to domain adaptation for different accents or geographic regions [ 27 , 28 ] , multilingual learning [ 29 , 30 ] , and recognition of out-of-vocabulary (OOV) words [ 31 ] . These studies, however, are focused on adult speech, leaving the question of the utility of EWC and SI in children ASR tasks unanswered. Our study seeks to address this knowledge gap. It is useful to keep in mind that children are not merely ‘scaled down adults’; their speech exhibits substantial variability not only between different speakers, but also within the same speaker over relatively short time-scale, caused by developmental changes. This necessitates further investigation and re-assessment of results obtained for adult speaker populations, important for practical children ASR applications. In our experiments, we simulate gradual drift in speaker population through a novel experimental pipeline (on the otherwise standard MyST corpus), detailed in Section 3 . Our experiments also address practical consideration related to which (and how many) ASR models one can retain during online learning. Before the experiments, however, we begin with self-contained description of the parameter regularization methodology. 2 Methods 2.1 Overview of Parameter-Based Continual Learning In continual learning, a model incrementally acquires knowledge across a series of tasks, with the aim of maintaining performance on previously learned tasks while integrating new information. A key challenge in this paradigm is catastrophic forgetting, where updates for new tasks overwrite important representations acquired earlier. Various approaches have been proposed to address this problem, including parameter regularization. The idea is to penalize large changes to previous model parameters, ensuring that new knowledge can be integrated without ’erasing’ essential information from earlier tasks. Both EWC and SI selectively penalize changes to parameters identified as critical for previous tasks. Their optimization objective has the form, \mathcal{L}(\theta)=\mathcal{L}_{\mathrm{new}}(\theta)+\beta\sum_{i}\omega_{i}\left(\theta_{i}-\theta_{i}^{*}\right)^{2}, (1) where \mathcal{L}_{\mathrm{new}}(\theta) is the loss on the current (new) task, \theta^{*} denotes parameters learned from previous tasks and \beta controls the degree of regularization. In EWC, \beta is set as \lambda/2 , where \lambda is the hyper-parameter controlling regularization strength, and can be interpreted within a Bayesian framework as imposing a Gaussian prior over the parameters. Accordingly, the quadratic penalty corresponds to the negative log-prior, and \beta reflects the inverse of the prior variance [ 12 ] . In contrast, SI adopts an empirically driven approach to regularization. Here, a global scaling factor \alpha serves a similar regularization role to that in EWC but is determined based on empirical observations rather than rooted on probabilistic principles [ 14 ] . In both cases, the importance of a parameter \theta_{i} is quantified by \omega_{i} : for EWC, \omega_{i} is expressed as F_{ii} , the diagonal of Fisher information matrix (FIM) that captures the sensitivity of the loss to changes in \theta_{i} ; for SI, it is denoted as \Omega_{i}^{T-1} , a cumulative measure reflecting the parameter’s contribution to previously learned tasks. We discuss these techniques in more detail in Sections 2.2 and 2.3 . 2.2 Elastic Weight Consolidation EWC is a continual learning technique

Read the complete text