Scaling few-shot spoken word classification with generative meta-continual learning
Louise Beyers, Batsirayi Mupamhi Ziki, Ruan van der Merwe · 2026
Read Scaling few-shot spoken word classification with generative meta-continual learning by Louise Beyers, Batsirayi Mupamhi Ziki, Ruan van der Merwe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13075.txt.
Introduction
Beyers, Ziki and van der Merwe scale few-shot spoken word classification to 1,000 classes with only five examples per class. Their Generative Meta-Continual Learning algorithm learns the classes sequentially and stays exceptionally stable. It does not always beat a repeatedly fully fine-tuned HuBERT model or a frozen HuBERT model with a retrained classifier head. But it matches the latter while adapting 2,000 times faster, having seen less than half the data for far less training time.
Catalog record
| CREATOR | Louise Beyers, Batsirayi Mupamhi Ziki, Ruan van der Merwe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20847 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13075
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scaling few-shot spoken word classification with generative meta-continual learning Louise Beyers, Batsirayi Mupamhi Ziki, Ruan van der Merwe Abstract Few-shot spoken word classification has largely been developed for applications where a small number of classes is considered, and so the potential of larger-scale few-shot spoken word classification remains untapped. This paper investigates the potential of a spoken word classifier to sequentially learn to distinguish between 1000 1000 classes when it is given only five shots per class. We demonstrate that this scaling capability exists by training a model using the Generative Meta-Continual Learning (GeMCL) algorithm and comparing it to repeatedly trained or finetuned baselines. We find that GeMCL produces exceptionally stable performance, and although it does not always outperform a repeatedly fully-finetuned HuBERT model nor a frozen HuBERT model with a repeatedly trained classifier head, it produces comparable performance to the latter while adapting 2000 2000 times faster, having been trained less than half of the data for two orders of magnitude less time. Figure 1: A single step in the meta-training procedure of GeMCL. Each task is an N -way- K -shot episode. The encoder, f_{\phi} , embeds the input. The support set is used to calculate the class statistics from the embeddings. The matrix \tilde{\mathcal{X}} contains the samples from the combined query set of all classes, totalling M samples, whereas the vector \tilde{\mathcal{Y}} contains the corresponding labels. The model produces a vector \hat{\mathcal{Y}} containing M predictions. The cross-entropy loss is calculated, and used to update the meta-parameters. 1 Introduction Spoken word classification is often researched in the context of keyword spotting (KWS) - specifically in the scenario where users are allowed to define their own keywords, and the model must adapt to the new words with only a few labelled shots ( 23 ; 18 ; 22 ; 3 ) . While this problem is usually formulated as an open-set classification problem ( 24 ; 16 ; 10 ; 17 ) , spoken word classification is generally closed-set ( 4 ; 22 ; 3 ) ; if we guarantee that all unseen words will be provided with labels from the user, then the open-set classification problem can be turned into a closed-set problem, with the addition of a continual learning aspect ( 22 ) . Spoken word classification evaluation in this setting is partially driven by the requirements of edge devices ( 6 ; 17 ; 18 ; 23 ) and is often tested on a small number of classes at a time, since a keyword system often does not require there to be many possible classes at a time ( 4 ; 10 ) and even at small numbers of classes, we observe limitations in performance ( 23 ) . However, if few-shot, continually learning spoken word classifiers are able to scale to large amounts of classes, the scope of the problems to which they may be applied expands significantly, from adding twenty new keywords to a keyword spotting system to potentially creating a data labelling flywheel for moderately-resourced languages. The requirements for such a system are twofold: good performance given access to only a moderate amount of data from the same distribution, and an unprecedented ability to generalise to words not seen in the training data. Meta-continual learning is a framework that fulfils these requirements as it lies at the intersection of meta-learning, traditionally used for few-shot classification ( 19 ; 7 ; 5 ) , and continual learning, and can be defined as learning how to continually learn ( 20 ) . Generative Meta-Continual Learning (GeMCL) ( 2 ) uses a generative classifier, and is a generalisation of prototypical networks ( 19 ) . There is a precedent for using prototypical networks for both versions of the user-defined KWS problem ( 17 ; 10 ; 16 ; 18 ; 4 ) particularly in practice because they take away the need to finetune. They have also been used to produce a model robust to numbers of shots and classes ( 4 ) . There is also a precedent for using meta-learning and meta-continual learning for few-shot continual spoken word classification to improve generalisation ( 22 ; 3 ) . Although meta-learning algorithms are often trained from scratch, they may be less data-hungry than infamously data-hungry foundation models such as HuBERT ( 8 ) and Wav2Vec2 ( 1 ) , which would make them a more reasonable alternative when we cannot expect to rely on transfer learning from out-of-distribution data. Another alternative investigated by ( 10 ) combines large self-supervised models with meta-trained projectors and classifier heads. GeMCL isolates the class-specific parameters and is therefore immune to catastrophic forgetting ( 2 ) . We therefore hypothesise that it may be able to generalise and scale beyond the approaches tried so far. We thus train the meta-continual learning algorithm GeMCL ( 2 ) on the Multilingual Spoken Words Corpus (MSWC) ( 14 ) dataset. In line with the findings of ( 9 ) on appropriate baselines for continual learning, we produce baselines by selecting two adaptation strategies and applying them to the popular pre-trained ( ( 24 ; 10 ; 13 ) ) HuBERT base model. For the first baseline we perform full finetuning of the pre-trained model, and for the second we train a classifier head on the pre-trained model. HuBERT base has significantly more data and compute available for training than our implementation of GeMCL, though its data source does not match the test distribution. GeMCL has access to data which is lower in quantity but more relevant to the task at hand. Comparing the models is therefore not only a comparison of algorithms, but of strategies: given enough data to train GeMCL but not enough data or compute to train HuBERT from scratch, is it more practical to finetune a large self-supervised model like HuBERT or is training GeMCL from scratch a viable option in terms of accuracy and compute? Our contribution is to answer this question by comparing the two algorithms in a setting which matches practical requirements: the models must perform continual learning, they only have access to few ( 5 ) shots per word to do so, and they must be able to scale the number of words they can classify. We are the first to our knowledge to report on the application of GeMCL on speech data, as well as the first to report results for few-shot spoken word classification for as many as 1000 classes at one time. Our contribution is also distinctive in that we test at multiple points during the continual learning process, so that we are able to report on per-class continual learning stability. 2 Methodology We discuss GeMCL, our baselines and specify how we evaluate for a continual learning scenario. 2.1 GeMCL We provide a short overview of the GeMCL algorithm; refer to ( 2 ) and ( 11 ) for more details. GeMCL consists of an encoder followed by a generative classifier that models the distribution of each class in the embedding space. In a continual learning setting, new data arrives sequentially and the distribution of a class evolves as new samples are observed. GeMCL adopts a Bayesian approach in which each class is represented