Does language matter for spoken word classification? A multilingual generative meta-learning approach
Batsirayi Mupamhi Ziki, Louise Beyers, Ruan van der Merwe · 2026
Read Does language matter for spoken word classification? A multilingual generative meta-learning approach by Batsirayi Mupamhi Ziki, Louise Beyers, Ruan van der Merwe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13084.txt.
Introduction
Ziki, Beyers and van der Merwe test whether language matters for few-shot spoken word classification, using generative meta-continual learning. They train monolingual models on English, German, French and Catalan, a bilingual English-German model, and a multilingual model on all four languages. The multilingual model performs best, but gaps between models are unexpectedly small. Hours of unique training data predict performance better than the number of languages included.
Catalog record
| CREATOR | Batsirayi Mupamhi Ziki, Louise Beyers, Ruan van der Merwe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18843 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13084
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Does language matter for spoken word classification? A multilingual generative meta-learning approach Batsirayi Mupamhi Ziki, Louise Beyers, Ruan van der Merwe Abstract Meta-learning has been shown to have better performance than supervised learning for few-shot monolingual spoken word classification. However, the meta-learning approach remains under-explored in multilingual spoken word classification. In this paper, we apply the Generative Meta-Continual Learning algorithm to spoken word classification. The generative nature of this algorithm makes it viable for use in application, and the meta-learning aspect promotes generalisation, which is crucial in a multilingual setting. We train monolingual models on English, German, French, and Catalan, a bilingual model on English and German, and a multilingual model on all four languages. We find that although the multilingual model performs best, the differences between model performance is unexpectedly low. We also find that the hours of unique data seen during training seems to be a stronger performance indicator than the number of languages included in the training data. 1 Introduction The goal of keyword spotting is to detect pre-defined keywords in continuous audio streams ( 8 ) . One application of this is to activate voice assistants when the keywords are spoken ( 12 ) . Keyword spotting models often require datasets that contain thousands of utterances for each keyword ( 11 ) , however, collecting data at this scale for low-resource languages is infeasible. As a result, it is ideal for these systems to be of a few-shot nature. A growing field of work shows that leveraging data from multiple languages can also improve performance on lower-resource languages ( 13 ; 10 ) . Mazumder et. al. ( 10 ) introduced a transfer learning method to perform few-shot keyword spotting in any language. They did this by training a multilingual encoder via supervised learning and using it to fine-tune multilingual keyword spotting model such that it can generalise to unseen languages. They also pointed out that the multilingual model outperformed the monolingual model on its own language and unseen languages. Another common approach to improving the performance in few-shot systems is to use meta-learning algorithms. Meta-learning is informally defined as learning how to learn ( 6 ) . As shown in ( 2 ; 3 ) , the model-agnostic meta-learning (MAML) ( 5 ) approach outperforms standard supervised learning and transfer learning in the few-shot setting. Generative meta-continual learning (GeMCL) ( 1 ) is an algorithm that lies in the intersection of meta-learning and continual learning used in classification tasks. It is a generalisation of and improvement on prototypical networks ( 14 ) , adopting a Bayesian approach in modelling the distribution of the classes in the dataset. Because GeMCL isolates class-specific parameters, it is immune to catastrophic forgetting ( 1 ) . Given GeMCL’s strong few-shot performance and its immunity to catastrophic forgetting, it stands to be a promising algorithm to investigate further for spoken word classification settings in both the low-resource and multilingual settings. Our contribution is to investigate applying GeMCL’s meta-learning procedure to train multilingual, bilingual and monolingual GeMCL models on the Multilingual Spoken Words Corpus (MSWC) dataset ( 11 ) , exploring whether the multilingual model provides additional benefit over the monolingual and bilingual models on unseen languages in spoken word classification. We show that: 1. On unseen languages, the monolingual and the bilingual models’ performance is comparable to the multilingual model’s performance. 2. For our setting, model performance may be more strongly linked to the number of hours of training data seen, rather than the number of languages a model is exposed to. 2 GeMCL We provide an overview of the Generative meta-continual learning (GeMCL) algorithm; for more details please refer to ( 1 ; 7 ) . GeMCL consists of an encoder, f_{\phi} , followed by the Bayes classifier, where each class c is modelled by a Gaussian. The encoder outputs an embedding of input \vec{x} in d -dimensional space. Each Gaussian has a mean, \vec{\mu}^{c}\in\mathbb{R^{d}} , and precision, \vec{\lambda}^{c}\in\mathbb{R}_{+}^{d} , whose estimates are updated based on the data available for that class. The uncertainty of the mean and precision are modelled by a Normal-Gamma distribution, P(\vec{\mu}^{c},\vec{\lambda}^{c}\mid\theta_{0})\propto\prod_{i=1}^{d}\mathcal{N}\big(\mu^{c}_{i}\mid\mu_{0},(\kappa_{0}\lambda^{c}_{i})^{-1}\big)\mathrm{Ga}\big(\lambda^{c}_{i}\mid\alpha_{0},\beta_{0}\big), (1) where \theta_{0}=\{\alpha_{0},\beta_{0},\kappa_{0},\mu_{0}\} are parameters of the Normal-Gamma distribution and represent our prior belief before seeing the embedded training samples of class c . However, in practice the prior of mean \vec{\mu}^{c} is assumed to be uninformative, therefore \kappa_{0}=\mu_{0}=0 . The Normal-Gamma distribution is the conjugate prior to the Gaussian likelihood. Therefore, the posterior remains in the same family. This allows closed-form updates for the Normal-Gamma parameters. Assume that each class c has K training samples: (\vec{x_{1}},\vec{x_{2}},\cdots,\vec{x_{K}}) . Then the updates to the Normal-Gamma parameters are, \displaystyle\kappa_{K}^{c} \displaystyle=K, (2) \displaystyle\vec{\mu}_{K}^{c} \displaystyle=\frac{1}{K}\sum_{i=1}^{K}\vec{z}_{i}, (3) \displaystyle\alpha_{K}^{c} \displaystyle=\alpha_{0}+\frac{K}{2}, (4) \displaystyle\boldsymbol{\beta}_{K}^{c} \displaystyle=\beta_{0}\mathbf{1}+\frac{1}{2}\sum_{i=1}^{K}(\mathbf{z}_{i}-\boldsymbol{\mu}_{K}^{c})\odot(\mathbf{z}_{i}-\boldsymbol{\mu}_{K}^{c}). (5) Here f_{\phi}(\vec{x_{i}})=\vec{z_{i}} for i\in\{1,2,\cdots,K\} and \odot denotes the Hadamard product. The predicted label {\hat{y}} is, {\hat{y}}=\underset{{y}}{\rm{argmax}}\,P\left(\vec{z}|\alpha_{K}^{{y}},\vec{\beta}_{K}^{{y}},\vec{\mu}_{K}^{{y}},\kappa_{K}^{{y}}\right), (6) where P is a Student’s t-distribution. Figure 1 illustrates the GeMCL procedure for learning the class statistics. GeMCL makes use of meta-learning to train and evaluate the encoder’s parameters and the prior parameters: \alpha_{0} and \beta_{0} . Collectively, these parameters are known as the meta-parameters. Meta-learning consists of two phases: meta-training and meta-testing. A single step in meta-training involves training and testing on a batch of tasks sampled from a task distribution. In our case, the tasks are classification tasks consisting of N classes each with K samples, known as a N -way- K -shot episode. The training samples of each class are known as the support set, while the test samples are known as the query set. During an N -way- K -shot episode, GeMCL makes use of the support sets to learn the statistics of all the classes using equations ( 2 ) - ( 5 ). Next, the model predicts the label of each sample in the query sets with equation ( 6 ). The cross-entropy loss is used to obtain the gradients to update the meta-parameters. In meta-testing, we evaluate the meta-parameters on unseen N -way- K -shot classification tasks, i.e., tasks that are not in the meta-training distribution. This process is similar to the meta-training phase; however, the meta-parameters remain frozen. Figure 1: An example of the GeMCL procedure for learning class statistics for a classification task with three classes and K samples in each class. 3 Empirical design In this section, we describe the MSWC dataset, the training procedure of the models, the encoder architecture of GeMCL, and the evaluation procedure. 3.1 MSWC dataset We make use of the MSWC dataset ( 11 ) . It is an audio dataset consisting of 23.4 million 1 -second audio clips of 340\,000 keywords across 50 languages. The dataset is split into train / dev / test splits by the original authors for each of the keywords, prioritising speaker diversity when making the splits. We do not make use of the dev set. In selecting the languages for training the multilingual models, the top four high-resource languages, English, German, French and Catalan, are used. We filter out any words that do not have at least five valid examples. The words are then randomly split into the meta-training set and meta-test set with a 70:30 split. Table 1 shows the meta-splits for the four training languages.