MauBERT : Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery
Angelo Ortiz Tandazo, Manel Khentout, Youssef Benchekroun, Thomas Hueber, Emmanuel Dupoux · 2025
Read MauBERT : Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery by Angelo Ortiz Tandazo, Manel Khentout, Youssef Benchekroun, Thomas Hueber, Emmanuel Dupoux (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-19612.txt.
Introduction
Speech technology for low-resource languages motivates MauBERT, built by Angelo Ortiz Tandazo and four co-authors. MauBERT extends the self-supervised HuBERT model across languages by teaching it articulatory features, the physical gestures behind speech sounds. Training continues on multilingual data covering 55 languages, with targets drawn from a mapping between phones and those articulatory features.
Tests using ABX discriminability, which checks whether sounds stay distinguishable across contexts, show MauBERT's representations hold up better than rival multilingual self-supervised models. The models also transfer to languages never seen in training and to casual speech, needing only about ten hours of speech for light fine-tuning. The team presents it as a workable recipe for giving speech models a phonetic head start.
Catalog record
| CREATOR | Angelo Ortiz Tandazo, Manel Khentout, Youssef Benchekroun, Thomas Hueber, Emmanuel Dupoux |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-19612
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MauBERT : Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery Angelo Ortiz Tandazo, Manel Khentout, Youssef Benchekroun, Thomas Hueber, Emmanuel Dupoux Abstract This paper introduces MauBERT , a multilingual extension of HuBERT that leverages articulatory features for robust cross-lingual phonetic representation learning. We continue HuBERT pre-training with supervision based on a phonetic-to-articulatory feature mapping in 55 languages. Our models learn from multilingual data to predict articulatory features or phones, resulting in language-independent representations that capture multilingual phonetic properties. Through comprehensive ABX discriminability testing, we show MauBERT models produce more context-invariant representations than state-of-the-art multilingual self-supervised learning models. Additionally, the models effectively adapt to unseen languages and casual speech with minimal self-supervised fine-tuning ( 10 10 hours of speech). This establishes an effective approach for instilling linguistic inductive biases in self-supervised speech models. 1 Introduction Is it possible to automatically discover the linguistic units of an unknown language from raw audio only? Doing so would be of great help to linguists or speech technologists working on low-resource or unwritten languages ( 6 ; 31 ; 51 ; 7 ; 52 ) , or to cognitive modellers trying to understand how children learn their native language before learning to read and write ( 25 ; 48 ) . This question has been addressed using a variety of approaches under the Zero Resource Speech Challenge series ( 45 ; 14 ; 15 ) , yielding impressive progress alongside unresolved questions. Much of this progress stems from advances in self-supervised learning (SSL) techniques ( 34 ; 4 ; 21 ) , which have produced speech representations that capture phonetic structure better than traditional features like MFCCs or mel filterbanks. This is evidenced by improved discriminability in the learnt representation spaces: two instances of the syllable ‘bit’ lie closer together than one instance of ‘bit’ and one instance of ‘bet’, even across different speakers ( 42 ; 41 ) . Further evidence comes from the success of quantisation of these representations, yielding low-bitrate discrete codes suitable for training generative language models that produce novel utterances in the target language ( 26 ; 5 ; 12 ; 39 ) . a. HuBERT-base Feature Extractor ❄ Transformer \cdots Downstream Modules Weighted Sum Projection BLSTM Feature Proj. \mathbf{\tilde{f}} sg Phone Model \mathbf{\tilde{p}} Gold features \mathbf{f} Gold phones \mathbf{p} Cross-entropy Cross-entropy b. Teacher Hb ❄ DM ❄ Threshold Target clusters \{\mathbf{c_{k}}\} Frequency Feature vectors by frequency rank Student HuBERT-base FE ❄ \mathbf{x} \mathbf{\tilde{x}} Transformer \mathbf{\tilde{z}} Cross-entropy Figure 1: a. Multilingual training. MauBERT -feat is trained to recognise ternary-valued articulatory features and phones using an encoder (HuBERT-base), downstream modules (weighted sum, up-projection, two-layer BLSTM, feature projection), and a phone model (two-layer perceptron); the feature states receive no gradients from the phone recognition loss due to the stop-gradient operator ( sg ). b. Self-supervised fine-tuning. Top: Offline clustering is applied to one of the layers of MauBERT , the teacher network, on an unseen language; bottom: the MauBERT Transformer, the student network, is then trained to predict the corresponding clusters of masked input. However, current approaches face two limitations. First, units discovered through speech SSL do not correspond one-to-one with linguistic units like phones, syllables or words. After clustering these units are typically shorter and more numerous than standard linguistic units: 2040\text{\,}\mathrm{ms} long vs. 70\text{\,}\mathrm{ms} for phonemes, and N= 1001000 vs. 3080 for phonemes ( 27 ; 40 ) . Moreover, they lack full invariance to speaker identity ( 11 ; 32 ) and phonetic context ( 20 ) , suggesting they capture acoustic events rather than abstract linguistic units. As a result, they produce codes with higher bitrates than phonemic transcriptions: about 100150\text{\,}\mathrm{bit}\text{/}\mathrm{s} versus 5070\text{\,}\mathrm{bit}\text{/}\mathrm{s} ( 26 ; 15 ) . Second, current SSL algorithms require massive amounts of clean speech: 21 uses 960 hours of clean English audio, 50 uses 90\text{\,}\mathrm{k} hours, and 9 uses 1\text{\,}\mathrm{M} hours. Such quantities are unavailable for low-resource languages, and notably, children acquire their language’s phonetics with far less than 1000\text{\,}\mathrm{h} of much noisier input. One avenue for improving SSL models involves pre-training universal models ( 10 ) , with recent work expanding both language coverage and training data ( 3 ; 50 ; 38 ; 9 ) . Inspired by the International Phonetic Alphabet (IPA), another research direction explores how phonetically-informed target signals influence learnt representations and their cross-lingual transferability ( 46 ; 29 ; 18 ) , suggesting that explicit phonological supervision enhances speech models’ cross-lingual capabilities. In this paper, we explore the hypothesis that standard SSL algorithms lack strong inductive biases necessary for learning invariant speech representations from limited audio data in new languages. Following the universal pre-training and phonetically-informed research lines, we propose transforming a monolingual pre-trained SSL model (specifically, HuBERT-base trained on English) into a universal SSL model with strong inductive biases by fine-tuning it on universal IPA phonemes and features across 55 diverse languages. We evaluate this model, coined MauBERT , on the ZRC 2017 challenge, which presents 5 languages with less than 10\text{\,}\mathrm{h} of training data (English, French, German, Mandarin, Wolof). To increase the evaluation’s diversity and validity, we extend the ZRC 2017 benchmark with 5 typologically diverse languages (Swahili, Tamil, Thai, Turkish, Ukrainian). Evaluation employs the within- and across-speaker ABX metrics from ZRC 2017 , supplemented with metrics measuring invariance to contextual allophony ( 20 ) . Our main contributions are twofold: (i) We demonstrate that multilingual supervised fine-tuning of HuBERT for articulatory feature or phone prediction creates robust multilingual phonetic representations with strong zero-shot transfer capabilities. (ii) Our resulting models enable effective adaptation to unseen languages and casual speech with minimal self-supervised fine-tuning, achieving strong speaker and contextual invariance in new languages with only 10\text{\,}\mathrm{h} of unlabelled data. As a by-product, our method also yields candidate phoneme and feature sets for unseen languages, with potential applications for linguistic analyses of low-resource languages. 2 Related Work Multilingual Speech Representation Learning. The field of multilingual speech processing has grown rapidly with large-scale semi- or self-supervised learning models that showed the potential for cross-lingual representation learning with little to no supervision ( 47 ; 10 ) . Recent studies have expanded language coverage ( 3 ) , diversified data sources ( 38 ) , and improved efficiency ( 50 ) and robustness to noise ( 9 ) . These multilingual SSL models build upon foundational work in self-supervised speech representation learning ( 4 ; 21 ; 8 ) and have been evaluated with multi-task frameworks like SUPERB ( 49 ; 43 ) . Meanwhile, other studies have explored the impact of phonetically-informed targets on learnt representations and their cross-lingual transferability ( 46 ; 29 ; 18 ) . Inspired by the downstream framework of SUPERB, this work extends HuBERT-base for articulatory feature prediction. Articulatory Features in Speech Processing. Early work established the use of articulatory features (AFs) in speech processing ( 13 ; 17 ; 16 ) , demonstrated that supervised learning can be used to automatically extract phonological features from raw (and continuous) speech ( 35 ; 22 ) and produced robust articulatory/phonological feature-based speech technologies ( 24 ; 28 ; 19 ) . The development of systematic feature inventories, particularly PanPhon ( 33 ) , has provided practical computational tools for cross-linguistic analysis. This has