Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering

Hillary Mutisya, John Mugane · 2026

Read Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering by Hillary Mutisya, John Mugane (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22723.txt.

Introduction

Mutisya and Mugane find word structure in Giriama, a Bantu tongue with only 91 labeled pattern sets, by joining transfer from Swahili with clustering done without labels. Their method gives noun classes to 2,455 words and finds two new, never-noted patterns: an a- form for Class 2 and a short k'- form. Checks on known verb sets confirm strong base-form accuracy.

Catalog record

CREATORHillary Mutisya, John Mugane
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20321
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22723

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering Hillary Mutisya, John Mugane Abstract We present a method for discovering morphological features in low-resource Bantu languages by combining cross-lingual transfer learning with unsupervised clustering. Applied to Giriama (nyf), a language with only 91 labeled paradigms, our pipeline discovers noun class assignments for 2,455 words and identifies two previously undocumented morphological patterns: an a- prefix variant for Class 2 (vowel coalescence—the merger of two adjacent vowels—of wa- , 95.1% consistency) and a contracted k’- prefix (98.5% consistency). External validation on 444 known Giriama verb paradigms confirms 78.2% lemmatization accuracy, while a v3 corpus expansion to 19,624 words (9,014 unique lemmas) achieves 97.3% segmentation and 86.7% lemmatization rates across all major word classes. Our ensemble of transfer learning from Swahili and unsupervised clustering, combined via weighted voting, exploits complementary strengths: transfer excels at cognate detection (leveraging ∼ \sim 60% vocabulary overlap) while clustering discovers language-specific innovations invisible to transfer. We release all code and discovered lexicons to support morphological documentation for low-resource Bantu languages. 1. Introduction 1.1 Motivation Morphological analysis is fundamental to linguistic documentation and natural language processing, yet most of the world’s 7,000+ languages lack comprehensive morphological resources. This is particularly acute for the Bantu language family (500+ languages, 300M+ speakers), where noun class systems—a defining typological feature—remain undocumented for many languages. Consider Giriama (Bantu E.72b in Guthrie’s classification, \sim 600,000 speakers, Kenya coast): despite being a living language with substantial speaker populations, only 91 morphological paradigms (annotated word-lemma pairs with grammatical features) exist in computational form. Standard supervised learning approaches would achieve poor coverage with such minimal data. Yet Giriama shares approximately 60% vocabulary with Swahili (a high-resource relative with 816+ paradigms), suggesting cross-lingual transfer may be viable. 1.2 Research Questions The discovery pipeline builds on BantuMorph ( 17 ) , a ByT5-small character-level model trained on 16 Bantu languages for morphological analysis. BantuMorph’s encoder maps words from any Bantu language into a shared embedding space where morphologically similar words—including cross-lingual cognates—cluster together. We exploit this property for zero-shot noun class discovery in Giriama and 15 other Bantu languages. This work addresses three key questions: 1. Zero-shot discovery: Can we discover morphological features in a low-resource language using minimal supervision ( n<100 paradigms)? 2. Language-specific innovation: Can unsupervised methods identify morphological patterns unique to the target language? 3. Method complementarity: How do cross-lingual transfer and unsupervised clustering complement each other for morphological discovery? 1.3 Contributions 1. Novel multi-method approach: We combine transfer learning (K-nearest neighbors in embedding space), unsupervised clustering (UMAP + K-means), and ensemble validation. 2. Empirical validation: On Giriama, we discover 2,455 noun class labels (27 \times increase) and validate the underlying model on 444 known paradigms (78.2% lemmatization accuracy). 3. Linguistic discoveries: Two previously undocumented Giriama patterns: the a- prefix variant (Class 2, 95.1% consistent) and the k’- contracted prefix (98.5% consistent). 4. Scalability: Our approach requires only a character-level pretrained model, a related high-resource language, and a small unlabeled corpus. 5. Open resources: Code, discovered lexicons, and visualizations. 2. Related Work 2.1 Morphological Analysis for Low-Resource Languages Supervised approaches ( 20 ; 12 ) require large annotated datasets. Semi-supervised methods reduce annotation burden but still require seed data ( 10 ; 4 ) . Unsupervised morphology induction ( 7 ; 5 ; 8 ) discovers structure without supervision but struggles with rare affixes. Cross-lingual transfer ( 1 ; 3 ; 16 ) exploits typological similarity, showing promise for related languages but missing language-specific innovations. Our work combines both approaches. 2.2 Bantu Language Morphology Bantu languages exhibit rich agglutinative morphology with noun class systems ( 14 ; 15 ) . Each noun belongs to one of 15–20 classes marked by prefixes that trigger agreement on verbs, adjectives, and other words in the sentence. Computational work on Bantu morphology includes analyzers for Swahili ( 9 ) , Zulu ( 19 ) , and recent neural methods ( 21 ) . Most of the 500+ Bantu languages remain understudied computationally. 2.3 Embedding-Based Morphology 18 and 6 show that contextualized embeddings capture morphosyntactic information. Character-level models ( 11 ; 22 ) handle morphological variation naturally. Cross-lingual embeddings ( 2 ) enable transfer. ByT5 ( 22 ) , our base model, operates at the character level and has shown strong cross-lingual transfer for morphologically rich languages. 3. Methodology Figure 1 illustrates our three-component pipeline. Swahili (816 labeled nouns) Giriama (7,812 sentences) Transfer Learning (KNN, K =5) Unsupervised Clustering (UMAP+K-means) Ensemble Voting (weighted, conf \geq 0.70) Noun Class Labels + Confidence Figure 1: Three-stage discovery pipeline. Row 1: Data sources (labeled Swahili nouns + unlabeled Giriama corpus). Row 2: Two complementary methods—KNN transfer finds cognates, clustering discovers innovations. Row 3: Weighted ensemble produces high-confidence noun class labels. 3.1 Problem Formulation Input: • M : Character-level pretrained model (ByT5) • L_{s} : High-resource source language with noun class labels (Swahili) • L_{t} : Low-resource target language (Giriama) • P_{s}=\{(w_{i},c_{i})\}_{i=1}^{N_{s}} : Labeled paradigms in L_{s} • C_{t} : Unlabeled corpus in L_{t} Output: Noun class assignments \hat{C}=\{(w_{j},\hat{c}_{j},\text{conf}_{j})\}_{j=1}^{N_{t}} for words in L_{t} . 3.2 Method 1: Transfer Learning via Cross-Lingual Projection Intuition: Related languages share cognates with similar embeddings; nearest neighbors likely have the same noun class. Listing 1: Transfer learning algorithm ⬇ 1. Extract embeddings : For each word w in Ls : es [ w ] = M . encode ( w ) For each word w in Lt : et [ w ] = M . encode ( w ) 2. For each target word wt : a . Find K =5 nearest source neighbors b . Vote for class via majority vote c . Confidence = vote_conf x sim_conf 3. Return : {( wt , c_pred , conf )} How it works: BantuMorph’s encoder maps words from any Bantu language into a 1,472-dimensional space where morphologically similar words cluster together. For a target-language word like Giriama akimbola , the 5 nearest Swahili neighbors might all be Class 2 plural forms (e.g., wanafunzi , watu ), yielding a Class 2 prediction with confidence proportional to neighbor agreement and cosine similarity. Advantages: High precision on cognates ( \sim 60% vocabulary overlap for Giriama–Swahili); interpretable; leverages labeled source data. Limitations: Cannot detect language-specific innovations absent from the source language (see Section 6 for how clustering addresses this gap). 3.3 Method 2: Unsupervised Clustering Intuition: Words in the same noun class cluster together in embedding space due to shared prefix patterns and agreement contexts (where nouns trigger matching prefixes on verbs and modifiers). Listing 2: Clustering algorithm ⬇ 1. Extract noun candidates from corpus Ct 2. Dimensionality reduction : - UMAP : reduce to d =50 dimensions 3. Cluster : - K - means with K =12 clusters 4. Analyze each cluster : - Extract prefix patterns ( first 1-3 chars ) - Map to noun class via prefix - class table 5. Return : {( w , c_cluster , consistency )} How it works: UMAP projects the high-dimensional embeddings to 50 dimensions preserving local structure. K-means ( K =12, matching the typical number of productive (actively used to form new words) Bantu noun classes) partitions words into clusters. Each cluster is mapped to a noun class by extracting the

Read the complete text