Improving Cross-Lingual Phonetic Representation of Low-Resource Languages Through Language Similarity Analysis
Minu Kim, Kangwook Jang, Hoirin Kim · 2025
Read Improving Cross-Lingual Phonetic Representation of Low-Resource Languages Through Language Similarity Analysis by Minu Kim, Kangwook Jang, Hoirin Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06810.txt.
Introduction
Kim, Jang, and Kim studied source language choice in this paper from twenty twenty-five. Speech tools for languages with little data often borrow training from richer languages. Picking the donor language, though, was usually done with little care. The team asked how sound-level similarity between languages shapes the payoff.
For phoneme recognition, training with sound-alike languages beat single-language training by a wide margin. It even topped a large self-supervised model. Within one language family, closer sound patterns helped, while distant ones hurt. Similarity across families mattered more than family ties themselves. The authors give a practical way to judge sound closeness before choosing donors.
Catalog record
| CREATOR | Minu Kim, Kangwook Jang, Hoirin Kim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-06810
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Cross-Lingual Phonetic Representation of Low-Resource Languages Through Language Similarity Analysis Thanks: This work was supported by the National Research Foundation of Korea grant funded by the Korea government (MSIT). (No. 2021R1A2C1014044) Minu Kim, Kangwook Jang, Hoirin Kim Abstract This paper examines how linguistic similarity affects cross-lingual phonetic representation in speech processing for low-resource languages, emphasizing effective source language selection. Previous cross-lingual research has used various source languages to enhance performance for the target low-resource language without thorough consideration of selection. Our study stands out by providing an in-depth analysis of language selection, supported by a practical approach to assess phonetic proximity among multiple language families. We investigate how within-family similarity impacts performance in multilingual training, which aids in understanding language dynamics. We also evaluate the effect of using phonologically similar languages, regardless of family. For the phoneme recognition task, utilizing phonologically similar languages consistently achieves a relative improvement of 55.6% over monolingual training, even surpassing the performance of a large-scale self-supervised learning model. Multilingual training within the same language family demonstrates that higher phonological similarity enhances performance, while lower similarity results in degraded performance compared to monolingual training. I Introduction There are approximately 7,000 languages in the world, but the number studied in the field of speech processing is limited. The primary reason for this is the scarcity of data, leading to the classification of these languages as low resource languages [ 1 ] . Given the limited data available for low resource languages, training models exclusively on these languages poses significant challenges. Some research [ 2 , 3 ] has shown that multilingual processing can enhance recognition for new languages by leveraging knowledge from multiple sources. However, effective source language selection remains underexplored. This study follows a similar approach by incorporating multiple languages alongside the target language for training. Furthermore, this work proposes a strategy for optimal source language selection in cross-lingual training for low-resource speech recognition. On the other hand, research in Natural Language Processing (NLP), particularly in text-based tasks, has focused on selecting languages for cross-lingual tasks based on linguistic similarity. Research like [ 4 ] evaluates how similarity impacts model performance, while the study by [ 5 ] ranks languages for training based on linguistic criteria. In most of cross-lingual speech recognition studies, however, source languages are often chosen randomly [ 6 , 7 , 8 ] or taken from the same language family without much consideration [ 9 , 10 ] . Though not their main focus, some studies have attempted to demonstrate the impact of source languages. For example, the study by [ 11 ] examines International Phonetic Alphabet (IPA) transcription based on similarity, and the research conducted by [ 12 ] shows that closely related languages perform better in cross-linguistic contexts. Other works [ 13 , 14 , 15 ] support using phonologically similar languages. Meanwhile, conflicting views on family-based learning exist: some report training within the same language family improves performance [ 16 ] , while others find training with other language family data can be effective [ 17 ] . However, existing research on cross-lingual speech recognition for low-resource languages has not mainly focused on the selection of languages itself, resulting in a lack of detailed analysis of its effects and the absence of specific metrics. This study aims to explore the reasoning behind these seemingly contradictory findings in cross-lingual speech recognition and seeks to determine how to effectively implement cross-lingual approaches to enhance performance in low-resource settings. By applying cross-lingual studies across diverse language families, this research will provide insights into the dynamics of language similarity and its impact on speech recognition. As noted by [ 18 ] , languages can be evaluated through corpus-based and typology-based approaches. In the corpus-based similarity assessment, we convert the text in the speech processing data into IPA and analyze the frequency distribution of phonemes [ 19 ] . The languages are evaluated as more similar if their phoneme distributions are alike. In the typology-based similarity assessment, we examine how similar the typological features are using the Grambank dataset [ 20 ] , which numerically records the typological characteristics of languages. In this study, we primarily utilize corpus-based similarity assessment, while typology-based similarity evaluation is employed as a supplementary method to examine how well it aligns with the similarity evaluation of data within the same language family. We compare three language families—Indo-Iranian (a branch of Indo-European), Turkic, and Afro-Asiatic—using both methods to assess the degree of similarity among languages within each family. As languages in the same family phylogenetically share the same root (ancestor), it can be a starting point for cross-lingual approach. Based on this analysis, we will examine the impact of language similarity in family-based multilingual learning. Additionally, we will explore how training language data selected based on phonological distribution similarity regardless of language family (obtained from corpus-based methods) affects cross-lingual learning. As we train a universal symbol system based on the IPA, no language-specific fine-tuning is conducted during multilingual training. Instead, the model is trained to produce the corresponding IPA when inputting speech data from various languages. For both monolingual and multilingual training, we utilize a Conformer-based model [ 21 ] . The results of our analysis reveal a clear trend: the performance of our model improves significantly when there is greater phonological similarity between the source and target languages within the same language family. This performance boost is particularly evident when the phonological characteristics of these languages are closely aligned. Conversely, with lower phonological similarity, performance drops, and in some cases, it can be worse than monolingual training, where the language pairs lack phonological advantages. Leveraging languages with high phonological similarity (regardless of family) yields a relative improvement of 55.6% over the monolingual approach, notably surpassing the performance of a large-scale self-supervised learning (SSL) model [ 22 ] . This advantage is particularly notable given that fine-tuning the SSL model requires extensive pre-training data and significantly larger parameters (300M vs. 43M), which underscores the effectiveness of our source selection approach. II Analysis Methods This chapter contains methods for constructing an appropriate training set to learn phonetic representations from the speech of the target language. The overall framework is illustrated in Fig. 1 . II-A Grapheme-to-Phoneme Model For low-resource languages, especially those at risk of extinction, IPA transcription is crucial as it captures the phonological essence of the language [ 23 ] , providing a key resource for research, revitalization, and education. Traditionally, human listeners manually transcribe spoken language into IPA, which is labor-intensive and time-consuming [ 24 ] . There have been efforts to automate this process [ 25 , 26 , 27 , 28 ] , and our study builds on those attempts. We transcribe text data into IPA using a Grapheme-to-Phoneme (G2P) model. II-B Similarity Metric In this section, we suggest two language similarity metrics: corpus-based and typology-based. Each methodology is used to evaluate the similarity among languages within a family, and specifically, the corpus-based approach is employed to select the phonetically most similar languages regardless of their language family. Corpus-based approach. We analyze unigram phoneme sequences by examining the frequencies of individual phonemes and measuring similarity using cosine similarity between phoneme distribution vectors \mathbf{p}_{A} and \mathbf{p}_{B}