Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world?
Tuukka Törö*, · 2025
Read Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world? by Tuukka Törö*, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08564.txt.
Introduction
Törö, Suni, and Šimko present a study from twenty twenty-five on what speech models reveal about language kinship. Languages relate through family lines, word stock, and contact between neighbors. The team take speech vectors from a model trained to name languages by ear. They study recordings of one hundred six world languages. They group the vectors and compare their gaps with family, word, and map distances. The speech gaps match the classic measures closely.
The match holds for both world-scale and local patterns. Charts of the links prove tricky, since language change is fluid. The authors discuss data size and space shape as method concerns. They see speech vectors as a scalable tool for language studies. Future work targets rare languages and social variation.
Catalog record
| CREATOR | Tuukka Törö*, |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08564
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world? Tuukka Törö*, Abstract Abstract Investigating linguistic relationships on a global scale requires analyzing diverse features such as syntax, phonology and prosody, which evolve at varying rates influenced by internal diversification, language contact, and sociolinguistic factors. Recent advances in machine learning (ML) offer complementary alternatives to traditional historical and typological approaches. Instead of relying on expert labor in analyzing specific linguistic features, these new methods enable the exploration of linguistic variation through embeddings derived directly from speech, opening new avenues for large-scale, data-driven analyses. This study employs embeddings from the fine-tuned XLS-R self-supervised language identification model voxlingua107-xls-r-300m-wav2vec , to analyze relationships between 106 world languages based on speech recordings. Using linear discriminant analysis (LDA), language embeddings are clustered and compared with genealogical, lexical, and geographical distances. The results demonstrate that embedding-based distances align closely with traditional measures, effectively capturing both global and local typological patterns. Challenges in visualizing relationships, particularly with hierarchical clustering and network-based methods, highlight the dynamic nature of language change. The findings show potential for scalable analyses of language variation based on speech embeddings, providing new perspectives on relationships among languages. By addressing methodological considerations such as corpus size and latent space dimensionality, this approach opens avenues for studying low-resource languages and bridging macro- and micro-level linguistic variation. Future work aims to extend these methods to underrepresented languages and integrate sociolinguistic variation for a more comprehensive understanding of linguistic diversity. Author summary Introduction Understanding and quantifying relationships among languages is a complex task, as languages differ across multiple dimensions, including phonology, syntax, morphology, lexicon, and prosody [ 1 ] . These features evolve at varying rates and are shaped by diverse influences such as internal diversification, contact with other languages, and broader sociolinguistic factors. Traditional approaches to analyzing linguistic variation typically rely on genetic and typological frameworks. Genetic approaches trace languages to shared ancestors and reconstruct proto-languages combining various sources of information such as linguistics, archaeology and human genetics, constructing assumed historical migration patterns of speaker communities [ 2 ] , while typological approaches classify languages based on structural similarities and differences [ 3 ] . Both rely on predefined assumptions about historical events, cross-linguistic categories or comparative concepts [ 4 , 5 ] , which can introduce interpretive biases. Furthermore, these approaches demand significant manual effort, and their dependence on human-curated linguistic labels limits their scalability. Languages are not static systems; they are dynamic, constantly evolving through processes of internal diversification and external influences. Variation manifests across multiple linguistic and typological dimensions, each evolving at different rates depending on cultural, social, and ecological contexts [ 1 ] . For instance, internal diversification occurs when languages diverge from their parent forms, potentially giving rise to entirely new languages. This process is often complemented by external contact with other languages, leading to linguistic change through borrowing, convergence, or the creation of entirely new language systems, such as creoles and pidgins. Traditional genetic approaches in historical linguistics typically represent language evolution using the family tree model, borrowing from biology [ 6 ] . This model conceptualizes languages as branching from a common proto-language spoken by an ancestral community, which eventually diverged due to geographic and social separation. While the tree model simplifies the complex web of interactions that characterize linguistic evolution, e.g., different rates of change of different linguistic features, it provides an easily accessible method to visualize global inter-relatedness among multiple languages and their groupings into families based on the analyzed set of features. To address the inherent limitations of family trees, alternative frameworks such as the wave model and network-based approaches have been developed. For instance, the NeighborNet method constructs phylogenetic networks instead of trees, allowing for intersecting connections that reflect the multifaceted influences shaping languages [ 7 ] . These models move beyond the linear, bifurcating narratives of traditional approaches to embrace the complex, intersecting relationships that define language evolution. In recent years, machine learning (ML) methods have offered new avenues for exploring language relationships, primarily by analyzing vast corpora of textual material [ 8 , 9 , 10 ] . By leveraging data-driven techniques, researchers can analyze linguistic patterns across large datasets without relying exclusively on predefined linguistic categories or historical assumptions. Such approaches not only complement traditional methods but also reveal new perspectives on the interplay of structure, history, and function in the evolution of human language. Text based ML approaches derived from, e.g., language identification (LID) or machine translation systems, use embeddings from self-supervised deep learning models for analyzing linguistic variation without reliance on predefined theoretical frameworks. The general assumption of this kind of ML assisted analyses–shared by the present account–is that the topological relationships among the numeric latent representations of linguistic material (or of the languages themselves) systematically reflect the underlying typological relations among the analyzed languages. One of the issues with comparing world languages based on the textual material is the necessary step of coordinating different orthographies, and either converting the texts to a shared (potentially phonetic) script or use embeddings that might include orthographic rather than linguistic information. This is not the case when dealing with spoken language directly rather than its very lossy and arbitrary transformation to a textual form. Additionally, many languages of interest lack a written form, meaning speech is the only viable medium for their analysis. Speech captures the natural variability and language change more effectively than text, offering richer insights into linguistic dynamics. Speech also provides a universal signal, making it feasible to build massively multilingual models without being limited by the orthographic constraints of individual languages [ 11 , 12 ] . These models can then be fine-tuned to specific tasks, such as speech recognition, or language identification in our case. In this paper we thus analyze the latent space embeddings of a fine-tuned self-supervised language identification (LID) model derived from purely speech material of over 100 languages. Specifically, we use embeddings from the XLS-R model, fine-tuned on the Voxlingua107 corpus [ 13 ] , to cluster languages based on their sound characteristics, such as phonetic, phonotactic and prosodic features implicitly present in the recordings. Using a corpus of recently collected spoken material, the method takes a synchronic view of languages, analyzing them as they are currently spoken, sidestepping many of the constraints and assumptions of traditional methods. This paper shares some reasoning–albeit being methodologically very different–with the experimental work presented in Skirgaard etal. [ 14 ] where participants listened to utterances in various languages and were asked to guess the language in a forced choice test, and their choice patterns were used to quantify similarities and differences among the analyzed languages. Our goal is to evaluate the extent to which the speech-based embeddings reflect known genealogical and geographical clusters and to identify patterns of linguistic variation that emerge from the model’s latent space. This analysis builds on previous work that has used embeddings for studying dialectal variation [ 15 , 16 , 17 , 18 ] and language family relationships [ 19 ] , extending it to a broader typological context. The current approach is also inspired by the Voxlingua107 datasets’ authors’ observations using an x-vector model that language embeddings can