From Isolates to Families: Using Neural Networks for Automated Language Affiliation
Frederic Blum, Steffen Herbold, Johann-Mattis List · 2025
Read From Isolates to Families: Using Neural Networks for Automated Language Affiliation by Frederic Blum, Steffen Herbold, Johann-Mattis List (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11688.txt.
Introduction
Frederic Blum, Steffen Herbold, and Johann-Mattis List test neural networks for automated language affiliation. Their 2025 arXiv paper addresses a core task in historical linguistics. Scholars normally assign languages to families by slow manual comparison. Large standard wordlists and grammar datasets now make automation plausible. The team trained models on data from over a thousand languages with known families.
Models fed only word data beat models fed only grammar data. That matches the classic view held by most linguists. Combining both data types gave the best results of all. Further tests showed the models can spot distant links between whole subgroups. They can also hint at possible relatives of isolate languages. The work opens a new path for large-scale family classification.
Catalog record
| CREATOR | Frederic Blum, Steffen Herbold, Johann-Mattis List |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11688
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Isolates to Families: Using Neural Networks for Automated Language Affiliation Frederic Blum, Steffen Herbold, Johann-Mattis List Abstract In historical linguistics, the affiliation of languages to a common language family is traditionally carried out using a complex workflow that relies on manually comparing individual languages. Large-scale standardized collections of multilingual wordlists and grammatical language structures might help to improve this and open new avenues for developing automated language affiliation workflows. Here, we present neural network models that use lexical and grammatical data from a worldwide sample of more than 1,000 languages with known affiliations to classify individual languages into families. In line with the traditional assumption of most linguists, our results show that models trained on lexical data alone outperform models solely based on grammatical data, whereas combining both types of data yields even better performance. In additional experiments, we show how our models can identify long-ranging relations between entire subgroups, how they can be employed to investigate potential relatives of linguistic isolates, and how they can help us to obtain first hints on the affiliation of so far unaffiliated languages. We conclude that models for automated language affiliation trained on lexical and grammatical data provide comparative linguists with a valuable tool for evaluating hypotheses about deep and unknown language relations. 1 Introduction One of the central tasks in historical linguistics is the grouping of languages into families. While this is often done to propose new language families, this task also includes the affiliation of individual languages into existing families. The affiliation of languages to a common language family or a subgroup within that family is usually carried out manually and relies on comparing individual languages in depth. The traditional ‘comparative method’ in historical linguistics uses lexical and grammatical features to assign individual languages to one of the more than 200 language families proposed so far Osthoff and Brugmann 1878 ; Anttila 1972 ; Durie and Ross 1996 . The main part of this comparison starts with the initial assumption that two languages are related. However, such family relationships do not come as given and must first be established Hoenigswald 1978 ; Nichols 1996 ; Donohue et al. 2012 ; Campbell 2017 . While the rise of computational methods in historical linguistics has largely replaced traditional techniques for subgrouping with computational approaches in phylogenetic reconstruction Greenhill et al. 2020 ; Wu et al. 2020 ; Blum and List 2023 , the classical workflow for language affiliation of the comparative method is still considered the state-of-the-art in the field. But given that the affiliation of languages to families can be viewed as a computational classification task , it is possible to model the task in a setting that benefits from a large number of digital cross-linguistic datasets that have recently been published ( List et al. 2022 ; Seifart et al. 2022 ; Wichmann et al. 2022 ; Skirgård et al. 2023 ; Blum et al. 2024b ) . We show how automated language affiliation can be implemented computationally, how this approach can recover known long-distance genealogical relationships, and how it can shed new light on the affiliation of linguistic isolates and small language families. We train models on known language families and affiliate languages of unknown classification with the existing ones while aiming for worldwide coverage. When reporting the results, we pay special attention to the success of classifying small language families. The supervised training approach enables us to build upon the vast existing knowledge of historical linguistics. Having a large baseline of classes corresponding to true language families makes it possible to go beyond individual comparisons and to compare against all families simultaneously. 2 Background Starting with the detection of the Indo-European ( Bopp 1816 ) and Uralic ( Gyarmathi 1799 ) families towards the beginning of the 19th century, linguists have been able to group the more than 7000 languages still spoken today into several hundred language families. However, while major parts of the traditional workflow of the comparative method have been intensively discussed and partially formalized, the first step of this workflow, the affiliation of languages to a family by proving their genetic relationship ( Durie and Ross 1996 , “proof of relationship”, see) still lacks formalization. Although many quantitative and qualitative approaches have been proposed throughout the 20th and 21st centuries, none except the comparative method has gained general acceptance. Some methods, such as the application of superficial “mass comparison” techniques to large wordlists Greenberg 1957 , have been heavily criticized because of their lack of standardized data and clear criteria, and ultimately fell out of vogue ( Campbell 1988 ) . Other methods, such as the proposal by Dolgopolsky 1964 , who suggested looking for matching consonant classes to identify potential cognates, were ignored by most scholars for a long time before they were re-adopted in different contexts. While scholars working in the traditional paradigm of the comparative method usually agree that lexical and grammatical evidence combined is best to prove language relationship ( Campbell and Poser 2008 ) , the former is given preference in those cases where grammatical evidence is hard to obtain ( Dybo and Starostin 2008 ) . Quantitative and statistical methods typically restrict themselves to either lexical or grammatical evidence. Quantitative methods that take lexical data as their primary source can be divided into two basic types, depending on the evidence they try to obtain. Some approaches concentrate on the regularity of sound correspondences, trying to show that pairs of related languages exhibit significantly more matches in sound correspondences than unrelated ones ( Ringe 1992 ; Kessler 2001 ; Blevins and Sproat 2021 ) . Other methods do not use specific sounds as observed in the languages in question and convert them to broader classes ( sound classes or consonant classes , as originally proposed by Dolgopolsky 1964 , see List 2014 ) to identify cognate words. These methods argue that words that share direct matches in a certain number of sound classes are likely to be etymologically related and that languages for which a certain number of matches can be observed are likely to be genetically related ( Baxter and Manaster Ramer 2000 ; Turchin et al. 2010 ; Kassian et al. 2023 ) . Among the latter approaches, the Automated Similarity Judgment Program (ASJP, https://asjp.clld.org , Wichmann et al. 2022 ) deserves special mention, given that it can be seen as a first attempt to automatically classify as many of the languages of the world as possible with the help of phylogenetic methods. Using a specific sound class alphabet by which speech sounds are reduced to 40 classes, ASJP computationally compares word forms in language pairs, using traditional methods for sequence alignment ( Wagner and Fischer 1974 ) , to infer distances between language pairs. These are later used to reconstruct a phylogenetic tree with the help of the Neighbor-joining algorithm ( Saitou and Nei 1987 ) . Quantitative methods that exclusively use grammatical data as their primary evidence have less frequently been proposed than their lexical counterparts. Despite this, Dunn et al. 2005 suggest that analyzing grammatical data could lead further back in time than the traditional comparative method. Today, these claims have lost supporters due to