PhiloBERTA: A Transformer-Based Cross-Lingual Analysis of Greek and Latin Lexicons
Rumi A. Allbert † [email protected] Wolfram Institute Makai L. Allbert † [email protected] Ralston College · 2025
Read PhiloBERTA: A Transformer-Based Cross-Lingual Analysis of Greek and Latin Lexicons by Rumi A. Allbert † [email protected] Wolfram Institute Makai L. Allbert † [email protected] Ralston College (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05265.txt.
Introduction
Rumi Allbert and Makai Allbert measure how Greek philosophical words kept their meaning as they passed into Latin, in a 2025 paper. Their transformer model, PhiloBERTA, embeds word pairs from classical texts and compares them by angular closeness. Word pairs with shared roots scored far closer than control pairs.
The effect held strongest for abstract terms, such as the Greek word for knowledge against its Latin heir. A significance test backed the pattern, and related pairs proved steady across contexts. The authors offer the method as a counting tool for how ideas traveled between the two traditions.
Catalog record
| CREATOR | Rumi A. Allbert † [email protected] Wolfram Institute Makai L. Allbert † [email protected] Ralston College |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-05265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PhiloBERTA: A Transformer-Based Cross-Lingual Analysis of Greek and Latin Lexicons Rumi A. Allbert † [email protected] Wolfram Institute Makai L. Allbert † [email protected] Ralston College Abstract We present PhiloBERTA, a cross-lingual transformer model that measures semantic relationships between ancient Greek and Latin lexicons. Through analysis of selected term pairs from classical texts, we use contextual embeddings and angular similarity metrics to identify precise semantic alignments. Our results show that etymologically related pairs demonstrate significantly higher similarity scores, particularly for abstract philosophical concepts such as \accpsili ς \acctonos µ (scientia) and ς \acctonos (iustitia). Statistical analysis reveals consistent patterns in these relationships ( p = 0.012 p=0.012 ), with etymologically related pairs displaying remarkable semantic preservation compared to control pairs. These findings establish a quantitative framework for examining how philosophical concepts progressed between Greek and Latin traditions, offering a new method for classical philological research. 1 Introduction Diachronic and cross-lingual semantic studies play an important role in understanding linguistic and conceptual evolution across time and cultures. In the domain of classical languages, such as Ancient Greek and Latin, this type of analysis is particularly valuable for examining complex historical texts, including philosophical writings. Philosophical terms in Ancient Greece often carried specialized meanings that evolved through centuries of theological and philosophical discourse. These texts shaped intellectual traditions for millennia, yet studying their nuanced semantic shifts and cross-lingual relationships presents unique computational challenges. A core difficulty lies in handling three fundamental complexities: 1) scarce parallel corpora for classical languages, 2) polysemy influenced by genre-specific usage, and 3) diachronic semantic changes across texts. Ancient Greek and Latin feature highly inflected morphology, significant lexical polysemy, and distinct syntactic structures from modern languages. Moreover, the texts under investigation—particularly philosophical works—encompass specialized vocabulary, overlapping conceptual terminology, and complex allusions. For instance, terms such as logos in Ancient Greek or anima in Latin exhibit semantic and conceptual evolution tied to changing philosophical interpretations over time. These characteristics push the boundaries of existing computational models, which are often trained on modern, high-resource languages and do not generalize well to classical languages. This underlines the necessity of developing customized models that incorporate diachronic information and adapt to cross-lingual alignment in resource-scarce historical contexts. The advent of contextualized embeddings through transformer architectures has opened new possibilities for handling these challenges. Models like mBERT, LASER, and XLM-R have demonstrated the ability to generate aligned latent representations for cross-lingual tasks and to uncover semantic relationships in low-resource languages. These techniques have been successfully applied to tasks such as sentence-level alignment of Ancient Greek translations into Latin [ 3 , 15 ] , multilingual lemmatization, and semantic retrieval [ 12 , 7 ] . However, these approaches often prioritize modern cross-lingual alignment tasks (e.g., French—English) and seldom tackle the additional complexity of diachronic semantic shifts, where meanings change over time as reflected in historical texts. Efforts to analyze diachronic semantic change, such as through embedding-based or Bayesian models, have proven effective for detecting lexical shifts in Ancient Greek and Latin [ 10 , 14 , 9 ] . Embedding methods like GASC [ 10 ] and lemma-based comparisons of historical corpora [ 14 ] have shown that temporal segmentation can reveal meaningful trends in word usage. While these methods provide valuable insights into diachronic patterns, they lack integration with cross-lingual approaches, limiting their utility for analyzing semantic alignment between Ancient Greek and Latin philosophical corpora. To date, few computational frameworks have unified these two areas: the unsupervised cross-lingual alignment of embeddings and the detection of diachronic semantic shifts. Existing research has also largely overlooked the philosophical domain, whose texts—dense with conceptual polysemy and intertextual references—pose unique challenges for computational analysis. Recent innovations, such as SPhilBERTa [ 12 ] and fine-tuned XLM-R [ 15 ] , have demonstrated the potential of multilingual transformers for analyzing domain-specific corpora. However, these models have yet to explicitly address diachronic modeling or tackle the interpretive complexity of philosophical semantics. This paper addresses these gaps by introducing a new transformer-based model specifically designed for cross-lingual and diachronic semantic alignment in Ancient Greek and Latin philosophical writings. The proposed model builds on the strengths of recent multilingual embedding techniques while incorporating advancements for handling sparse, diachronic corpora and domain-specific terminology. By integrating cross-lingual alignment capabilities with the ability to track temporal semantic shifts, this model aims to advance the computational analysis of classical languages, providing new tools for uncovering the evolution of philosophical ideas and concepts across historical and linguistic divides. Beyond presenting this model, the work makes four key contributions to the field: • An evaluation framework that combines angular similarity analysis with visualization techniques to isolate semantic relationships between languages. • Empirical validation that etymologically related pairs exhibit systematic preservation ( \sigma=0.003 ) while maintaining appropriate differentiation in control pairs ( \sigma=0.023 ). • Demonstration of robust preservation in abstract philosophical concepts (e.g., \accpsili ς \acctonos µ– scientia : 0.820, ς \acctonos – iustitia : 0.814). By focusing on these contributions, our approach combines cross-lingual alignment and diachronic modeling. It highlights the interpretive insights that can be gained from studying philosophical semantics in Ancient Greek and Latin. 2 Background Modern computational philology builds upon three foundational pillars: contextual language models, cross-lingual alignment techniques, and diachronic semantic analysis. Let \mathcal{C}=\{c_{1},...,c_{n}\} represent a corpus of ancient texts where each context c_{i}\in\mathbb{R}^{d} is encoded through transformer layers f_{\theta}:\mathcal{V}^{*}\rightarrow\mathbb{R}^{d} , with \mathcal{V} being the vocabulary. For bilingual term pairs (g,l)\in\mathcal{P} where g\in\mathcal{V}_{g} (Greek) and l\in\mathcal{V}_{l} (Latin), the semantic similarity metric becomes: s(g,l)=\frac{1}{|\mathcal{C}_{g}||\mathcal{C}_{l}|}\sum_{c_{g}\in\mathcal{C}_{g}}\sum_{c_{l}\in\mathcal{C}_{l}}\cos(f_{\theta}(c_{g}),f_{\theta}(c_{l})) (1) where \mathcal{C}_{g},\mathcal{C}_{l} are contextual instances of terms g and l . This formulation addresses polysemy through contextual averaging while handling data sparsity via parameter sharing in \theta . Table 1 : Embedding approaches for classical languages Static (FastText) Contextual (BERT) OOV Rate 18.7% 4.2% POS Accuracy 76.3 85.1 Cross-lingual r 0.31 0.68 The problem setting introduces two key constraints: 1) Alignment asymmetry where |\mathcal{C}_{g}|\gg|\mathcal{C}_{l}| for 73% of pairs, requiring inverse frequency weighting w_{i}=1/\log(|\mathcal{C}_{i}|+1) during training. 2) Genre-induced variance modeled as: \text{Var}(s(g,l))=\alpha\sigma^{2}_{\text{genre}}+(1-\alpha)\sigma^{2}_{\text{temporal}}+\epsilon (2) with \alpha=0.63 estimated through our ANOVA ( F=8.92,p=0.002 ). Our approach differs from prior work by explicitly modeling missing genre metadata through adversarial dropout layers during training. Multilingual knowledge distillation [ 6 ] provides the framework for cross-lingual transfer, where teacher logits z_{t}\in\mathbb{R}^{|\mathcal{V}_{e}|} (English) guide student model f_{\theta} through: \mathcal{L}_{\text{distill}}=\frac{1}{n}\sum_{i=1}^{n}\|z_{t}^{(i)}-f_{\theta}(c_{g}^{(i)})\|^{2}+\lambda\text{KL}(p_{t}\|p_{\theta}) (3) This enables alignment without parallel Greek-Latin corpora, leveraging English as a pivot language. Our ablation studies show this reduces required parallel data by 83% compared to direct alignment approaches. 3 Related Work Prior work in classical language modeling falls into three categories: monolingual embeddings, cross-lingual alignment, and philological applications. Riemenschneider and Frank (2023) established state-of-the-art monolingual performance through their 100M-word multilingual corpus, achieving 0.89 accuracy on POS tagging versus our PhiloBERTA’s 0.85 ( \Delta=-4.5\% , p=0.03 ). However, their explicit avoidance of modern language contamination limits cross-lingual capabilities, yielding only 0.62 translation search accuracy compared to our 0.92 ( +48\% improvement). This trade-off between linguistic purity and utility mirrors debates in low-resource ML - our synthetic training paradigm confirms that contamination can be beneficial when properly controlled through adversarial filtering ( \beta=0.73 , SE=0.15 ). For cross-lingual alignment, Krahn et al.(2023) pioneered knowledge distillation for Ancient Greek using parallel biblical texts. Although their approach achieves 0.96 accuracy on