From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages
Matthias Schöffel Esteban Garces Arias · 2026
Read From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages by Matthias Schöffel Esteban Garces Arias (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09147.txt.
Introduction
Matthias Schöffel and co-author Esteban Garces Arias compare ways to tag parts of speech in medieval Romance texts, a job made hard by shifting spelling, rich word forms, and scarce labeled data. Their 2026 study covers medieval Occitan, Catalan, and French, and pits older rule-based and statistical taggers against modern open language models. Settings include prompting with no examples, prompting with a few, single-language tuning, and transfer across languages.
Across historically grounded datasets, the language model approaches beat the older taggers, with fine-tuning and multilingual training giving the largest gains. Transfer across languages helps under-resourced varieties a lot, and focused two-language training can beat broader mixes for some targets. The authors stress that closeness between languages and dataset traits shape what works best. They release code, models, and processed datasets, giving digital humanities scholars a practical guide for tagging old texts.
Catalog record
| CREATOR | Matthias Schöffel Esteban Garces Arias |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.09147
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages Matthias Schöffel Esteban Garces Arias Abstract Part-of-speech (POS) tagging for Medieval Romance languages remains challenging due to orthographic variation, morphological complexity, and limited annotated resources. This paper presents a systematic empirical evaluation of large language models (LLMs) for POS tagging across three medieval varieties: Medieval Occitan, Medieval Catalan, and Medieval French. We compare traditional rule-based and statistical taggers with modern open-source LLMs under zero-shot prompting, few-shot prompting, monolingual fine-tuning, and cross-lingual transfer learning settings. Experiments on historically grounded datasets show that LLM-based approaches consistently outperform traditional taggers, with fine-tuning and multilingual training yielding the largest improvements. In particular, cross-lingual transfer learning substantially benefits under-resourced varieties, while targeted bilingual training can outperform broader multilingual configurations for specific target languages. The results highlight the importance of linguistic proximity and dataset characteristics when designing transfer strategies for historical NLP. These findings provide empirical insights into the applicability of modern neural methods to medieval text processing and provide practical guidance for deploying LLM-based POS tagging pipelines in digital humanities research. All code, models, and processed datasets are released for reproducibility 1 1 1 https://github.com/msch38/medieval-romance-pos/tree/main . 1 Introduction The computational analysis of historical texts plays an increasingly important role in digital humanities, enabling large-scale investigation of linguistic change, cultural transmission, and textual variation. A fundamental prerequisite for such analyses is reliable linguistic annotation, with part-of-speech (POS) tagging serving as a key preprocessing step for downstream tasks including syntactic parsing, semantic analysis, and diachronic linguistic modeling ( 18 ; 6 ) . POS tagging for Medieval Romance languages presents particular challenges. These varieties exhibit substantial orthographic instability, dialectal diversity, and complex morphological systems, while annotated corpora remain comparatively small and domain-specific ( 23 ) . An overview of the geographic distribution as well as the spelling characteristics in all languages is shown in Figure 1 . As a result, computational tools originally developed for modern standardized languages often show limited robustness when applied to medieval textual material. (a) Map of Medieval Occitan and Medieval Catalan variations, 13th century ( 4 ) . Spelling variants Modern/Known spelling Medieval French deffendre défendre (engl. ’to defend’) joinncture jointure (engl. ’knuckle’) sun son (engl. ’his’) pruz preux (engl. ’brave’) Medieval Catalan ssaber saber (engl. ’to know’) Ffrança França (engl. ’France’) hòmens homes (engl. ’men’) jóvens joves (engl. ’youth’) Medieval Occitan deceplina disciplina, disiplina, desiplina (engl. ’discipline’) falssa fals (engl. ’false’) liech / lech lloc (engl. ’place’) fuoc / foc foc (engl. ’fire’) (b) Spelling characteristics across medieval language variations. Figure 1: Geographic distribution and spelling characteristics of medieval Romance languages (13th century). Left: regional variation of Medieval Occitan and Medieval Catalan. Right: spelling variants across Medieval French, Medieval Catalan, and Medieval Occitan. Traditional approaches to historical POS tagging have relied on rule-based systems and statistical models adapted from modern resources. While tools such as COLaF 2 2 2 https://colaf.huma-num.fr/deucalion/occ-cont and UDPipe 3 3 3 https://lindat.mff.cuni.cz/services/udpipe/ provide useful baselines, their performance can degrade in the presence of spelling variation, lexical sparsity, and genre-specific linguistic patterns. Recent advances in large language models (LLMs) offer new opportunities to address these challenges by leveraging contextual representations learned from large-scale multilingual corpora. However, systematic evaluation of LLM-based approaches for medieval Romance POS tagging remains limited, particularly with respect to prompting strategies, fine-tuning regimes, and multilingual transfer effects. This study provides a comparative evaluation of traditional and neural methods across three medieval Romance datasets representing different genealogical branches and textual domains: Medieval Occitan, Medieval Catalan, and Medieval French. We investigate the following research questions: 1. How do LLM-based approaches compare to established POS tagging tools for medieval Romance varieties? 2. To what extent do prompting strategies and decoding configurations influence tagging performance? 3. Can cross-lingual transfer learning improve performance across related historical languages? Our contributions are threefold. First, we present a controlled empirical comparison of traditional taggers and modern LLMs across multiple training and inference regimes. Second, we analyze the effectiveness of bilingual and trilingual transfer configurations, highlighting the role of linguistic proximity in multilingual historical NLP. Third, we provide practical recommendations for selecting modeling strategies under different resource constraints. Together, these contributions aim to support the development of more robust annotation pipelines for historical text collections. 2 Related Work Computational processing of historical language has evolved from early rule-based approaches toward statistical and neural methods that better accommodate linguistic variation and limited supervision. Foundational surveys ( 18 ) highlight key challenges in historical NLP, including orthographic instability, domain specificity, and data scarcity. Subsequent work has explored normalization and annotation strategies for early language stages, demonstrating the importance of adapting preprocessing and modeling techniques to non-standard linguistic input ( 21 ) . For Romance historical varieties, research has focused on developing specialized corpora and task-specific annotation tools. Studies on Classical and Early Modern French show that domain-adapted models can substantially improve lemmatization and POS tagging accuracy compared to general-purpose systems ( 5 ) . Neural approaches have further expanded these capabilities. Work on historical text normalization and lemmatization demonstrates that sequence-to-sequence and contextualized models can effectively capture variation-rich morphological patterns ( 3 ; 15 ; 13 ) . Related efforts in OCR post-correction and handwritten text recognition highlight the broader applicability of neural pipelines in historical document processing ( 24 ; 14 ; 9 ; 27 ; 17 ; 20 ) . Recent studies have begun to investigate large language models in historical linguistic contexts. Prompt-based experiments suggest that pretrained multilingual models can transfer useful linguistic knowledge to low-resource historical varieties, although systematic comparisons across training regimes remain scarce ( 23 ; 22 ) . In parallel, research on cross-lingual transfer learning indicates that multilingual training can improve performance on under-resourced languages when genealogical or typological proximity is present ( 12 ; 16 ) . Building on this line of work, the present study provides a unified evaluation of traditional taggers, prompting-based LLM inference, fine-tuning, and multilingual transfer learning for medieval Romance POS tagging. By jointly analyzing multiple datasets and transfer configurations, we aim to clarify the relative strengths of these approaches in realistic historical NLP scenarios. 3 Methodology We design a comparative experimental framework to evaluate traditional and neural approaches to part-of-speech (POS) tagging for medieval Romance languages. The study considers five experimental settings: direct application of traditional taggers, prompting-based large language model (LLM) inference, monolingual fine-tuning, bilingual cross-lingual transfer learning, and trilingual multilingual training. This setup enables systematic analysis of how varying levels of supervision and multilingual exposure affect tagging performance. In particular, the bilingual and trilingual configurations allow us to examine the role of linguistic relatedness and dataset composition in cross-lingual transfer. An overview of the experimental configuration is provided in Table 3 . 3.1 Datasets We employ three historically grounded datasets representing distinct medieval Romance varieties and textual domains, summarized in Table 1 . Further dataset references are available in the accompanying GitHub repository. The three resources differ substantially in size, genre, and century of composition, which we revisit when interpreting cross-lingual transfer results in Section 4 and as a limitation in the closing discussion. Dataset Language Genre Tokens NAF Med. Occitan (14th c.) Literary 45,457 CAT Med. Catalan (13th c.) Chronicle 59,359 Chauliac Med. French (15th