ParsiPy: NLP Toolkit for Historical Persian Texts in Python

Farhan Farsi, Parnian Fazel, Sepand Haghighi, Sadra Sabouri, Farzaneh Goshtasb, Nadia Hajipour, Ehsaneddin Asgari, Hossein Sameti · 2025

Read ParsiPy: NLP Toolkit for Historical Persian Texts in Python by Farhan Farsi, Parnian Fazel, Sepand Haghighi, Sadra Sabouri, Farzaneh Goshtasb, Nadia Hajipour, Ehsaneddin Asgari, Hossein Sameti (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17810.txt.

Introduction

Farhan Farsi and seven co-authors release a Python toolkit for computers working with historical Persian texts, in this 2025 paper. Old languages resist standard text tools: scripts vary, texts survive in fragments, and digital forms lack standards. Their toolkit, ParsiPy, targets the pipeline from raw text to analysis.

Modules split text into tokens, reduce words to their dictionary forms, tag grammar roles, convert between sounds and transliteration, and build word embeddings. The team demonstrates the kit on Parsig, the corpus of Middle Persian, the language of pre Islamic Iran. Processing those texts shows the modules working end to end. The authors offer ParsiPy to computational philology at large, noting the design can adapt to other ancient languages whose digital preservation lags behind.

Catalog record

CREATORFarhan Farsi, Parnian Fazel, Sepand Haghighi, Sadra Sabouri, Farzaneh Goshtasb, Nadia Hajipour, Ehsaneddin Asgari, Hossein Sameti
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17810

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ParsiPy: NLP Toolkit for Historical Persian Texts in Python Farhan Farsi, Parnian Fazel, Sepand Haghighi, Sadra Sabouri, Farzaneh Goshtasb, Nadia Hajipour, Ehsaneddin Asgari, Hossein Sameti Abstract The study of historical languages presents unique challenges due to their complex orthographic systems, fragmentary textual evidence, and the absence of standardized digital representations of text in those languages. Tackling these challenges needs special NLP digital tools to handle phonetic transcriptions and analyze ancient texts. This work introduces ParsiPy 1 1 1 https://github.com/openscilab/parsipy , an NLP toolkit designed to facilitate the analysis of historical Persian languages by offering modules for tokenization, lemmatization, part-of-speech tagging, phoneme-to-transliteration conversion, and word embedding. We demonstrate the utility of our toolkit through the processing of Parsig (Middle Persian) texts, highlighting its potential for expanding computational methods in the study of historical languages. Through this work, we contribute to computational philology, offering tools that can be adapted for the broader study of ancient texts and their digital preservation. 1 Introduction Ancient languages serve as windows into the past, offering valuable insights into human history and the evolution of communication. The connection between language and culture has long been recognized, with scholars using ancient languages such as Greek Ostwald (2009) , Italian Lomas (2013) , and Latin Farrell (2001) to uncover the social contexts of historical civilizations. These languages not only preserve cultural heritage but also provide a lens for studying the development of linguistic structures and thought patterns Kaplan (2013) . Despite significant advancements in Natural Language Processing (NLP), which have transformed the study of modern languages, the application of these technologies to ancient languages remains underexplored Magueresse et al. (2020) . As preliminary attempts, some researchers have tailored NLP tools developed for Pre-modern English Johnson et al. (2021b) and Sumerian Guzman-Soto and Liu (2023) , yet many historically significant languages, such as Old Persian and Middle Persian (Pārsīg), still lack sufficient computational resources and tools. Expanding NLP research to include these underserved languages can help bridge critical gaps in historical linguistics while contributing to the preservation of invaluable cultural knowledge. Pārsīg, represents one such language Haug (1870) . Despite its historical importance as a bridge between ancient Iranian languages and modern Persian (see Appendix A for more details), Pārsīg has received minimal attention in computational linguistics. Its challenges include a highly limited digital corpus, complex writing system variations, and the absence of standardized computational resources for processing Pārsīg texts in the originally written form. To address this gap, we introduce ParsiPy, the first NLP toolkit in Python specifically designed for processing Pārsīg. Our framework includes tools for word embeddings, lemmatization, tokenization, and part-of-speech (POS) tagging. Our POS tagging system includes three models—Hidden Markov Model with Viterbi decoding, logistic regression, and random forest. Pārsīg was written in multiple scripts, but the Book Pahlavi script, widely used in Zoroastrian texts, lacks a Standard Unicode encoding 2 2 2 https://www.unicode.org/standard/unsupported.html . As a result, most digital resources rely on phonetic transcriptions. To address this, ParsiPy includes a phoneme-to-transliteration module with rule-based and LSTM models. We also provide a tool for converting this transliteration to Book Pahlavi. Future work could develop Unicode support, enabling broader computational applications. ParsiPy addresses the challenges of processing Pārsīg texts by using rule-based and statistical methods, which are more effective than large language models for this low-resource language. As a foundational NLP toolkit, ParsiPy enhances computational analysis of Pārsīg, supports digital research, and serves as a model for similar efforts in other ancient languages. The code is available as a Python package on GitHub. In this paper, we outline the structure of the Pārsīg language in Section 2 , followed by related works on NLP toolkits for ancient languages in Section 3 . Section 4 details the system design of ParsiPy, while Section 5 describes the dataset used for training and evaluation. We then assess our toolkit in Section 6 and discuss future research directions in Section 7 . Figure 1: An example of Pārsīg text in original written form from Andarze Azarabade Mehrsepandan database. It reads “ ān uzīd frāmōš kun ud ān nē mad ēstēd rāy tēmār bēš ma bar ” and it means “ Forget what is gone and do not worry about what has not yet come. ” 2 Pārsīg Language Structure Pārsīg, the language of the Sassanian Empire (224–651 CE), an ancestor of modern Persian (Farsi), has a unique linguistic structure that can be divided by specific features in phonology, morphology, syntax, and orthography. In this section, we go through its specific characteristics. Phonology and Orthography of Pārsīg is similar to that of modern Persian, though there are important historical phonetic differences. The Pārsīg alphabet consists of only fourteen letters to represent the entire range of sounds, as illustrated in the Appendix. Consequently, several letters possess multiple phonetic values. This variation in phonetic values presents challenges in reading Pārsīg.The difficulty is further compounded by the different shapes the letters can take, depending on their position in the word MacKenzie (1971) . A significant portion of Pārsīg words is written using Aramaeograms (known as uzwārišn), where words of Aramaic origin are spelled using Pārsīg characters Farzaneh Goshtasb and Ghayoomi (2023) . Morphology of Pārsīg is primarily inflectional, with both nouns and verbs marked for grammatical roles such as case, tense, and mood. Pārsīg originally had two cases: one reserved for the grammatical subject, and the other for all other syntactic functions (oblique). These cases are commonly referred to as the ‘direct’ case (used for the subject of the sentence) and the ‘oblique’ case (used for objects, indirect objects, and other syntactic functions) Brown (2005) . Verbs are inflected for various grammatical features such as tense, mood, and person Brunner (1977) . Additionally, Pārsīg verbs often include the use of verbal particles and suffixes to convey different meanings and functions, which can make morphological analysis complex. Another common feature is enclitic pronouns, short pronoun-like elements that attach to words to show possession or objects, which can make segmentation tasks complex. Syntax of Pārsīg follows a Subject-Object-Verb (SOV) word order Mohammad Dabir Moghaddam (2014) , but this structure is flexible depending on context or emphasis. This variability makes syntactic parsing more challenging. The language also uses prepositions and postpositions, and relative clauses often form with subordinators, requiring tools to detect clause boundaries accurately. Semantic and Lexical Features. The vocabulary of Pārsīg includes many loanwords from Aramaic Shaked (2005) , which creates challenges for distinguishing between native and borrowed words. Also, due to the script’s lack of vowel markings, polysemy (words with multiple meanings) and tomography (identical spellings with different pronunciations) present challenges. These features complicate tasks like word sense disambiguation and machine translation. Developing NLP tools for Pārsīg requires addressing these unique linguistic features. Techniques such as character-level models for handling logograms, graph-based parsing for non-fixed word order, and morphological analyzers for suffix-rich structures can be particularly effective. This paper uses an excerpt from a Zoroastrian manuscript Goshtasb and Hajipour (2022) , originally written in P=ars=ig, as an example. Figure 1 shows the original handwritten text. The passage is from Andarze Azarabade Mehrsepandan , a collection of life advice, with an English translation: Forget what is gone and do not worry about what has not yet

Read the complete text