Lemmatization as a Classification Task: Results from Arabic across Multiple Genres

Mostafa Saeed 1, Nizar Habash 1 · 2025

Read Lemmatization as a Classification Task: Results from Arabic across Multiple Genres by Mostafa Saeed 1, Nizar Habash 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-18399.txt.

Introduction

Mostafa Saeed and Nizar Habash tackle Arabic lemmatization in this two thousand twenty-five paper. Lemmatization cuts words back to their dictionary base forms. It counts much for Arabic, with rich word forms and spelling that can be read more than one way. Tools now in use face mixed standards and cover few kinds of text. The team puts forth two ways, both treating it as sorting into a tag set: lemma, part of speech, and an English gloss.

One way draws on machine translation; the other groups by meaning. The team built a new test set. It spans many kinds of text; older sets were brought to one standard next to it. Character-level sequence-to-sequence models did well. But they could give lemmas only, not the tag set, and at times made up odd forms. The sorting and grouping ways gave stronger, clearer output and set marks.

Catalog record

CREATORMostafa Saeed 1, Nizar Habash 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-18399

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Lemmatization as a Classification Task: Results from Arabic across Multiple Genres Mostafa Saeed 1, Nizar Habash 1 Abstract Lemmatization is crucial for NLP tasks in morphologically rich languages with ambiguous orthography like Arabic, but existing tools face challenges due to inconsistent standards and limited genre coverage. This paper introduces two novel approaches that frame lemmatization as classification into a Lemma-POS-Gloss (LPG) tagset, leveraging machine translation and semantic clustering. We also present a new Arabic lemmatization test set covering diverse genres, standardized alongside existing datasets. We evaluate character-level sequence-to-sequence models, which perform competitively and offer complementary value, but are limited to lemma prediction (not LPG) and prone to hallucinating implausible forms. Our results show that classification and clustering yield more robust, interpretable outputs, setting new benchmarks for Arabic lemmatization. \setcode utf8 1 Introduction Lemmatization is the process of mapping a word to a base form that abstracts away from its inflectional variants. Lemmatization has played an important enabling technology role in many NLP applications, including machine translation Conforti et al. 2018 , information retrieval Semmar et al. 2006 , parsing Seddah et al. 2010 , text classification Abdelrahman et al. 2021 and summarization El-Shishtawy and El-Ghannam 2014 . Despite the shift toward large language models, lemmatization remains essential for tasks involving morphologically rich languages and requiring interpretability, such as readability assessment Al Khalil et al. 2018 ; Liberato et al. 2024 or automated error detection Belkebir and Habash 2021 . Table 1: Eight possible Lemma-POS-Gloss analyses for the Arabic word \< عقد> Eqd . Transliteration is in Buckwalter 2002 ’s scheme. Lemmatization is especially challenging in morphologically rich languages like Arabic due to complex morphology and optional diacritics. Table 1 presents multiple out-of-context analyses of a single word, varying in diacritization, lemma, POS, and English gloss (as a proxy for sense). Previous lemmatization approaches rely on morphological analyzers and ranking models Roth et al. 2008 , sequence-to-sequence (seq2seq) generation Bergmanis and Goldwater 2018a ; Zalmout and Habash 2020 , or edit-based tagging Gesmundo and Samardžić 2012 ; Kondratyuk et al. 2018a . However, these methods often focus only on the lemma form, lack generalization across domains, and rely on narrow lexical resources or genre-specific training data. In this work, we propose a broader and more interpretable framing of lemmatization as classification into a rich Lemma-POS-Gloss (LPG) tagset. Our contributions are: First , we introduce two novel approaches that classify into LPG labels, one leveraging machine translation of source sentences and dictionary glosses, and the other using LPG semantic clustering. Second , we present a new multi-genre Arabic lemmatization test set, covering underexplored domains such as novels and children’s stories. Our experiments demonstrate that LPG-based classification and clustering approaches outperform prior systems that resolve most morphosyntactic ambiguity Inoue et al. 2021b , offering superior accuracy and robustness. We also evaluate character-level seq2seq models, which perform competitively and provide complementary benefits, but are limited to lemma-only (not LPG) prediction and often hallucinate implausible forms. Hybrid models that combine seq2seq and classification techniques further boost performance. All code, models, and annotations will be released to support continued research in Arabic lemmatization. The paper is organized as follows: § 2 covers linguistic background, § 4.2 reviews related work, § 4 describes the dataset, § 5 outlines our methods, and § 6 presents the evaluation results. Unique Avg All Top Ambig \downarrow Recall Analyses 15.5 1.3 91.3% LPG 2.7 1.3 52.8% 96.2% LP 2.5 1.2 52.6% 98.8% L 2.0 1.2 42.4% 99.6% Table 2: Avg # of unique entries of CAMeL Tools analyzer and disambiguator on the ATB Dev set in terms of full morphological analyses, Lemmas ( L ), Part-of-Speech ( P ) and Gloss ( G ) combinations. All refers to all returned unique values per word; and Top refers to all remaining values after filtering with the POS Tagger. Ambig \downarrow shows the effect of the POS Tagger. Recall shows the maximal potential accuracy for each representation combination. 2 Linguistic Background Arabic is morphologically and orthographically rich, with optional diacritics and multiple word forms contributing to ambiguity in both meaning and structure. Previous research has focused on lemma alone (L) or lemma with POS (LP), but none have examined the more complex Lemma, POS, and Gloss (LPG). This study aims to fill this gap while evaluating simpler variations for completeness. We use the CAMeL Tools analyzer-and-disambiguator system as our baseline Inoue et al. 2021b ; Obeid et al. 2020 , which returns a set of ranked morphological analyses per word, including gender, number, clitics, POS, and 37 other features. While this helps resolve many morphosyntactic ambiguities, it does not fully disambiguate lemma or sense, which are often given the same rank. For instance, for \< بعقدها> bEqdhA , the model correctly rules out a verbal interpretation, but ambiguity remains among nominal readings such as ‘in her contract,’ ‘necklace,’ or ‘complexes’ (see Table 1 ). As shown in Table 2 , the analyzer initially produced an average of 15 analyses per word on the dev set. Restricting to the top-ranked disambiguator output reduced this to 1.3 analyses per word on average, a 91.3% reduction via morphosyntactic feature tagging. For LPG selection, ambiguity was reduced by 52.8% but capped at 96.2% recall. The LPG space remains significantly more ambiguous than LP or L alone, making full disambiguation substantially more challenging. Table 3: Examples highlighting differences in lemma representations across the data sets we synchronize. 3 Related Work 3.1 Lemmatization Resources In Arabic lemmatization, morphological dictionaries and analyzers serve as the primary resources for nearly all previous works in this task Maamouri et al. 2010 ; Boudchiche et al. 2017 ; Taji et al. 2018a ; Jarrar et al. 2024 . These analyzers or dictionaries extract the lemmas in an out-of-context manner based on morphosyntactic features. While they provide a strong foundation for lemmatization, their reliance on predefined linguistic rules limits adaptability to contextual variations. In this paper, we make use of the CALIMA-S31 analyzer Taji et al. 2018a , which extends SAMA31 Maamouri et al. 2010 , and is used inside CAMeL Tools Obeid et al. 2020 . Several benchmark datasets exist for Arabic lemmatization, including the Penn Arabic Treebank Maamouri et al. 2004 , Zaebuc Habash and Palfreyman 2022 , Wiki News Mubarak 2018 , Salma Jarrar et al. 2024 , Quranic Dukes and Habash 2010a , and Nemlar Yaseen et al. 2006 . However, most are heavily skewed toward the news genre, limiting their applicability to diverse linguistic contexts. In addition, inconsistencies in lemma definitions and diacritic conventions complicate fair comparisons across systems Elgamal et al. 2024 . Table 3 highlights some of the differences using three example lemmas. To address this, we apply a synchronization method to standardize lemma and diacritic representations, enabling more consistent evaluation. We also introduce a new multi-genre benchmark dataset to expand coverage beyond news and support more comprehensive assessment of lemmatization approaches. Most of the aforementioned datasets are included in our evaluation to ensure broad generalization. 3.2 Lemmatization Approaches Lemmatization has been tackled through various paradigms. One common approach relies on morphological dictionaries, framing lemmatization as the selection of the correct lemma from a predefined lexicon Jarrar et al. 2024 ; Mubarak 2017 ; Jongejan and Dalianis

Read the complete text