A POS-tagged dataset of Uzbek word lemmas and stems for morphological and NLP research Authors: Maksud Sharipov Published: 2026 ABSTRACT AbstractThis article presents a comprehensive, multi-domain dataset of stems and lemmas for the Uzbek language, systematically categorized by parts-of-speech (POS). The dataset contains 46,307 unique stems and 64,052 corresponding lemmas classified into 12 distinct grammatical categories, including independent, auxiliary, and intermediate word classes. Data were methodically extracted from authoritative sources such as the Explanatory Dictionary of the Uzbek Language and various digital text corpora. Following a rigorous right-to-left suffix stripping methodology, words were segmented to isolate the base stems, and their corresponding canonical dictionary forms (lemmas) were identified. The finalized linguistic data is structured in a machine-readable tabular (.xlsx) format consisting of three essential columns: stem, lemmas, and POS tag. This dataset provides a foundational open-source resource for natural language processing (NLP) applications in the morphologically complex, agglutinative Uzbek language, actively supporting the development, training, and benchmarking of rule-based and machine-learning tools such as lemmatizers, stemmers, and POS taggers.Keywords: Uzbek, Inflexion, Sound change, Lemma, Lemmatization, Stem, Stemming, Linguistic supply Specifications Table SubjectComputer Science, Natural Language Processing, Computational LinguisticsSpecific subject areaAI model training process, Lemmatization using lexicons, Stemming using lexicons, machine translation, morphological analysis.Type of dataTable (.xlsx format)Analyzed, ProcessedData collectionThe data compilation process involved systematically extracting and categorizing Uzbek stems and lemmas according to their respective parts of speech. A comprehensive linguistic database was processed to isolate 46,307 unique stems and 64,052 corresponding lemmas spanning 12 distinct grammatical categories. The finalized data was structured into a three-column format (stem, lemmas, POS tag) to ensure machine-readability and immediate applicability for natural language processing applications.Data source locationUrgench State University named after Abu Rayhan Biruni, Urgench, Uzbekistan.Data accessibilityPlatform: GitHubRepository name: UzbekLemmaStems-POS-DatasetDirect URL to data: https://github.com/MaksudSharipov/UzbekLemmaStems-POS-DatasetDOI: https://doi.org/10.5281/zenodo.20638468Licence: Apache-2.0 licenseInstructions for accessing these data: The dataset is openly accessible to the public and can be downloaded directly from the provided GitHub repository.Related research articleNone Value of the Data The following are the values of the presented data: •Provides a massive and comprehensive dataset of Uzbek stems and lemmas, comprising 46,307 unique stems and 64,052 corresponding lemmas categorized across 12 distinct parts of speech (POS). This addresses the critical need for large-scale, open-source morphological resources in the morphologically complex and under-resourced Uzbek language [1,2]. Provides a massive and comprehensive dataset of Uzbek stems and lemmas, comprising 46,307 unique stems and 64,052 corresponding lemmas categorized across 12 distinct parts of speech (POS). This addresses the critical need for large-scale, open-source morphological resources in the morphologically complex and under-resourced Uzbek language [1,2]. •Facilitates the development and evaluation of NLP tools, including stemmers, lemmatizers, morphological analyzers, and POS taggers. Researchers and software developers can utilize this tabular dataset as a reliable dictionary or lookup table to train, test, and benchmark rule-based systems and artificial intelligence models [3]. Furthermore, its structured tabular format ensures seamless integration with standard modern NLP frameworks and libraries (such as NLTK, spaCy, and Hugging Face) for custom pipeline development. Facilitates the development and evaluation of NLP tools, including stemmers, lemmatizers, morphological analyzers, and POS taggers. Researchers and software developers can utilize this tabular dataset as a reliable dictionary or lookup table to train, test, and benchmark rule-based systems and artificial intelligence models [3]. Furthermore, its structured tabular format ensures seamless integration with standard modern NLP frameworks and libraries (such as NLTK, spaCy, and Hugging Face) for custom pipeline development. •Supports Information Retrieval (IR) and text normalization tasks where mapping inflected words to their base forms (lemmas) or stems is crucial for reducing vocabulary size and improving search engine performance [[2], [3], [4]]. Supports Information Retrieval (IR) and text normalization tasks where mapping inflected words to their base forms (lemmas) or stems is crucial for reducing vocabulary size and improving search engine performance [[2], [3], [4]]. •Enables cross-linguistic and comparative studies within the Turkic language family. The structured morphological data can assist in transfer learning, cross-lingual NLP tool building, and typological analyses among related agglutinative languages such as Turkish, Kazakh, and Kyrgyz [5]. Enables cross-linguistic and comparative studies within the Turkic language family. The structured morphological data can assist in transfer learning, cross-lingual NLP tool building, and typological analyses among related agglutinative languages such as Turkish, Kazakh, and Kyrgyz [5]. Background The Uzbek language is a widely spoken agglutinative language characterized by a highly complex morphology, where new words and grammatical forms are created by attaching multiple suffixes to a stem. Despite its linguistic richness, Uzbek is still considered a low-resource language in the domain of Natural Language Processing (NLP) [6]. Fundamental text processing tasks, such as lemmatization and stemming, rely heavily on mapping inflected word forms to their dictionary forms (lemmas) or unchangeable stems [2]. However, existing morphological tools for Uzbek frequently depend on constrained lexicons or limited rules, reducing their overall applicability and accuracy [1]. The primary motivation for compiling this dataset was to address this resource gap by systematically curating a large-scale, multi-POS repository of Uzbek stems and lemmas. By isolating 46,307 stems and 64,052 lemmas into 12 grammatical categories, this dataset provides an essential foundational framework. It is specifically designed to function as a comprehensive linguistic database and ground-truth lexicon to support dictionary-based, rule-based, and AI-driven morphological analysis algorithms, ultimately accelerating the advancement of computational linguistics for the Uzbek language. While the foundational lexical items are derived from existing secondary sources, the novel contribution of this work lies in the extensive data transformation—specifically, the rigorous suffix stripping, orthographic normalization, and structured alignment into a newly accessible, machine-readable format. Data Description The dataset provides a comprehensive, POS-tagged collection of Uzbek stems and lemmas. The main directory of the dataset, named stem_lemma_pos, is structured into sub-categories based on the three fundamental word super-classes in the Uzbek language: •mustaqil (Independent): Contains data for independent or content word classes (Noun, Verb, Adjective, Adverb, Numeral, Pronoun). mustaqil (Independent): Contains data for independent or content word classes (Noun, Verb, Adjective, Adverb, Numeral, Pronoun). •yordamchi (Auxiliary): Contains data for auxiliary or function word classes (Auxiliary, Conjunction, Particle). yordamchi (Auxiliary): Contains data for auxiliary or function word classes (Auxiliary, Conjunction, Particle). •oraliq (Intermediate): Contains data for intermediate word classes (Modal, Interjection, Imitation). oraliq (Intermediate): Contains data for intermediate word classes (Modal, Interjection, Imitation). The data is provided in a tabular format as an Excel (.xlsx) file, encoded in UTF-8 to ensure full compatibility with the Uzbek Latin alphabet. The dataset is complete, meaning there are no missing values; every stem is mapped to at least one corresponding lemma and possesses exactly one POS tag. The dataset is strictly organized into three columns to ensure machine-readability and ease of use in NLP pipelines: •Column 1 (Stem): Represents the base segment or stem isolated from the words. Each row contains exactly one unique stem. Column 1 (Stem): Represents the base segment or stem isolated from the words. Each row contains exactly one unique stem. •Column 2 (Lemmas): Contains a list of dictionary headwords (lemmas) that correspond to the stem. A single stem can be associated with multiple lemmas (representing derivational variants and polysemy), which are separated by commas within the same cell. To assist users in selecting the primary lemma, these lists are ordered by frequency of occurrence in the source corpora. Column 2 (Lemmas): Contains a list of dictionary headwords (lemmas) that correspond to the stem. A single stem can be associated with multiple lemmas (representing derivational variants and polysemy), which are separated by commas within the same cell. To assist users in selecting the primary lemma, these lists are ordered by frequency of occurrence in the source corpora. •Column 3 (POS): Indicates the grammatical part-of-speech category for the given stem and its associated lemmas. Each row is assigned exactly one POS tag. Column 3 (POS): Indicates the grammatical part-of-speech category for the given stem and its associated lemmas. Each row is assigned exactly one POS tag. Table 1 presents the quantitative distribution of the dataset. It details the exact count of unique stems and their corresponding lemmas across all 12 grammatical categories. The entire corpus consists of 46,307 stems and 64,052 lemmas. Table 1Statistical distribution of stems and lemmas across different part-of-speech (POS) categories.Table 1 dummy alt textPart of Speech (POS)Stem CountLemma CountNoun (Ot)Verb (Fe'l)Adjective (Sifat)Adverb (Ravish)Numeral (Son)Pronoun (Olmosh)Auxiliary (Ko'makchi)Conjunction (Bog'lovchi)Particle (Yuklama)Modal (Modal)Imitation (Taqlid)Interjection (Undov)39,5274,0031,4397113520834225852419648,12411,6692,227996416488342258524196Total46,30764,052 Statistical distribution of stems and lemmas across different part-of-speech (POS) categories. To familiarize the reader with the structure and contents of the files, Table 2 provides a representative sample of the data entries extracted from the dataset. Table 2A sample excerpt from the dataset illustrating the structure of stems, multiple lemmas, and POS tags.Table 2 dummy alt textStemLemmasPart of Speech (POS)a’zoaartsenabadobodobro‘ob’ektochpovpoypoygirpoylaa’zo, a’zolikaartsenabad, abadiy, abadiya, abadiyan, abadiyat, abadiylikobodobro‘ob’ekt, ob’ektivoch, ochiq, ochiqarli, ochiqcha, ochiqchasiga, ochiqgo‘ypov, povullapoy, poylapoygir, poygirlapoyla, poylan, poylash, poylat, poylatil, poylatqiz, poylattirNounNounNounAdjectiveAdjectiveAdjectiveAdjectiveVerbVerbVerbVerb A sample excerpt from the dataset illustrating the structure of stems, multiple lemmas, and POS tags. Experimental Design, Materials and Methods The initial lexical inventory was compiled from authoritative Uzbek language references, yielding a raw, uncleaned total of 72,140 headwords and inflected forms—prior to any deduplication or normalization—extracted primarily from the Explanatory Dictionary of the Uzbek Language (EDUL)1. This was supplemented by carefully curated digital text sources to ensure broad coverage of contemporary usage across multiple domains. All extracted strings were normalized to a consistent Uzbek Latin (UZ-Latin) orthography, and duplicate entries were systematically removed. Each lexical item was subsequently assigned one of twelve Part-of-Speech (POS) tags, which were further grouped into three super-classes: Independent (Noun, Adjective, Verb, Adverb, Numeral, Pronoun), Auxiliary (Auxiliary, Conjunction, Particle), and Intermediate (Modal, Interjection, Imitation). POS annotation and morphological feature assignment were conducted using a compact, Universal Dependencies (UD)-compatible tagset tailored to the agglutinative morphology of the Uzbek language. The morphological decomposition followed a rigorous right-to-left segmentation approach (suffix stripping). Starting from the right edge of each word, grammatical units such as inflectional suffixes and derivational endings were sequentially identified. A specific notation policy was enforced during this phase: segments were separated by backslashes, while a trailing forward slash (/) was used exclusively as a terminal delimiter to mark the absolute word boundary. The forward slash does not represent a morphological segment itself (e.g., MORPH1\MORPH2.../). Furthermore, rare inherited or borrowed prefixes were explicitly flagged with an asterisk (*) before the segmented chain, reserving this symbol strictly for prefix signaling (e.g., *prefix\stem.../). It is important to emphasize that this specific notation policy was strictly an intermediate mechanism utilized during the morphological decomposition phase to accurately track morpheme boundaries. Once the base forms were successfully isolated and verified, all segmentation markers (backslashes, trailing slashes, and asterisks) were computationally removed. To illustrate this morphological decomposition process, Fig. 1 provides a traceable example of the pipeline, demonstrating the transformation of a raw inflected word into its finalized dataset entry. The complete inventory of stripped affixes, alongside the visual block diagram detailing the exact algorithmic segmentation rules and exception-handling logic, is freely available in Supplementary Material 1. The transformation pipeline of a raw inflected word into its finalized dataset entry. Following segmentation, the stem (the morphologically reduced base) was isolated. These stems reflect the complex morphotactics of the Uzbek language and may occasionally differ from the standard citation form. Concurrently, the canonical dictionary headword representing the semantic core was recorded as the lemma. In instances where multiple related lemmas corresponded to a single surface form, they were documented as an ordered, comma-separated list within the dataset. To accelerate the annotation process while maintaining high reliability, trained human annotators were assisted by rule-based candidate analyses generated from grammatical rules and a growing lexicon of validated lemma-stem pairs. Affixes were strictly categorized into inflectional and derivational groups, with the order of affixation explicitly preserving the sequential morpheme attachment typical of Turkic languages. Ambiguous cases, such as word forms with multiple plausible POS categories or competing lemma assignments, were resolved through consensus among multiple expert reviewers. The annotation team consisted of two primary annotators and one senior expert reviewer. To ensure the reliability of the manual annotation before the consensus phase, an inter-annotator agreement (IAA) analysis was conducted on a random sample of 5% of the dataset, yielding a Cohen’s Kappa coefficient of 0.84 for POS tagging and 0.81 for lemma assignment, indicating a high level of annotator consistency. Also, the raw absolute agreement rate between annotators was 88.5% for POS tags and 86.2% for lemma assignments across the stratified sample, confirming robust consensus even within low-frequency grammatical categories. Furthermore, the 5% random sample utilized for the Inter-Annotator Agreement (IAA) analysis was strictly stratified by POS category. This stratification ensured that low-frequency grammatical classes, such as Interjections and Particles, were proportionately evaluated alongside high-frequency categories like Nouns and Verbs. Finally, the resource assembly underwent stringent quality control. This included automated regex-based checks for segmentation patterns, duplicate detection, and tag validity. Frequency-based audits were conducted to identify systematic labeling drift, and targeted manual re-evaluations were performed on rare or highly complex morphological patterns to ensure dataset-wide consistency across the full vocabulary. Limitations Although the dataset provides extensive coverage of Uzbek stems and lemmas across 12 part-of-speech categories, several limitations should be acknowledged. First, the resource is lexicon-based and primarily derived from dictionary entries and curated textual sources; therefore, it may not fully represent newly emerging vocabulary, neologisms, domain-specific terminology, or informal language usage found in social media and conversational texts. Second, while the dataset records stem–lemma relationships and POS categories, it does not include complete morphological feature annotations (e.g., case, number, tense, person, or mood) for individual word forms. Third, due to the rich derivational morphology of Uzbek, certain stems may correspond to multiple semantically related lemmas, and although expert review was employed to ensure consistency, some ambiguity may remain in rare linguistic cases. Finally, the dataset is limited to Uzbek written in the Latin script and does not explicitly cover Cyrillic orthographic variants. Despite these limitations, the resource constitutes one of the largest openly available collections of Uzbek stems and lemmas and provides a valuable foundation for morphological analysis, lemmatization, stemming, and broader NLP research. Ethics Statement The author confirms that the current work does not involve human subjects, animal experiments, or any data collected from social media platforms. Furthermore, no surveys, questionnaires, or primary empirical elicitation methods were employed in the creation of this dataset. All the lexical resources were gathered from publicly available linguistic texts and dictionaries. Credit Author Statement Maksud Sharipov: Conceptualization, Methodology, Data Curation, Formal analysis, Resources, Writing - Original Draft, Administration. Acknowledgements This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors would like to thank the professors and master’s students from Urgench state university, who are expert linguists in the Uzbek language, for their valuable contribution to the manual annotation of the dataset. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.dib.2026.113072. Specific details of the corpus: 5-volume edition, edited by A. Madvaliyev, 2006–2008, National Encyclopedia of Uzbekistan State Scientific Publishing House. Supplementary materials Data Availability GitHubUzbekLemmaStems-POS-Dataset (Original data) GitHubUzbekLemmaStems-POS-Dataset (Original data) References Associated Data Supplementary Materials Data Availability Statement GitHubUzbekLemmaStems-POS-Dataset (Original data) GitHubUzbekLemmaStems-POS-Dataset (Original data)