TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks
Özay Ezerceli, Gizem Gümüşçekiçci, Tuğba Erkoç, Berke Özenç · 2025
Read TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks by Özay Ezerceli, Gizem Gümüşçekiçci, Tuğba Erkoç, Berke Özenç (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08376.txt.
Introduction
Ozay Ezerceli and co-authors introduce TurkEmbed, a Turkish language embedding model built to beat current options on inference and similarity tasks. Turkish models often train on machine translated data sets, which can blunt accuracy and semantic depth. TurkEmbed instead combines diverse data sources with advanced training methods, including matryoshka representation learning, which packs useful information into embeddings of several sizes for resource limited settings.
On the Turkish semantic similarity benchmark, correlation scores improve clearly over prior models. The model also passes the previous state of the art system, Emrecan, on Turkish natural language inference and similarity sets, with gains of one to four percent. Faster coding adds practical value for downstream Turkish applications.
Catalog record
| CREATOR | Özay Ezerceli, Gizem Gümüşçekiçci, Tuğba Erkoç, Berke Özenç |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-08376
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks Thanks: Citation : Özay Ezerceli, Gizem Gümüşçekiçci, Tuğba Erkoç, Berke Özenç. "TurkEmbed: Turkish Embedding Model on Natural Language Inference & Sentence Text Similarity Tasks." 2025 IEEE 11th International Conference on Advances in Software, hardware and Systems Engineering (ASYU), 2025. DOI: 10.1109/ASYU67174.2025.11208511 Özay Ezerceli, Gizem Gümüşçekiçci, Tuğba Erkoç, Berke Özenç Abstract This paper introduces TurkEmbed, a novel Turkish language embedding model designed to outperform existing models, particularly in Natural Language Inference (NLI) and Semantic Textual Similarity (STS) tasks. Current Turkish embedding models often rely on machine-translated datasets, potentially limiting their accuracy and semantic understanding. TurkEmbed utilizes a combination of diverse datasets and advanced training techniques, including matryoshka representation learning, to achieve more robust and accurate embeddings. This approach enables the model to adapt to various resource-constrained environments, offering faster encoding capabilities. Our evaluation on the Turkish STS-b-TR dataset, using Pearson and Spearman correlation metrics, demonstrates significant improvements in semantic similarity tasks. Furthermore, TurkEmbed surpasses the current state-of-the-art model, Emrecan, on All-NLI-TR and STS-b-TR benchmarks, achieving a 1-4% improvement. TurkEmbed promises to enhance the Turkish NLP ecosystem by providing a more nuanced understanding of language and facilitating advancements in downstream applications. K eywords Semantic text similarity \cdot matryoshka representation \cdot embedding model \cdot natural language inference \cdot downstream task 1 Introduction Natural Language Processing (NLP) is considered a branch of computational linguistics that focuses on enabling machines to understand, interpret, and generate human language [ 14 ] . It has a wide research area and many diverse applications, the most popular ones are sentiment analysis [ 10 ] , machine translation, and sarcasm detection [ 13 ] . Many of these applications are popular topics that are currently being investigated to find better approaches to ongoing challenges. With recent advances in technology, the NLP field has seen remarkable advancements and there is a continuous need for improvement. NLP applications primarily rely on embeddings to represent words or sentences. An embedding is a numerical representation of words, phrases, or sentences. It is used to convert textual data into numerical vector representations that preserve semantic and syntactic properties. Thus, the performance of embedding systems plays a crucial role in determining the success of NLP systems. Word embeddings are categorized as non-contextual and contextual, each with distinct impacts on NLP models like TurkEmbed. Non-contextual embeddings, such as Word2Vec [ 23 ] , GloVe [ 27 ] , and FastText [ 4 ] , assign fixed representations to words, ignoring variations in meaning based on context [ 1 ] . For instance, the word "bank" is represented identically whether it refers to a "riverbank" or a "financial institution." This limitation makes non-contextual embeddings inadequate for advanced tasks like Natural Language Inference (NLI) and Semantic Textual Similarity (STS), which require contextual understanding [ 22 ] . Contextual embeddings, such as BERT [ 8 ] , ELMo [ 28 ] , and T5 [ 29 ] , address this by generating dynamic representations that adapt to a word’s usage within a sentence, enabling nuanced interpretation. This is crucial for Turkish, a morphologically rich language. TurkEmbed leverages techniques like matryoshka representation learning [ 20 ] to optimize contextual embeddings, overcoming challenges in Turkish morphology and syntax. These advancements position TurkEmbed as a state-of-the-art model for Turkish NLP, highlighting the importance of contextual embeddings in capturing semantic nuances. Embedding models are often language-dependent, with multilingual versions available but potentially less effective for low-resource languages like Turkish due to the need for generalization across multiple languages. While most embedding models are built for English due to the abundance of resources, Turkish NLP research relies on multilingual models or a limited number of Turkish-specific models, which can hinder performance in tasks requiring a deep understanding of semantic relationships; even models like bert-base-turkish-cased-mean-nli-stsb-tr [ 9 ] have room for improvement in semantic similarity tasks. We introduce TurkEmbed, a novel and enhanced Turkish embedding model designed to overcome existing limitations in the Turkish language. The methodology involves combining diverse datasets with advanced training techniques, notably Matryoshka representation learning, and selecting base models from the MTEB leaderboard [ 24 ] . TurkEmbed’s performance was evaluated on semantic similarity tasks using the Turkish STSb [ 3 ] and STS22 [ 16 ] datasets, showing superior results compared to current state-of-the-art models. The main contributions include the TurkEmbed model itself, which excels on NLI and STS tasks, the demonstration of Matryoshka learning’s efficacy for Turkish, and a thorough evaluation on benchmarks like All-NLI-TR, STSb-TR, and STS22-Crosslingual-STS, ultimately aiming to advance Turkish NLP capabilities. The remainder of this paper is organized as follows: Section 2 reviews related works on Turkish embedding models. Section 3 details the methodology employed in developing TurkEmbed. Section 4 describes the experimental setup, and Section 5 presents the results and discussion. Finally, Section 6 concludes the paper and outlines future research directions. 2 Related Work The Turkish language, characterized by its agglutinative morphology and rich vocabulary, presents a unique and compelling challenge for natural language processing (NLP). The development of effective word embedding models is paramount to overcoming these linguistic complexities and enabling robust NLP tools for Turkish. Contextual embedding models like BERT [ 37 ] and ELMo [ 28 ] significantly changed NLP by overcoming the limitations of non-contextual approaches. ELMo employed deep bidirectional language models for context-aware representations, whereas BERT used the Transformer architecture with bidirectional encoders to achieve state-of-the-art results across many NLP tasks. BERT’s capacity to consider the full context of a word was particularly advantageous for Turkish, facilitating a more nuanced understanding of meaning in complex sentences. Research confirmed that BERT-based models outperformed earlier non-contextual methods in various Turkish NLP tasks, highlighting the crucial role of contextual awareness for the language. Building upon the success of BERT, Turkish-specific BERT models are developed and trained on large-scale Turkish corpora like the Boun Web Corpus [ 33 ] and the Huawei Corpus [ 35 ] . These models, pre-trained on extensive Turkish text, offered enhanced performance on downstream tasks due to a better understanding of language-specific nuances. Furthermore, research has explored adapting and fine-tuning these models for specific Turkish NLP tasks. [ 9 ] ’s work, for example, utilized machine-translated datasets to fine-tune BERT-based models for Turkish NLI and STS, establishing initial benchmarks for these tasks and highlighting the feasibility of cross-lingual transfer learning for Turkish NLP. The YTU Cosmos Lab [ 19 ] further introduced Turkish BERT models trained on a sizable corpus of 75GB, compiled from various sources to enhance the diversity and representativeness of the data. These models aimed to improve performance on downstream tasks such as text classification and named entity recognition. Although they provided a solid foundation, they required significant computational resources for training and did not specifically address tasks like NLI and STS. Beyond core NLP tasks, adaptations of existing architectures have also emerged. Turkish-ColBERT [ 34 ] adapted the ColBERT architecture, initially designed for English information retrieval, to the Turkish language, showcasing the adaptability of advanced retrieval models to morphologically complex languages. It was fine-tuned on the machine-translated MS MARCO dataset [ 25 ] , utilizing over 500,000 translated queries and passages. While it demonstrated improved retrieval performance, the reliance on