Solving Word-Sense Disambiguation and Word-Sense Induction with Dictionary Examples
2025
Read Solving Word-Sense Disambiguation and Word-Sense Induction with Dictionary Examples (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04328.txt.
Introduction
Škvorc and Robnik-Šikonja showed in twenty twenty-five how word lists can train chat models to tell word senses apart in a small tongue. Many small tongues lack the large marked sets such tools need. Word lists, by contrast, hold rich sense facts that are oft left unused.
They turn list sample lines into pairs and ask if a word keeps the same sense in both. A tool trained on that pair task can then do the full sense tasks. Pairs are made by chat models from the list lines. Their tools beat past tools on all three tasks, shown on the Slovene tongue.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-04328
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Solving Word-Sense Disambiguation and Word-Sense Induction with Dictionary Examples Abstract Many less-resourced languages struggle with a lack of large, task-specific datasets that are required for solving relevant tasks with modern transformer-based large language models (LLMs). On the other hand, many linguistic resources, such as dictionaries, are rarely used in this context despite their large information contents. We show how LLMs can be used to extend existing language resources in less-resourced languages for two important tasks: word-sense disambiguation (WSD) and word-sense induction (WSI). We approach the two tasks through the related but much more accessible word-in-context (WiC) task where, given a pair of sentences and a target word, a classification model is tasked with predicting whether the sense of a given word differs between sentences. We demonstrate that a well-trained model for this task can distinguish between different word senses and can be adapted to solve the WSD and WSI tasks. The advantage of using the WiC task, instead of directly predicting senses, is that the WiC task does not need pre-constructed sense inventories with a sufficient number of examples for each sense, which are rarely available in less-resourced languages. We show that sentence pairs for the WiC task can be successfully generated from dictionary examples using LLMs. The resulting prediction models outperform existing models on WiC, WSD, and WSI tasks. We demonstrate our methodology on the Slovene language, where a monolingual dictionary is available, but word-sense resources are tiny. Keywords: large language models, word-sense disambiguation, word-sense induction, word-in-context, dictionary examples Solving Word-Sense Disambiguation and Word-Sense Induction with Dictionary Examples Tadej Škvorc and Marko Robnik-Šikonja University of Ljubljana, Faculty of Computer and Information Science Večna pot 113, 1000 Ljubljana, Slovenia [email protected],[email protected] Abstract content 1. Introduction State-of-the-art approaches in most natural language processing (NLP) tasks such as machine translation Dabre et al. 2020 , text generation Iqbal and Qureshi 2022 , and word-sense disambiguation Bevilacqua et al. 2021 currently rely on pre-trained large language models (LLMs). In spite of LLMs’ versatility, for many complex tasks, we still require relatively large, labeled datasets to fine-tune the models. While such datasets are available in English and other well-resourced languages, they rarely exist in the majority of other, less-resourced languages. Consequently, NLP tools in such languages often support significantly fewer tasks compared to English. Finding alternative approaches to create the necessary data is therefore a research of high significance and practical relevance. One way to address the lack of data is to automatically generate training instances using LLMs. A key advantage of pre-trained LLMs is their self-supervised training, i.e. that they can be trained on unlabelled text Shorten et al. 2021 . Such generative approaches are usually task-specific and generate new data based on existing examples or domain knowledge. For text, this was traditionally done with rule-based approaches and, recently, through auxiliary LLMs to augment existing data. Such models take a (weakly related) text and extend it in a way that tries to mimic human-generated language. We argue that traditional linguistic resources, such as dictionaries, available in many less-resourced languages, can be a source of relevant texts and information. Recently, generative LLMs have advanced significantly and models such as LaMDA Thoppilan et al. 2022 , GPT-3 Brown et al. 2020 , GPT-4 OpenAI Team 2023 and Llama-3 Dubey et al. 2024 show excellent performance in many tasks. In comparison to traditional language models, LLMs are trained on larger datasets, contain larger amounts of network parameters, and are human-aligned with instruction fine-tuning, allowing them to generate texts that closely match those produced by humans. Furthermore, existing work has shown that such models can be used to solve specific tasks without the need for manually annotated data with the so-called in-context learning. Examples include solving grade-school math assignments Cobbe et al. 2021 and generating code in a variety of programming languages Liu et al. 2024 . This makes them potentially useful for a variety of tasks that currently suffer from a lack of large annotated datasets. Figure 1: A flowchart demonstrating our methodology of using WiC for WSD & WSI. Dictionary definitions and usage snippets can be extended into WiC dataset that is used to create a same-sense classifier. This WiC classifier can be applied to all different word senses to solve the WSD task. If no sense is matched, we have a candidate for a new sense, i.e. addressing the WSI task. The WSD dataset can be replaced with examples generated from a dictionary for less-resourced languages. In the word-sense disambiguation (WSD) task, a model tries to identify the correct meaning of a word in a given context, especially when the word has multiple possible meanings or senses. This task relies on a sense inventory and is crucial for understanding human language, as many words are polysemous and their correct interpretation depends on the specific context in which they are used. WSD is important for applications such as machine translation, information retrieval, and (linguistic) text analyses. Unlike WSD, which maps words to known senses from a dictionary or database, word-sense induction (WSI) discovers and clusters possible senses or meanings directly from text corpora. This enables handling new word senses in languages with sense inventories and is useful in languages or domains where sense-annotated resources are not available. Most existing research in NLP focuses on English and most LLMs are primarily trained on English texts. However, as several LLMs intentionally or unintentionally contain small amounts of training data in other languages, their performance on these languages currently remains underexplored. In our work, we examine whether LLMs could be helpful in improving NLP tasks in a less-resourced language based on commonly available linguistic resources. Specifically, we examine if GPT-3.5 can augment and extend training data for Slovene word-sense disambiguation (WSD) and word-sense induction (WSI) based on dictionary entries. We start with a dataset of dictionary entries, where each word sense contains examples that are too short and incomplete for training WSD models using contextual models, such as BERT Devlin et al. 2019 . We show that LLMs can be used to extend these short dictionary examples into complete sentences while maintaining a correct sense of the given word. An additional problem with models for the WSD task in this setting is that they are difficult to evaluate, as WSD models traditionally require a unified sense inventory in order to match words to senses. Such inventories are unavailable in many languages or may not match well the senses captured by LLMs. To address this, we use an alternative Word-in-Context (WiC) classification task presented by Martelli et al. 2021a . In this task, a model is presented with a pair of sentences containing the same target lemma and has to determine whether its sense differs between the two sentences. We show that if data generated by LLM is used in the WiC task, the resulting models exhibit an improved classification performance compared to models that do not use such data. Finally, we show that models trained on the WiC tasks can be adapted to solve WSD and WSI tasks through pairwise comparison with sense-annotated data. An outline of our approach is depicted in Figure 1 . Our main contributions are as follows: 1. A novel methodology to utilize information from