An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs

2026

Read An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05400.txt.

Introduction

Deshan Sumanathilaka and team test low-setting disambiguation models. Disambiguation challenges rare senses. Their machine and energy demands limit scalability. FEWS fine-tunes eight small models. Chain-of-Thought matches GPT-4-Turbo. Gemma-3-4B and Qwen-3-4B outperform FEWS older models. Fool Me If You Can confirms adaptability. Reasoning-centric fine-tuning delivers correct low-setting disambiguation.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48502
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05400

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs Abstract Word Sense Disambiguation (WSD) remains a key challenge in Natural Language Processing (NLP), especially when dealing with rare or domain-specific senses that are often misinterpreted. While modern high-parameter Large Language Models (LLMs) such as GPT-4-Turbo have shown state-of-the-art WSD performance, their computational and energy demands limit scalability. This study investigates whether low-parameter LLMs ( < 4B parameters) can achieve comparable results through fine-tuning strategies that emphasize reasoning-driven sense identification. Using the FEWS dataset augmented with semi-automated, rationale-rich annotations, we fine-tune eight small-scale open-source LLMs (e.g. Gemma and Qwen). Our results reveal that Chain-of-Thought (CoT)-based reasoning combined with neighbour-word analysis achieves performance comparable to GPT-4-Turbo in zero-shot settings. Importantly, Gemma-3-4B and Qwen-3-4B models consistently outperform all medium-parameter baselines and state-of-the-art models on FEWS, with robust generalization to unseen senses. Furthermore, evaluation on the unseen “Fool Me If You Can” dataset confirms strong cross-domain adaptability without task-specific fine-tuning. This work demonstrates that with carefully crafted reasoning-centric fine-tuning, low-parameter LLMs can deliver accurate WSD while substantially reducing computational and energy demands. Keywords: Word Sense Disambiguation, Low-parameter LLMs, Reasoning-driven Fine-tuning An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs Deshan Sumanathilaka, Nicholas Micallef, Julian Hough Department of Computer Science, Swansea University Fabian Way, Crymlyn Burrows, Swansea, Wales, UK {t.g.d.sumanathilaka, nicholas.micallef, julian.hough}@swansea.ac.uk Abstract content 1. Introduction With recent advances in large language models (LLMs), Natural Language Processing (NLP) tasks such as machine translation, information retrieval, question answering, sentiment analysis, and summarization have achieved near-human performance in modern applications 16 ; 55 . However, to perform such linguistic tasks effectively, word sense disambiguation (WSD) plays a major role, as ambiguity can lead to incorrect information and the generation of misinformation 52 . For example, during the COVID-19 pandemic, the term “ positive ” could indicate a confirmed infection or express optimism in a general context which are two drastically different meanings that require contextual understanding to avoid incorrect inferences. Understanding the meaning of a word in a sentence or phrase and disambiguating the correct sense is crucial for achieving accurate results, particularly when tackling the lexical ambiguity of an ambiguous word. Working with words that have multiple meanings (polysemy) is a major challenge for current NLP systems, as highly polysemous words are difficult to disambiguate due to the nature of usage. For example, according to BabelNet 1 1 1 https://babelnet.org/ , the word ‘ bank ’ has 49 meanings, including 40 as a nouns and 9 as a verb. However, in common usage, the senses “financial institution” (money bank) and “sloping land beside a body of water” (river bank) dominate. In contrast, senses including “a flight maneuver” (highly domain-specific to aviation) or “a slope in the turn of a road or track” (used mainly in engineering, road design) are rare in everyday usage. In domain-specific situations, models often struggle to disambiguate less common senses effectively 4 . Recent studies on WSD using large language models (LLMs) have revealed promising findings, including significant improvements in disambiguating rare senses and leveraging broader contextual information, paving the way for enhanced language understanding and more accurate disambiguation. For instance, 54 evaluated multiple large-parameter LLMs for disambiguation using few-shot and knowledge-base-combined methods, highlighting the strengths of LLMs for WSD. However, other studies also reveal important limitations. 9 demonstrated that while LLMs perform well in zero-shot scenarios, they cannot surpass current state-of-the-art methods without fine-tuning, suggesting that existing approaches do not fully exploit LLM capabilities or generalize across diverse WSD settings. These gaps highlight the need for research that addresses scalability, efficiency, and more effective use of LLMs for robust and generalizable disambiguation. In particular, incorporating proper inner reasoning to guide decision making is crucial, motivating the present study on reasoning-driven WSD with low-parameter LLMs. Inspired by these observations, we design and evaluate a smaller, energy-efficient LLMs for WSD with three primary design strategies: 1. Given a word in a sentence, the LLM must generate the correct definition. 2. Given a word in a sentence and possible senses, the model reasons using the neighbouring context closely related to the disambiguation process to identify the correct sense. 3. Given a word in a sentence and possible senses, the model reasons why a particular sense is correct and why other senses are not, to determine the correct meaning. Following these design principles, we fine-tune energy-efficient, low-parameter open-source models with fewer than 4B parameters 5 . Our study reveals that a chain-of-thought (CoT)-based reasoning process combined with neighbouring context analysis performs comparably to large models like GPT-4-Turbo, achieving high performance on unseen data. Furthermore, our advanced reasoning approach requires only 10% of the training data compared to the second approach, yet performed similarly, demonstrating its effectiveness. The main contributions of this work are: • Building three reasoning datasets using a semi-automated approach and releasing them as open-source resources for future research. 2 2 2 Reasoning dataset: https://huggingface.co/datasets/deshanksuman/Reasoning_WSD_dataset 3 3 3 Reasoning dataset for Verb: https://huggingface.co/datasets/deshanksuman/Advance_Reasoning_for_Verb_WSD • Proposing and implementing lightweight adapters following a novel EAD (Exploration, Analysis and Disambiguation) framework that can be efficiently used for WSD with small-scale models. • Evaluating eight open-source small-parameter models with the proposed approaches and achieving superior performance compared to all medium-parameter models. The rest of the paper is organized as follows: background and related work, which discusses the WSD evolution and related studies; methodology, which outlines the approach used to conduct this study; results and observations, followed by conclusions and future directions. 2. Background and Related Work 2.1. Evolution of WSD WSD has a long history, evolving from rule-based methods to deep learning and LLM approaches. Early rule-based systems relied on hand-crafted grammatical and syntactic rules to exploit the local context of ambiguous words 14 , but these methods were labor-intensive, domain-specific, and difficult to scale to complex languages 44 . Knowledge-based (KB) approaches emerged with resources such as WordNet, dictionaries, and thesauri incorporating algorithms such as Lesk to measure the sense overlap between gloss definitions and the surrounding context 31 . Advanced KB solutions such as random walks over extended lexical graphs including Extended WordNet 2 and graph-based methods that combined word embeddings with contextual information were explored 18 . Exploiting synset definitions, hypernymy relations, and contextual features improved the accuracy 25 , with frameworks like the Synset Relation-Enhanced Framework (SREF) achieving state-of-the-art KB WSD 62 . Filtering unnecessary semantic information 27 and incorporating dependency parsing to extract more precise contextual knowledge 40 were also prominent KB innovations. With annotated corpora, supervised Machine Learning became the most common research approach 30 ; 3 ; 20 . These methods exploited features, namely target word morphology, Part of Speech (POS) tags, syntactic dependencies, and collocations. To achieve efficient WSD, supervised neural architectures incorporated richer lexical and gloss-based information to the training pipeline. Gloss-augmented neural networks jointly encoded glosses and context were examined 36 . Other approaches to supervised WSD explored multiple-sense identification 42 , stacked BiLSTMs with attention 28 , and context-dependent modeling 26 . There was a major shift in WSD research with the arrival of Transformer architectures 61 , which enabled deep, attention-based contextual modeling. Transformer-based models such as BERT and GPT achieved state-of-the-art performance on WSD tasks 23 , demonstrating strong generalization

Read the complete text