LANE: Lexical Adversarial Negative Examples for Word Sense Disambiguation
Jader Martins Camboim de Sá, Jooyoung Lee, Cédric Pruski, Marcos Da Silveira · 2025
Read LANE: Lexical Adversarial Negative Examples for Word Sense Disambiguation by Jader Martins Camboim de Sá, Jooyoung Lee, Cédric Pruski, Marcos Da Silveira (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-11234.txt.
Introduction
Jader Martins Camboim de Sá, Jooyoung Lee, and their colleagues propose LANE, an adversarial training strategy that sharpens how neural language models pin down a word's sense in context. Such models often lean on the whole sentence's gist and miss local cues, so fine-grained meaning slips away. LANE deliberately shifts the learning focus onto the target word itself.
The method crafts hard negative examples by marking alternate words in training sentences, forcing the model to separate like sentences that differ only in the marked word. On sense disambiguation and semantic change benchmarks, it beat standard contrastive baselines. Checks show the learned forms capture subtle meaning gaps, and the method fits existing frameworks.
Catalog record
| CREATOR | Jader Martins Camboim de Sá, Jooyoung Lee, Cédric Pruski, Marcos Da Silveira |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-11234
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LANE: Lexical Adversarial Negative Examples for Word Sense Disambiguation Jader Martins Camboim de Sá, Jooyoung Lee, Cédric Pruski, Marcos Da Silveira Abstract Fine-grained word meaning resolution remains a critical challenge for neural language models (NLMs) as they often overfit to global sentence representations, failing to capture local semantic details. We propose a novel adversarial training strategy, called LANE, to address this limitation by deliberately shifting the model’s learning focus to the target word. This method generates challenging negative training examples through the selective marking of alternate words in the training set. The goal is to force the model to create a greater separability between same sentences with different marked words. Experimental results on lexical semantic change detection and word sense disambiguation benchmarks demonstrate that our approach yields more discriminative word representations, improving performance over standard contrastive learning baselines. We further provide qualitative analyses showing that the proposed negatives lead to representations that better capture subtle meaning differences even in challenging environments. Our method is model-agnostic and can be integrated into existing representation learning frameworks. 1 Introduction wsd ( wsd ), the task of identifying the precise meaning of a word in context, remains central to deep semantic understanding. It underpins applications such as lscd ( lscd ) 7 and word similarity tasks 1 . For example, wsd systems aim to determine whether the word “crazy” denotes insanity (as in That’s a crazy man ) or excitement (as in That’s crazy, man ), and whether “bank” refers to a financial institution or a river edge. While these words can appear in similar contexts, its particular usage can modulate completely different meanings. Modern neural language models like XLM-Roberta achieve impressive results in this task, but their final word representation often overfit to global context rather than encoding a word’s specific sense 17 . They solve the task by capturing topical cues, for instance, inferring that “bank,” “loan,” and “interest” signal finance, but can misrepresent a word’s fine-grained contribution 40 23 . This limitation surfaces in cases like heavy rain vs. heavy traffic: both imply “a lot,” yet with different nuances, intensity/volume versus density/severity. Such failures to capture precise contextual meaning hinder performance in downstream tasks requiring genuine semantic nuance, like neologism identification 24 . A widely adopted strategy to improve lexical sensitivity in wsd is target-word highlighting, where the word of interest is marked (e.g., with special tokens) before being encoded by a language model 4 . The underlying assumption is that explicit marking encourages the model to attend more directly to the lexical semantics of the word. Yet, as our analysis reveals ( Figure 1 ), this assumption is fragile: embeddings of the same sentence remain nearly identical regardless of which word is highlighted, indicating that the model often resolves the task using only the surrounding context 17 . Figure 1: Comparison of traditional (Baseline) and lane learned representations. Colors represent anchor (gray), positive (blue), and adversarial negative (red). In practice, this results in nearly indistinguishable representations for semantically distinct words, while instances of the same sense across different contexts may be mapped to more distant vectors. Such behavior illustrates the persistence of shortcut learning 32 , where models exploit superficial contextual regularities rather than grounding their predictions in the lexical meaning of the marked token. This imbalance produces a collapsed representation space, where embeddings concentrate in a narrow region irrespective of the highlighted word. The consequence is reduced lexical separability and weak generalization on tasks requiring sensitivity to word-level sense, especially in contexts that are semantically similar. Taking this into consideration, in this paper, we investigate How new approaches can ensure that the representation space meaningfully reflects the contribution of the target word? To address this, we introduce the LANE framework, which biases representations toward lexical relevance. LANE generates adversarial negatives by substituting the highlighted word while keeping the sentence context unchanged. These hard negatives share the full contextual frame of the anchor sentence but differ in the target token, forcing the model to focus on the word’s semantics rather than relying solely on surrounding context. By presenting nearly identical sentences that differ only in the marked word, LANE sharpens semantic boundaries and produces more discriminative representations. This encourages the model to attend precisely to the lexical contribution of the target word, resulting in embeddings that better capture fine-grained word meaning. We evaluate lane across both English and multilingual datasets, spanning a diverse set of architectures. Results show consistent improvements in lexical representation quality and cross-domain generalization, demonstrating that targeted lexical supervision can enhance robustness even in high-resource, context-rich settings. Furthermore, the method’s agnostic nature allows it to be incorporated into existing training pipelines with minimal computational overhead or implementation complexity, making it broadly applicable for multilingual lexical tasks. 2 Related Work Representing fine-grained word meaning has long been a longlasting challenge in natural language processing. Early approaches tackled this problem through lexical resources such as WordNet 26 and the creation of sense-annotated corpora to support supervised wsd methods 30 2 13 . With the advent of deep learning, contextualized word embeddings such as ELMo 28 and BERT 8 became standard, enabling substantial gains in wsd and related tasks by dynamically adjusting representations according to context. Yet despite these advances, contextualized models often fail to distinguish subtle word-level meaning differences, tending instead to overfit to sentence-level semantics 9 . This weakness is particularly visible in lscd , which requires tracking the semantic shifts of individual words across time and domains. While static embedding approaches offered initial baselines 34 22 , more recent work has leveraged contextualized embeddings to capture dynamic variation 35 15 11 . Nevertheless, isolating the semantics of the target word from the broader discourse remains an open challenge. To address these limitations, recent research has turned to contrastive learning as a way to induce more discriminative semantic representations. Inspired by advances in computer vision 5 , contrastive objectives have been adapted to NLP, yielding stronger sentence and word embeddings for tasks such as semantic similarity and clustering 10 . Within lexical semantics, several directions have emerged. One line of work leverages multilingual pretraining on large-scale, diverse corpora to improve contextual coverage of word senses 4 42 . Another incorporates auxiliary supervision, such as in-context sense induction, to encourage sense separation 12 27 16 18 . A third explores data augmentation, for example by altering input structure: Martelli et al. proposed swapping sentence order in a cross-encoder 21 , though this strategy does not extend to bi-encoder architectures where contrastive losses are typically computed. Complementary to these efforts, adversarial strategies aim to construct more challenging negatives to prevent representational collapse. Prior work has explored lightweight perturbations such as case alternation 39 , synonym and antonym substitution 38 , and token replacements with masked language model predictions 6 . Multilingual adversarial signals have further been obtained through cross-lingual links in Wikipedia, which align English sentences with low-resource counterparts while introducing cross-lingual semantic contrasts 17 . While these methods increase robustness and sharpen decision boundaries, they typically operate at the sentence level, focusing on global semantic differences rather than directly enforcing separability of meanings for a specific word in context. Our work addresses this gap by combining the strengths of