Bhav-Net: Knowledge Transfer for Cross-Lingual Antonym vs Synonym Distinction via Dual-Space Graph Transformers

Samyak S. Sanghvi · 2025

Read Bhav-Net: Knowledge Transfer for Cross-Lingual Antonym vs Synonym Distinction via Dual-Space Graph Transformers by Samyak S. Sanghvi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15792.txt.

Introduction

Samyak Sanghvi tackles a word puzzle across languages in a 2025 paper. Some word pairs mean the same, while others mean the exact reverse. Opposites are tricky because they share a topic yet clash in sense. Sanghvi builds a network that learns these links in two spaces at once.

One space pulls same meaning pairs together, while the other keeps opposites close in a linked way. Language encoders feed graph layers that pass lessons from rich models to leaner ones. Tests span eight languages, from English and German to Dutch and Russian. Scores rival top rival systems, and the layout stays easy to read. The work aids translation and search tools.

Catalog record

CREATORSamyak S. Sanghvi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bhav-Net: Knowledge Transfer for Cross-Lingual Antonym vs Synonym Distinction via Dual-Space Graph Transformers Samyak S. Sanghvi Abstract Antonym vs synonym distinction across multiple languages presents unique computational challenges due to the paradoxical nature of antonymous relationships—words that share semantic domains while expressing opposite meanings. This work introduces Bhav-Net, a novel dual-space architecture that enables effective knowledge transfer from complex multilingual models to simpler, language-specific architectures while maintaining robust cross-lingual antonym-synonym distinction capabilities. This approach combines language-specific BERT encoders with graph transformer networks, creating distinct semantic projections where synonymous pairs cluster in one space while antonymous pairs exhibit high similarity in a complementary space. Through comprehensive evaluation across eight languages (English, German, French, Spanish, Italian, Portuguese, Dutch, and Russian), we demonstrate that semantic relationship modeling transfers effectively across languages. The dual-encoder design achieves competitive performance against state-of-the-art baselines while providing interpretable semantic representations and effective cross-lingual generalization. 1 1 1 The complete implementation of Bhav-Net, including training scripts, evaluation code, and model checkpoints, is available at: https://github.com/SamyakSS83/bhavnet 1 Introduction Semantic relationship detection remains a cornerstone challenge in natural language processing, with particular complexity arising in antonym vs synonym distinction. Unlike synonyms, which exhibit both semantic similarity and distributional alignment, antonyms present a fundamental paradox: they occupy shared semantic domains yet express diametrically opposite meanings. Consider word pairs like "hot/cold" or "love/hate"—these terms frequently co-occur in similar contexts despite their contrasting semantics, making traditional distributional approaches inadequate for distinguishing antonymous from synonymous relationships. The multilingual dimension adds substantial complexity to this challenge. While recent advances in cross-lingual language models offer promising avenues for semantic understanding across languages, existing approaches typically treat all semantic relationships uniformly, failing to capture the nuanced differences between synonymy and antonymy across diverse linguistic structures. This work addresses two critical research questions: 1. Knowledge Transfer : How can semantic relationship understanding be effectively transferred from complex multilingual models to simpler, more efficient architectures without significant performance degradation? 2. Cross-Lingual Generalization : How do antonym-synonym modeling capabilities generalize across languages with varying linguistic characteristics and resource availability? I propose Bhav-Net, 2 2 2 Named after the Sanskrit word ”bhava” meaning sentiment or emotion. a dual-space neural architecture that tackles these challenges through explicit separation of synonymous and antonymous relationship modeling. Our key contribution lies in demonstrating that semantic relationship patterns can be effectively transferred across languages while respecting language-specific characteristics. The main contributions of this work are: 1. A novel dual-space architecture enabling effective knowledge transfer from complex multilingual models to simpler, language-specific networks 2. Comprehensive cross-lingual evaluation demonstrating consistent antonym-synonym distinction across eight languages 3. Empirical analysis revealing that performance variations across languages stem primarily from embedding model quality rather than architectural limitations 4. Open-source implementation and model weights facilitating reproducible research in multilingual semantic relationship detection 2 Related Work 2.1 Antonym vs Synonym Distinction and Semantic Relationships The task of distinguishing antonyms from synonyms has been a longstanding challenge in computational linguistics, primarily due to the distributional hypothesis limitations. Traditional approaches to antonym detection have largely relied on distributional hypotheses, assuming that words appearing in similar contexts share semantic properties. However, this assumption fails for antonyms, which often share contexts while expressing opposite meanings Mohammad and Turney 2013 . Early work by Lin 1998 demonstrated that purely distributional methods struggle with antonym-synonym distinction, leading to the development of more sophisticated approaches. Pattern-based methods emerged as a promising solution to this challenge. Nguyen et al. 2017a introduced AntSynNET, a neural network model that exploits lexico-syntactic patterns from syntactic parse trees to distinguish antonyms and synonyms. Their work established important benchmarks for English antonym-synonym distinction and demonstrated the effectiveness of incorporating syntactic information. This foundational work provided the English dataset used in our evaluation, consisting of balanced synonym and antonym pairs across different parts of speech. Recent neural approaches have shown significant promise in this domain. Nguyen et al. 2017b introduced siamese networks for antonym detection, while Schwartz et al. 2015 proposed symmetric pattern-based approaches that capture coordinations and other linguistic patterns. The work of Vulić et al. 2018 demonstrated that post-hoc specialization of word embeddings could improve antonym detection through lexical entailment modeling, though these methods typically focus on monolingual settings. The state-of-the-art approach, ICE-NET Ali et al. 2024 , employs interlaced encoder networks to capture relation-specific properties of antonym and synonym pairs. Their method addresses the limitations of existing approaches by modeling symmetry, transitivity, and trans-transitivity properties inherent in semantic relationships. ICE-NET achieves superior performance by explicitly modeling these mathematical properties of semantic relations, setting new benchmarks for antonym vs synonym distinction. 2.2 Cross-Lingual Semantic Modeling and Multilingual Approaches Cross-lingual semantic relationship modeling has gained significant attention with the advent of multilingual language models. The fundamental challenge lies in transferring semantic understanding across languages with varying morphological, syntactic, and semantic structures. Conneau et al. 2020 showed that cross-lingual word embeddings can capture semantic relationships across languages through unsupervised alignment methods, while Ruder et al. 2019 provided comprehensive analysis of cross-lingual representation learning approaches, highlighting both opportunities and limitations. The work of Artetxe et al. 2018 demonstrated that semantic relationships can transfer across languages through multi-step alignment techniques, though their focus remained primarily on synonymy rather than the more challenging antonym-synonym distinction. Their generalization framework provides important insights into cross-lingual semantic transfer, but lacks the specialized architectures needed for oppositional relationship modeling. More recently, Reimers and Gurevych 2020 showed that sentence-level embeddings from multilingual models exhibit cross-lingual semantic consistency through knowledge distillation techniques. This work opened avenues for relationship-specific modeling across languages, demonstrating that semantic patterns learned in one language can be effectively transferred to others when appropriate architectural constraints are imposed. However, existing multilingual approaches typically treat all semantic relationships uniformly, failing to account for the unique challenges posed by antonym-synonym distinction. The paradoxical nature of antonyms—sharing distributional contexts while expressing opposite meanings—requires specialized architectural considerations that most cross-lingual models do not address. 2.3 Knowledge Transfer and Model Distillation in NLP Knowledge transfer from complex to simpler models has become increasingly important for practical deployment of NLP systems. Hinton et al. 2015 introduced the fundamental concept of knowledge distillation for neural networks, demonstrating that smaller models can learn to mimic the behavior of larger, more complex teachers. This paradigm has proven particularly valuable in NLP, where large pre-trained models offer superior performance but are computationally prohibitive for many applications. Building on this foundation, Sanh et al. 2019 demonstrated the effectiveness of knowledge distillation for language models, showing that DistilBERT could retain 97% of BERT’s performance while being 60% smaller. This work established important precedents for task-agnostic knowledge transfer in language understanding. Recent advances have focused on task-specific knowledge transfer. Jiao et al. 2020 explored progressive knowledge distillation for natural language understanding tasks, while Sun et al. 2019 introduced patient knowledge distillation that allows the student model to learn at its own pace. These approaches demonstrate that specialized capabilities can be effectively transferred to smaller models when the distillation process accounts for task-specific requirements. However, most existing work focuses on general language understanding rather than specific semantic relationships. The transfer of antonym-synonym distinction capabilities presents unique challenges because it requires maintaining fine-grained semantic distinctions that are easily lost during distillation. Our work extends this paradigm to multilingual semantic

Read the complete text