PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation
Linhan Xia, Mingzhan Yang, Guohui Yuan, Shengnan Tao, Yujing Qiu, Guo Yu, Kai Lei* · 2025
Read PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation by Linhan Xia, Mingzhan Yang, Guohui Yuan, Shengnan Tao, Yujing Qiu, Guo Yu, Kai Lei* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00968.txt.
Description
Xia, Yang, Yuan, Tao, Qiu, Yu, and Lei present in twenty twenty-five a model that picks the right meaning of a word from its context. Many words carry several senses, and software must choose the one a sentence intends. Past models used a standard language engine but leaned on either word-level or sentence-level meaning, not both, which capped their scores.
Their model fuses both levels through a multi-head attention design. A batch training trick reuses the correct senses of other target words in the same batch as negative examples. That cuts training inputs and computing cost sharply. Tests beat two known baseline systems on the standard score. The authors share both gains as the core advance.
Catalog record
| CREATOR | Linhan Xia, Mingzhan Yang, Guohui Yuan, Shengnan Tao, Yujing Qiu, Guo Yu, Kai Lei* |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00968