PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation

Linhan Xia, Mingzhan Yang, Guohui Yuan, Shengnan Tao, Yujing Qiu, Guo Yu, Kai Lei* · 2025

Read PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation by Linhan Xia, Mingzhan Yang, Guohui Yuan, Shengnan Tao, Yujing Qiu, Guo Yu, Kai Lei* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00968.txt.

Description

Xia, Yang, Yuan, Tao, Qiu, Yu, and Lei present in twenty twenty-five a model that picks the right meaning of a word from its context. Many words carry several senses, and software must choose the one a sentence intends. Past models used a standard language engine but leaned on either word-level or sentence-level meaning, not both, which capped their scores.

Their model fuses both levels through a multi-head attention design. A batch training trick reuses the correct senses of other target words in the same batch as negative examples. That cuts training inputs and computing cost sharply. Tests beat two known baseline systems on the standard score. The authors share both gains as the core advance.

Catalog record

CREATORLinhan Xia, Mingzhan Yang, Guohui Yuan, Shengnan Tao, Yujing Qiu, Guo Yu, Kai Lei*
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00968

Browse the full library · About the Tower of Babel