Visual Word Sense Disambiguation with CLIP through Dual-Channel Text Prompting and Image Augmentations
Shamik Bhattacharya 1 2 , Daniel Perkins 1 2 , Yaren Dogan 3 , Vineeth Konjeti 3 , Sudarshan Srinivasan 4 , Edmon Begoli 3 4 · 2026
Read Visual Word Sense Disambiguation with CLIP through Dual-Channel Text Prompting and Image Augmentations by Shamik Bhattacharya 1 2 , Daniel Perkins 1 2 , Yaren Dogan 3 , Vineeth Konjeti 3 , Sudarshan Srinivasan 4 , Edmon Begoli 3 4 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06799.txt.
Description
Shamik Bhattacharya and five co-authors take on words that can mean more than one thing. A word like "bank" can point to a river edge or a place for money, and text alone may not settle it. Their fix pairs each word sense with pictures, using the CLIP model to place words and images in one shared space.
Two kinds of text hints enrich the word side: hints about meaning, and hints about photos, plus near words from WordNet. On the image side, the team tweaks each picture at test time, then picks the picture that lines up best with the text. On a 2023 test set, the enriched hints lifted the main score from about 0.72 to 0.76. Extra word lists and hints in many tongues, by contrast, tended to blur meaning.
Catalog record
| CREATOR | Shamik Bhattacharya 1 2 , Daniel Perkins 1 2 , Yaren Dogan 3 , Vineeth Konjeti 3 , Sudarshan Srinivasan 4 , Edmon Begoli 3 4 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06799