NLPrompt: Noise-Label Prompt Learning for Vision-Language Models
Bikang Pan, Qun Li, Xiaoying Tang, Wei Huang, Zhen Fang, Feng Liu, Jingya Wang, Jingyi Yu, Ye Shi · 2024
Read NLPrompt: Noise-Label Prompt Learning for Vision-Language Models by Bikang Pan, Qun Li, Xiaoying Tang, Wei Huang, Zhen Fang, Feng Liu, Jingya Wang, Jingyi Yu, Ye Shi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01256.txt.
Description
Bikang Pan and eight co-authors tackled noisy labels in training image-text models in a 2024 paper. Such models learn from paired images and captions, often tuned with short prompts. Real data sets carry wrong labels, which can drag prompt learning down.
One part of their method swaps the usual loss for one that is less swayed by wrong labels, which theory shows can mute noisy samples. A second part sorts the data into clean and noisy piles by matching images to text-based anchors. The clean pile gets the standard loss and the noisy pile gets the sturdy one. Across many noise settings, the combined method, named NLPrompt, scored clear gains over rivals.
Catalog record
| CREATOR | Bikang Pan, Qun Li, Xiaoying Tang, Wei Huang, Zhen Fang, Feng Liu, Jingya Wang, Jingyi Yu, Ye Shi |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01256