NLPrompt: Noise-Label Prompt Learning for Vision-Language Models

Bikang Pan, Qun Li, Xiaoying Tang, Wei Huang, Zhen Fang, Feng Liu, Jingya Wang, Jingyi Yu, Ye Shi · 2024

Read NLPrompt: Noise-Label Prompt Learning for Vision-Language Models by Bikang Pan, Qun Li, Xiaoying Tang, Wei Huang, Zhen Fang, Feng Liu, Jingya Wang, Jingyi Yu, Ye Shi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01256.txt.

Description

Bikang Pan and eight co-authors tackled noisy labels in training image-text models in a 2024 paper. Such models learn from paired images and captions, often tuned with short prompts. Real data sets carry wrong labels, which can drag prompt learning down.

One part of their method swaps the usual loss for one that is less swayed by wrong labels, which theory shows can mute noisy samples. A second part sorts the data into clean and noisy piles by matching images to text-based anchors. The clean pile gets the standard loss and the noisy pile gets the sturdy one. Across many noise settings, the combined method, named NLPrompt, scored clear gains over rivals.

Catalog record

CREATORBikang Pan, Qun Li, Xiaoying Tang, Wei Huang, Zhen Fang, Feng Liu, Jingya Wang, Jingyi Yu, Ye Shi
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01256

Browse the full library · About the Tower of Babel