CaptionFool: Universal Image Captioning Model Attacks
Swapnil Parekh · 2026
Read CaptionFool: Universal Image Captioning Model Attacks by Swapnil Parekh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00529.txt.
Description
Parekh presents CaptionFool, a universal adversarial attack that fools transformer captioning models into writing attacker chosen captions, including offensive text. Changing only seven of 577 image patches, about one percent of the image, the attack succeeds in nearly all trials. It can also plant slang terms crafted to slip past content moderation filters. The work exposes serious flaws in deployed vision language systems and presses the case for stronger defences.
Catalog record
| CREATOR | Swapnil Parekh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30037 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.00529