CaptionFool: Universal Image Captioning Model Attacks

Swapnil Parekh · 2026

Read CaptionFool: Universal Image Captioning Model Attacks by Swapnil Parekh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00529.txt.

Description

Parekh presents CaptionFool, a universal adversarial attack that fools transformer captioning models into writing attacker chosen captions, including offensive text. Changing only seven of 577 image patches, about one percent of the image, the attack succeeds in nearly all trials. It can also plant slang terms crafted to slip past content moderation filters. The work exposes serious flaws in deployed vision language systems and presses the case for stronger defences.

Catalog record

CREATORSwapnil Parekh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30037
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.00529

Browse the full library · About the Tower of Babel