REVEAL : Multi-turn Evaluation of Image-Input Harms for Vision LLMs
Madhur Jindal, Saurabh Deshpande · 2025
Read REVEAL : Multi-turn Evaluation of Image-Input Harms for Vision LLMs by Madhur Jindal, Saurabh Deshpande (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04673.txt.
Description
Jindal and Deshpande introduce REVEAL, a framework for testing image-based harms in vision language models over long chats. Models that see and read raise new safety risks. Old safety tests assume text-only, single-turn exchanges. REVEAL mines images on its own and builds adversarial cases from them. It stretches attacks across many turns with a crescendo strategy. Judge models, including GPT-4o, then grade the harm in replies.
Five leading vision models were put through the pipeline. The tests show how guardrails erode as a chat wears on. Single-turn checks miss that slow slide. The frame is built to scale across models and image sets.
Catalog record
| CREATOR | Madhur Jindal, Saurabh Deshpande |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04673