REVEAL : Multi-turn Evaluation of Image-Input Harms for Vision LLMs

Madhur Jindal, Saurabh Deshpande · 2025

Read REVEAL : Multi-turn Evaluation of Image-Input Harms for Vision LLMs by Madhur Jindal, Saurabh Deshpande (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04673.txt.

Description

Jindal and Deshpande introduce REVEAL, a framework for testing image-based harms in vision language models over long chats. Models that see and read raise new safety risks. Old safety tests assume text-only, single-turn exchanges. REVEAL mines images on its own and builds adversarial cases from them. It stretches attacks across many turns with a crescendo strategy. Judge models, including GPT-4o, then grade the harm in replies.

Five leading vision models were put through the pipeline. The tests show how guardrails erode as a chat wears on. Single-turn checks miss that slow slide. The frame is built to scale across models and image sets.

Catalog record

CREATORMadhur Jindal, Saurabh Deshpande
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-04673

Browse the full library · About the Tower of Babel