SpurLens: Automatic Detection of Spurious Cues in Multimodal LLMs
Parsa Hosseini [email protected], Sumit Nawathe [email protected], Mazda Moayeri [email protected], Sriram Balasubramanian [email protected], Soheil Feizi [email protected] · 2025
Read SpurLens: Automatic Detection of Spurious Cues in Multimodal LLMs by Parsa Hosseini [email protected], Sumit Nawathe [email protected], Mazda Moayeri [email protected], Sriram Balasubramanian [email protected], Soheil Feizi [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08884.txt.
Introduction
Multimodal language models can name the dog in a photo for the wrong reason: the grass behind it. Parsa Hosseini and co-authors build a pipeline, SpurLens, that catches such spurious cues automatically. It pairs a strong vision-language model with open-set object detectors. Together they hunt for background elements the model secretly depends on. No human labeling is needed.
The findings sting. Removing a spurious cue can collapse recognition accuracy. Worse, the cues can trigger outright hallucination, inventing objects more than ten times as often. Prompt tricks and reasoning prompts soften the problem only partly. The authors argue that multimodal systems need this kind of audit as standard practice. A model that answers from the scenery, not the subject, will fail exactly where trust matters most.
Catalog record
| CREATOR | Parsa Hosseini [email protected], Sumit Nawathe [email protected], Mazda Moayeri [email protected], Sriram Balasubramanian [email protected], Soheil Feizi [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 140 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08884