Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent
Christy Li, Josep Lopez Camuñas, Jake Thomas Touchet, Jacob Andreas, Agata Lapedriza, Antonio Torralba, Tamar Rott Shaham · 2025
Read Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent by Christy Li, Josep Lopez Camuñas, Jake Thomas Touchet, Jacob Andreas, Agata Lapedriza, Antonio Torralba, Tamar Rott Shaham (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21704.txt.
Introduction
Christy Li and six co-authors send an AI agent to interrogate vision models about their secret habits. A recognition model may lean on an unintended visual cue, a background, a texture, a watermark, and such hidden reliance breeds overfitting and spurious success that only shows up when conditions change.
Their agent works like a scientist: it proposes hypotheses about which attributes drive a model's predictions, designs experiments to test them, and grades its own conclusions with a self-evaluation protocol. When the evidence contradicts a finding, the agent reflects, discards, and starts a new round. On a fresh benchmark of one hundred thirty models rigged with known dependencies across eighteen categories, self-reflection steadily improves the agent's detection and clearly beats non-reflective versions. The agent also exposes genuine attribute reliance inside leading models, including a major vision encoder and a popular object detector.
Catalog record
| CREATOR | Christy Li, Josep Lopez Camuñas, Jake Thomas Touchet, Jacob Andreas, Agata Lapedriza, Antonio Torralba, Tamar Rott Shaham |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21704
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent Christy Li, Josep Lopez Camuñas, Jake Thomas Touchet, Jacob Andreas, Agata Lapedriza, Antonio Torralba, Tamar Rott Shaham Abstract When a vision model performs image recognition, which visual attributes drive its predictions? Detecting unintended reliance on specific visual features is critical for ensuring model robustness, preventing overfitting, and avoiding spurious correlations. We introduce an automated framework for detecting such dependencies in trained vision models. At the core of our method is a self-reflective agent that systematically generates and tests hypotheses about visual attributes that a model may rely on. This process is iterative: the agent refines its hypotheses based on experimental outcomes and uses a self-evaluation protocol to assess whether its findings accurately explain model behavior. When inconsistencies arise, the agent self-reflects over its findings and triggers a new cycle of experimentation. We evaluate our approach on a novel benchmark of 130 models designed to exhibit diverse visual attribute dependencies across 18 categories. Our results show that the agent’s performance consistently improves with self-reflection, with a significant performance increase over non-reflective baselines. We further demonstrate that the agent identifies real-world visual attribute dependencies in state-of-the-art models, including CLIP’s vision encoder and the YOLOv8 object detector. † † footnotetext: Website: https://christykl.github.io/saia-website/ 1 Introduction Computer vision models trained on large-scale datasets have achieved remarkable performance across a broad range of recognition tasks, often surpassing human accuracy on standard benchmarks ( He et al. 2016 ; Dosovitskiy et al. 2020 ; Tan and Le 2019 ; Russakovsky et al. 2015 ) . However, strong benchmark results can obscure underlying vulnerabilities. In particular, models may achieve high accuracy using prediction strategies that are non-robust or non-generalizable. These include relying on object-level characteristics such as pose or color ( Geirhos et al. 2018 ) , contextual cues like background scenery or co-occurring objects ( Xiao et al. 2021 ; Alcorn et al. 2019 ) , and demographic traits of human subjects ( Wilson et al. 2019 ; Wang et al. 2019 ; Rosenfeld et al. 2018 ) . Such visual dependencies may result in overfitting, reduced generalization, and performance disparities in real-world usage ( Hendrycks et al. 2021 ; Recht et al. 2019 ; Taori et al. 2020 ; Wiles et al. 2022 ) . Figure 1 : Attribute Reliance Detection. We use a self-reflective agent to produce natural-language descriptions of the visual attributes that a model relies on to recognize or detect a given concept. For each target concept (e.g., vase , teacher , or pedestrians ), the agent first conducts hypothesis testing to reach a candidate description and then validates the description’s predictiveness of actual model behavior through a self-evaluation protocol. The top row shows the agent’s generated explanations. The bottom rows show images predicted to elicit high (green) or low (red) model responses, along with their actual model scores. Results are shown for different target concepts across an object recognition model with a controlled attribute reliance (left), CLIP (middle), and YOLOv8 (right). Existing methods take various approaches to discover visual attributes that drive model predictions. These include saliency-based methods that highlight input regions associated with a prediction ( Simonyan et al. 2013 ; Selvaraju et al. 2017 ; Kindermans et al. 2017 ) , feature visualizations that map activations to human-interpretable patterns ( Olah et al. 2017 ) , and concept-based attribution methods that evaluate sensitivity to predefined semantic concepts ( Kim et al. 2018 ; Ghorbani et al. 2019 ; Mu and Andreas 2020 ) . While powerful for visualizing local behaviors, these approaches often rely on manual inspection and assume access to a fixed set of predefined concepts, limiting their ability to scale to modern models with complex behaviors. In this paper, we introduce a fully automated framework designed to detect visual attribute reliance in pretrained vision models. Given a pretrained model and a target visual concept (e.g., an image classifier selective for the object vase ), our method identifies specific image features that systematically influence the model’s predictions, even when these features fall outside the model’s intended behavior (e.g., the classifier relies on flowers to detect the vase , Fig. 1 ). At the core of our approach is a self-reflective agent (implemented with a backbone multimodal LLM) that treats the task as a scientific discovery process. Rather than relying on a predefined set of candidate attributes, the agent autonomously formulates hypotheses about image features that the model might rely on, designs targeted tests, and updates its beliefs based on observed model behavior. In contrast to previous interpretability agents ( Schwettmann et al. 2023 ; Rott Shaham et al. 2024 ) , the self-reflective agent does not stop after generating an initial finding but rather actively evaluates how well it aligns the model’s behavior on unseen test cases. Importantly, this evaluation is self-contained and does not require any ground-truth knowledge about the model’s attribute dependencies. Instead, the agent generates two sets of test images: one where the candidate attribute is present (which is expected to elicit high prediction scores from the model) and one where the attribute is absent (which is expected to elicit low prediction scores). When discrepancies arise between the expected and actual model behaviors, the agent reflects on its assumptions, identifies gaps or inconsistencies in its current understanding, and initiates a new hypothesis-testing loop. We show that the agent’s ability to reason about attribute reliance significantly improves with self-reflection rounds (see Sec. 5 ). To quantitatively evaluate our method, we introduce a novel benchmark of 130 object recognition models, each constructed with a well-defined intended behavior and an explicitly injected attribute dependency. The benchmark spans 18 types of visual reliances, inspired by vulnerabilities known to exist in vision models ( Dreyer et al. 2023 ; Geirhos et al. 2018 ; Geirhos et al. 2020 ; Buolamwini and Gebru 2018 ; Xiao et al. 2021 ; Wang et al. 2019 ; Singh et al. 2020 ) . These include object-level attributes (e.g., color, material), contextual dependencies (e.g., background, co-occurring object state), and demographic associations. Together with an automated evaluation protocol, this benchmark provides a controlled environment for evaluating visual attribute reliance detection methods. Our method is model-agnostic and can be applied to any vision model that assigns scores to input images. To demonstrate its versatility, we evaluate it on both our controlled benchmark and state-of-the-art pretrained models. Across a range of visual reliance types, our experiments show that our self-reflective agent consistently outperforms non-reflective baselines. Moreover, it successfully uncovers previously unreported attribute dependencies in pretrained models (Fig. 1 ). For example, it identifies that the CLIP-ViT vision encoder ( Radford et al. 2021 ) recognizes teachers based on classroom backgrounds and that YOLOv8 ( Jocher et al. 2023 ) , trained for object detection for autonomous driving, relies on the presence of crosswalks to detect pedestrians. These findings highlight the efficacy of our method as a scalable tool for detecting hidden dependencies in pretrained models deployed in real-world scenarios. 2 Related Work Revealing Visual Attribute Reliance in Vision Models. Prior work has explored methods to uncover the visual cues that drive model predictions. One common strategy is to manipulate input features to isolate model sensitivities, such as shape