Metonymy in vision models undermines attention-based interpretability

Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Massimiliano Mancini, Diego Marcos · 2026

Read Metonymy in vision models undermines attention-based interpretability by Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Massimiliano Mancini, Diego Marcos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06095.txt.

Introduction

Ananthu Aniraj and four co-authors test a founding assumption behind part based interpretability in vision models. Part centric attention is meant to make a model explain itself through object parts. That works only if the internal feature for a part mostly encodes that part's image region, a locality assumption the authors set out to measure.

Using part based attribute annotations, they find modern pretrained vision transformers break locality badly. Each part's features leak information from the whole object, a visual metonymy that makes attention based explanations unfaithful. The team also bound the problem with a two stage design that blocks leakage by construction. That disentangled extraction improved attribute driven part discovery across tasks, confirming the practical cost of the leakage.

Catalog record

CREATORAnanthu Aniraj, Cassio F. Dantas, Dino Ienco, Massimiliano Mancini, Diego Marcos
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06095

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Metonymy in vision models undermines attention-based interpretability Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Massimiliano Mancini, Diego Marcos Abstract Part-based reasoning is a classical strategy to make a computer vision model directly focus on the object parts that are relevant to the downstream task. In the context of deep learning, this also serves to improve by-design interpretability, often by using part-centric attention mechanisms on top of a latent image representation provided by a standard, black-box model. This approach is based on a locality assumption: that the latent representation of an object part encodes primarily information about the corresponding image region. In this work, we test this basic assumption, measuring intra-object leakage in vision models using part-based attribute annotations. Through a comprehensive experimental evaluation, we show that modern pretrained vision transformers violate the locality assumption and exhibit a strong intra-object leakage, in which each part encodes information from the whole object, a visual metonymy that compromises the faithfulness of attention-based interpretable-by-design methods for part-based reasoning, ultimately rendering them uninterpretable. In addition, we establish an upper bound using a two-stage approach that prevents leakage by design. We then show that this inherently disentangled feature extraction improves attribute-driven part discovery on a variety of tasks, confirming the practical impact of intra-object leakage. Our results uncover a neglected issue affecting the interpretability of part-based representations, such as those in CBMs relying on part-centric concepts, highlighting that two-stage approaches offer a promising way to mitigate it. 1 Introduction Part-based reasoning has been pursued as a strategy for visual recognition long before deep learning became commonplace [ 14 , 10 , 13 ] . Many modern concept-based interpretability methods, although not explicitly frame themselves as part-based, employ concepts that are inherently tied to object parts. By grounding predictions in such localized object parts, models offer users a natural entry point to inspect and verify the decision process. This principle underlies a range of recent approaches, from fine-grained recognition methods that learn part-centric attention [ 60 , 38 , 47 ] to Concept Bottleneck Models (CBMs) [ 27 , 8 , 45 ] , which predict human-readable attributes, such as “has a black bill” or “has red legs” in the context of bird species identification, as an intermediate step before mapping to class labels. The interpretability of all such methods rests on a shared, often implicit, assumption: that the latent representation of a given part encodes primarily information about the corresponding image region. However, a model trained on bird photos may learn a consistent co-occurrence between beak colour and leg colour, entangling their representations such that information about one is encoded in the features of the other, a form of visual metonymy , in which some element of an object comes to stand for the whole. If this locality assumption is violated in this way, then the reasoning exposed to the user is no longer faithful, regardless of how interpretable it appears. Indeed, there is growing evidence that concept representations, such as those in CBMs, leak information beyond their intended scope [ 34 ] and fail to respect locality [ 19 , 42 ] : a prediction for “has a black bill” may be heavily influenced by features outside the bill region. This resists straightforward mitigation [ 34 ] , and higher-quality concept predictors that are causally influenced only by the relevant object part are a necessary part of any solution [ 16 , 46 ] . Figure 1 : Intra-object leakage: Modern pretrained vision models violate locality. Early masking via two-stage approaches can mitigate this issue. Mean average precision (mAP) results for part attribute prediction using either the head or wing parts with a DINOv3 backbone. With late masking, the wrong part achieves almost as high mAP (anti-diagonal, red squares) as the correct part (diagonal, green squares). This problem represents a form of shortcut learning [ 15 ] , where models exploit readily available statistical regularities rather than the intended causal features. While prior work has extensively studied background bias , where models spuriously associate objects with their contextual surroundings [ 5 , 15 , 1 ] , and developed attention-based methods to suppress it [ 31 , 54 , 12 , 9 ] , there is comparatively little discussion of leakage within the foreground object itself. Yet several families of methods, including CBMs, part prototype networks, and unsupervised part discovery, all rely on applying part-centric attention mechanisms to deep feature maps whose representations are already entangled by the large receptive fields of modern architectures. Even if such attention maps visually highlight the correct part region (e.g., the beak), the underlying features may still encode information leaked from other parts, undermining faithfulness. This phenomenon, illustrated in Fig. 1 using actual attribute prediction results from a DINOv3 model, is particularly critical in high-stakes domains such as medical imaging, where reliance on a correlated but unrelated object structure could lead to dangerous misdiagnoses. In this paper, we address this gap by proposing a benchmark to measure intra-object leakage and a way forward, based on two-stage transformer models, that yield better part-centric representations by design. To make this problem measurable, we propose a metric for intra-object leakage based on datasets with part-level attribute annotations. Using this metric, we evaluate several state-of-the-art self-supervised backbones, including DINO and DINOv2, across three diverse datasets. Despite evidence that such models preserve enough locality for tasks like semantic segmentation [ 52 ] , we find that intra-object leakage is pervasive, putting into question the faithfulness of attention-based interpretability approaches built on top of them. We then study two-stage transformer models, in which attention maps are applied to the original image rather than to deep features, and show that this architectural change substantially reduces intra-object leakage, leading to significant improvements in part discovery. In summary, our contributions are: • A benchmark for intra-object leakage using part attribute annotations. • A comprehensive evaluation showing that intra-object leakage is widespread in modern pretrained backbones. • Evidence that two-stage feature extraction mitigates this leakage, improving attention-based part discovery. 2 Related Work Shortcut learning and feature coupling. Deep neural networks are prone to shortcut learning [ 15 ] , exploiting non-causal statistical regularities rather than causal structure. A well-studied manifestation is background bias, where an object’s classification becomes spuriously tied to its context [ 5 , 1 , 15 ] : a model may learn to recognize “cow” by detecting “pasture” [ 5 ] , and in extreme cases achieve high accuracy even when the foreground object is removed entirely, as long as the same biases are present in the test set [ 61 , 58 ] . This problem persists even in large-scale Vision-Language Models [ 53 ] , indicating that scaling alone is not enough to mitigate it. Attention-based approaches have been proposed to focus models on the foreground information, improving generalization [ 49 ] and efficiency [ 25 ] , and several works have aimed at encouraging locality in learned representations [ 59 , 29 , 44 ] . However, these previous efforts mainly target information leakage between image background and foreground. In this work, we show that even models with improved locality still suffer from intra-object leakage between part representations. Object coupling is a variant

Read the complete text