Image Reconstruction as a Tool for Feature Analysis
Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova, Andrey Kuznetsov · 2025
Read Image Reconstruction as a Tool for Feature Analysis by Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova, Andrey Kuznetsov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07803.txt.
Introduction
Eduard Allakhverdov and co-authors present a 2025 computing paper on reading the inner features of vision models by rebuilding images from them. Vision encoders power many modern systems, yet how they store what an image contains stays murky. The authors turn the encoder around and decode its features back into pictures.
They compare two sibling model families that differ only in training goal. Encoders trained on image-based tasks kept far more picture detail than those trained by matching images to text. The method ranks a range of encoders by how much information their features hold. Twisting the feature space in set ways changed the rebuilt images in predictable ways. One surprise: rotations in feature space, not shifts in the picture, steer how color is stored.
Catalog record
| CREATOR | Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova, Andrey Kuznetsov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07803
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Image Reconstruction as a Tool for Feature Analysis Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova, Andrey Kuznetsov Abstract Vision encoders are increasingly used in modern applications, from vision-only models to multimodal systems such as vision-language models. Despite their remarkable success, it remains unclear how these architectures represent features internally. Here, we propose a novel approach for interpreting vision features via image reconstruction. We compare two related model families, SigLIP and SigLIP2, which differ only in their training objective, and show that encoders pre-trained on image-based tasks retain significantly more image information than those trained on non-image tasks such as contrastive learning. We further apply our method to a range of vision encoders, ranking them by the informativeness of their feature representations. Finally, we demonstrate that manipulating the feature space yields predictable changes in reconstructed images, revealing that orthogonal rotations — rather than spatial transformations — control color encoding. Our approach can be applied to any vision encoder, shedding light on the inner structure of its feature space. The code and model weights to reproduce the experiments are available in GitHub. 1 Introduction In recent years, vision encoders have become a crucial component of machine-learning pipelines serving as generalized image representations. They serve as specialized modules for solving computer-vision tasks, aggregating information for image generators (image priors), and powering modern vision–language models. The primary objective of these encoders is to capture all the semantic and object-level information in an image and transfer this knowledge to downstream tasks. Although experiments can show which model performs best on a specific downstream task, important questions remain: How do hidden representations encode features? What is the relationship between these representations and the original images? Can we manipulate an image by influencing its features – and vice versa? Answering these questions sheds new light on the interpretability of vision encoders and gives us confidence when choosing and working with their feature outputs. This, in turn, allows for more informed decisions about which encoder is best suited to a particular downstream application. In this work, we introduce a novel reconstruction-based method to evaluate the quality of features extracted from input images. Our approach reconstructs images from hidden representations, revealing exactly what types of information are preserved within a model’s internal layers. Our contributions are as follows: 1. A novel interpretability metric: We propose an approach for comparing image-feature interpretability based on the visual quality of reconstructed images. 2. Analysis of training objectives: We perform a detailed comparison between two models with identical architectures – SigLIP ( Zhai et al., 2023 ) and SigLIP2 ( Tschannen et al., 2025 ) – whose main difference is their training objective. We demonstrate that SigLIP2 produces significantly higher-fidelity reconstructions than SigLIP. 3. Models study: We investigate multiple vision-encoder designs — varying in image resolution, training objectives, and hidden dimensions — to understand how design choice and peculiarities in model training affect the amount of information each layer retains. 4. Feature-space transformations: We reveal that applying rotations in the embedding space corresponds to interpretable color transformations in the reconstructed images, pointing toward novel, semantically grounded editing operations. 5. We release the code to reproduce the results of the experiments available via GitHub repo 1 1 1 https://fusionbrainlab.github.io/feature_analysis/ . 2 Related work Recent research has proposed various approaches to interpret the internal representations of vision encoders such as ViT Dosovitskiy et al., 2021 , CLIP Radford et al., 2021a , DINO Caron et al., 2021a , SAM Kirillov et al., 2023 , and others. These methods can be grouped into three major categories based on their interpretability strategy. 2.1 Attention and concept emergence analysis Many works focus on analyzing activations to determine which neurons or layers encode specific image concepts—ranging from simple elements like text and colors to more complex structures such as objects and semantics. These approaches typically inspect self-attention maps or neuron activations to reveal what concepts are learned at different stages of transformer-based vision models. For example, Caron et al., 2021b demonstrated that attention heads in self-supervised ViTs can function as unsupervised object detectors, effectively segmenting semantically meaningful regions. Dorszewski et al., 2025 performed neuron labeling across ViT layers and found a clear progression from low-level features (e.g., edges, color) to high-level concepts (e.g., object parts, categories). More recently, Darcet et al., 2024 introduced register tokens to eliminate pathological attention patterns and restore clear, interpretable attention in large ViTs. Together, these studies highlight how semantic and spatial regularities can naturally emerge in transformer attention mechanisms — especially under self-supervised or architecture — aware training regimes. 2.2 Representational similarity and probing tasks Representational similarity analysis and probing tasks offer large-scale insights into feature spaces. A widely used technique is Centered Kernel Alignment (CKA) Kornblith et al., 2019 , which quantifies the similarity between representations from two layers (or two models) on the same inputs — remaining invariant to orthogonal transformations and isotropic scaling. By computing CKA across all layer pairs, researchers visualize a model’s internal structure as a heatmap. Raghu et al., 2021 applied CKA to compare ViT and ResNet, discovering that ViT representations remain highly homogeneous across layers, whereas ResNet exhibits a more hierarchical progression with strong locality among adjacent layers. Beyond comparing architectures, CKA has been used to study different training paradigms: supervised versus self-supervised ResNets, or CLIP’s image encoder versus a supervised ViT. These analyses consistently find that supervised models concentrate class-discriminative information in deeper layers, while self-supervised models distribute various attributes throughout the network. Kornblith et al., 2019 further showed that supervised networks of different architectures converge to remarkably aligned final representations on ImageNet, despite diverging early-layer features. 2.3 Latent space manipulation Latent manipulation techniques operate directly on internal feature vectors — such as StyleSpace Wu et al., 2021 or CLIP Radford et al., 2021b embeddings — to generate counterfactual or modified visual outputs. For instance, StylEx optimizes GAN latents to identify minimal, independent changes in features that affect classifier decisions (e.g., altering the background or object shape) Lang et al., 2021 . Kazemi et al., 2024 performed directional edits in CLIP’s embedding space to test compositionality and memorization, enabling “cat astronaut” image synthesis. Work on CLIP inversion also demonstrates that certain semantic directions correspond to recognizable visual attributes, which can be amplified or suppressed. By enabling fine-grained causal analysis of visual attributes, these methods facilitate the creation of controlled visual counterfactuals and deepen our understanding of model reliance on particular features. Prior works have explored inversion-based interpretability for convolutional encoders. Mahendran and Vedaldi, 2016 proposed feature inversion by iteratively optimizing input pixels so that a CNN’s activations match those of a target image, revealing which patterns each layer encodes, but their gradient-descent approach is too slow for real-time analysis and is tailored to convolutional architectures. Dosovitskiy and Brox, 2016 advanced this research direction by training a feed-forward decoder to reconstruct images from CNN features in a single pass, yielding much faster and higher-fidelity inversion; however, their focus remained on convolutional networks, and they did not examine how different pretraining objectives affect invertibility or the semantic geometry of the feature space. In contrast, our method delivers high-fidelity, real-time reconstruction for transformer-based encoders, enables a direct comparison of contrastive versus multimodal training objectives, and — by learning orthogonal operators