Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations
Dahee Kwon, Sehyun Lee, Jaesik Choi · 2025
Read Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations by Dahee Kwon, Sehyun Lee, Jaesik Choi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01728.txt.
Introduction
Dahee Kwon and co-authors hunt for concept circuits inside vision models in a 2025 paper. Deep vision systems build ideas by composing single neurons across many layers. Since each concept is spread thin, finding where one concept lives is hard but vital work.
Their method, Granular Concept Circuit, builds one circuit per concept tied to a query. It grows the circuit step by step, judging links by both function and meaning. Many circuits can be found on their own for one image query. The result is a fine-grained map of what the model uses to decide. Such maps help auditors check that a model sees the right cues, not shortcuts.
Catalog record
| CREATOR | Dahee Kwon, Sehyun Lee, Jaesik Choi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01728
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations Dahee Kwon, Sehyun Lee, Jaesik Choi Abstract Deep vision models have achieved remarkable classification performance by leveraging a hierarchical architecture in which human-interpretable concepts emerge through the composition of individual neurons across layers. Given the distributed nature of representations, pinpointing where specific visual concepts are encoded within a model remains a crucial yet challenging task. In this paper, we introduce an effective circuit discovery method, called Granular Concept Circuit (GCC) * * * Code is available at https://github.com/daheekwon/GCC . , in which each circuit represents a concept relevant to a given query. To construct each circuit, our method iteratively assesses inter-neuron connectivity, focusing on both functional dependencies and semantic alignment. By automatically discovering multiple circuits, each capturing specific concepts within that query, our approach offers a profound, concept-wise interpretation of models and is the first to identify circuits tied to specific visual concepts at a fine-grained level. We validate the versatility and effectiveness of GCCs across various deep image classification models. 1 Introduction In deep image classification models, representations evolve hierarchically—from low-level features like points and edges in early layers, to mid-level patterns such as contours and textures, and finally to high-level semantic representations of objects and scenes [ 3 , 4 , 26 , 10 ] . Throughout this hierarchy, concepts, defined as meaningful representations comprehensible to humans, emerge through the composition of individual neurons, with each neuron contributing to progressively abstract representations [ 17 , 18 ] . However, much of the existing research in deep learning interpretation has focused on associating individual neurons with specific concepts [ 3 , 4 , 20 , 16 ] . These studies typically interpret neurons in isolation without explicitly considering relational structures among them. While this single-neuron approach provides valuable insights into isolated concept representations, it inherently overlooks the distributed nature of neural encoding, where concepts are inherently represented across multiple neurons and layers. Indeed, this form of distributed representation closely resembles how the human brain stores and recalls information [ 31 , 39 , 15 ] , leading to extensive studies on the mechanisms underlying these neural connections [ 5 , 19 , 32 , 30 ] . Building upon these neuroscience-inspired insights and the previously discussed neuron interpretability methods, recent research has begun to trace how visual concepts are represented across multiple neurons and layers. For example, VCC [ 22 ] represents layer-wise concepts using Concept Activation Vectors (CAVs) and analyzes their connectivity; however, its misalignment with the network structure makes it difficult to localize where specific concepts emerge precisely. Rajaram et al. 2024 , hereafter referred to as ADVC, extend the connectivity-based methods of Olah et al. 2020 by iteratively computing and refining the connectivity strength between adjacent layers. Nevertheless, these methods remain constrained by their focus on class-relevant relationships, limiting deeper understanding and interpretation of the models. To analyze models more comprehensively, it is essential to capture concepts beyond class-specific ones with finer granularity and identify where and how each concept is encoded. Importantly, existing approaches do not aim to subdivide these concept representations or analyze them in a fine-grained manner, fundamentally differing from our objective. CRP VCC ADVC GCC Circuit ✖ ⚫ ⚫ ⚫ Neuron-based ⚫ ✖ ⚫ ⚫ Concept Specificity - ✖ ✖ ⚫ \# of Input 1 class few versatile \# of Output Circuit - 1 1 \geq 1 Table 1 : Comparative Analysis of CRP [ 1 ] , ADVC [ 29 ] , VCC [ 22 ] , and GCC (Ours). Building on this, we introduce Granular Concept Circuits (GCC), an effective method for discovering visual concept circuits that leverage the distributed nature of representations to capture concepts relevant to a given query. To form a concept circuit, we assess inter-neuron connectivity by examining both their dependencies and semantic alignment, quantified using two scores: Neuron Sensitivity Score ( S_{NS} ) and Semantic Flow Score ( S_{SF} ). By iteratively identifying strongly connected neurons that represent coherent concepts, our method automatically constructs a visual concept circuit for each query. This results in multiple circuits, each encoding a distinct concept relevant to the query. The proposed circuit discovery method can be applied to various scenarios: from a single query to multiple queries, and from exploring common concepts to identifying unique concepts among them. For convenience, we refer to both the proposed method and the resulting circuit as Granular Concept Circuit (GCC). Our main contributions are summarized as follows: • We propose a method to automatically discover Granular Concept Circuits, where each circuit encodes a distinct concept relevant to the given query. • We iteratively identify connected neurons based on two proposed scores, S_{NS} and S_{SF} , to construct circuits representing coherent concepts. • Our method enables flexible circuit construction based on the number of queries, concept types, and user-defined objectives. 2 Related work Figure 1 : Illustration of Neuron Sensitivity Score ( S_{NS} ) and Semantic Flow Score ( S_{SF} ). (a) S_{NS} is computed by masking the source neuron and evaluating its connections (Eq. ( 2 )). (b) S_{SF} quantifies semantic alignment across layers, with stronger alignment (orange) indicating stable concept interpretability (Eq. ( 3 )). (c) Score distribution and corresponding neuron connections, with highly activated samples shown for selected neurons. Legend: ( purple ) low S_{NS} , low S_{SF} ; ( green ) high S_{NS} , low S_{SF} ; ( orange ) high S_{NS} , high S_{SF} . 2.1 Concept-based Interpretability Analyzing visual concepts learned by deep neural networks has become essential for understanding and interpreting their decision-making processes. Previous methods can be broadly categorized based on whether they analyze concepts individually or explicitly examine relationships between multiple concepts. Single-concept approaches can be divided by how concepts are represented: neuron-based and vector-based approaches. Neuron-based methods—such as NetDissect [ 3 ] , CLIP-Dissect [ 25 ] , RDR [ 6 ] , FALCON [ 20 ] , and MILAN [ 16 ] —aim to identify the semantic meanings associated with individual or small groups of neurons. These methods offer intuitive interpretations but are often insufficient for explaining complex model decisions that involve interactions among multiple concepts. In contrast, vector-based methods—such as Concept Activation Vectors (CAVs) [ 21 , 13 ] —represent concepts as activation directions spanning multiple neurons, enabling more abstract and generalizable representations. However, they often depend on manually labeled concept examples and struggle with capturing fine-grained semantics. To address this limitation, relation-based approaches have emerged, explicitly capturing interactions between multiple concepts. For example, HINT [ 36 ] identifies collaborative neurons within a single layer to represent specific concepts using CAV, but the detailed mechanisms underlying their interactions across layers to form hierarchical representations remain uncertain. CRAFT [ 11 ] leverages non-negative matrix factorization to trace and decompose concepts across multiple layers, but because it starts from the final classifier layer, it is limited to extracting concepts directly used for classification. CRP [ 1 ] computes neuron relevance scores based on conditional dependencies between adjacent layers, but its focus on pairwise layer-wise relevance limits its ability to capture circuit dynamics. 2.2 Visual Circuit Discovery As interest in understanding relationships between concepts grows, researchers are increasingly exploring ways to identify these relationships