Navigating the Concept Space of Language Models
Wilson E. Marcílio-Jr, Danilo Medeiros Eler · 2026
Read Navigating the Concept Space of Language Models by Wilson E. Marcílio-Jr, Danilo Medeiros Eler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23524.txt.
Introduction
Marcílio-Jr and Eler present Concept Explorer, a system for browsing the concepts inside a language model. Sparse autoencoders split model activity into thousands of readable features. Checking them one by one or by top examples makes wide exploration slow. Their system arranges feature summaries as layered neighborhood maps. Users zoom from broad clusters down to fine local groups. That layout supports finding, comparing, and relating concepts at scale. On SmolLM2 features, it surfaces clean top-level themes and rare concepts that older workflows miss.
Catalog record
| CREATOR | Wilson E. Marcílio-Jr, Danilo Medeiros Eler |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29479 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23524
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Navigating the Concept Space of Language Models Wilson E. Marcílio-Jr, Danilo Medeiros Eler Abstract Sparse autoencoders (SAEs) trained on large language model activations output thousands of features that enable mapping to human-interpretable concepts. The current practice for analyzing these features primarily relies on inspecting top-activating examples, manually browsing individual features, or performing semantic search on interested concepts, which makes exploratory discovery of concepts difficult at scale. In this paper, we present Concept Explorer , a scalable interactive system for post-hoc exploration of SAE features that organizes concept explanations using hierarchical neighborhood embeddings. Our approach constructs a multi-resolution manifold over SAE feature embeddings and enables progressive navigation from coarse concept clusters to fine-grained neighborhoods, supporting discovery, comparison, and relationship analysis among concepts. We demonstrate the utility of Concept Explorer on SAE features extracted from SmolLM2, where it reveals coherent high-level structure, meaningful subclusters, and distinctive rare concepts that are hard to identify with existing workflows. 1 Introduction Sparse autoencoders (SAEs) trained on internal activations of large language models have recently emerged as an effective approach for discovering latent features that align with human-interpretable concepts ( 2 ; 6 ; 10 ; 11 ) . By enforcing sparsity in a bottleneck representation, SAEs encourage a decomposition of activation space into a large set of relatively independent feature directions, many of which correspond to syntactic, semantic, or functional patterns. This paradigm has enabled mechanistic interpretability studies that operate at a scale beyond individual neurons. At the same time, advances in SAE training have substantially increased the number of learned features. Contemporary SAEs frequently contain hundreds of thousands to millions of latent dimensions ( 12 ; 5 ) . While this scale improves representational coverage, it introduces a fundamental analysis challenge: only a small fraction of features can be examined manually, yet there is currently no principled mechanism for organizing or surveying the full feature set. Consequently, the dominant bottleneck has shifted from feature learning to feature discovery. Common workflows involve selecting individual features, retrieving their top-activating contexts, using an LLM to infer their semantics, and employ heuristics to identify interesting concepts ( 3 ; 11 ) . Such heuristics include ranking features by activation frequency, selectivity, or magnitude, and performing similarity search around known features ( 13 ) . Although effective for validating specific hypotheses, these strategies are inherently local and query-driven. They provide limited support for global reasoning over the feature space and do not naturally facilitate open-ended discovery of concept families, identification of rare or isolated features, or analysis of relationships among large groups of features. On the other hand, single-level interaction embeddings like neuropedia.org or the one utilized by 3 make feature discovery difficult due to visualization clutter. Given a large collection of SAE features together and their textual explanations, an effective visual representation should support coarse-to-fine exploration, preserve neighborhood relationships among related features, and scale to millions of items. This representation should provide both a global overview and local inspection through interactive analysis. We introduce Concept Explorer 1 1 1 Code and application will be released after double-blind revision. , a scalable interactive system for post-hoc exploration of SAE features based on hierarchical neighborhood embeddings. Feature explanations are embedded into a multi-resolution manifold constructed using HUMAP ( 7 ) , yielding a hierarchy of increasingly coarse representations. This hierarchy enables progressive navigation from high-level concept groupings to fine-grained neighborhoods, while preserving both local and global structure. Concept Explorer supports identification of dominant concept families, discovery of rare or disconnected features, and examination of relationships among concepts. Our work does not propose a new SAE training objective or architecture. Instead, we focus on the downstream problem of organizing and exploring the outputs of existing SAE methods. 2 Background 2.1 Sparse Autoencoders for Large Language Model Representations Sparse autoencoders (SAEs) provide a scalable method for decomposing high-dimensional activation vectors of large language models (LLMs) into sparse, interpretable latent features. Given an activation vector h\in\mathbb{R}^{D} from a fixed LLM layer, an SAE learns a sparse code z\in\mathbb{R}^{K},K\gg D , and a reconstruction \hat{h} such that h\approx\hat{h} . Then, an encoder–decoder parameterization is used: \displaystyle z \displaystyle=f(W_{e}h+b_{e}), (1) \displaystyle\hat{h} \displaystyle=g(W_{d}z+b_{d}), (2) where W_{e}\in\mathbb{R}^{K\times D} , W_{d}\in\mathbb{R}^{D\times K} , and f and g are often the same activation functions. The model minimizes the reconstruction loss with a sparsity constraint, which ensures that only a small subset of features contribute to each reconstruction. SAEs are trained on large corpora of LLM activations with the base model frozen. The learned features can be interpreted by identifying inputs that maximize individual latent activations or by performing causal interventions using the decoder directions. 3 HUMAP for hierarchical neighborhood embedding HUMAP constructs a hierarchy of subsets based on landmarks and the corresponding fuzzy similarity graphs, enabling visualization of multiple resolution manifolds through repeated application of UMAP ( 8 ) . Given X^{(0)}=\{x_{i}\}_{i=1}^{n}\in\mathbb{R}^{D} and a nested sequence X^{(0)}\supset X^{(1)}\supset\cdots\supset X^{(L)} , HUMAP computes an embedding Y^{(\ell)}=\{y_{i}^{(\ell)}\}\in\mathbb{R}^{2} at each level \ell . Then, for each x_{i},x_{j}\in X^{(\ell)} , it defines connection strength weights ( p_{ij} ) using the UMAP kernel. Finally, these weights are then used to induce a Markov transition matrix T_{ij}=\frac{p_{ij}}{\sum_{m\in N_{k}(i)}p_{im}}. (3) Landmarks are selected by ranking nodes according to visit frequency after random walks on T , which generates level X^{(\ell+1)} . For each landmark u\in , a representation neighborhood is encoded in a sparse matrix R and lardmark similarities are computed using Equation 4 , and each level embedding Y^{(\ell)} is obtained by applying UMAP to S . S=1-\frac{R^{\top}R}{max(R^{\top}R)}. (4) Finally, random walks create a region of influence at level X^{(\ell)} for each landmark in level X^{(\ell+1)} , enabling hierarchical drill-down. That is, one can project on X^{(\ell)} the landmarks of interest in X^{(\ell+1)} , essentially projecting the region of influence (more data) from the selected points. 4 Methods In order to provide scalable and interactive concept finding and analysis we built Concept Explorer on top of a hierarchical neighborhood embedding computed using HUMAP. In the following, we explain the design and provide a Use Case in Section 5 . 4.1 Concept Explorer We defined the following questions to drive the visualization and interaction design: • Q1 : What are the main concepts in the dataset? • Q2 : What are the rarest concepts? • Q3 : How do concepts interact (overlap and influence)? Figure 1 gives an overview of the interface: a multi-level canvas showing HUMAP projections at each level ( b , c ), a detail panel containing explanation text and top activation contexts for the selected feature ( d ), and controls for annotating, merging or reprojecting regions of interest ( e , f ). Figure 1: Concept Explorer user interface. The right panel shows HUMAP projections at two levels; the left panel shows explanation content, top-k activation contexts and annotation controls for a selected feature. The middle panel shows the projection level being explored. To answer Q1 , users build a hierarchy with an adjustable number of landmarks per level (the more landmarks the finer is the granularity). Since HUMAP preserves locality and influence regions, similar features remain spatially close across adjacent levels; users can therefore iterate by building more levels and by inspecting the textual explanations and top-k contexts for each