Transcoder-based Circuit Analysis for Interpretable Single-Cell Foundation Models
Sosuke Hosokawa, Toshiharu Kawakami, Satoshi Kodera, Masamichi Ito, Norihiko Takeda · 2025
Read Transcoder-based Circuit Analysis for Interpretable Single-Cell Foundation Models by Sosuke Hosokawa, Toshiharu Kawakami, Satoshi Kodera, Masamichi Ito, Norihiko Takeda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14723.txt.
Introduction
Sosuke Hosokawa and four co-authors apply circuit analysis in 2025 to single cell foundation models, large models trained on transcriptome data for tasks such as cell type labelling. Such models beat older methods, yet their decision process is opaque beside classic differential expression analysis. Clinicians and biologists need reasons.
Transcoders, a tool from language model interpretability, extract readable decision circuits from a model. The team trains a transcoder on the cell2sentence foundation model. Extracted circuits highlight which gene patterns drive a model call, in forms researchers can inspect. Interpretable single cell models could earn trust in perturbation prediction and annotation pipelines where a wrong silent call misleads downstream experiments.
Catalog record
| CREATOR | Sosuke Hosokawa, Toshiharu Kawakami, Satoshi Kodera, Masamichi Ito, Norihiko Takeda |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-14723
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Transcoder-based Circuit Analysis for Interpretable Single-Cell Foundation Models Sosuke Hosokawa, Toshiharu Kawakami, Satoshi Kodera, Masamichi Ito, Norihiko Takeda Abstract Single-cell foundation models (scFMs) have demonstrated state-of-the-art performance on various tasks, such as cell-type annotation and perturbation response prediction, by learning gene regulatory networks from large-scale transcriptome data. However, a significant challenge remains: the decision-making processes of these models are less interpretable compared to traditional methods like differential gene expression analysis. Recently, transcoders have emerged as a promising approach for extracting interpretable decision circuits from large language models (LLMs). In this work, we train a transcoder on the cell2sentence (C2S) model, a state-of-the-art scFM. By leveraging the trained transcoder, we extract internal decision-making circuits from the C2S model. We demonstrate that the discovered circuits correspond to real-world biological mechanisms, confirming the potential of transcoders to uncover biologically plausible pathways within complex single-cell models. 1 Introduction In recent years, single-cell foundation models (scFMs) such as cell2sentence (C2S) Levine2024C2S and Geneformer Theodoris2023 have garnered significant attention in the field of computational biology. These models adapt techniques from large language models (LLMs) in natural language processing, combining pre-training on large-scale transcriptome data corpora to learn general gene-gene relationships with task-specific fine-tuning on smaller datasets Theodoris2023 ; Cui2024 . While these models have achieved state-of-the-art performance on various single-cell analysis tasks including cell-type annotation and cellular response prediction, their low interpretability, stemming from the inherent nature of neural network algorithms, remains a significant challenge. This is particularly crucial in single-cell analysis models, where biological interpretation of model predictions is essential, making improved interpretability in scFMs an urgent priority. A major goal in efforts to elucidate the internal mechanisms of large-scale models like LLMs and scFMs includes identifying internal circuits: the combinations of features that determine model behavior (circuit tracing) transcoder_circuits_nips2024 ; Elhage2021Framework . In single-cell analysis models, discovered internal circuits could potentially lead to new discoveries when connected with biological insights. This pursuit of understanding model internal mechanisms falls under the research field of mechanistic interpretability, which has recently attracted considerable attention. In the domain of natural language processing, mechanistic interpretability of LLMs has emerged as a major research topic, with various methods being proposed. Among these, sparse autoencoders (SAEs) huben2024sparse and their variant, transcoders transcoder_circuits_nips2024 , have gained attention as methods that can resolve the "polysemanticity" within LLMs and transform internal representations into interpretable features. Transcoders, in particular, have been shown to extract input-invariant and highly interpretable features by training neural networks with wide, sparsely activated intermediate layers that replace MLP layers, enabling the extraction of model internal circuits. Figure 1: Pipeline for transcoder-based circuit tracing in scFMs. We train a transcoder on each MLP and attribute across features and attention to recover a sparse computational subgraph (circuit) that underlies cell-type predictions. The recovered features align with known endothelial biology (e.g., VWF, PTPRB, SPARCL1). In this work, we apply transcoders to the C2S model, a state-of-the-art scFM, to extract its internal circuits and biologically interpret the circuit components (Figure 1 ). Our contributions are summarized as follows: • We apply transcoders to scFMs (for the first time to our knowledge), demonstrating their effectiveness for mechanistic interpretability of scFMs. • We establish correspondences between extracted circuit components and biological knowledge, showing that biologically plausible circuits can be extracted. • We discuss challenges and future directions when applying transcoders to scFMs for mechanistic interpretability. The remainder of this paper is organized as follows. We first explain the background methods of scFMs and transcoders along with their use for circuit analysis. We then discuss case studies of experiments applying transcoders to the C2S model. Subsequently, we summarize related work on interpretability in single-cell analysis models, and finally present conclusions and future perspectives. 2 Single-cell Foundation Models Single-cell foundation models (scFMs) are transformer-based models pre-trained on large-scale transcriptome data. These models process input by ranking genes within each cell based on their expression levels and other factors, then arranging genes in rank order to form gene sequences. Prominent examples include Geneformer, scGPT, and cell2sentence (C2S). ScFMs exhibit several architectural variations: • Architecture type : Models can be either encoder-based or decoder-based. • Tokenization method : Some models tokenize at the gene level, while others leverage natural language tokenizers to process gene sequences represented as natural language strings. • Gene ranking methods : Different approaches exist for ranking genes Theodoris2023 ; Levine2024C2S , which represents a unique challenge specific to scFMs. In this work, we focus on the C2S model, which employs a decoder-based architecture and utilizes natural language tokenization. C2S leverages the Pythia Biderman2023Pythia architecture and tokenizer, pre-trained on 57 million human and mouse cells from scRNA-seq data, along with biological literature abstracts Levine2024C2S . This approach enables the model to capture both gene expression patterns and broader biological knowledge from scientific texts. 3 Transcoders and Circuit Tracing 3.1 The Need for Transcoders: Resolving Polysemanticity Understanding the internal representations of LLMs faces a fundamental challenge known as polysemanticity : the phenomenon where individual neurons or weights simultaneously encode multiple distinct concepts or functions Elhage2022ToyModels . For instance, a single neuron might strongly respond to both Japanese city names and gene names. This polysemanticity makes it difficult to disentangle and understand which parts of the weight matrices represent specific concepts through direct observation. To resolve or mitigate polysemanticity and reconstruct the internals of large models into human-interpretable units, methods such as sparse autoencoders (SAEs) and their variant, transcoders, have proven effective huben2024sparse ; transcoder_circuits_nips2024 . 3.2 Sparse Autoencoders and Transcoders 3.2.1 Sparse Autoencoders (SAE) SAEs consist of an encoder that transforms an input vector \mathbf{x}\in\mathbb{R}^{d} into a higher-dimensional activation vector \mathbf{z}\in\mathbb{R}^{l}_{\geq 0} (where l>d ), and a decoder that reconstructs the original dimension: \displaystyle\mathbf{z} \displaystyle=\mathrm{ReLU}(\mathbf{W}_{\mathrm{enc}}\mathbf{x}+\mathbf{b}_{\mathrm{enc}}) (1) \displaystyle\hat{\mathbf{x}} \displaystyle=\mathbf{W}_{\mathrm{dec}}\mathbf{z}+\mathbf{b}_{\mathrm{dec}} (2) Typical SAEs use the same LLM hidden state for both encoder input and decoder output, and are trained to minimize the following loss function: \mathcal{L}=||\hat{\mathbf{x}}-\mathbf{x}||_{2}^{2}+\lambda||\mathbf{z}||_{1} (3) where the first term represents reconstruction error, the second term is a sparsity penalty that encourages sparse activations, and \lambda is a hyperparameter controlling the L1 weight. 3.2.2 Transcoders Transcoders are a variant of SAEs that learn on the input and output of each transformer layer’s MLP rather than on the same hidden state: \mathcal{L}=||\hat{\mathbf{x}}-\mathrm{MLP}(\mathbf{x})||_{2}^{2}+\lambda||\mathbf{z}||_{1} (4) This formulation enables transcoders to approximate the transformer’s MLP, decomposing MLP neurons into interpretable components. 3.2.3 Key Differences between SAEs and Transcoders While standard SAEs are trained to reproduce hidden states and extract input-dependent features, transcoders approximate specific modules within transformers (the MLPs) and thus extract input-invariant features. For explaining general model behavior, input-invariant features are preferable; therefore, transcoders are better suited for extracting circuits within transformers. 3.3 Circuit Tracing with Transcoders Recent work has proposed methods for tracing circuits within LLMs using transcoders. We outline the key components below. 3.3.1 Attribution Between Transcoder Feature Pairs The contribution of transcoder feature i in layer l to feature j in layer l^{\prime} is computed as: z^{(l,i)}(x)\times\big(f^{(l,i)}_{\mathrm{dec}}\cdot f^{(l^{\prime},j)}_{\mathrm{enc}}\big) (5) where z^{(l,i)}(x) represents the input-dependent activation level, and the dot product (f_{\mathrm{dec}}\cdot f_{\mathrm{enc}}) is an input-independent fixed value. Here, f^{(l,i)}_{\mathrm{enc}/\mathrm{dec}} denotes feature vector i of the transcoder encoder/decoder in layer l , corresponding to row vectors of \mathbf{W}_{\mathrm{enc}} and column vectors of \mathbf{W}_{\mathrm{dec}} respectively. This decomposition allows separate treatment of "input-independent