Sparse Autoencoders Map Brain–LLM Alignment onto Cortical Semantic Topography

Dongxin Guo, Jikun Wu, Siu Ming Yiu · 2026

Read Sparse Autoencoders Map Brain–LLM Alignment onto Cortical Semantic Topography by Dongxin Guo, Jikun Wu, Siu Ming Yiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23035.txt.

Introduction

Guo, Wu, and Yiu use sparse autoencoders to explain why middle layers of language models best predict human brain responses to language. Decomposing GPT-2 XL plus Llama-3.1-8B into tens of thousands of interpretable features per layer, they find semantic features alone recover 94 percent of peak encoding performance. Five semantic subcategories drawn from three neuroscience programs map onto distinct brain regions, confirmed by a formal convergence test. The features also predict human reading times beyond lexical controls, and generalize across English, Chinese, and French.

Catalog record

CREATORDongxin Guo, Jikun Wu, Siu Ming Yiu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63939
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23035

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sparse Autoencoders Map Brain–LLM Alignment onto Cortical Semantic Topography Dongxin Guo, Jikun Wu, Siu Ming Yiu Abstract Intermediate layers of large language models (LLMs) best predict human brain responses to language, one of the most robust findings in computational neurolinguistics, yet why remains mechanistically unexplained. We address this gap by bridging sparse autoencoders (SAEs) from mechanistic interpretability with neural encoding models, decomposing GPT-2 XL and Llama-3.1-8B into 16K–32K interpretable features per layer. A human-validated taxonomy ( κ ≥ 0.74 \kappa\geq 0.74 ) reveals that semantic features alone recover 94% of peak encoding performance ( r = 0.285 r{=}0.285 ), substantially exceeding variance-matched baselines ( p < 0.001 p{<}0.001 , d = 1.31 d{=}1.31 ). Beyond this aggregate dominance, we test a novel cortical topography prediction: five semantic subcategories derived a priori from three independent neuroscience programs should map onto distinct brain regions. A formal convergence test confirms this alignment (Spearman ρ = 0.72 \rho{=}0.72 , p < 0.001 p{<}0.001 ; hypergeometric p = 0.007 p{=}0.007 ), demonstrating that SAE-discovered features recapitulate known cortical semantic organization at a granularity inaccessible to prior methods. SAE features further predict human reading times beyond lexical controls ( Δ \Delta logLik = 38.4 {=}38.4 , p < 0.001 p{<}0.001 ), and an exploratory prediction-error analysis provides preliminary evidence that the brain additionally encodes unexpected semantic content. Results generalize across English, Chinese, and French. 1 Introduction What computational properties of language representations make them predictive of human brain activity? This question, rooted in decades of neurolinguistic theory ( 25 ; 40 ; 32 ; 52 ) , has gained new urgency as large language models (LLMs) predict neural responses with remarkable accuracy ( 59 ; 24 ; 57 ) . 65 demonstrated that LLM-optimized stimuli causally drive the brain’s language network, and 20 argued that this network constitutes a natural kind whose properties align with LLM-discovered representations. The “direct fit” perspective ( 26 ) suggests this alignment arises because both brains and models are optimized for similar computational objectives. Yet a fundamental puzzle remains. Brain-predictive power is not uniform across depth: intermediate layers consistently outperform early and late layers ( 11 ; 63 ; 65 ) . This inverted-U is among the most robust findings in computational neurolinguistics ( 46 ) . But why do intermediate layers best align with the brain? Three non-mutually-exclusive accounts have been proposed. (1) Predictive processing: under hierarchical predictive coding ( 58 ; 21 ; 14 ) , neural responses reflect prediction errors; intermediate layers may generate the most brain-like predictions ( 22 ; 28 ) . (2) Feature discovery: LLMs discover rich linguistic features as a byproduct of prediction ( 1 ) ; intermediate layers contain the richest semantics before late-layer specialization. (3) Geometric artifact: intermediate layers may simply have higher-dimensional representations that are more linearly decodable ( 36 ) . Distinguishing what the brain represents from how it computes requires decomposing representations into interpretable features. Sparse autoencoders (SAEs) provide exactly this capability ( 31 ; 62 ) . We bridge mechanistic interpretability and neural encoding to provide a feature-level decomposition of the intermediate-layer advantage. We use “mechanistic” here in the sense of mechanistic interpretability (decomposing representations into interpretable features), not as a claim of full causal-mechanistic explanation. Our contributions: 1. Primary empirical (novel): We derive five semantic subcategories a priori from the independent neuroscience programs of Huth et al., Binder et al., and Deniz et al., then test whether SAE features recapitulate their predicted cortical topography. A formal convergence test (Spearman rank correlation \rho{=}0.72 , p{<}0.001 ; hypergeometric overlap p{=}0.007 ) confirms significant alignment between the predicted and observed subcategory \times region patterns, demonstrating that SAE-derived features capture neurally meaningful semantic distinctions at a granularity unavailable to prior methods. 2. Methodological (enabling): We train SAEs on every fourth layer of GPT-2 XL and Llama-3.1-8B, extracting 16K–32K features per layer with a validated five-way taxonomy ( \kappa\geq 0.74 for substantive categories). We empirically demonstrate that SAEs outperform simpler word-level semantic norm annotations for cortical topography mapping (§ 5.2 ). 3. Supporting empirical (confirmatory + exploratory): Three converging analyses (encoding models, variance partitioning, and activation patching with variance-matched controls) confirm at feature-level resolution that semantic content dominates brain–LLM alignment, extending prior aggregate findings ( 34 ; 1 ) . Behavioral validation shows SAE semantic features predict human reading times (§ 5.3 ). An exploratory prediction-error analysis provides preliminary evidence that the brain may additionally encode unexpected semantic content. Central claim. SAE-extracted semantic features simultaneously explain the well-known intermediate-layer advantage in brain–LLM alignment and a cortical topography that converges with established neuroscientific accounts of semantic organization: a single feature-level account spanning two previously disconnected empirical regularities. Prior work by 34 and 1 established that semantic content, specifically the lexical-semantic dimension contrasted with syntactic structure, dominates brain alignment using category-level probes and aggregate measures. Our SAE decomposition advances beyond these predecessors in four ways: (i) feature-level granularity (16K–32K individual features per layer vs. {\sim} 3–5 broad categories) enabling the subcategory \times brain-region interaction analysis (§ 4.4 ); (ii) a priori subcategories and formal convergence testing against independent neuroscience programs (§ 4.5 ); (iii) behavioral validation showing these features predict human reading times (§ 5.3 ); and (iv) an exploratory test of semantic prediction errors (§ 5.4 ). Our finer-grained decomposition further reveals that the brain-predictive variance attributable to semantic content is captured by contextual SAE features rather than by lexical features alone (Table 2 ), refining the lexical-semantic emphasis of 34 . Together, these contributions transform the question from whether semantics drives alignment to which specific semantic features drive alignment where in the brain . 2 Background and Related Work The intermediate-layer advantage. 59 systematically documented that intermediate layers best predict brain responses. 11 confirmed this across 100+ subjects; 24 found shared principles using ECoG. The earliest layer-wise alignment work used ELMo and BERT ( 33 ; 63 ) . 46 showed earlier layers correspond to fast processing signals while later layers align with N400 amplitudes, connecting to work by 50 demonstrating that LLM surprisal predicts single-trial N400 amplitudes. 35 demonstrated analogous hierarchical correspondence in vision. What drives alignment? A central debate concerns which properties of LLM representations are responsible for their brain-predictive power. One line of work emphasizes lexical-semantic content : 34 showed that lexical-semantic information dominates LM–brain alignment when contrasted with syntactic structure, building on earlier work disentangling syntax and semantics in brain responses ( 9 ) . A second line emphasizes feature discovery and contextualization : 1 argued that brain-predictive features emerge as a byproduct of next-word prediction; 54 showed alignment depends on multiple linguistic dimensions across modalities; and 66 demonstrated that fine-grained semantic annotations improve brain encoding. A third strand examines training and scale : 29 found brain alignment in models trained on only {\sim} 100M words, 55 showed training dynamics shape brain fit, and 18 showed geometric alignment in multilingual embeddings. Beyond raw activations, 47 showed XAI attributions outperform raw activations, and 38 deconstructed attention heads into functionally specialized components. Methodologically, the representational similarity analysis tradition ( 37 ; 53 ) offers a complementary view; the THINGS initiative ( 27 ) and Algonauts challenges ( 13 ) have advanced shared evaluation for object semantics, though we focus on

Read the complete text