Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Models
Youngji Roh, Hyunjin Cho, Jaehyung Kim · 2026
Read Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Models by Youngji Roh, Hyunjin Cho, Jaehyung Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00029.txt.
Introduction
Roh, Cho and Kim argue that massive activations in large language models are not mere glitches but interpretable functional units from domain specialisation. A few feature axes dwarf the rest, and a simple size-based rule picks out domain-critical axes with no training. Analysis shows they act as semantic sensors for symbolic, quantitative or own-domain patterns. Steering only those axes, named Critical Dimension Steering, beats whole-axis steering in domain adaptation and jailbreaking tests.
Catalog record
| CREATOR | Youngji Roh, Hyunjin Cho, Jaehyung Kim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13100 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.00029
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Embracing Anisotropy: Turning Massive Activations into Interpretable Control Knobs for Large Language Models Youngji Roh, Hyunjin Cho, Jaehyung Kim Abstract Large Language Models (LLMs) exhibit highly anisotropic internal representations, often characterized by massive activations , a phenomenon where a small subset of feature dimensions possesses magnitudes significantly larger than the rest. While prior works view these extreme dimensions primarily as artifacts to be managed, we propose a distinct perspective: these dimensions serve as intrinsic interpretable functional units arising from domain specialization. Specifically, we propose a simple magnitude-based criterion to identify Domain-Critical Dimensions in a training-free manner. Our analyses reveal that such dimensions behave as interpretable semantic detectors for symbolic/quantitative patterns or domain-specific terms. In addition, we introduce Critical Dimension Steering , which applies activation steering exclusively to the identified dimensions. Empirical results show that this approach outperforms conventional whole-dimension steering in domain adaptation and jailbreaking scenarios. 1 1 1 We will release the codes upon acceptance. 1 Introduction Despite the versatile capabilities of Large Language Models (LLMs) across diverse tasks, prior research indicates that their internal activations exhibit a distinct lack of isotropy ( ethayarajh2019representationanisotropic ; rudman2023stable ) . In particular, analyses on Transformer-based models reveal that representations do not uniformly utilize the embedding space dimensions ( ding2022isotropycalibration ; razzhigaev2024anisotropytransformer ) . This anisotropy, characterized by a highly non-uniform distribution of activations, is closely linked to the presence of a few outlier dimensions exhibiting variances orders of magnitude larger than others ( hammerl2023anisotropyoutlier ; dettmers2022int8 ) . sun2024massive characterizes this phenomenon as massive activations , where a small set of features attains values tens of thousands of times larger ( e.g. , 100,000\times ) than the remainder. Figure 1: Overview of the study. Domain-Critical Dimensions identified via activation magnitude act as interpretable functional units. We also introduce Critical Dimension Steering, which intervenes only on the identified dimensions for precise model control. A substantial body of literature argues that this representational imbalance prevents models from fully exploiting their capacity, potentially obscuring meaningful linguistic distinctions ( gao2019repdeg ; timkey2021rogue ) . Numerous studies have proposed to algorithmically suppress the influence of extreme dimensions in order to promote representational isotropy ( wei2022outliersupp ; he2024minimisingoutlier ) . Conversely, some perspectives suggest that anisotropy is a natural and potentially beneficial byproduct of stochastic gradient descent ( zhu2018anisotropicsgd ) , arguing that preserving these characteristics is crucial for maintaining downstream performance ( rudman2023stable ) . However, both perspectives largely treat these extreme dimensions as artifacts to be managed, either suppressed or preserved, rather than as features whose intrinsic functional role should be examined. In this work, we propose a distinct perspective: we hypothesize the emergence of extreme dimensions as a direct consequence of functional specialization . Consequently, these dimensions likely to encode domain-specific knowledge, serving as intrinsic interpretable functional units that can be leveraged for better model control. Unlike interpretation methods such as probe-based classifiers ( burns2023discoveringlatent ; li2023inference ) or sparse autoencoders (SAEs) ( cunningham2023sae ; bricken2023monosemanticity ) , which require learning additional parameters to map internal states to concepts or disentangle polysemantic features, our perspective allows us to uncover specialized features directly from the pre-trained model by exploiting simple statistics without additional training. Based on this hypothesis, we present a method for identifying specialized, domain-specific critical dimensions in § 2 . Our approach is grounded in two key observations regarding feature dimensions: (1) sparsity , where a small subset of dimensions plays a crucial role in model performance, and (2) extremity , where domain-discriminative dimensions naturally emerge with high-magnitude activations. Building on these insights, we propose a simple yet effective approach to identify Domain-Critical Dimensions by selecting the top- k dimensions based on activation magnitude. These dimensions substantially overlap with the ground-truth critical dimensions discovered through computationally expensive iterative masking and evaluation, indicating that functional importance can be inferred from activation statistics without supervision. Given the functional importance of these dimensions, in § , we conduct qualitative analyses to investigate their semantic roles by examining token-level activation patterns. These analysis reveals that individual domain-critical dimensions behave as highly interpretable feature detectors, as they are selectively activated on coherent concepts. For example, dimension 1046 strongly activates on mathematical terms ( + , x , \infty ), dimension 2106 on specific biological terms ( _ATP , _NAD , _phosphorylation ), and dimension 334 on the topic keywords ( _mathematics , _biology ). 2 2 2 Dimension indices here refer to Gemma-2-2b-it. Finally, we demonstrate that domain-critical dimensions can be utilized for better model control (§ ). We introduce Critical Dimension Steering (CDS) , a targeted inference-time intervention that applies activation steering— i.e. , modifying internal activations during the forward pass ( turner2023steering ; rimsky2024steering ) —only to the top- k dimensions identified by our method. As a sparse set of dimensions dominates behavior within a target domain, manipulating activations along these specific axes serves as more effective control knobs than influencing the entire latent space. We validate the efficacy of CDS in two distinct applications. In a domain adaptation setup using MMLU benchmark ( hendrycks2021measuring ) , CDS improves domain-specific accuracy over whole-dimension steering baseline in 34 out of 57 subjects. In a jailbreaking setup with AdvBench ( zou2023universal ) , applying CDS increases attack success rate to 92%, surpassing 84% achieved by whole-dimension steering baseline. 2 Existence and Identification of Domain-Critical Dimensions In this section, we present a method for identifying the top- k dimensions critical to a target domain. In § 2.1 , we first observe the existence of two types of feature dimensions: functionally critical dimensions and domain-discriminative dimensions. In § 2.2 , we then leverage these properties to identify the final domain-critical dimensions using simple statistical indicators. Problem setup. We consider pre-trained LLM composed of L Transformer layers and hidden dimension D . Given an input token sequence \mathbf{x}=[x_{1},\dots,x_{T}] , the model first maps tokens to the input embedding {h}_{0}\in\mathbb{R}^{T\times D} . Each layer l takes the hidden state {h}_{l-1} as input and produces {h}_{l} via residual connection, where the sub-layer output is added to the input. We refer to activations as scalar entries of {h}_{l} , and dimensions that encode significant semantic information or functional roles as feature dimensions . Our goal is to identify domain-critical dimensions \mathcal{I}_{\text{dcd}}\subset\{1,\dots,D\} , feature dimensions that are both specialized to a target domain and critical for preserving performance within that domain. Throughout this work, we operationalize a domain using the MMLU benchmark ( hendrycks2021measuring ) , which comprises 57 subjects spanning STEM, humanities, and social sciences. For each subject, we sample 100 prompts from the test split and partition them into two disjoint subsets: 50 prompts as an identification set and 50 prompts as an evaluation set . The identification set is used to extract \mathcal{I}_{\text{dcd}} , while the evaluation set is reserved for validating the identified dimensions and for downstream evaluations. 2.1 Intriguing Feature Dimensions of LLMs Existence of critical dimensions. We first observe sparse dimensions that are functionally critical . Intuitively, one way to estimate the importance of a specific dimension is measuring the performance degradation when target dimension is removed from the inference process. If all dimensions contribute equally to the model’s performance, masking a single dimension among thousands should result in