What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
Stephen Cheng, Sarah Wiegreffe, Dinesh Manocha University of Maryland, College Park, Correspondence: [email protected] · 2026
Read What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal by Stephen Cheng, Sarah Wiegreffe, Dinesh Manocha University of Maryland, College Park, Correspondence: [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08524.txt.
Introduction
Cheng, Wiegreffe, and Manocha ask what steering vectors actually change inside language models, using refusal as a test case. A multi token activation patching study finds varied steering methods use interchangeable circuits at the same layer, acting mainly via the OV circuit and largely bypassing the QK circuit. Freezing attention scores costs little performance, and steering vectors can be cut by 85 to 96 percent while keeping most of their effect.
Catalog record
| CREATOR | Stephen Cheng, Sarah Wiegreffe, Dinesh Manocha University of Maryland, College Park, Correspondence: [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68178 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.08524
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal Stephen Cheng, Sarah Wiegreffe, Dinesh Manocha University of Maryland, College Park, Correspondence: [email protected] Abstract Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works– specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal . We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit– freezing all attention scores during steering drops performance by only ~8.75% across two model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 90-99% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions. † † footnotetext: ∗ Equal contribution. † † footnotetext: † Code will be released upon publication. 1 Introduction Figure 1: We analyze which components in language models are responsible for propagating refusal steering. Whereas an unsteered model (red) complies with harmless prompts and refuses harmful prompts, refusal steering can be used bidirectionally to enforce refusal on harmless prompts or jailbreak the model on harmful prompts (green). In Section 6.3 , we find that steering a model while freezing all attention weights to their unsteered activations has a negligible effect on steering (blue), indicating that the refusal vector largely ignores the QK circuit. Aligning large language models to behave in accordance with human intent is a central challenge in deploying these systems safely Anwar et al. 2024 . Steering vectors have emerged as a lightweight model alignment technique that acts on the model’s hidden activations at inference time Zou et al. 2025 . This approach has been applied across a range of alignment-relevant tasks, including reducing hallucinatory behavior Chen et al. 2025 ; Rimsky et al. 2024 , controlling persona and style Subramani et al. 2022 ; TurnTrout et al. 2023 , and enhancing reasoning Venhoff et al. 2025 . Results on recent benchmarks demonstrate competitive performance against fine-tuning and prompting baselines Wu et al. 2025a . Despite their growing adoption, we lack a mechanistic understanding of how steering vectors interact with model components to produce behavioral shifts. In addition to advancing our scientific knowledge of LLMs, understanding these mechanisms can allow practitioners to assess steering robustness, diagnose failure cases Braun et al. 2025 , and inform the design of steering interventions with better concept expression or reduced degradation Da Silva et al. 2025 . To address this gap, we conduct a case study on steering vectors for a critical capability– refusal within the context of LLM jailbreaking Wei et al. 2023 . Refusal steering has been shown to be highly effective at encouraging or discouraging refusal responses Arditi et al. 2024 , making it a natural first target for a mechanistic analysis on steering. We propose to extend traditional mechanistic interpretability techniques, typically applied only to standard LLM inference runs, to steered inference runs, in order to better characterize steering vectors’ effectiveness. Our contributions are: 1. We propose a generalizable multi-token activation patching approach that extends circuit discovery to steered generations. We find that steering vectors obtained through different methodologies leverage highly interchangeable circuits ( \gtrsim 90\% overlap). 2. Refusal steering interacts with attention primarily through the OV circuit. On the other hand, freezing all attention scores (QK circuit) drops performance by only 8.75% . We introduce the steering value vector decomposition, which is semantically interpretable even when the steering vector itself is not. 3. We leverage our findings to sparsify refusal steering vectors up to 90-99% while mostly retaining performance. These steering methodologies converge on a small shared subset of important dimensions. 2 Related Works Refusal Steering and Steering Methods Arditi et al. 2024 demonstrate that the concept of refusal can be represented by a single direction, which can be used to jailbreak Xu et al. 2024 models on harmful prompts and induce refusal on harmless prompts. Subsequent work has further explored refusal steering, including reducing false refusals Lee et al. 2025 ; Wang et al. 2025 and characterizing the geometry of refusal directions Wollschläger et al. 2025 . Following prior work, we learn steering vectors to undo refusal on harmful prompts, which allows us to assess the robustness of LLM safety alignment. Learning-based steering methodologies Wu et al. 2025a ; Wu et al. 2025b ; Sun et al. 2025 have also achieved competitive performance against fine-tuning and prompting baselines. Whereas prior works focus on developing better refusal steering methods, we study how these vectors mechanistically interact with model components. Circuit Discovery Prior work in circuit discovery focuses on identifying model behaviors through counterfactual prompt templates Zhang and Nanda 2024 . These behaviors include indirect object identification Wang et al. 2023 , addition Stolfo et al. 2023 , and multiple choice question answering Wiegreffe et al. 2025 . Whereas existing circuit discovery approaches operate on single-token tasks, we extend activation patching to multi-token steered generation. The most closely related work is Sinii et al. 2025 , who apply causal analysis to reasoning steering vectors. However, their analysis is limited to the last two layers of the LLM, which does not reflect conventional steering applied most effectively in middle layers, and they study only one steering methodology. 3 Preliminaries 3.1 Data and Models Data To learn steering vectors, we construct harmless instruction and harmful instruction datasets, D_{safe} and D_{harm} . Following Arditi et al. 2024 , for D_{harm} , we select harmful prompts from adversarial datasets AdvBench Zou et al. 2023 , MaliciousInstruct Huang et al. 2023 , TDC2023 Mazeika et al. 2022 , and HarmBench Mazeika et al. 2024 . For D_{safe} , we randomly select harmless prompts from Alpaca Taori et al. 2023 . D_{harm} and D_{safe} each consist of train-validation splits of 128 train samples (standard for steering vectors, which are data-efficient) and 32 validation samples. For our harmful and harmless test sets, we use 100 harmful prompts from JailbreakBench Chao et al. 2024 and 100 randomly-selected harmless prompts from Alpaca, respectively. Models We use Gemma 2 2B Instruct Team et al. 2024 and Llama 3.2 3B Instruct Grattafiori et al. 2024 , two representative open LLMs. 3.2 Refusal Steering Activation Addition Given a language model with hidden activation \mathbf{h}^{\ell}\in\mathbb{R}^{d} at layer \ell and a refusal steering vector \mathbf{s}\in\mathbb{R}^{d} with dimension d , activation addition steering Turner et al. 2023 is formulated as \mathbf{h}^{\ell}\leftarrow\mathbf{h}^{\ell}+\alpha\cdot\mathbf{s} (1) where \alpha is a scalar steering coefficient. \mathbf{s} is added with \alpha>0 at every token position to induce refusal and subtracted with \alpha<0 to induce compliance. We study multi-token steering Chen et al. 2025 ; Wu et al. 2025a , where the steering vector is repeatedly added to each decoded token. Difference-in-Means DIM Turner et al. 2023 ; Rimsky et