Analysing the Safety Pitfalls of Steering Vectors
Yuxiao Li, Alina Fastowski, Efstratios Zaradoukas, Bardh Prenkaj, Gjergji Kasneci · 2026
Read Analysing the Safety Pitfalls of Steering Vectors by Yuxiao Li, Alina Fastowski, Efstratios Zaradoukas, Bardh Prenkaj, Gjergji Kasneci (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24543.txt.
Introduction
Li and co-authors audit the safety risks of activation steering in large language models. Steering vectors, often built by contrastive activation addition, shape model behaviour without changing weights, but their safety effects are little studied. Using JailbreakBench under one protocol, they find steering reliably shifts jailbreak success rates. Depending on direction, attack success can rise by up to 57 percent or fall by up to 50 percent. They trace this to overlap between steering vectors and the latent directions of refusal behaviour, exposing a control-versus-safety trade-off.
Catalog record
| CREATOR | Yuxiao Li, Alina Fastowski, Efstratios Zaradoukas, Bardh Prenkaj, Gjergji Kasneci |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52603 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.24543
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Analysing the Safety Pitfalls of Steering Vectors Yuxiao Li, Alina Fastowski, Efstratios Zaradoukas, Bardh Prenkaj, Gjergji Kasneci Abstract Activation steering has emerged as a powerful tool to shape LLM behavior without the need for weight updates. While its inherent brittleness and unreliability are well-documented, its safety implications remain underexplored. In this work, we present a systematic safety audit of steering vectors obtained with Contrastive Activation Addition (CAA), a widely used steering approach, under a unified evaluation protocol. Using JailbreakBench as benchmark, we show that steering vectors consistently influence the success rate of jailbreak attacks, with stronger amplification under simple template-based attacks. Across LLM families and sizes, steering the model in specific directions can drastically increase (up to 57 % 57\% ) or decrease (up to 50 % 50\% ) its attack success rate (ASR), depending on the targeted behavior. We attribute this phenomenon to the overlap between the steering vectors and the latent directions of refusal behavior. Thus, we offer a traceable explanation for this discovery. Together, our findings reveal the previously unobserved origin of this safety gap in LLMs, highlighting a trade-off between controllability and safety. Disclaimer: This manuscript may contain potentially harmful model outputs. 1 Introduction Figure 1: Activation steering erodes LLM safety. For Qwen 14B, without steering, a model refuses harmful input (ASR: 4%, A). Steering towards "Self-Awareness" alone compromises safety (ASR: 42%, C). Critically, combining steering with simple attacks leads to a near-complete collapse of safety (ASR: 80%, B), revealing a severe safety-controllability trade-off in LLMs. Activation steering provides a cost-efficient way for controlling Large Language Models’ (LLMs) behavior at inference time 54 ; 42 ; 28 ; 9 . By directly manipulating LLMs activations, steering can guide high-level attributes, such as enhancing truthfulness 21 , mitigating sycophancy 28 , shaping political leanings 20 , and even improving complex reasoning 44 . Despite its conceptual elegance and initial successes, the reliability of steering remains a significant challenge. Recent studies highlight its poor generalization, limited effectiveness to specific tasks, and substantial variability 38 ; 8 ; 34 ; 6 . These fundamental reliability issues question the real-world advantages of steering over simple prompting 51 . However, the existing line of research has defined and analyzed steering brittleness primarily through the lens of utility and reliability : does the steering vector reliably produce its intended effect without breaking the LLMs’ general capabilities? Motivated by the gap between steering’s promise of control and the emerging evidence of its unreliability, this paper investigates a more dangerous dimension of its brittleness: safety . Our central research question is: How does steering impact the safety alignment of LLMs ? We conduct a systematic safety audit of steering as a technique, treating it as an inherently fragile process with predictable safety pitfalls. Our central finding is that steering’s primary safety gap is its systematic erosion of the model’s safety alignment, increasing the success rate of otherwise weak, prompt-level jailbreak attacks. Our contributions are the following: (1) We conduct the a systematic safety audit of activation steering spanning six models from three families and sizes (3B–32B). Using simple, template-based attacks as a diagnostic tool, we find that steering systematically alters the ASR, with some behaviors causing drastic increases (up to 57%) or decreases (up to 50%). (2) We trace these safety issues to a mechanistic origin. Our analysis reveals that this phenomenon is correlated with the directional overlap between steering vectors and the model’s refusal behavior direction. (3) We provide causal evidence for this mechanism and demonstrate a potential mitigation strategy. By ablating the refusal-aligned component from steering vectors, we consistently mitigate the vector’s impact on ASR, providing causal validation for the geometric interference hypothesis. (4) We establish a fundamental trade-off between controllability and safety. With our work, we aim to draw attention to a new class of vulnerabilities and motivate a research agenda towards inherently safe steering methods. 2 Related Work Activation Steering and its Brittleness. Building on mechanistic interpretability studies 45 ; 12 ; 14 ; 7 ; 25 ; 29 , activation steering is first proposed as a lightweight paradigm for modifying LLMs’ behavior without altering the parameters 37 ; 54 . Existing approaches operate at different levels of the model architecture: steering vectors computed from activation differences 42 ; 28 , direct interventions on attention head outputs 21 ; 52 , and methods based on sparse autoencoders that extract interpretable feature directions in the residual stream activations 9 . While steering has been applied to different tasks 20 ; 35 ; 11 ; 44 , prior work has revealed significant challenges regarding its reliability and generalization. Studies show that steering suffers from high variability, poor out-of-distribution generalization 38 , and frequent ineffectiveness 34 . Its effectiveness is often limited to specific task types 8 and is less successful when steering multiple behaviors at once 43 . The underlying cause may be tied to the geometric coherence of activation differences 6 , and prompting comparisons that question its utility over simpler baselines like prompting 51 . While these works largely frame brittleness as a limitation for utility, we adopt a safety-focused perspective and investigate a more dangerous pitfall, the erosion of safety alignment due to activation steering. Safety Alignment and its Brittleness. Safety alignment in instruction-tuned LLMs is primarily achieved through refusal training, where models are trained to reject unsafe or disallowed requests 26 ; 3 ; 4 ; 33 . However, this alignment is often brittle 5 ; 17 ; 49 , as the model’s underlying unsafe capabilities are merely suppressed, not erased, resulting in a fragile safety loophole 31 ; 36 ; 47 . This brittleness is exposed by jailbreak studies, where prompt-level attacks have been shown to bypass safety alignment 46 ; 16 ; 1 ; 55 . At a deeper level, representation-level analysis reveals how refusal behaviors are encoded and can be subverted in activation space 13 ; 19 ; 22 . Interestingly, 2 find that refusal behavior is mediated by a single direction in the model’s residual stream activations. Similarly, 50 and 27 argue that refusal is not one-dimensional, but instead is controlled by higher-dimensional directions. We extend this representational view by showing that activation steering consistently shifts attack success rate. Specifically, we show that steering vectors partially align with the one-dimensional refusal direction, and use this geometric overlap to examine how activation shifts can push the model off its safety manifold. 3 Preliminaries Activation Steering Intervention. Let f_{\theta} be a decoder-only LLM with transformer layers \{1,\dots,L\} . For an input x and layer \ell , denote the residual stream activation of token position i by h_{\ell}^{(i)}(x)\in\mathbb{R}^{d_{\ell}} . Furthermore, let p be a prompt consisting of |p| tokens. We denote as i\in\{1,...,|p|\} the token position in p . At inference time, we apply steering during the forward pass on the prompt tokens by adding a fixed direction to these activations. Given a steering vector v_{\ell,\tau} and a scalar multiplier m\in\mathbb{R} controlling its strength, the steered activation is: \tilde{h}_{\ell}^{(i)}(x)\;=\;h_{\ell}^{(i)}(x)\;+\;mv_{\ell,\tau}. (1) Here v_{\ell,\tau} is associated with some behavioral trait \tau , and m determines the sign and magnitude of the intervention. We describe the choices of behavior vectors v_{\ell,\tau} , layers \mathcal{L} , multipliers m , and evaluation