Refusal Direction is Universal Across Safety-Aligned Languages
Xinpeng Wang, Mingyang Wang, Yihong Liu, Hinrich Schütze, Barbara Plank · 2025
Read Refusal Direction is Universal Across Safety-Aligned Languages by Xinpeng Wang, Mingyang Wang, Yihong Liu, Hinrich Schütze, Barbara Plank (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17306.txt.
Introduction
Xinpeng Wang and four co-authors show that the refusal direction in language models is universal across safety aligned languages. Refusal in English models can be mediated by a single direction in activation space. Whether that holds beyond English was unknown, yet safe behavior matters in every language a model serves.
A multilingual safety dataset is built by translating malicious and benign English prompts into 14 languages. A refusal vector extracted from English alone bypasses refusals in other languages with near perfect effect and no fine tuning. Vectors from any safety aligned language transfer to the rest. Parallel geometry of the vectors in embedding space explains the transfer and exposes a cross lingual jailbreak route.
Catalog record
| CREATOR | Xinpeng Wang, Mingyang Wang, Yihong Liu, Hinrich Schütze, Barbara Plank |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17306
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Refusal Direction is Universal Across Safety-Aligned Languages Xinpeng Wang, Mingyang Wang, Yihong Liu, Hinrich Schütze, Barbara Plank Abstract Refusal mechanisms in large language models (LLMs) are essential for ensuring safety. Recent research has revealed that refusal behavior can be mediated by a single direction in activation space, enabling targeted interventions to bypass refusals. While this is primarily demonstrated in an English-centric context, appropriate refusal behavior is important for any language, but poorly understood. In this paper, we investigate the refusal behavior in LLMs across 14 languages using PolyRefuse , a multilingual safety dataset created by translating malicious and benign English prompts into these languages. We uncover the surprising cross-lingual universality of the refusal direction: a vector extracted from English can bypass refusals in other languages with near-perfect effectiveness, without any additional fine-tuning. Even more remarkably, refusal directions derived from any safety-aligned language transfer seamlessly to others. We attribute this transferability to the parallelism of refusal vectors across languages in the embedding space and identify the underlying mechanism behind cross-lingual jailbreaks. These findings provide actionable insights for building more robust multilingual safety defenses and pave the way for a deeper mechanistic understanding of cross-lingual vulnerabilities in LLMs. 1 1 1 We make our code publicly available at https://github.com/mainlp/Multilingual-Refusal . † † ∗ Equal Contribution. † † † The work was done prior to joining Amazon. 1 Introduction LLMs are increasingly deployed across a wide range of real-world applications ( Kaddour et al., 2023 ; Yang et al., 2024 ; Raza et al., 2025 ) . To ensure their safe use, LLMs are expected to exhibit a refusal mechanism , the ability to obey to non-harmful request but refuse harmful, unethical, or policy-violating requests ( Bai et al., 2022 ) . This capability is typically instilled via reinforcement learning from human feedback (RLHF) ( Ouyang et al., 2022 ; Christiano et al., 2017 ; Dai et al., 2024 ) and other alignment strategies ( Yuan et al., 2024 ; Wallace et al., 2024 ; Xu et al., 2024a ) . Despite these efforts, LLMs remain vulnerable to jailbreak attacks, including adversarial prompt engineering ( Wei et al., 2023 ; Zou et al., 2023 ; Liu et al., 2024a ; Tao et al., 2024 ) , where carefully crafted inputs trigger unsafe outputs, and targeted fine-tuning ( Yang et al., 2023 ; Lermen et al., 2024 ; Zhan et al., 2024 ) , which undermines safety constraints through parameter updates. Notably, cross-lingual jailbreaks have emerged as a growing concern ( Yong et al., 2024 ; Li et al., 2024a ; Deng et al., 2024 ) , where prompts in non-English languages bypass refusal mechanisms that succeed in English, raising critical questions about the multilingual refusal mechanism in LLMs. Recent work has revealed that refusal behavior in LLMs is encoded within the model’s activation space ( Arditi et al., 2024 ; Wang et al., 2025b ) . Specifically, a low-dimensional subspace – often well-approximated by a single vector known as the refusal direction – captures the model’s tendency to refuse certain prompts. This insight has enabled controlled bypassing or reinforcement of refusals through simple vector operations. However, these findings have largely been limited to English, leaving a critical question unanswered: How universal are refusal directions across languages? Refusal is a core pragmatic function present in all human languages, although its surface form may vary across linguistic and cultural contexts ( Brown, 1987 ; Beebe et al., 1990 ) . Prior work suggests that LLMs often share internal representations across languages ( Artetxe et al., 2020 ; Wei et al., 2021 ; Hua et al., 2024 ; Brinkmann et al., 2025 ) and often rely on English as an implicit pivot in their reasoning processes ( Wendler et al., 2024 ; Wang et al., 2025a ; Yong et al., 2025 ) . These findings motivate our hypothesis: that refusal, as a pragmatic feature, may also generalize across languages – either in an English-centric way (i.e., the refusal direction learned in English transfers to other languages ), or more strongly, universally (i.e., refusal directions derived from any language covered within the LM’s abilities are approximately equivalent ). To evaluate this hypothesis, we perform a series of activation-based interventions across multiple languages. To enable this cross-linguistic analysis, we develop PolyRefuse , a dataset containing translated harmful prompts across 14 linguistically diverse languages. We first extract refusal directions with English prompts and assess their effectiveness in modulating refusal behavior in other languages. We then derive refusal directions from three typologically diverse safety-aligned languages and assess their cross-lingual transferability. 2 2 2 We refer to languages that exhibit stable and robust refusal responses – i.e., those resistant to jailbreak attempts and aligned with safety objectives – as safety-aligned languages (cf. § 3.1 ). Our experiments support the hypothesis, demonstrating a certain universality of refusal directions across safety-aligned languages . To better understand the underlying cause of this transferability and why cross-lingual jailbreaks still succeed, we analyze the geometric structure of refusal directions and harmfulness representations across languages in the models’ embedding space. We find that refusal vectors are approximately parallel across languages, explaining the effectiveness of cross-lingual vector-based interventions. However, models often fail to separate harmful and harmless prompts in non-English languages. This insufficient separation weakens refusal signals and leaves models vulnerable to jailbreaks. These findings contribute to a deeper mechanistic understanding of how LLMs encode and generalize refusal behavior across languages. By revealing the language-agnostic nature of refusal directions, we also provide actionable insights for developing stronger, more reliable multilingual safety defenses. 2 Related Work LLM Safety and Refusal Mechanism In AI safety research, various efforts have been made to prevent LLMs from responding to malicious queries. Notable approaches include supervised fine-tuning (SFT) ( Bianchi et al., 2024 ) and reinforcement learning from human feedback (RLHF) ( Bai et al., 2022 ) . To evaluate the effectiveness of these safety measures, researchers have developed comprehensive safety evaluation datasets. While these datasets initially focused on English ( Zou et al., 2023 ; Mazeika et al., 2024 ; Xie et al., 2025 ) , recent work has expanded to include multilingual evaluations, revealing concerning vulnerabilities in non-English contexts ( Shen et al., 2024 ; Yong et al., 2024 ; Wang et al., 2024 ) . Furthermore, researchers have begun investigating the internal mechanisms that enable LLMs to recognize and refuse harmful requests. Studies examining model representations ( Xu et al., 2024b ; Li et al., 2024b ) and have identified specific “refusal directions” in the embedding space ( Arditi et al., 2024 ; Marshall et al., 2025 ) . However, these mechanistic interpretability studies have predominantly focused on English, leaving cross-lingual aspects of refusal mechanisms largely unexplored. This paper addresses this gap by investigating how refusal mechanisms function across different languages. Multilingual Alignment. A central goal in multilingual natural language processing (NLP) is to develop language-agnostic representations that enable generalization across linguistic boundaries – commonly referred to as cross-lingual transfer ( Libovický et al., 2020 ; Wei et al., 2021 ; Chang et al., 2022 ) . Early research primarily focuses on aligning static word embeddings using bilingual dictionaries or parallel corpora ( Lample et al., 2018a ; Lample et