Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification

Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng · 2026

Read Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification by Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06226.txt.

Introduction

Toxic outputs persist inside language models even after output-level fixes. Zenghao Duan and 10 co-authors propose removing the problem at its source in a 2026 paper on detoxification. Alignment changes what a model prefers to say, but the underlying toxic regions in its parameters remain, leaving it open to adversarial prompts.

Their analysis shows removed toxic vectors can be rebuilt from combinations of harmless ones, so the whole toxic subspace must go. Sampling noise also corrupts layer-by-layer extraction. Their GLOSS method identifies a global toxic subspace in the feed-forward parameters and suppresses it. On models such as Qwen3, GLOSS achieved top detoxification while preserving general abilities, without large-scale retraining.

Catalog record

CREATORZenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-06226

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng Abstract Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content, restricting their safe deployment. While traditional methods (e.g., alignment) adjust output preferences, they fail to eliminate underlying toxic regions in parameters, leaving models vulnerable to adversarial attacks. Prior mechanistic studies characterize toxic regions as "toxic vectors" or "layer-wise subspaces", yet our analysis identifies critical limitations: i) Removed toxic vectors can be reconstructed via linear combinations of non-toxic vectors, demanding targeting of entire toxic subspace; ii) Contrastive objective over limited samples inject noise into layer-wise subspaces, hindering stable extraction. These highlight the challenge of identifying robust toxic subspace and removing them. Therefore, we propose GLOSS ( GL obal t O xic S ubspace S uppression), a lightweight method that mitigates toxicity by identifying and eliminating this global subspace from FFN parameters. Experiments on LLMs (e.g., Qwen3) show GLOSS achieves SOTA detoxification while preserving general capabilities without requiring large-scale retraining. WARNING: This paper contains context which is toxic in nature. 1 Introduction Large language models (LLMs) have shown impressive capabilities in various domains ( 1 ; 35 ) . However, they also pose risks of toxicity generation, which may lead to undesirable effects in real-world applications ( 16 ) . To mitigate toxicity, traditional detoxification methods based on fine-tuning and reinforcement learning ( 21 ; 26 ) have been widely adopted to improve LLM safety. Despite these efforts, aligned models remain vulnerable to adversarial attack prompts ( 37 ) , as these methods only align model behavior without effectively removing the underlying toxic content from the models. Consequently, recent research has focused on analyzing the internal mechanisms of LLMs to identify the specific regions that generate toxicity. Recent studies have primarily attributed toxic generation to the feed-forward networks (FFNs) within LLM, leading to two distinct theoretical frameworks. First, exemplified by 14 , identifies the toxic region as toxic vectors and suggests that alignment methods ( 26 ) mitigates toxicity by bypassing these vectors. In contrast, represented by ProFS ( 30 ) , proposes that toxicity resides in layer-wise toxic subspaces, which are identified through embedding differences between toxic and non-toxic prompt pairs. However, the connection and limitations of these frameworks remain unexplored. Figure 1: Motivation for global toxic subspace. (a) Toxic vectors can be reconstructed from non-toxic vectors via linear combinations. (b) Layer-wise subspaces suffer from noise due to limited samples. (c) Global toxic subspace provides stable, layer-invariant representation. To address this gap, we conduct systematic analysis on GPT-2 Medium and Qwen3-0.6B-Base, yielding the following key findings, as shown in Figure 1 . i) Since FFNs operate as linear combinations of value vectors ( 9 ) , the magnitude and sign of activation coefficient significantly influence toxicity expression. Even after toxic vectors are suppressed or removed, toxic content can still be reconstructed through non-toxic vectors ( \mathsection 3.1 ). ii) FFN exhibit varying capacity for toxicity modeling across layers. It makes layer-wise extraction methods susceptible to noise interference from limited samples, hindering stable extraction of layer-wise toxic subspaces ( \mathsection 3.2 ). This hinders robust toxic subspace identification. Inspired by the consistent information flow facilitated by residual connections ( 7 ) , we identify a global toxic subspace shared across layers, which yields a stable toxic representation invariant to layer-specific variations ( \mathsection 3.3 ). Motivated by the above analysis, we propose GLOSS ( GL obal t O xic S ubspace S uppression), a lightweight detoxification method that requires neither large-scale data nor model retraining ( \mathsection 4 ). GLOSS operates through a three-stage process: First, it extracts candidate toxic directions from each layer by applying SVD to activation differences between toxic and non-toxic input pairs. Second, it ranks all candidate directions globally and selects those with high toxicity scores to ensure only meaningful toxic directions are retained. Finally, it extracts principal components from the selected directions to construct a unified global toxic subspace. GLOSS suppresses toxicity by projecting the weights of each FFN module onto the orthogonal complement of this subspace, effectively removing toxic components while preserving the model’s general capabilities. We conduct extensive experiments to evaluate GLOSS on RealToxicityPrompts ( 8 ) and PolyglotoxicityPrompts ( 13 ) across six LLMs of varying sizes and architectures ( \mathsection 5 ). Our experimental results demonstrate that GLOSS achieves lower toxicity scores than ProFS and other baselines while maintaining the model’s general capabilities, thereby validating our hypothesis that removing the global toxic subspace enables more effective detoxification. Notably, despite requiring fewer training samples, both GLOSS and ProFS substantially outperform supervised safety fine-tuning (SSFT) and direct preference optimization (DPO), demonstrating the effectiveness of safety mechanism analysis compared to traditional fine-tuning paradigms. In summary, our contributions are the following: i) We provide a systematic analysis revealing the limitations of existing toxic vector and layer-wise subspaces perspectives, and identify the global toxic subspace as a more robust representation of toxic region. ii) We propose GLOSS, a lightweight detoxification method that extracts and removes the global toxic subspace without requiring model retraining. iii) We demonstrate through extensive experiments that GLOSS achieves superior detoxification performance while maintaining model capabilities across diverse LLMs. (a) Enhance activation (b) Reverse activation (c) Suppress activation Figure 2: Toxicity changes under different vector activation operations in Qwen3. (a) Enhanced activations amplify toxic vectors by factor 10; (b) Reversed activations flip signs based on cosine similarity to toxic direction; (c) Suppressed activations scale down top- k toxic vectors. 2 Preliminaries In this section, we analyze FFN components and introduce the vocabulary space projection method. FFN as a linear combination of vectors. Large language models consist of stacked Transformer layers ( 31 ) , each containing multi-head self-attention (MHSA) and feed-forward network (FFN) blocks with residual connections and normalization. Given an input sequence \mathbf{w}=\langle w_{0},\dots,w_{t}\rangle , the model maps each token w_{i} to an embedding \mathbf{e}_{i}\in\mathbb{R}^{d} using the embedding matrix E . At each layer \ell , the FFN receives the hidden state \mathbf{x}_{i}^{\ell}\in\mathbb{R}^{d} corresponding to token i and produces an intermediate output \mathbf{o}_{i}^{\ell}=\mathrm{FFN}^{\ell}(\mathbf{x}_{i}^{\ell})\in\mathbb{R}^{d} . The updated representation after applying the FFN and residual connection is \tilde{\mathbf{x}}_{i}^{\ell}=\mathbf{x}_{i}^{\ell}+\mathbf{o}_{i}^{\ell}\in\mathbb{R}^{d} . FFNs function as linear combinations of value vectors ( 9 ) . We focus on the two-layer MLP formulation (e.g., GPT-2), deferring the three-layer variant (e.g., Qwen3) to Appendix C.1 . Let W_{K}^{\ell},W_{V}^{\ell}\in\mathbb{R}^{d_{m}\times d} denote the input and output projection matrices, respectively, and let f(\cdot) be a non-linear activation. For one token with hidden state \mathbf{x}^{\ell}\in\mathbb{R}^{d} (omit the token index for readability), the FFN first computes activation weights \mathbf{m}^{\ell} and then produces the output \mathrm{FFN}^{\ell}(\mathbf{x}^{\ell}) : \displaystyle\mathrm{FFN}^{\ell}(\mathbf{x}^{\ell}) \displaystyle=(\mathbf{m}^{\ell})^{\top}W_{V}^{\ell}=\sum_{i=1}^{d_{m}}m_{i}^{\ell}\,\mathbf{v}_{i}^{\ell} (1) \displaystyle\mathbf{m}^{\ell} \displaystyle=f\!\left(W_{K}^{\ell}\,\mathbf{x}^{\ell}\right)\in\mathbb{R}^{d_{m}} (2) where m_{i}^{\ell}=f(\mathbf{k}_{i}^{\ell}\cdot\mathbf{x}^{\ell})\in\mathbb{R}^{d} with \mathbf{k}_{i}^{\ell} the i -th row of W_{K}^{\ell} , and \mathbf{v}_{i}^{\ell}\in\mathbb{R}^{d} the i -th row of W_{V}^{\ell} . Equations ( 2 ) make explicit that the FFN output is a weighted sum of value vectors. Interpreting vectors in the vocabulary space. To interpret the semantic meaning of a vector \mathbf{u}\in\mathbb{R}^{d} in the embedding space, we project it into the vocabulary space using the output embedding matrix E=[\mathbf{e}_{1},\ldots,\mathbf{e}_{|\mathcal{V}|}]^{\top}\in\mathbb{R}^{|\mathcal{V}|\times d} , where \mathcal{V} denotes the vocabulary

Read the complete text