Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoders
Agam Goyal, Vedant Rathi, William Yeh, Yian Wang, Yuen Chen, Hari SundaramSiebel School of Computing and Data ScienceUniversity of Illinois Urbana-Champaign · 2025
Read Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoders by Agam Goyal, Vedant Rathi, William Yeh, Yian Wang, Yuen Chen, Hari SundaramSiebel School of Computing and Data ScienceUniversity of Illinois Urbana-Champaign (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14536.txt.
Introduction
Agam Goyal, Vedant Rathi, William Yeh, and a wider team wrote this machine learning paper, published in 2025. It cleans toxic language out of large language models by steering the model's inner activity, using sparse autoencoders to find the exact directions tied to toxicity.
Most detox methods patch the surface and can be dodged by jailbreak prompts. Sparse autoencoders split a model's activity into tidy, readable features. The team found the features linked to profanity and slurs, then pushed activity away from them at three strengths. On two models, strong steering cut toxicity by up to a fifth more than rival methods, though fluency suffered on the smaller model. Standard knowledge tests stayed flat, so the model's skills survived. The paper also warns that overly split features blunt such fixes, and draws lessons for safer deployment.
Catalog record
| CREATOR | Agam Goyal, Vedant Rathi, William Yeh, Yian Wang, Yuen Chen, Hari SundaramSiebel School of Computing and Data ScienceUniversity of Illinois Urbana-Champaign |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14536
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Breaking Bad Tokens: Detoxification of LLMs Using Sparse Autoencoders Agam Goyal, Vedant Rathi, William Yeh, Yian Wang, Yuen Chen, Hari SundaramSiebel School of Computing and Data ScienceUniversity of Illinois Urbana-Champaign Abstract Large language models (LLMs) are now ubiquitous in user-facing applications, yet they still generate undesirable toxic outputs, including profanity, vulgarity, and derogatory remarks. Although numerous detoxification methods exist, most apply broad, surface-level fixes and can therefore easily be circumvented by jailbreak attacks. In this paper we leverage sparse autoencoders (SAEs) to identify toxicity-related directions in the residual stream of models and perform targeted activation steering using the corresponding decoder vectors. We introduce three tiers of steering aggressiveness and evaluate them on GPT-2 Small and Gemma-2-2B, revealing trade-offs between toxicity reduction and language fluency. At stronger steering strengths, these causal interventions surpass competitive baselines in reducing toxicity by up to 20 % 20\% , though fluency can degrade noticeably on GPT-2 Small depending on the aggressiveness. Crucially, standard NLP benchmark scores upon steering remain stable, indicating that the model’s knowledge and general abilities are preserved. We further show that feature-splitting in wider SAEs hampers safety interventions, underscoring the importance of disentangled feature learning. Our findings highlight both the promise and the current limitations of SAE-based causal interventions for LLM detoxification, further suggesting practical guidelines for safer language-model deployment. 1 1 1 Code: https://github.com/CrowdDynamicsLab/SAE-Detoxification . ‡ ‡ footnotetext: Both authors contributed equally. 1 Introduction Large language models (LLMs) are increasingly being used in human-facing settings such as chatbots, academic tutors, mental-health assistants, content-moderation tools, and social simulations Dam et al. (2024) ; Furumai et al. (2024) ; Stade et al. (2024) ; Park et al. (2024) ; Zhan et al. (2025) ; Han et al. (2024c) ; Chuang et al. (2024a) . However, the diverse data that gives these models their impressive capabilities also exposes them to the toxicity and biases inherently present in human-generated content on which they are trained Sheng et al. (2019) ; Gehman et al. (2020) ; Jain et al. (2024) . Figure 1: SAE-based LLM Detoxification: We extract the activations from the residual stream of the model after the transformer block of Layer N . Using sparse autoencoders (SAEs), we decompose activations to identify toxic dimensions and perform targeted interventions before the steered activations enter Layer N+1 . Model developers incorporate various safeguards to prevent harmful outputs such as methods like supervised fine-tuning (SFT), preference tuning methods such as Proximal Policy Optimization (PPO) Schulman et al. (2017) and Direct Preference Optimization (DPO) Rafailov et al. (2023) , and machine unlearning (MU) methods Yao et al. (2024) ; Liu et al. (2025) . However, research has shown that these safety measures often lead to superficial shortcuts rather than actual modifications Lee et al. (2024) ; Łucki et al. (2024) , making them vulnerable to circumvention through relatively simple techniques like strategic prompting and fine-tuning Gehman et al. (2020) ; Deshpande et al. (2023) ; Luong et al. (2024) . Further, preference-tuning of models is prohibitively expensive and requires large-scale, high-quality preference-data which is difficult to collect in practice Strubell et al. (2019) ; Ziegler et al. (2019) ; Ouyang et al. (2022) . Finally, these techniques are uninterpretable, which is a key limitation that hinders development of a deeper understanding of how to prevent these behaviors in models and enhance alignment Anwar et al. (2024) . As a result, this fundamental tension between model capability and safety continues to challenge responsible deployment of LLMs across sociotechnical systems. Mechanistic Interpretability (MI) techniques allow for the identification of specific human-interpretable concepts and subsequent steering of model behavior, which holds great potential for enhancing model safety Sharkey et al. (2025) . A key assumption in this line of work is the Linear Representation Hypothesis which states that model representations encode human-interpretable concepts in linear subspaces Mikolov et al. (2013) ; Bolukbasi et al. (2016) ; Elhage et al. (2022) ; Park et al. (2023) ; Nanda et al. (2023) . Sparse Autoencoders (SAEs) are a tool that leverage this to decompose model activations into meaningful concepts, providing dual benefits of interpretability and the ability to perform targeted steering along the dimension of the chosen concept Templeton et al. (2024) ; O’Brien et al. (2024) ; Gao et al. (2024) ; Karvonen et al. (2024) . In practice, SAEs could be used during inference time as ‘suppression heads’ in order to mitigate harmful behavior. However, despite this potential their usefulness for safety applications such as detoxification remains unexplored. In this work, we make two key contributions : \mathbf{\medbullet} We present the first comprehensive evaluation of SAEs for detoxification of LLMs. In contrast to prior work that has primarily focused on utility of SAEs on abstract concepts Templeton et al. (2024) ; Wu et al. (2025) without rigorous assessment of their practical utility for safety applications, we provide an in-depth analysis of how effectively SAEs can mitigate toxic outputs in real-world scenarios. We accomplish this by identifying and steering using toxicity-related features within SAEs trained on the residual stream at different layers of language models. This contribution advances our understanding of interpretable safety mechanisms and provides concrete evidence for when and how SAEs can be effectively deployed in production systems. \mathbf{\medbullet} We introduce a three-tiered steering approach that enables precise granularity in applying causal interventions for detoxification of language models at the levels of input sequences and tokens. In contrast to prior detoxification work that has primarily focused on reducing toxicity without sufficient consideration for maintaining model fluency and general capabilities, our approaches prioritize both safety and functionality as essential requirements for deployed systems. We accomplish this through our feature ablation and steering experiments across multiple layers of models. This provides actionable insights for selecting appropriate detoxification strategies based on their specific requirements and downstream applications. The core motivation of our work is to provide the first comprehensive evaluation of the application of SAEs for detoxification of LLMs, and demonstrate a strong safety use case where SAEs performs well. Key Findings: Through an extensive study on GPT-2 Small and Gemma-2-2B, we find that while SAE-based steering significantly reduces toxicity compared to existing detoxification methods—especially at higher steering strengths—this improvement may come at the cost of reduced fluency, depending on the underlying model and SAE used. Model capability upon steering on the other hand is not hampered. We also show how feature splitting effects in larger SAEs can be detrimental to detoxification performance and explore ways to mitigate this effect using features in Gemma-2-2B. Overall, our work shows the promise of using SAE-based interpretable approach to LLM detoxification, while also highlighting key challenges that may arise in using these techniques and outlining promising directions for future research in actionable interpretability for AI safety. 2 Background and Related Work 2.1 Large Language Model Safety LLMs today fundamentally exist as sociotechnical systems deeply embedded within human social contexts Dhole (2023) ; Dam et al. (2024) ; Chuang et al. (2024b) ; Han et al. (2024c) ; Goyal et al. (2025) . This means that challenges surrounding safety of LLM deployment cannot be addressed through purely technical means and instead demand holistic approaches that recognize the complex interplay between technological capabilities and societal dynamics Sartori and