LLM Safety From Within: Detecting Harmful Content with Internal Representations
Difan Jiao, Yilun Liu, Ye Yuan, Zhenwei Tang, Linfeng Du, Haolun Wu, Ashton Anderson · 2026
Read LLM Safety From Within: Detecting Harmful Content with Internal Representations by Difan Jiao, Yilun Liu, Ye Yuan, Zhenwei Tang, Linfeng Du, Haolun Wu, Ashton Anderson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18519.txt.
Introduction
Jiao and six colleagues present SIREN, a lightweight guard model that uses safety features spread across a language model's internal layers. Top guard models rely solely on final-layer signals, missing those internals. SIREN finds safety neurons by linear probing and joins them through a tuned layer mix, building a harmfulness checker from model insides without changing the base model. It clearly beats open-source guard models across benchmarks while using 250 times fewer trainable weights, transfers better to unseen benchmarks, and enables streaming checks with higher run-time speed.
Catalog record
| CREATOR | Difan Jiao, Yilun Liu, Ye Yuan, Zhenwei Tang, Linfeng Du, Haolun Wu, Ashton Anderson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45995 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18519
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLM Safety From Within: Detecting Harmful Content with Internal Representations Difan Jiao, Yilun Liu, Ye Yuan, Zhenwei Tang, Linfeng Du, Haolun Wu, Ashton Anderson Abstract Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 × 250\times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection. Our code is available at https://github.com/CSSLab/SIREN . Content Warning: This paper discusses content safety using datasets containing harmful language. 1 Introduction Large language models (LLMs) are now deployed at scale ( 35 ; 3 ; 11 ) and face a persistent content safety challenge: users can submit harmful prompts, and models can generate harmful responses ( 55 ) . To mitigate the risks stemming from this, LLM guardrails have become essential, with safety-specialized guard models emerging as a mainstream solution ( 17 ; 15 ; 50 ) . These models, typically fine-tuned from open-source LLM backbones on both user prompts and model responses, perform harmfulness detection as a generative classification task by decoding from the terminal layer of the model ( 17 ; 15 ; 50 ) . However, this reliance on the terminal layer overlooks rich safety-relevant features encoded throughout the model. Recent work has revealed that LLM internal representations encode rich specialized features, and leveraging these representations offers substantial performance improvements in classification tasks ( 13 ; 20 ; 25 ) . Moreover, several studies demonstrate that the internal representations of LLM encode fine-grained concepts for content safety ( 52 ; 51 ; 21 ) . Yet these findings have not been systematically translated into practical safeguard models. This gap presents an opportunity: can we harness the LLM internal representations to build better content harmfulness detectors? Figure 1: Comparison of LLM safeguard approaches. (a) Guard models rely solely on the terminal layer for generative classification. (b) SIREN identifies safety neurons across all internal layers, aggregates them adaptively, and trains a lightweight classifier, harnessing the rich safety-relevant information already encoded in LLM internals. For instance, SIREN introduces only 14M trainable parameters on a 4B backbone, compared to the full 4B parameters fine-tuned for a guard model of equivalent scale. In this work, we leverage internal safety-relevant features via a two-stage framework named SIREN ( S afeguard with I nternal RE presentatio N ) as shown in Figure 1 . First, SIREN employs linear probing ( 2 ) to localize safety-relevant features within each layer, supported by the linear representation hypothesis which posits that semantic concepts are often linearly represented in LLMs ( 16 ; 36 ) . We term features exhibiting high salience for content safety classification as safety neurons of each layer. As empirical evidence shows that cross-layer integration of internal neurons yields substantial performance gains ( 46 ; 20 ) , in the second stage, we aggregate safety neurons across all layers to train a lightweight classifier for harmfulness detection. We employ a layer-weighted aggregation strategy, as prior work shows that LLMs exhibit hierarchical learning structures in which different layers encode features at different levels and contribute unequally to a given task ( 42 ; 39 ; 25 ) . Specifically, we compute layer weights based on the validation performance of layer-wise linear probes, then concatenate the weighted activations of safety neurons across all layers. Such a design requires no modifications to the underlying LLM, enabling SIREN to operate as a plug-and-play component. We systematically evaluate the performance of our framework against state-of-the-art open-source guard models across three dimensions: efficacy, generalizability, and efficiency. First, with 250 \times fewer parameters, SIREN trained on general LLMs substantially outperforms the counterpart guard models fine-tuned from the exact same backbone. Second, we show that SIREN generalizes to unseen benchmarks on reasoning traces and harmfulness detection in streaming mode , a setting not seen during SIREN’s training where models are required to classify content safety in real-time as text is generated token-by-token. Third, SIREN offers remarkable efficiency, as inference requires just a single forward pass compared to autoregressive generative classification in guard models. Our contributions are two-fold: • We propose SIREN, a plug-and-play guard model that harnesses LLM internal representations for harmfulness detection. • Through evaluation across multiple benchmarks, we demonstrate that SIREN surpasses existing safeguard models in performance, generalization, and efficiency. 2 Related work 2.1 LLM Safety Systems and Guardrails The large-scale deployment of LLMs necessitates high-performing safety mechanisms to mitigate harmful content generation. Current mainstream approaches to content safety detection can be broadly categorized into two paradigms: discriminative classifiers and generative guard models. Discriminative classifiers emerged primarily in the pre-LLM era. Representative safeguard solutions leverage encoder-only transformer models fine-tuned with specialized classification heads for toxicity and hate speech detection. In particular, early work adapted BERT ( 6 ) and RoBERTa ( 30 ) for these tasks ( 34 ; 53 ) . For instance, 5 introduced HateBERT, retrained on abusive content from Reddit to improve hate speech detection. Similarly, 53 applied toxicity-specific fine-tuning strategies to RoBERTa. More recently, ShieldHead ( 44 ) and HSF ( 37 ) train lightweight classifiers on last-layer hidden states of LLMs for decoding-time safety filtering and jailbreak detection, respectively. Generative guard models have emerged as the dominant paradigm with the rise of instruction-tuned LLMs, reformulating safety detection as a generative classification task. Llama Guard ( 17 ) pioneered this approach by fine-tuning Llama-2-7B and later Llama-3 series on a safety taxonomy to classify both prompts and responses. Recent advances include WildGuard ( 15 ) , which targets malicious intent and jailbreak detection, and Qwen3Guard ( 50 ) , currently representing the state-of-the-art with notable performance in both content safety classification and streaming harmfulness detection. Other prevalent specialized safeguard models, including ShieldGemma ( 48 ) , NemoGuard ( 9 ) , and PolyGuard ( 24 ) , also demonstrate significant capability in content safety classification while being fine-tuned from open-source general LLM backbones. Both conventional paradigms, however, share a common limitation: they primarily rely on terminal-layer representations, either through classification heads or the generative decoder’s final outputs, neglecting the rich safety-relevant features encoded across internal layers. Also, generative guards incur additional computational costs due to the autoregressive token generation during inference. 2.2 Leveraging LLM Internals for Content Safety Empirical evidence across diverse tasks demonstrates that intermediate layers of LLMs encode richer task-relevant features than terminal-layer representations or generative outputs alone. Studies have successfully leveraged internal activations for sentiment analysis ( 40 ; 20 ) , factual knowledge retrieval ( 16 ; 33 ) , and question answering ( 41 ; 13 ) . These findings motivate investigating whether similar advantages hold for content safety. A