The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams
Isaac Llorente-Saguer · 2026
Read The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams by Isaac Llorente-Saguer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27412.txt.
Introduction
Llorente-Saguer presents LatentBiopsy, a way to spot harmful prompts without training. The method reads hidden activations inside a language model. It first maps the main direction of two hundred safe prompts. Each new prompt is scored by its angle from that direction. Odd angles get flagged as suspect. Tests cover six small Qwen model variants. Scores reach at least 0.937 AUROC for harmful prompts. Harmful prompts cluster at a very tight angle. That pattern survives even when refusal is removed.
Catalog record
| CREATOR | Isaac Llorente-Saguer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43620 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27412
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams Isaac Llorente-Saguer Abstract We present LatentBiopsy, a training-free method for detecting harmful prompts by analysing the geometry of residual-stream activations in large language models. Given 200 safe normative prompts, LatentBiopsy computes the leading principal component of their activations at a target layer and characterises new prompts by their radial deviation angle θ \theta from this reference direction. The anomaly score is the negative log-likelihood of θ \theta under a Gaussian fit to the normative distribution, flagging deviations symmetrically regardless of orientation. No harmful examples are required for training. We evaluate two complete model triplets from the Qwen3.5-0.8B and Qwen2.5-0.5B families: base, instruction-tuned, and abliterated (refusal direction surgically removed via orthogonalisation). Across all six variants, LatentBiopsy achieves AUROC ≥ \geq 0.937 for harmful-vs-normative detection and AUROC = = 1.000 for discriminating harmful from benign-aggressive prompts (XSTest), with sub-millisecond per-query overhead. Three empirical findings emerge. First, geometry survives refusal ablation: both abliterated variants achieve AUROC at most 0.015 below their instruction-tuned counterparts, establishing a geometric dissociation between harmful-intent representation and the downstream generative refusal mechanism. Second, harmful prompts exhibit a near-degenerate angular distribution ( σ θ ≈ 0.03 \sigma_{\theta}\approx 0.03 rad), an order of magnitude tighter than the normative distribution ( σ θ ≈ 0.27 \sigma_{\theta}\approx 0.27 rad), preserved across all alignment stages including abliteration. Third, the two families exhibit opposite ring orientations at the same depth: harmful prompts occupy the outer ring in Qwen3.5-0.8B but the inner ring in Qwen2.5-0.5B, directly motivating the direction-agnostic scoring rule. 1 Introduction Detecting harmful prompts before model response generation is a prerequisite for safe deployment of large language models. Existing methods divide into two broad families. Input-space filters based on perplexity ( 5 ; 1 ) are effective against adversarial suffix attacks but fail on semantically fluent jailbreaks. Supervised safety classifiers such as Llama Guard ( 4 ) achieve strong performance but require large curated datasets of harmful and harmless pairs, and per-model retraining whenever the target model changes. A parallel body of work in representation engineering has established that residual streams encode semantic content as geometric structure ( 13 ) . Linear directions for safety concepts have been extracted from contrastive safe/harmful pairs ( 13 ; 6 ) , and PCA of hidden states has been shown to separate harmful from harmless queries when safety-prompt manipulations define the reference ( 12 ) . Crucially, 2 demonstrated that refusal behaviour in aligned models is mediated by a single linear direction in the residual stream, and that orthogonalising this direction renders the model unable to refuse. These methods share a common dependency: they require harmful examples or deliberate safety-prompt manipulations to define the reference geometry. We ask whether the safe distribution alone defines a reference geometry from which harmful prompts deviate detectably. We answer affirmatively, and report a finding that sharpens the safety implications: LatentBiopsy detects harmful intent even in abliterated models that are constitutionally incapable of producing refusals. The model’s internal encoding of harmful semantic intent is geometrically distinct from the circuits that generate refusal text. Harm recognition and refusal generation are separable mechanisms. The practical consequence is direct: stripping a model’s refusal behaviour does not erase the latent signal available to an external detector. A secondary finding concerns the structure of the deviation itself. The two tested model families exhibit opposite ring orientations at the same residual-stream depth; harmful prompts are more angular from PC1 in Qwen3.5-0.8B and more aligned with PC1 in Qwen2.5-0.5B. This family-level reversal, together with known layer-level reversal within each model, establishes that a fixed directional threshold on \theta is architecturally unreliable and motivates the symmetric anomaly score. Contributions. 1. Training-free angular anomaly detector. LatentBiopsy builds a normative reference exclusively from 200 safe activations and scores every prompt by the negative log-likelihood of its angular deviation \theta from the leading normative principal component. The score is symmetric around the normative mean, requiring no knowledge of the ring direction and no harmful data. 2. Geometry survives refusal ablation in both tested model families. Abliterated variants of both Qwen3.5-0.8B and Qwen2.5-0.5B achieve AUROC h/b =1.000 and AUROC h/n within 0.015 of their instruction-tuned counterparts, establishing a geometric dissociation between harmful-intent representation and the refusal mechanism across two independent model families. 3. Opposite ring orientations across families. At layer 20, Qwen3.5-0.8B places harmful prompts at mean \theta\approx 1.80 rad vs. normative \mu_{0}\approx 1.17 rad (outer ring), while Qwen2.5-0.5B places harmful prompts at mean \theta\approx 1.34 rad vs. normative \mu_{0}\approx 1.82 rad (inner ring). The anomaly score s(x) correctly identifies both configurations without architectural knowledge. 4. Near-degenerate harmful compactness and K{=}1 sufficiency. Harmful prompts occupy a near-degenerate angular cluster ( \sigma_{\theta}^{\mathrm{harm}}\approx 0.03 – 0.05 rad, one order of magnitude tighter than \sigma_{\theta}^{\mathrm{norm}} ), and a single reference direction ( K{=}1 ) dominates multi-directional baselines at every layer in every model. 2 Related Work Input-space defences. Perplexity-based filters ( 5 ; 1 ) are computationally cheap and effective against adversarial suffix attacks, but rely on surface-form anomaly and cannot detect semantically fluent jailbreaks that score normally under any language model. Supervised safety classifiers. Llama Guard ( 4 ) and its successors fine-tune a language model on large labelled datasets of safe and harmful pairs, achieving strong per-category precision. The approach requires curated harmful data and per-model retraining, making it expensive to adapt to new architectures or harm taxonomies. Representation engineering and linear probes. 13 extract linear control directions from contrastive safe/harmful pairs and demonstrate that semantic intent is encoded as rich geometric structure in the residual stream. 6 fit linear probes on labelled activation data and intervene at inference time. 12 use optimised safety-prompt manipulations to visualise hidden-state separation and shift representations along a refusal direction. All three approaches require either labelled harmful examples or deliberate safety-prompt manipulations to define the reference; LatentBiopsy removes that dependency entirely. Refusal as a single linear direction. 2 show that refusal behaviour in aligned models is mediated by a single linear direction in the residual stream. LatentBiopsy provides independent quantitative support for the near-one-dimensional geometry ( K{=}1 dominance) while extending the picture: the safety representation that LatentBiopsy exploits is not the refusal direction itself, since it survives abliteration. Angular biomarkers. The theta biomarker concept was introduced for medical diagnostics to flag anomalous patient profiles via angular deviation in a clinical feature space ( 7 ) . LatentBiopsy translates this geometric principle to LLM residual streams, adapting it to the direction-agnostic scoring requirement imposed by layer- and family-dependent ring orientation. 3 Preliminaries: Geometry of the Residual Stream Modern causal language models process tokens through a sequence of transformer blocks communicating via a central residual stream. Let f_{\ell}(x)\in\mathbb{R}^{D} denote the activation vector of the residual stream at layer \ell for the final token of an input prompt x . The Linear Representation Hypothesis ( 8 ) posits that neural networks encode high-level concepts as spatial directions within this D -dimensional space. Under this view, semantic identity is defined by orientation, while concept intensity correlates with magnitude along that direction. Euclidean distance conflates semantic identity with concept intensity. Furthermore, in standard Pre-LayerNorm transformer architectures, the residual stream accumulates un-normalised outputs from consecutive layers, causing the \ell_{2} norm