What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics

Sofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal(✉) · 2026

Read What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics by Sofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal(✉) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25182.txt.

Introduction

Jailbreak prompts coax aligned language models into answers they were trained to refuse. Sofiia Nikolenko and three colleagues look for the fingerprint inside the model. They track how token level predictive entropy, the model's uncertainty, evolves across layers of a frozen model read through the logit lens.

Simple summaries of prompt entropy, its mean or variance, carry almost no signal about an attack. Features that capture how entropy moves across token positions, such as rank based trend scores, separate attacks far better. The signal is not spread evenly through the network. It concentrates in the intermediate layers and fades at the final layer, so mid network representations show jailbreak structure most clearly. The pattern holds across Llama, Qwen, and Gemma models and several adversarial benchmarks, with no extra training.

Catalog record

CREATORSofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal(✉)
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25182

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics Sofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal(✉) Abstract Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model’s internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced. 1 Introduction A jailbreak in Large Language Models (LLMs) is a prompting strategy that attempts to circumvent built-in safety mechanisms by manipulating instructions, role-play, or embedding hidden malicious intent. Understanding and mitigating jailbreak behavior is critical, as successful attacks can lead to unsafe outputs, policy violations, data leakage, or the misuse of connected tools. Reliable detection of jailbreak attempts is therefore essential for the safe deployment of LLMs. Existing jailbreak-detection approaches include rule-based filters for known attack patterns [ 7 ] , supervised classifiers that categorize prompts as benign or adversarial [ 1 , 44 ] , anomaly-detection methods that flag unusual inputs [ 14 ] , response-based detectors that inspect model outputs [ 21 ] , and internal-signal methods [ 39 , 42 , 11 , 20 , 40 ] that identify attacks from hidden model representations. While recent internal-signal approaches suggest that adversarial and benign prompts may be separable in latent space, it remains unclear which properties of internal dynamics encode harmful intent and how this signal evolves across token positions and model depth. Figure 1: Token-level predictive entropy trajectories at a representative intermediate layer (L22) of Llama-3.1-8B for safe and adversarial (jailbreak) prompts. While aggregate entropy levels are comparable, the jailbreak prompt exhibits a pronounced monotonic trend in entropy evolution across token positions, motivating the use of trajectory-based features. In this work, we build on the latter perspective and conduct a training-free analysis of internal model dynamics rather than relying solely on input text, generated outputs, or fine-tuned classifiers. Specifically, we analyze intermediate-layer, token-level predictive entropy trajectories to study how uncertainty evolves across token positions and model depth. Our central hypothesis is that jailbreak prompts do not merely alter overall uncertainty levels, but induce structured dynamics in how predictive entropy changes as tokens unfold . Static prompt-level summary statistics (e.g., mean or variance) may obscure such localized effects, whereas trajectory-based features can capture systematic monotonic trends across positions. Figure 1 illustrates this phenomenon on a representative intermediate layer of Llama-3.1-8B. While prompt-level entropy levels appear broadly similar for safe and adversarial inputs, their token-level trajectories differ systematically: jailbreak prompts exhibit structured monotonic trends in entropy evolution, as quantified by Kendall’s \tau (a rank-based measure of directional consistency), whereas benign prompts show comparatively flatter patterns. This token-level view is important because jailbreak behavior often arises from localized interactions between particular tokens and the model’s internal state, which can be obscured by prompt-level averaging [ 20 ] . These interactions may produce progressive shifts in confidence as the prompt develops, rather than uniformly high or low uncertainty. By treating entropy as a dynamic quantity and examining its evolution across depth, we aim to identify where in the network jailbreak-relevant structure becomes most pronounced. Our empirical findings (Section 5 ) support this hypothesis. Dynamic features that quantify monotonic entropy trends consistently separate jailbreak from benign prompts, while static aggregate measures are comparatively weak. Moreover, this signal is not uniformly distributed across layers: it is concentrated at intermediate depth and degrades at the final layer, suggesting that jailbreak-relevant structure emerges in mid-network semantic representations before being partially transformed by output-specific processing. Our contributions are threefold: 1. We introduce a training-free analysis for studying jailbreak-related signal in internal model dynamics , using intermediate-layer token-level predictive entropy trajectories rather than prompt text, generated outputs, or fine-tuned classifiers. 2. We show that the useful discriminative signal is primarily dynamic rather than static : prompt-level aggregate entropy statistics are comparatively weak, while features that capture positional entropy evolution, such as monotonicity and rank-based trends, provide stronger and more consistent separation between jailbreak and benign prompts. 3. We demonstrate that jailbreak prompts induce structured, depth-dependent patterns in intermediate representations , with the discriminative signal concentrated at intermediate layers and degrading at the final layer. Experiments on Llama [ 15 ] , Qwen [ 3 ] , and Gemma [ 36 ] across AdvBench, HarmBench, and StrongREJECT show consistent separation without additional training, providing a diagnostic characterization of this signal rather than a finished detector Paper structure. Section 2 situates our approach within prior jailbreak detection and internal-signal analyses. We then introduce our proposed framework in Section 3 , beginning with a formal problem formulation (Section 3.1 ), followed by intermediate-layer entropy extraction via the logit lens (Section 3.2 ), static and dynamic feature construction (Section 3.3 ), and the evaluation protocol (Section 3.4 ). Section 4 describes the experimental setup, including evaluated models, datasets, and benchmark configurations, and Section 5 presents the empirical findings, covering static versus dynamic comparisons, depth ablations, cross-model generalization, and adversarial control analyses. We conclude in Section 6 . Additional layer-wise metrics, extended ablations, an analysis of why JailbreakBench-benign prompts reduce separability (including prompt-level diagnostics and an intermediate-layer distribution comparison), full layer-wise directional AUROC tables for Llama [ 15 ] , Qwen [ 3 ] , and Gemma [ 36 ] , and implementation details are optionally provided in the Supplementary Material. 2 Related Work Jailbreak defense in LLMs. Prior studies on jailbreak defense have focused on external prompt-level filters [ 43 ] , supervised classifiers over user inputs, anomaly-detection methods [ 14 , 16 ] , and response-based detectors that inspect generated outputs [ 21 ] . These methods differ in the evidence they use –prompt text, model outputs, or external classifiers– but most rely on engineered prompt features or generated responses. Perplexity-based detection [ 2 , 17 ] is a training-free baseline, but can produce a high false positive rate on benign inputs. Perturbation-based defenses, such as SmoothLLM [ 33 ] , are robust but require multiple forward passes per prompt. More recent output-distribution methods detect jailbreaks from the first generated tokens in a single pass [ 10 , 6 ] . In contrast, our approach is training-free and operates directly on internal activations of a frozen model, using a single forward pass to derive a continuous detection score from intermediate uncertainty dynamics. Internal representations and safety-related signals. Recent studies emphasize that aligned language models

Read the complete text