Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes
Mohammed Alshaalan, Miguel R. D. Rodrigues · 2026
Read Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes by Mohammed Alshaalan, Miguel R. D. Rodrigues (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19966.txt.
Introduction
Mohammed Alshaalan and Miguel Rodrigues treat fluent jailbreak suffixes as a change in a language model's token-by-token surprise stream. Their training-free detector standardises each token's entropy against a baseline from the system prompt and applies a one-sided CUSUM test to spot the onset. Across six open-weight chat models it beats windowed perplexity baselines, and as a gate for Llama Guard it cuts guard calls sharply while keeping detection quality.
Catalog record
| CREATOR | Mohammed Alshaalan, Miguel R. D. Rodrigues |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78263 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19966
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes Mohammed Alshaalan, Miguel R. D. Rodrigues Abstract Optimization-based adversarial suffixes can jailbreak aligned large language models (LLMs) while remaining fluent, weakening static and windowed perplexity-based detectors. We cast adversarial suffix detection as an online change-point detection problem over the token-level next-token entropy stream. Using the LLM system prompt to estimate a robust baseline, we standardize user-token entropies and apply a one-sided CUSUM statistic. The resulting detector, CPD Online (CPD), is model-agnostic, training-free, runs online, and localizes the adversarial suffix onset. On a benchmark of 1,012 1{,}012 optimization-based suffix attacks (GCG, AutoDAN, AdvPrompter, BEAST, AutoDAN-HGA) and 1,012 1{,}012 perplexity-controlled benign prompts, CPD improves F1 over the strongest windowed-perplexity baseline on all six open-weight chat models (LLaMA-2-7B/13B, Vicuna-7B/13B, Qwen2.5-7B/14B). On LLaMA-2-7B at the canonical CUSUM setting ( k = 0 k{=}0 ), CPD reaches AUROC 0.88 0.88 and F1 0.82 0.82 . Beyond prompt-level detection, CPD concentrates 79.6 % 79.6\% of its triggers inside the adversarial suffix, versus 17 17 – 46 % 46\% for windowed perplexity. Finally, when used as a lightweight gate for LLaMA Guard, CPD reduces guard calls by 17 17 – 22 % 22\% on a high-volume, benign-dominated deployment while preserving guard-level detection quality. 1 Introduction Large language models (LLMs) are deployed in conversational assistants, code generators, and increasingly as decision-making components in multi-agent systems. Despite extensive alignment and safety training, they remain susceptible to jailbreak prompts and adversarial suffixes that coerce them into violating safety policies while appearing benign to a casual observer. Modern attack algorithms can optimize discrete token sequences to transfer across models and evade naive defenses, raising the stakes for robust runtime monitoring ( 26 ; 11 ) . Optimization-based adversarial suffixes are particularly challenging compared to human-crafted jailbreak prompts, as they can be generated automatically at scale and explicitly adapt to a model’s behavior. The base task prompt may be a benign factual question or an explicitly malicious request, e.g., for step-by-step weapon construction, that an aligned model would normally refuse. Jailbreaks can also be produced by rewriting the user request itself using persuasive strategies rather than by appending an optimized suffix ( 27 ) . New automated attacks generate the suffix by discrete token optimization such as Greedy Coordinate Gradient (GCG) ( 31 ) . More recent pipelines such as AutoDAN, AdvPrompter, BEAST, and AutoDAN-HGA extend this idea, often explicitly targeting low perplexity and naturalness while still inducing harmful behaviour ( 29 ; 20 ; 23 ; 15 ) . Such attacks can keep global perplexity within the range of benign prompts and distribute their effect over many tokens, making them difficult to detect with simple scalar heuristics. Existing defenses fall into two broad families. Statistical detectors compute a scalar anomaly score on the input text, most often global perplexity (PP) or windowed perplexity (WPP). These methods are simple and cheap, but they assume that adversarial suffixes are statistically unlikely and visibly disfluent; as soon as attacks prioritize fluency, their separability degrades ( 10 ; 2 ) . Safety classifiers deploy an auxiliary model, often a finetuned LLM such as LLaMA Guard, to judge whether a prompt or response violates safety policies ( 9 ; 17 ) . Guard models achieve strong benchmark scores but introduce substantial latency, memory, and serving complexity, and can themselves be directly targeted by optimization-based attacks. In addition, these approaches typically operate at the prompt level and do not attempt to localize where an adversarial suffix begins within the input. In this work we take a sequential view of the problem. As an LLM processes a prompt one token at a time, it emits a sequence of next-token distributions. We observe that user inputs, whether normal or malicious requests, exhibit an entropy distribution similar to that of the LLM system prompt, whereas optimization-based adversarial suffixes often induce a persistent shift in the token-entropy stream, even when global PP remains inconspicuous. This suggests treating the entropy sequence as a one-dimensional time series and casting adversarial suffix detection as an online change-point detection problem. Concretely, we use the fixed system prompt to define a deployment-specific entropy baseline and then apply a robust CUSUM-style detector to standardized user token entropies. The detector runs online over the user segment, raising an alarm when the cumulative deviation from the baseline exceeds a threshold. This yields both a prompt-level decision and a token-level estimate of where the suffix starts. The procedure is largely model-agnostic in that it requires only access to token-level next-token probabilities (or logits), it requires no additional training or auxiliary networks, and adds only a small amount of arithmetic on top of the existing forward pass. We refer to this entropy-CUSUM detector as CPD Online , and use CPD as shorthand in prose. Contributions. Our main contributions are: 1. We formulate adversarial suffix detection in LLMs as an online change-point detection problem on token-level entropy streams, and we show empirically that CPD achieves higher prompt-level F1 than the strongest fair WPP baseline on all six base LLMs studied here. 2. We instantiate CPD as a one-sided Page-CUSUM detector on standardized token entropies, using median and median absolute deviation (MAD) statistics from the fixed system prompt as the baseline, with operating thresholds selected by simple sweeps over prompt-level F1 and false-positive rates. The method is lightweight, training-free, and uses only quantities already available in a standard forward pass. 3. On a perplexity-matched benchmark of 1{,}012 adversarial and 1{,}012 benign prompts per model (TyDiQA+OpenOrca), we show CPD Online consistently improves prompt-level F1 over PP and WPP across all six base LLMs and provides token-level localization of the suffix onset. For example, on LLaMA-2-7B at the canonical k{=}0 setting CPD reaches AUROC 0.88 and F1 0.82 (versus PP AUROC 0.46 and best WPP F1 0.74 ), and concentrates 79.6\% of its triggers inside the suffix versus 17 – 46\% for WPP baselines. 4. We study CPD Online’s interaction with LLaMA Guard as a prompt-level safety classifier and show how CPD Online can act as a gate to focus guard calls on prompts whose entropy dynamics are most suspicious, trading off detection performance against compute. To our knowledge, this is the first work to apply online change-point detection within a single request (one prompt’s system+user token stream), at the token level, to detect and localize adversarial suffixes in deployed LLMs. We also release the code here: https://github.com/cpdonline/cpdonline . Conflict of Interest Disclosure. The authors are affiliated with University College London and declare no financial conflicts of interest. The models evaluated in this work are open-weight third-party releases; the authors have no commercial or advisory relationship with any of the developing organizations. 2 Method We formalize an online, training-free detector that monitors the token-level next-token entropy induced by a user prompt and raises an alarm when the entropy dynamics exhibit a sustained deviation from a deployment-specific baseline estimated from a fixed system prompt. 2.1 Problem Setup and Notation Let the LLM vocabulary be \mathcal{V} . Each request consists of a fixed system prompt and a user message: \mathbf{x}^{\text{sys}}=(x^{\text{sys}}_{1},\dots,x^{\text{sys}}_{m}),\qquad\mathbf{x}^{\text{usr}}=(x^{\text{usr}}_{1},\dots,x^{\text{usr}}_{T}), and the full input is their concatenation \mathbf{x}=(\mathbf{x}^{\text{sys}}\|\mathbf{x}^{\text{usr}}) . We consider two classes of user messages: • Benign: \mathbf{x}^{\text{usr}} contains only a base task prompt. • Adversarial: \mathbf{x}^{\text{usr}}