SafeDream: Safety World Model for Proactive Early Jailbreak Detection
Bo Yan, Weikai Lin, Yada Zhu, Song Wang · 2026
Read SafeDream: Safety World Model for Proactive Early Jailbreak Detection by Bo Yan, Weikai Lin, Yada Zhu, Song Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16824.txt.
Introduction
Yan and colleagues propose SafeDream, a lightweight world-model framework for proactive early jailbreak detection. Multi-turn jailbreaks erode safety alignment across innocuous turns, succeeding over 90 percent of the time. Existing methods need costly weight changes, judge turns independently, and detect attacks only after harmful content appears. SafeDream encodes hidden states into a compact safety representation, accumulates weak risk signals, and rolls out attack and benign futures for early alarms. Across three benchmarks against 8 baselines, it detects attacks just over one turn before compliance with competitive false positive rates.
Catalog record
| CREATOR | Bo Yan, Weikai Lin, Yada Zhu, Song Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51836 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16824
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SafeDream: Safety World Model for Proactive Early Jailbreak Detection Bo Yan, Weikai Lin, Yada Zhu, Song Wang Abstract Multi-turn jailbreak attacks progressively erode LLM safety alignment across seemingly innocuous conversation turns, achieving success rates exceeding 90% against state-of-the-art models. Existing alignment-based and guardrail methods suffer from three key limitations: they require costly weight modification, evaluate each turn independently without modeling cumulative safety erosion, and detect attacks only after harmful content has been generated. To address these limitations, we first formulate the proactive early jailbreak detection problem with a new metric, detection lead , that measures how early an attack can be detected before the LLM complies. We then propose SafeDream , a lightweight world-model-based framework that operates as an external module without modifying the LLM’s weights. SafeDream introduces three components: (1) a safety state world model that encodes LLM hidden states into a compact safety representation and predicts how it evolves across turns, (2) CUSUM detection that accumulates weak per-turn risk signals into reliable evidence, and (3) contrastive imagination that simultaneously rolls out attack and benign futures in latent space to issue early alarms before jailbreaks occur. On three multi-turn jailbreak benchmarks (XGuard-Train, SafeDialBench, SafeMTData) against 8 baselines, SafeDream achieves the best detection timeliness across all benchmarks (1.06–1.20 turns before compliance) while maintaining competitive false positive rates and outperforming baselines in detection quality. Fig. 1: SafeDream achieves the best Lead–FPR trade-off and outperforms all baselines across benchmarks. (a) Lead vs. FPR on XGuard: Lead measures how many turns earlier a jailbreak is detected, and FPR is the false alarm rate. SafeDream (blue star) achieves the best trade-off between Lead and FPR. (b,c) Radar charts on XGuard and SafeMTData comparing SafeDream against baselines across five metrics (axes individually normalized for readability). SafeDream dominates all baselines on all metrics across both benchmarks. 1 Introduction Large language models (LLMs) ( 27 ) can be exploited for harmful purposes such as generating misinformation or illegal instructions. Despite safety mechanisms like alignment training and guardrails ( 7 ; 5 ) , multi-turn jailbreak attacks remain highly effective ( 15 ) : by spreading malicious intent across seemingly innocuous turns, they gradually erode LLM safety barriers until the model complies. Crescendo ( 21 ) achieves a 98% binary success rate against GPT-4, FITD ( 25 ) averages 94% Attack Success Rate (ASR) across seven LLMs, and STAR ( 12 ) reaches 94.5% Safety Failure Rate (SFR) against GPT-4o. Recent work has explored various approaches to counter jailbreak attempts. Alignment-based methods such as Circuit Breaker ( 30 ) , X-Boundary ( 14 ) , and TARS ( 10 ) modify LLM weights to resist attacks. Guardrail methods such as WildGuard ( 5 ) and GuardReasoner ( 13 ) deploy external classifiers to evaluate each turn independently. However, these approaches suffer from three key limitations. First, alignment-based methods require access to and modification of model weights, incurring significant computational overhead and precluding deployment on closed-source models. Second, both paradigms typically evaluate each turn independently, making them unable to defend against multi-turn attacks where the LLM complies only after a gradual safety erosion accumulated across multiple turns. Third, neither paradigm formulates the problem of anticipating jailbreaks before they succeed; by the time these methods detect an attack signal, the LLM may have already generated harmful content. To address these three limitations, we reformulate the multi-turn jailbreak detection problem (Section 2 ). Unlike previous formulations, we explicitly incorporate detection lead time , measuring how many turns earlier a jailbreak attempt can be detected before the LLM actually complies. We then propose SafeDream , a world-model-based safety module that enables proactive, early jailbreak detection in multi-turn dialogues. SafeDream addresses the three limitations above: it operates as a lightweight external module without modifying the LLM’s weights ( solving limitation 1 ); it models safety state dynamics across turns rather than evaluating each turn independently ( solving limitation 2 ); and it proactively detects attacks before the LLM complies through contrastive imagination ( solving limitation 3 ). SafeDream consists of three key components. First, we introduce a Safety State World Model : a lightweight Transformer that predicts future safety states from the safety state history and user actions, operating entirely in a low-dimensional safety state space (Section 3.1 ). Second, we employ Cumulative Sum (CUSUM) detection to aggregate risk signals across multiple turns, demonstrating how multi-turn safety erosion can be effectively captured through sequential evidence accumulation (Section 3.2 ). Third, in ambiguous situations, in order to detect jailbreaks as early as possible, we use our Safety State World Model to conduct contrastive imagination , which rolls out possible attack and benign futures and enables early alarms before a jailbreak actually occurs (Section 3.3 ). We evaluate SafeDream on three multi-turn jailbreak benchmarks (XGuard-Train ( 18 ) , SafeDialBench ( 2 ) , SafeMTData ( 20 ) ) against 8 baselines. Our key contributions are: • We formulate the proactive early jailbreak detection problem with a new metric, detection lead , that measures how early attacks can be detected before the LLM complies. • We propose SafeDream , a world-model-based framework consisting of a safety state world model, CUSUM detection, and contrastive imagination for proactive early jailbreak detection. • SafeDream achieves the best detection lead across all benchmarks (1.06–1.20 turns before compliance) and the highest detection quality with only an external module, even outperforming baselines that modify LLM weights. • Ablation studies confirm the effectiveness of our design: contrastive imagination is the primary driver of early detection and CUSUM is the key for false alarm suppression. Fig. 2: SafeDream overview. At each turn t , the frozen LLM provides hidden-state observations h_{t} , which are projected into a safety-grounded latent space s_{t} via the concept cone. The safety state z_{t} is obtained by enriching s_{t} through cross-attention. A lightweight Transformer learns transition dynamics by predicting z_{t+1} conditioned on user actions. A discriminator maps each z_{t} to a risk score r_{t} . CUSUM accumulates risk scores across turns to detect safety state shifts; when evidence is ambiguous (gray zone), contrastive imagination rolls out both attack and benign futures to quantify vulnerability and enable early detection. 2 Problem Formulation Multi-turn dialogue and safety model. We study jailbreak detection given an ongoing dialogue between a user and an LLM, the goal is to decide whether the user is steering the conversation toward harmful content before the LLM complies. At turn t , the user sends utterance u_{t} and the LLM generates response y_{t} . We define the observation o_{t}=(u_{1},y_{1},\ldots,y_{t-1},u_{t}) as the conversation prefix immediately before y_{t} is generated. A safety model f(o_{t})\in\{0,1\} infers whether jailbreak intent is present; if f(o_{t})=1 , the LLM refuses to respond. Attack model and compliance point. In a multi-turn jailbreak, an adversarial user progressively guides the LLM toward harmful content through seemingly innocuous messages, gradually eroding the LLM’s safety alignment until the model fully complies. We call this critical point the compliance point c , formally defined as the first turn at which the majority of independent safety classifiers flag the LLM’s response as harmful (see Section 3.4 for the labeling procedure). Detecting the attack after the compliance point ( t>c ) is too late, as harmful content has been generated. Proactive