Ellipsoid Control: A White-list Jailbreak Defense via Benign Latent Modeling
Luoyu Chen, Weiqi Wang, Zhiyi Tian, Feng Wu, Ahmed Asiri, Shui Yu · 2026
Read Ellipsoid Control: A White-list Jailbreak Defense via Benign Latent Modeling by Luoyu Chen, Weiqi Wang, Zhiyi Tian, Feng Wu, Ahmed Asiri, Shui Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24552.txt.
Introduction
Luoyu Chen and colleagues defend language models against jailbreaks from the opposite direction: instead of learning from harmful examples, they protect benign behavior. Their test-time Ellipsoid Control nudges activations toward refusal with projected gradient descent, while an ellipsoid fitted to benign data limits distortion of benign geometry. Across several models, attacks, and benign tasks, the defense strengthened safety while preserving utility better than blacklist-style methods. The 2026 white-list approach thus avoids dependence on incomplete, shifting harmful samples.
Catalog record
| CREATOR | Luoyu Chen, Weiqi Wang, Zhiyi Tian, Feng Wu, Ahmed Asiri, Shui Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66690 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24552
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Ellipsoid Control: A White-list Jailbreak Defense via Benign Latent Modeling Luoyu Chen, Weiqi Wang, Zhiyi Tian, Feng Wu, Ahmed Asiri, Shui Yu Abstract Representation engineering (RepE) defenses have shown strong robustness against jailbreak attacks on large language models (LLMs). However, these methods fundamentally rely on black-list supervision : they learn jailbreak-to-refusal activation transformations from harmful or jailbreak data that are inherently incomplete and continuously evolving. Hence, the performance of RepE-based defenses becomes tightly coupled to the quality and coverage of collected harmful samples, leaving models vulnerable to unseen attacks. This reliance also obscures the distinction between defenses that fit known harmful distributions and defenses that protect a benign latent region without estimating the harmful distribution. We adopt the opposite, the white-list perspective, by leveraging the accessibility and abundance of benign data. The goal is to elicit refusal on arbitrary inputs while ensuring that harmless inputs are not falsely rejected. This shifts the core research question to: How can we design a robust benign-latent preservation mechanism such that the benign latent distribution remains intact while refusal is elicited? To answer this, we propose Ellipsoid Control, a test-time defense. It performs projected gradient descent that can elicit refusal on arbitrary inputs, aiming to improve defense effectiveness. At the same time, an anisotropic benign-geometry ellipsoid is fitted from abundant benign data to constrain the update to minimize distortion of the benign latent geometry. This tight constraint helps preserve model utility. Across multiple LLMs, jailbreak attacks, benign tasks, and safety-boundary evaluations, Ellipsoid Control consistently enhances safety while better preserving utility, demonstrating the effectiveness of the white-list approach for jailbreak defense. I Introduction Large language models (LLMs) have rapidly migrated from research prototypes to decision-critical infrastructure, powering code assistants, knowledge bases, and multimodal agents [ 29 , 19 , 22 ] . As these systems are increasingly deployed in sensitive environments, their trustworthiness depends on a broad set of privacy and security mechanisms. For example, private transformer inference protects user prompts and model computations during outsourced inference [ 8 ] ; privacy-enhanced federated learning protects distributed industrial training data [ 9 ] ; and scalable zero-knowledge proof systems support verifiable computation for machine-learning functions [ 7 ] . These system-level mechanisms are essential for data confidentiality, distributed privacy, and computation integrity. However, a new inference-time threat has emerged: jailbreak , where an aligned model can be manipulated into producing unsafe content. An expanding arsenal of jailbreak techniques—carefully crafted prompts that suppress a model’s built-in refusal behavior and coerce the model into disclosing step-by-step bomb recipes, medical malpractice advice, or hateful propaganda—threatens both mechanistic reliability and public trust in AI systems [ 31 ] . To defend against constantly evolving jailbreaks, the LLM safety community has found representation engineering (RepE) to be a powerful tool for robust defense: it manipulates hidden activations so that harmful inputs are steered toward meaningless or refusal states in latent space while benign inputs remain intact [ 35 , 27 , 1 ] . However, existing RepE-based defenses face two critical limitations. The first and most fundamental limitation ( L1 ) is that they operate under a black-list supervision paradigm: jailbreak-to-refusal activation transformations are learned from harmful or jailbreak data [ 27 , 35 , 25 ] . Yet the harmful latent space is inherently unbounded, and jailbreak strategies continuously evolve and may have previously unseen latent distributions. Consequently, any transformation learned from finite harmful samples cannot fully protect the full latent space due to the unbounded nature of harmful representations, and cannot reliably generalize to unseen attacks that exist in very different latent regions. Concretely, RepE defenses—including Circuit Breaker (CB) [ 35 ] , latent adversarial training (LAT) [ 27 ] , and refusal-direction steering defenses such as CAST and Jailbreak-Antidote [ 16 , 25 ] —all rely on paired harmful and benign datasets, and they are highly vulnerable to white-box embedding-space attacks that route prompts through the under-protected latent space to bypass learned transformations and restore harmful behaviors [ 24 ] . In other words, their novelty and robustness are limited by a black-list fitting objective: the learned transformation is only as complete as the harmful samples used to estimate it. The nature of unbounded harmful latent space and generalization gap motivates a shift toward a white-list approach: using easily accessible, diverse, and abundant benign data to build a white-list defense, by eliciting refusal for arbitrary inputs while minimally affecting benign latent geometry. However, adopting a white-list approach builds upon robust benign preservation, which exposes the second limitation ( L2 ): existing RepE defenses preserve benign utilities only point-wise , failing to capture the global structure of the benign latent distribution and causing overfitting to the seen benign samples. As a result, they often exhibit exaggerated safety behaviors. For example, Circuit Breaker fine-tuned LLMs have very high over-refusal rates (around 50%) on safety boundary datasets such as ORBench [ 4 ] , while the original models typically have less than 10% over-refusal rates. Thus, these observations highlight the need for a defense paradigm that both avoids reliance on harmful data and preserves the entire benign latent geometry rather than at a point-wise level. This naturally leads to our core research question: How can we design a robust benign latent preservation mechanism such that the benign latent distribution remains intact while refusal is elicited on arbitrary inputs? To answer this, we propose Ellipsoid Control , an inference-time defense that inversely elicits the refusal response ( gradient descent step ) for a suspected input under benign constraint ( projection step ). Ellipsoid Control eliminates the reliance on harmful data coverage and preserves the geometry of benign representations comprehensively. More specifically, to resolve L1 , instead of learning the jailbreak-to-refusal transformation on finite harmful data, we use projected gradient descent to route the representation of any inspected input to yield refusal responses under benign constraint. This makes EC different from standard PGD robustness methods and harmful-to-refusal representation steering: the PGD objective is not used to train a universal attack-specific transformation, but is solved per inspected input under a benign-data-derived feasible set. To resolve L2 , we use a high-dimensional anisotropic ellipsoid to fit the full benign data for the comprehensive geometric structure, with each axis in the latent space representing a unique semantic direction; during the projection step, we can efficiently calculate a closed form to constrain the update to minimize distortion of the benign latent geometry. Formally, if the benign hidden states admit the spectral decomposition H=U\Sigma V^{\top} , EC constrains the drift on every benign semantic axis by \sigma_{k}\|\Delta u_{k}\|\leq\epsilon , thereby protecting high-energy benign directions more tightly than weakly populated directions. In general, EC keeps the overall benign geometry intact while finding a descent path for the jailbreak input only. We conducted extensive experiments for evaluating the safety and utility goal of Ellipsoid Control . For jailbreak defense effectiveness, we first test it on Harmbench [ 18 ] with some classic jailbreak attack methods. To measure the robustness under white-box embedding space attack, we also tested it with SoftPrompt [ 24 ] that successfully breaks alignment on numerous RepE-based defense methods. For benign utility preservation, we first test it on MT-Bench and MMLU for benign task performance as how