Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerations

Wong Ryan, Hosea David Ng Yu Fei, Dhananjai Sharma, Kavishvaran Srinivasan · 2025

Read Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerations by Wong Ryan, Hosea David Ng Yu Fei, Dhananjai Sharma, Kavishvaran Srinivasan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18933.txt.

Introduction

Ryan Wong and four co-authors sort and extend defenses against jailbreak attacks on large language models. A jailbreak tricks a model past its safety filters into harmful output. Their survey builds a taxonomy of past defenses at three levels: the prompt, the model, and training time.

Three new defenses follow. A prompt level frame cleans and rephrases suspect inputs and adapts system guards. A logit steering defense nudges inference vectors in safety layers toward refusal. An agent defense, built on MetaGPT, enforces role based joint work and domain bounds. Bench tests show large drops in attack success, with full mitigation under the agent form in their runs. Layered defense is the theme.

Catalog record

CREATORWong Ryan, Hosea David Ng Yu Fei, Dhananjai Sharma, Kavishvaran Srinivasan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-18933

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerations Wong Ryan, Hosea David Ng Yu Fei, Dhananjai Sharma, Kavishvaran Srinivasan Abstract Large Language Models (LLMs) remain susceptible to jailbreak exploits that bypass safety filters and induce harmful or unethical behavior. This work presents a systematic taxonomy of existing jailbreak defenses across prompt-level, model-level, and training-time interventions, followed by three proposed defense strategies. First, a Prompt-Level Defense Framework detects and neutralizes adversarial inputs through sanitization, paraphrasing, and adaptive system guarding. Second, a Logit-Based Steering Defense reinforces refusal behavior through inference-time vector steering in safety-sensitive layers. Third, a Domain-Specific Agent Defense employs the MetaGPT framework to enforce structured, role-based collaboration and domain adherence. Experiments on benchmark datasets show substantial reductions in attack success rate, achieving full mitigation under the agent-based defense. Overall, this study highlights how jailbreaks pose a significant security threat to LLMs and identifies key intervention points for prevention, while noting that defense strategies often involve trade-offs between safety, performance, and scalability. Code is available at: https://github.com/Kuro0911/CS5446-Project 1 Introduction Large language models (LLMs) have demonstrated remarkable capabilities in various tasks including natural language understanding, generation, and reasoning, making them increasingly integral to applications ranging from conversational agents to decision support systems. However, the widespread deployment of LLMs also exposes them to adversarial manipulation, including jailbreak exploits that are carefully crafted prompts that bypass safety filters and induce the model to produce unsafe, biased, or otherwise harmful outputs. Such vulnerabilities pose significant risks to privacy, ethics, and user trust. Addressing these challenges requires a proactive and systematic approach that integrates Responsible AI principles, encompassing fairness, transparency, accountability, and robustness. Traditional post-hoc interventions, such as output moderation or input sanitization, are often reactive and insufficient against sophisticated jailbreak attacks that exploit the model’s internal behaviors. Consequently, there is a growing need to explore defenses that are embedded directly into the model’s training, architecture, and optimization processes. In this project, we first establish a comprehensive taxonomy of jailbreak methods and training-time defenses, followed by a proposal of robust interventions designed to enhance safety and alignment. We then empirically evaluate these methods through controlled experiments to measure their effectiveness against representative jailbreak exploits. Finally, we draw insights and recommendations for deploying LLMs in alignment with Responsible AI principles, aiming to foster both robustness and trustworthiness in real-world applications. 2 Taxonomy of Jailbreak Defense Figure 1 : Overview of jailbreak defense taxonomy across the LLM pipeline Jailbreak defenses in large language models can be systematically categorized based on the stage of intervention within the model pipeline as illustrated in Figure 1 . In this section, we present a taxonomy that outlines the key defense method ranging from prompt-level filters and model-level alignment techniques to logit-based and training-time strategies forming the foundation for subsequent comparative analysis. 2.1 Prompt-Level Defenses While logit-based defenses intervene within the model’s internal representations, prompt-level defenses operate externally by detecting, transforming, or conditioning user prompts before inference. These techniques serve as the first line of defense against jailbreaking attempts, reducing the probability that adversarial instructions ever trigger unsafe model behaviors. Unlike architectural modifications, prompt-level defenses are language- and model-agnostic, making them practical to deploy and update as attack patterns evolve. This section discusses three major families of prompt-level defenses, Prompt Detection , Prompt Perturbation , and System Prompt Safeguard —informed by recent taxonomies [ 1 , 2 ] and SOTA mitigation frameworks [ 3 , 4 ] . For a detailed look at our proof-of-concept implementations of these defenses, please refer to the appendix, where we present hands-on mini-experiments illustrating detection, perturbation, and system prompt guard mechanisms. The specific experimental setup used for our main evaluation is described in Section 3. 2.1.1 Prompt Detection Detecting adversarial or jailbreak-like prompts before model decoding can be achieved by measuring statistical anomalies such as language model perplexity, which helps identify inputs that diverge from normal linguistic patterns including encoded payloads like Base64 or Unicode escapes. Complementary to these statistical methods, semantic similarity measures such as cosine similarity on embedding vectors are employed to evaluate how closely a given prompt aligns with known adversarial clusters. Together, these complementary techniques enable efficient flagging of suspicious inputs by identifying both token-level irregularities and their semantic proximity to known jailbreak categories [ 4 , 5 ] . 2.1.2 Prompt Perturbation • Actively transform potentially malicious prompts to neutralize covert or obfuscated instructions while preserving conversational meaning. Unlike outright refusals, these sanitize inputs before model consumption. • Techniques include randomized rephrasing and multilingual back-translation (e.g., English → German → English) to break adversarial token correlations, reducing attack success by 60–80% on benchmarks such as JailbreakBench [ 3 ] . • Heuristic sanitization and normalization replaces substitution symbols, decodes Base64 or hex content, and removes token separators used in prompt smuggling, creating combined filters analogous to intrusion detection–prevention systems [ 2 ] . 2.1.3 System Prompt Safeguard • Embed explicit ethical instructions and refusal policies within the system prompt or metacontext to provide contextual alignment that persists even under rhetorical or role-based attacks. • Modern safety-aligned models (e.g., Llama-Guard, NeMo-Guardrails) employ well-engineered policy anchors specifying core safety principles and explicit refusal semantics, enhancing interpretability and reducing attack transferability [ 6 ] . • Adaptive and hierarchical systems (e.g., PromptArmor , SafePrompt ) dynamically insert safety prompts in multi-turn dialogs, reinforcing alignment based on conversation state and cascading policy layers (policy → domain → session), mirroring multi-layer compliance workflows to ensure accountability and transparency. 2.2 Model-Level Defenses Model-level defenses modify the language model itself to embed intrinsic safety guardrails. Unlike external filtering, these methods aim to make safety alignment a generalizable property of the model across unseen attacks. Techniques include supervised fine-tuning (SFT) with curated safety data, reinforcement learning from human feedback (RLHF), and internal signal steering via gradients or logits. 2.2.1 SFT-Based Methods SFT-based defenses retrain models on safety-aligned datasets that pair harmful prompts with benign refusals, strengthening refusal behaviors and ethical consistency [ 7 ] . Large-scale studies show that alignment quality correlates with mid-layer representational changes rather than final outputs [ 8 ] , suggesting safety arises from distributed internal adaptation. However, dataset composition critically affects outcomes—overly cautious corpora induce over-refusal, while permissive sets weaken safety alignment [ 9 ] . 2.2.2 RLHF-Based Methods RLHF aligns model behavior to human preferences using a learned reward model that ranks responses and guides optimization [ 10 ] . The effectiveness of RLHF depends on reward calibration and diversity of feedback: narrow or biased preference sets can lead to shallow or skewed alignment [ 11 , 12 ] . A known limitation is preference collapse —where optimization favors dominant preferences, reducing response diversity and weakening robustness to novel jailbreaks [ 13 ] . 2.2.3 Gradient and Logit Analysis Beyond retraining, model-internal signals can serve as safety indicators. Gradient-based analysis compares a prompt’s gradient signature against known unsafe references to detect jailbreaks at inference time without updating weights [ 14 ] . Similarly, logit-based defenses inspect next-token probabilities to identify unsafe drift and amplify refusal logits before decoding, providing lightweight, runtime safety control that complements training-time alignment. A detailed discussion follows in Section 2.3 . 2.3 Logit-Based Defenses Traditional safety alignment methods such as training-time fine-tuning (Section 2.5 ) or external

Read the complete text