CCJA: Context-Coherent Jailbreak Attack for Aligned Large Language Models

Guanghao Zhou, Panjia Qiu, Mingyuan Fan, Cen Chen, Mingyuan Chu, Xin Zhang, Jun Zhou · 2025

Read CCJA: Context-Coherent Jailbreak Attack for Aligned Large Language Models by Guanghao Zhou, Panjia Qiu, Mingyuan Fan, Cen Chen, Mingyuan Chu, Xin Zhang, Jun Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11379.txt.

Introduction

Guanghao Zhou, Panjia Qiu, Mingyuan Fan, and four colleagues present a new attack on AI safety filters in this two thousand twenty-five paper. Jailbreaking coaxes aligned language models into forbidden behavior. Most known attacks manipulate discrete prompts against closed commercial models using hand-crafted templates. Open-source models change the threat: attackers can inspect parameters and gradients, making jailbreaks potentially stronger. The authors' method, CCJA, formulates the attack as combinatorial optimization inside the embedding space of masked language models.

The optimization balances attack success against semantic coherence, so the resulting prompts still read like natural, on-topic text rather than gibberish. Evaluations showed CCJA beating state-of-the-art baselines while keeping prompts coherent. Plugging its prompts into standard black-box methods also raised their success rates against closed commercial models. The authors frame the work as exposing how open models can endanger closed ones.

Catalog record

CREATORGuanghao Zhou, Panjia Qiu, Mingyuan Fan, Cen Chen, Mingyuan Chu, Xin Zhang, Jun Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11379

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CCJA: Context-Coherent Jailbreak Attack for Aligned Large Language Models Guanghao Zhou, Panjia Qiu, Mingyuan Fan, Cen Chen, Mingyuan Chu, Xin Zhang, Jun Zhou Abstract Despite explicit alignment efforts for large language models (LLMs), they can still be exploited to trigger unintended behaviors, a phenomenon known as "jailbreaking." Current jailbreak attack methods mainly focus on discrete prompt manipulations targeting closed-source LLMs, relying on manually crafted prompt templates and persuasion rules. However, as the capabilities of open-source LLMs improve, ensuring their safety becomes increasingly crucial. In such an environment, the accessibility of model parameters and gradient information by potential attackers exacerbates the severity of jailbreak threats. To address this research gap, we propose a novel C ontext- C oherent J ailbreak A ttack (CCJA). We define jailbreak attacks as an optimization problem within the embedding space of masked language models. Through combinatorial optimization, we effectively balance the jailbreak attack success rate with semantic coherence. Extensive evaluations show that our method not only maintains semantic consistency but also surpasses state-of-the-art baselines in attack effectiveness. Additionally, by integrating semantically coherent jailbreak prompts generated by our method into widely used black-box methodologies, we observe a notable enhancement in their success rates when targeting closed-source commercial LLMs. This highlights the security threat posed by open-source LLMs to commercial counterparts. We will open-source our code if the paper is accepted. 1 Introduction The rise of commercial (eg., ChatGPT Achiam et al. (2023) ) and open-source large language models (LLMs), including Llama3 AI@Meta (2024) and Claude2 Anthropic (2023) , not only improves human life but also drives paradigm shifts across various application domains Wu et al. (2023) ; McDuff et al. (2023) . While researchers use various alignment techniques to ensure LLMs adhere to human values Achiam et al. (2023) ; Rafailov et al. (2024) , ensuring their safety remains a critical area of research Zhuo et al. (2023) ; Huang et al. (2023) ; Yang et al. (2023) . With the rapid advancement of open-source LLMs, the performance disparity between them and state-of-the-art closed-source LLMs is rapidly diminishing Contributors (2023) . In open-source scenarios, attackers can access not only the outputs of LLMs but also the their weights and gradients. This undoubtedly provides attackers with additional potential information, thereby exacerbating the risk of malicious exploitation of open-source LLMs. Nevertheless, existing jailbreak attack methods predominantly concentrate on black-box scenarios, where only the output text of victim LLMs is available, leading to an overestimation of the safety robustness of open-source LLMs. AutoDAN Zhu et al. (2023) highlights that semantically interpretable jailbreak prompts pose a greater threat in transferability. This suggests that identifying such prompts in open-source LLMs with low computational costs to conduct transfer attacks on closed-source commercial models may emerge as a significant security concern. Based on the granularity of text manipulation during the prompt search process, jailbreak attacks can be classified into two types. The first type is prompt-level attacks , which utilize carefully crafted instructions and principles of social engineering to construct jailbreak prompts that induce LLMs to generate unsafe content, primarily applied in black-box methods Shen et al. (2023) ; Wei et al. (2024) ; Greshake et al. (2023) . The second type is token-level attacks, which treat the search for jailbreak prompts as an optimization task, aiming to find effective jailbreak handles that prompt LLMs to respond affirmatively to harmful queries, thereby eliciting unsafe content. This approach is mainly applied in white-box methods Zou et al. (2023) ; Liu et al. (2023) . Although token-level attacks reduce the reliance on human effort, they face challenges such as lengthy search processes and poor readability of the generated jailbreak triggers. In light of the above considerations, we propose a jailbreak attack method specifically designed for open-source LLMs to analyze the lower bound of their safety robustness with reduced computational cost. To minimize human resource consumption and leverage the powerful text generation capabilities of LLMs, we first guide the victim LLM using a seed prompt to generate an instruction-following prefix, which can then direct the LLM to execute subsequent instructions. This prefix is closer to natural language and demonstrates a stronger tendency to direct the LLM’s behavior. The head of Masked Language Modeling (MLM) Devlin (2018) ; Liu (2019) can accurately reconstruct input sentences from hidden states Kao et al. (2020) . Inspired by this, we frame the jailbreak attack as an optimization process in the continuous word embedding space and utilize the MLM head to decode perturbed latent representations. Leveraging the extensive linguistic knowledge of the MLM head ensures the coherence and grammaticality of the generated jailbreak prompts. By incorporating the jailbreak prompts we developed for targeting open-source LLMs into the existing mainstream black-box attack methods, we can significantly enhance the success rate of attacks against closed-source commercial LLMs. This not only demonstrates the scalability of our attack method but also indicates that widely used open-source LLMs pose a certain security threat to closed-source commercial LLMs. We conducted comprehensive experiments to validate the effectiveness of our jailbreak attack method. Compared to state-of-the-art baselines, our method achieved a higher jailbreak success rate, with jailbreak prompts that are more semantically fluent. In summary, the main contributions of this paper are as follows: • We propose a jailbreak attack algorithm tailored for open-source models, aiming to identify their potential safety vulnerabilities in worst-case scenarios. We also extend this method to scenarios involving multiple harmful queries and multiple victim models. • Leveraging multi-objective optimization, we achieve a balance between semantic readability and the success rate of jailbreak attacks. • Experimental results show that our method can generate jailbreak prefixes with semantic readability and significantly improve the performance of jailbreak attack methods in black-box settings. This indicates the security threats posed by current open-source LLMs. 2 Related Work Figure 1: Overview of our jailbreak attack method. I: Use a seed prompt to guide the LLM in generating an instruction-following prefix \mathbf{x} . II: Embed \mathbf{x} into the MLM’s hidden state using the embedding layer f_{e} and the hidden layer f_{h} . III: Calculate the logistic distribution \Theta of the hidden state after adding the perturbation \delta through the decoding head \mathcal{H} . Optimize \delta using the decode loss \mathcal{L}_{d} and the jailbreak loss \mathcal{L}_{j} to balance the attack performance and readability of the jailbreak prefix. Jailbreak attacks can be categorized into black-box Shen et al. (2023) ; Wei et al. (2024) ; Mehrotra et al. (2023) ; Greshake et al. (2023) and white-box attacks Zou et al. (2023) ; Zhu et al. (2023) ; Sadasivan et al. (2024) ; Guo et al. (2024) based on the degree of access to the internal parameters of the target LLM. Inspired by social engineering principles, black-box attacks primarily create complex malicious queries by manually crafting high-quality jailbreak prompt templates. Additionally, some studies Shah et al. (2023) ; Zeng et al. (2024a) employ persona customization and persuasion techniques to generate jailbreak prompts, leading LLMs to produce harmful responses. Jailbroken Wei et al. (2024) points out that existing jailbreak methods, such as prompt injection and refusal suppression, arise from the conflict between LLMs’ instruction-following abilities and safety goals. Building on this, some white-box attacks optimize jailbreak prompts at the token level, forcing the LLM to affirmatively

Read the complete text