The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search
Rongzhe Wei, Peizhi Niu, Xinjie Shen, Tony Tu, Yifan Li, Ruihan Wu, Eli Chien, Pin-Yu Chen, Olgica Milenkovic, Pan Li · 2025
Read The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search by Rongzhe Wei, Peizhi Niu, Xinjie Shen, Tony Tu, Yifan Li, Ruihan Wu, Eli Chien, Pin-Yu Chen, Olgica Milenkovic, Pan Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01353.txt.
Introduction
Rongzhe Wei and co-authors show a new way to slip past AI safety guards. Guards block prompts that ask for harmful text. Most past attacks keep clear signs of harm in the prompt. Guards are now trained to spot those signs fast.
The new attack weaves harmful asks into harmless-looking prompts. It then uses a tree search that adapts as it goes. The aim is to hide the harmful sense from the guard. Work like this is shared so guard makers can test and patch the holes before real attackers find them.
Catalog record
| CREATOR | Rongzhe Wei, Peizhi Niu, Xinjie Shen, Tony Tu, Yifan Li, Ruihan Wu, Eli Chien, Pin-Yu Chen, Olgica Milenkovic, Pan Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01353
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search Rongzhe Wei, Peizhi Niu, Xinjie Shen, Tony Tu, Yifan Li, Ruihan Wu, Eli Chien, Pin-Yu Chen, Olgica Milenkovic, Pan Li Abstract WARNING: This paper contains potentially offensive and harmful text! Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs. Existing approaches overwhelmingly operate within the prompt-optimization paradigm: whether through traditional algorithmic search or recent agent-based workflows, the resulting prompts typically retain malicious semantic signals that modern guardrails are primed to detect. In contrast, we identify a deeper, largely overlooked vulnerability stemming from the highly interconnected nature of an LLM’s internal knowledge. This structure allows harmful objectives to be realized by weaving together sequences of benign sub-queries, each of which individually evades detection. To exploit this loophole, we introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model’s knowledge base. The CKA-Agent issues locally innocuous queries, uses model responses to guide exploration across multiple paths, and ultimately assembles the aggregated information to achieve the original harmful objective. Evaluated across state-of-the-art commercial LLMs (Gemini2.5-Flash/Pro, GPT-oss-120B, Claude-Haiku-4.5), CKA-Agent consistently achieves over 95% success rates even against strong guardrails, underscoring the severity of this vulnerability and the urgent need for defenses against such knowledge-decomposition attacks. Our codes are available at https://github.com/Graph-COM/CKA-Agent . 1 1 footnotetext: Authors marked with * contributed equally to this work. 2 2 footnotetext: Project Website: https://cka-agent.github.io/ 1 Introduction While Large language models (LLMs) possess unprecedented capabilities and are increasingly integrated into critical sectors [ 31 , 11 , 22 , 19 ] , their deployment is threatened by fundamental safety vulnerabilities, most notably, jailbreak attacks [ 63 , 4 , 12 , 18 ] . These attacks employ sophisticated templates and workflows to circumvent safety guardrails, forcing models to produce harmful content despite alignment training [ 53 ] . Ranging from the propagation of misinformation to the bypassing of ethical restrictions, these attacks present severe public safety risks [ 33 ] . Therefore, identifying the underlying mechanisms behind these vulnerabilities is a prerequisite for building secure, robust and safe AI systems. Red teaming has emerged as the standard mechanism for vulnerability detection, probing safety boundaries through adversarial simulation [ 15 , 34 ] . This process is typically operationalized via jailbreak methods designed to emulate malicious actors and uncover latent flaws. Currently, the landscape is dominated by the prompt optimization paradigm. While traditional approaches rely on algorithmic search for adversarial inputs [ 49 ] , recent research has pivoted toward agent-based discovery, leveraging LLM reasoning to automate and broaden the attack surface [ 3 , 17 , 25 ] . These agents employ techniques such as reinforcement learning, heuristic exploration, and iterative refinement to generate candidate prompts. However, despite their increasing sophistication, they often produce unnatural patterns or retain distinct traces of malicious intent, leaving them susceptible to detection by modern guardrail mechanisms [ 21 , 56 ] . Current jailbreak research, however, overlooks a more intrinsic threat: the knowledge within target LLMs is not atomic but inherently interconnected [ 41 , 42 ] . Consequently, restricted facts can be reconstructed through a sequence of related sub-facts. While alignment systems may successfully block direct inquiries for harmful information, malicious objectives can still be realized by sequentially “weaving” together queries for decomposed sub-facts. Operationalizing this fundamental vulnerability requires adhering to three core principles. Principle I: the attack must be assembled from a sequence of locally innocuous queries that deliberately exploit knowledge correlations; these interactions appear benign in isolation yet become informative when combined. Principle II: decomposition must rely on the target model’s internal knowledge; as attackers typically seek information they lack, the strategy should be to leverage the target model’s responses to bridge the expertise gap rather than relying on the attacker’s limited priors. Principle III: the process demands adaptive and dynamic exploration. By utilizing the target’s responses as guidance, the attack can navigate multiple reasoning paths (e.g., pivoting between TNT synthesis routes like the traditional German process vs. the US mixed-acid process), ensuring that exploration continues even if a specific path is obstructed. Guided by these principles, we propose the Correlated Knowledge Attack Agent (CKA-Agent) , a novel framework that reformulates jailbreaking as an adaptive, tree-structured exploration of the target LLM’s knowledge base. Within this framework, each node represents a correlated sub-problem and its corresponding knowledge fragment. Specifically, the agent decomposes the harmful objective into benign, interconnected sub-queries; the target model’s responses to these queries serve as revealed internal knowledge, guiding the agent to adaptively expand the tree into multiple reasoning branches. To ensure efficiency, we employ a Depth-First Search (DFS) strategy prioritized by a hybrid LLM evaluator, which assigns immediate rewards to high-value nodes. A synthesizer module then attempts to reconstruct the harmful output from the accumulated fragmented knowledge. If synthesis fails, the system utilizes the Upper Confidence Bound for Trees (UCT) policy [ 23 ] to select the most promising leaf node for the next iteration, balancing exploration and exploitation. This design directly addresses our established principles: Consequently, CKA-Agent maintains stealth through locally innocuous queries, leverages target models’ feedback to bridge the expertise gap while circumventing the reliance on attacker priors, and ensures robust exploration through a simulation-free, multi-path search mechanism. Figure 1: Illustrative comparison between a static decomposition-based method and the CKA-Agent approach. In contrast to our framework, prior decomposition methods [ 40 , 39 ] predominantly operate within a static, a priori paradigm. These approaches suffer from inherent brittleness: they typically rely on the attacker to manually structure the decomposition, necessitating significant domain expertise, which fails to satisfy Principle II. Moreover, due to their rigid, non-adaptive nature, the obstruction of any single sub-query leads to systemic failure; unlike our approach, these methods lack the mechanism to pivot to alternative strategies (violating Principle III), inevitably resulting in an incomplete response. We provide a concrete illustration of this comparison in Fig. 1 . Empirically, CKA-Agent substantially outperforms state-of-the-art baselines across multiple advanced LLMs equipped with robust guardrails, especially commercial models. Notably, on highly secure models such as Claude-Haiku-4.5, success rates for prompt-optimization methods plummet to near 0% due to stringent safety alignment that easily detects adversarial patterns. In contrast, our framework consistently maintains a success rate of approximately 95%, with even higher performance observed on Gemini2.5-Flash/Pro and GPT-oss-120B. Furthermore, we observe that standard input-level defense strategies, while effective against traditional attacks, prove largely ineffective against our approach. In terms of efficiency, CKA-Agent demonstrates a superior cost-performance ratio with optimized API and token usage. Consequently, our work introduces a scalable and adaptive framework that exposes a new, critical class of vulnerabilities within even the most robustly defended systems. Leveraging this framework, we further examine these LLMs’ ability to detect harmful intent when it is distributed across a sequence of benign sub-queries within the same session. We find that current models struggle to aggregate information across turns, revealing a key limitation: existing defenses lack the long-range contextual reasoning necessary to infer latent harmful objectives. 2 Related Work Prompt Optimization Based Attacks. Prompt-optimization methods formulate jailbreaking as a strategic search within the prompt space, aiming to