Conjunctive Prompt Attacks in Multi-Agent LLM Systems

Nokimul Hasan Arif, Qian Lou, Mengxin Zheng · 2026

Read Conjunctive Prompt Attacks in Multi-Agent LLM Systems by Nokimul Hasan Arif, Qian Lou, Mengxin Zheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16543.txt.

Introduction

Arif, Lou and Zheng study conjunctive prompt attacks in multi-agent systems. Safety work covers single agents, but routing between agents creates missed attack surfaces. A trigger key in the query and a hidden template in one compromised agent look benign alone. Together through routing, they activate harmful behavior. The attacker changes neither weights nor the client agent, controlling trigger placement and template use. Across star, chain and DAG topologies, routing-aware optimization raised attack success over baselines. False activations stayed low, and defenses did not stop the attack.

Catalog record

CREATORNokimul Hasan Arif, Qian Lou, Mengxin Zheng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60525
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16543

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Conjunctive Prompt Attacks in Multi-Agent LLM Systems Nokimul Hasan Arif, Qian Lou, Mengxin Zheng Abstract Most LLM safety work studies single-agent models, but many real applications rely on multiple interacting agents. In these systems, prompt segmentation and inter-agent routing create attack surfaces that single-agent evaluations miss. We study conjunctive prompt attacks , where a trigger key in the user query and a hidden adversarial template in one compromised remote agent each appear benign alone but activate harmful behavior when routing brings them together. We consider an attacker who changes neither model weights nor the client agent and instead controls only trigger placement and template insertion. Across star, chain, and DAG topologies, routing-aware optimization substantially increases attack success over non-optimized baselines while keeping false activations low. Existing defenses, including PromptGuard, Llama-Guard variants, and system-level controls such as tool restrictions, do not reliably stop the attack because no single component appears malicious in isolation. These results expose a structural vulnerability in agentic LLM pipelines and motivate defenses that reason over routing and cross-agent composition. Code is available at https://github.com/UCF-ML-Research/ConjunctiveAgents . 1 Introduction Foundation models such as Large Language Models (LLMs) are increasingly deployed as agentic systems rather than standalone chat models. In these systems, multiple specialized agents collaborate through task decomposition, communication, and tool use, enabling stronger performance on complex tasks and driving adoption in assistants, copilots, and autonomous workflows ( Sidahmed et al. 2024 ; Roy et al. 2025 ; Lou et al. 2022b ; Hsu et al. 2022 ; Lou et al. 2022a ; Li et al. 2025a ) . Flight Agent Hotel Agent blackbox User Trip Planning Agent input prompt Flight Database Hotel Database collect data (vulnerable) collect data (vulnerable) Figure 1: Normal multi-agent LLM pipeline without adversarial manipulation. A user interacts with a client (orchestrator) agent that decomposes the request into subtasks and routes them to specialized remote agents (e.g., flight and hotel agents), each connected to external tools or databases. Remote agents operate as black boxes to the client, exposing only their natural-language interfaces. In a typical agentic LLM system, shown in Figure 1 , a client agent decomposes a user request into subtasks, routes them to specialized remote agents, and aggregates the results, often through tool calls or database access. This modular design is powerful, but it also creates attack surfaces that single-agent evaluations do not capture ( Du 2025 ; Li et al. 2025a ; Li et al. 2025c ; Asthana et al. 2025 ) . The key security challenge is that behavior in a multi-agent pipeline depends not only on the user prompt, but also on segmentation, routing, and the hidden wrappers or system prompts of remote agents. As a result, prompt injection can become more dangerous in agentic systems, where seemingly benign content may indirectly influence privileged components or tool-using agents through agent-to-agent communication ( Wen et al. 2025 ; Hossain et al. 2025 ; Triedman et al. 2025 ) . We focus on an underexplored supply-chain threat in which remote agents are black boxes to the client: their weights, prompts, or wrapper templates may be proprietary, externally hosted, or provided by third parties ( Boisvert et al. 2025 ) . In this setting, an attacker does not modify model weights or the client agent, but tampers with a hidden template inside one remote agent ( Mo et al. 2025 ) . The resulting attack is conjunctive : a trigger key in the user query and the injected template each appear benign in isolation, yet together can activate harmful behavior only when routing brings them to the same compromised agent ( Wang et al. 2025b ) . This vulnerability is inherently topology-dependent. A trigger is effective only if it reaches the compromised agent, and different communication structures—such as star, chain, and DAG topologies—induce different routing dynamics and exposure patterns ( Yu and Wang 2010 ; Wang et al. 2025c ) . This also explains why existing defenses often fail: prompt- and output-level guard models inspect isolated messages, but conjunctive activation emerges only after cross-agent composition, when individually benign-looking components align at the compromised agent ( Touvron et al. 2023 ) . In this paper, we study conjunctive prompt attacks in multi-agent LLM systems under explicit communication topologies. We introduce a topology- and routing-aware attack framework that operates purely at the prompt level by optimizing trigger placement, template placement, and routing bias, without modifying model weights. Across star, chain, and DAG topologies, we show that routing-aware optimization substantially strengthens attacks over non-optimized baselines while maintaining low false activation. Our contributions are threefold. First, we formalize a realistic black-box threat model for conjunctive prompt attacks in which compromise arises from the interaction between user-side triggers and a tampered remote-agent template. Second, we develop a topology- and routing-aware prompt-level optimization framework for constructing such attacks without changing model weights or the client agent. Third, we show empirically that these attacks transfer across topologies and backbones, while widely used guard models and system-level controls remain poorly matched to the underlying cross-agent activation mechanism. 2 Related Work Prompt injection and indirect injection. Prompt injection is commonly framed as a failure mode in which adversarial strings embedded in user inputs or retrieved content override developer intent ( Zheng et al. 2022 ; Greshake et al. 2023 ; Xue et al. 2024 ; Lou et al. 2024 ) . The threat becomes sharper in retrieval-augmented and tool-using settings, where untrusted external content can steer privileged behavior and enable indirect prompt injection ( Wang et al. 2025a ) . Recent work also highlights the tension between aggressive filtering and usability, showing that simple trigger-based defenses can over-reject benign inputs and degrade normal task performance ( Li and Liu 2025 ; Lou et al. 2023 ; Zheng et al. ; Al Ghanim et al. 2023 ) . Safety in agentic and multi-agent systems. A growing literature argues that safety results from single-model chat settings do not directly transfer to agentic systems, where behavior emerges from message passing, tool use, and routing across multiple agents ( Yu et al. 2025 ; Kavathekar et al. 2025 ) . Multi-agent pipelines introduce additional attack surfaces, including delegation boundaries, coordination failures, and topology-dependent exposure that changes which components observe which information and when ( Liang et al. 2025 ; Zhu et al. 2025 ; Xu et al. 2025 ; Krawiecka and de Witt 2025 ) . Jailbreak evaluation and attack metrics. Prior jailbreak and red-teaming work typically measures attack success rate (ASR) for a single model invocation under different prompt strategies and threat models ( Ntais 2025 ; Rahman et al. 2025 ; Al Ghanim et al. 2024 ) . In agentic systems, however, success depends not only on whether malicious content exists, but also on whether it reaches the relevant component and produces a harmful downstream effect ( Boisvert et al. 2025 ; Hazan et al. 2025 ) . This motivates our four-regime evaluation (clean, key-only, template-only, both), which treats success as an end-to-end compromise rather than a local refusal failure. Guardrails and prompt-injection defenses. A parallel line of work proposes detectors, guard models, and sanitization schemes for prompt injection ( Li and Liu 2025 ) . Recent results are especially relevant to our setting.

Read the complete text