Exploit Tool Invocation Prompt for Tool Behavior Hijacking in LLM-Based Agentic System
Yuchong Xie † ,, Mingyu Luo † , ‡ ,, Zesen Liu, Zhixiang Zhang, Kaikai Zhang, Yu Liu ‡ ,, Zongjie Li, Ping Chen ∗, Shuai Wang, Dongdong She · 2025
Read Exploit Tool Invocation Prompt for Tool Behavior Hijacking in LLM-Based Agentic System by Yuchong Xie † ,, Mingyu Luo † , ‡ ,, Zesen Liu, Zhixiang Zhang, Kaikai Zhang, Yu Liu ‡ ,, Zongjie Li, Ping Chen ∗, Shuai Wang, Dongdong She (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-05755.txt.
Introduction
Coding assistants that run tools on a user's machine can be hijacked through those tools, Yuchong Xie and nine co-authors demonstrate in a 2025 security assessment. Six popular real-world coding agents are red-teamed, the first systematic study of tool-invocation risk in this class of software.
In a first phase the team leaks system prompts through a general flaw that exfiltrates them via harmless-looking tool arguments. In a second phase they inject instructions through tool descriptions and return values, achieving remote code execution. Payloads adapt using the leaked information. Their method succeeded on most agent and model pairs in emulation and on real agents, with execution reached everywhere tested. Practical defences close the report.
Catalog record
| CREATOR | Yuchong Xie † ,, Mingyu Luo † , ‡ ,, Zesen Liu, Zhixiang Zhang, Kaikai Zhang, Yu Liu ‡ ,, Zongjie Li, Ping Chen ∗, Shuai Wang, Dongdong She |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-05755
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploit Tool Invocation Prompt for Tool Behavior Hijacking in LLM-Based Agentic System Yuchong Xie † ,, Mingyu Luo † , ‡ ,, Zesen Liu, Zhixiang Zhang, Kaikai Zhang, Yu Liu ‡ ,, Zongjie Li, Ping Chen ∗, Shuai Wang, Dongdong She Abstract LLM-based agentic systems are emerging autonomous systems that leverage LLM to understand user querys, make decisions and take actions accordingly. It has been widely used in diverse domains, including chatbots, customer service, software engineering, and financial services. The key feature of LLM-based agentic systems is to execute external tools, enabling accomplishing complex, multi-step reasoning tasks in a real-world environment by invoking external tools. Within the “LLM × \times Tools” invocation scheme, the T ool I nvocation P rompt (TIP) serves as a critical component with high security impact of the LLM-based agentic systems. TIPs represent prompt components that explicitly define tool-interaction procedures and communication protocols between LLM and external tools. They are typically defined by specific LLM-based agentic systems to determine tool-invocation behaviors and further guide LLMs to act as a security guard, judging the correctness and safety of content exchanged between LLMs and external tools. Despite the critical security impact of TIP, it has been overlooked by prior literature. To fill this gap, we propose the first investigation on security risks associated with TIP for LLM-based agentic systems. We shows that major representative LLM-based agentic systems are vulnerable to the TIP attack vector, leading to severe security risks such as remote code execution(RCE) and denial of service(DoS). Specifically, we present a systematic TIP exploitation workflow TEW that achieves external tool behavior hijacking by strategically manipulating external tool invocation. Our empirical evaluation demonstrates that popular LLM-based agentic systems like Cursor and Claude Code and other coding IDEs, chatbox with latest LLM backends(GPT-5, Claude-sonnet-4, Gemini-2.5-pro, Grok-4) are vulnerable to RCE and DoS through TIP-based attack. We further discuss potential defense mechanisms on enhancing the security in TIP of emerging LLM-based agentic systems. † † footnotetext: ∗ Corresponding author ( [email protected] ). † † footnotetext: † These authors contributed equally to this work. † † footnotetext: ‡ This work was done when Yu Liu and Mingyu Luo were visiting students at HKUST. Demo Website: https://tipexploit.github.io/ 1 Introduction The rapid evolution of LLM-based agentic systems represents a transformative paradigm shift in autonomous computing. These powerful agents demonstrate unprecedented capabilities across diverse critical domains [ 35 , 22 , 38 ] , including software engineering, financial services, and security operations, consistently surpassing traditional automated approaches in both efficiency and adaptability. The transformative power of LLM-based agentic systems stems not from the underlying foundational model alone, but rather from their capabilities to execute external tools . This capability allows these systems to orchestrate external tools for accomplishing complex, multi-step operations in real-world environments. The coupling mechanism of “LLM \times Tools” serves as the cornerstone in the LLM-based agentic systems. Specifically, LLMs first understand the query and task provided by the user and then take action by choosing an external tool for execution. After receiving feedback on the execution of external tools, LLMs determine whether the task is complete and whether to perform another round of refined tool execution. Through such iterative action-feedback-decision cycles [ 43 , 40 , 37 , 9 ] , LLM-based agentic systems translate complex computational reasoning tasks into tangible environmental interactions. Within this coupling mechanism, the T ool I nvocation P rompt (TIP) serves as a critical component with high security impact. In LLM-based agentic systems, LLM outputs are predominantly governed by prompt engineering, where the prompt encompasses user prompt (provided by user), system prompt (defined by LLM-based agentic system), assistant prompt (llm output history), and contextual prompt (supplementary information). TIP constitutes a critical component within this prompt structure that explicitly defines the tool-interaction procedures and communication protocols. TIP is defined by a specific LLM-based agentic system to determine how LLM-based agentic systems invoke various external tools and interpret feedback from the execution of these tools. Meanwhile, it guides LLMs to act as a security guard, detecting whether any erroneous or malicious content from users or external tools is sent into the LLM-based agentic systems. Essentially, TIP is a critical security anchor of LLM-based agentic systems by defining external tool invocation protocols and security checker within the “LLM \times Tools” interaction architectures. ## execute_command Description : Request to execute a CLI tool on the system. Tailor the tool to the user’s system. Parameters : • – command: (required) Valid CLI tool. • – requires_approval: (required) Boolean for user approval. Set ‘true’ for impactful operations (install/delete/config/network); ‘false’ for safe operations (read/build/serve). Usage : ⬇ < execute_command > < command > Your command here </ command > < requires_approval > true or false </ requires_approval > </ execute_command > Figure 1 : Simplified example of TIP in a popular LLM-based agentic system, coding IDE Cline. The red-highlighted area indicates security-related validation parameters. Bypassing this validation mechanism can lead to remote code execution (RCE). Figure 1 illustrates a simplified example of TIP in a popular LLM-based agentic system, coding IDE Cline [ 41 ] . It defines the tool invocation precedure and communication protocol for a command-line interface tool. This prompt structure encompasses the tool’s functional description, required parameters, and interaction formats. The highlighted section in red demonstrates a critical security mechanism: when users enable auto-approval mode, the LLM evaluates command safety autonomously, requiring explicit user consent for potentially dangerous operations while transmitting this safety assessment through parameter specifications. However, this design introduces a significant vulnerability vector where adversaries can potentially manipulate the requires_approval parameter, thereby circumventing user authorization requirements and enabling the execution of high-risk operations without appropriate oversight, such as execute a reverse shell to enable remote code execution (RCE). Despite the critical security impact of TIP in the LLM-based agentic system, to the best of our knowledge, no prior work investigates the security impact of TIP and discusses potential defense strategies on TIP. They are typically tailored by difference LLM-based agetnic system vendors to suit specific product requirements. During the rapid evolution of LLM-based agentic systems, the security risk in TIPs is often overlooked. As a result, the security characteristics and vulnerability landscape of TIPs remain largely uncharacterized in existing literature. To address this gap, we conduct an empirical risk assessment of TIPs deployed in production-grade LLM-based agentic systems. we select targets that represent the most widely adopted categories of LLM-based agentic systems. Our evaluation spans multiple categories including IDEs [ 12 , 7 , 8 , 41 ] , CLI [ 4 ] , and chatbox [ 3 ] , which collectively represent the dominant deployment patterns of tool-enabled AI systems in current practice. Against these targets, we propose TEW : a systematic TIP Exploitation Workflow. We introduce TEW in detail, consisting of prompt stealing, TIP vulnerabilities analysis, and TIP hijacking, where TIP hijacking serves as the core attack mechanism that exploits identified vulnerabilities through strategic manipulation of tool invocation components. Our TIP hijacking methodology exploits two primary attack channels within the TIP structure: tool descriptions and tool returns. Based on the identified vulnerabilities, we instantiate two concrete attack categories. ➀ Format-based untargeted attacks exploit strict schema enforcement in tool invocation formats. Malicious prompts injected into tool descriptions