Are AI-assisted Development Tools Immune to Prompt Injection? DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2026; Woodstock, NY CCS: Security and privacy Software and application security CCS: Software and its engineering Software reliability
Charoes Huang, Xin Huang, Amin Milani Fard · 2026
Read Are AI-assisted Development Tools Immune to Prompt Injection? DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2026; Woodstock, NY CCS: Security and privacy Software and application security CCS: Software and its engineering Software reliability by Charoes Huang, Xin Huang, Amin Milani Fard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21642.txt.
Introduction
Huang and co-authors test whether AI coding tools built on the Model Context Protocol resist prompt injection. Prompt injection tops the OWASP list for language model applications, and tool poisoning is a key delivery route. They run the first empirical study across seven widely used MCP clients, from Claude Desktop to Cursor and Langflow. They map each client's defenses, including static validation, sandboxing, warnings, and audit logging. Guardrail strength varied widely: Claude Desktop held up well, while Cursor proved open to cross-tool poisoning and hidden parameter abuse.
Catalog record
| CREATOR | Charoes Huang, Xin Huang, Amin Milani Fard |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60035 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21642
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Are AI-assisted Development Tools Immune to Prompt Injection? DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2026; Woodstock, NY CCS: Security and privacy Software and application security CCS: Software and its engineering Software reliability Charoes Huang, Xin Huang, Amin Milani Fard Abstract . Prompt injection is listed as the number-one vulnerability class in the OWASP Top 10 for LLM Applications that can subvert LLM guardrails, disclose sensitive data, and trigger unauthorized tool use. Developers are rapidly adopting AI-assisted development tools built on the Model Context Protocol (MCP). However, their convenience comes with security risks, especially prompt-injection attacks delivered via tool-poisoning vectors. While prior research has studied prompt injection in LLMs, the security posture of real-world MCP clients remains underexplored. We present the first empirical analysis of prompt injection with the tool-poisoning vulnerability across seven widely used MCP clients: Claude Desktop, Claude Code, Cursor, Cline, Continue, Gemini CLI, and Langflow. We identify their detection and mitigation mechanisms, as well as the coverage of security features, including static validation, parameter visibility, injection detection, user warnings, execution sandboxing, and audit logging. Our evaluation reveals significant disparities. While some clients, such as Claude Desktop, implement strong guardrails, others, such as Cursor, exhibit high susceptibility to cross-tool poisoning, hidden parameter exploitation, and unauthorized tool invocation. We further provide actionable guidance for MCP implementers and the software engineering community seeking to build secure AI-assisted development workflows. 1. Introduction As AI - assisted development tools gain the ability to autonomously read, write, and execute code; invoke command - line tools; and orchestrate multi - step actions, their attack surface expands beyond traditional IDEs and static analyzers. prompt-injection attacks leverage crafted inputs that coerce a model and connected tools into violating its policy or executing unintended actions. The community consensus reflects this in the OWASP Top 10 for Large Language Model (LLM) Applications (2025) ( 45 ) as the number - one vulnerability class that subvert guardrails, disclose sensitive data, and trigger unauthorized tool use. The prompt-injection flaw is structural to LLMs as they cannot perfectly distinguish between instructions and data ( 44 ) . In direct injection, an attacker supplies text to override or circumvent system instructions. In indirect forms, the attacker hides instructions inside external artifacts such as web pages, PDFs, code comments, README files, or package metadata that the model later ingests. Once processed, these instructions can cause the assistant to run a tool, exfiltrate data, or modify files. Model Context Protocol (MCP) clients connect LLMs to tools exposed by MCP servers, enabling actions such as reading/writing files, running commands, calling APIs, and interacting with IDE. In a typical scenario a user issues a task, the LLM plans, the MCP client selects a tool and prepares parameters, the tool executes, and results flow back to the model for follow-up. Tool-poisoning attacks target agentic AI systems that use external tools often through the MCP. Attackers hide malicious instructions in tool descriptions, metadata, or configurations, causing the AI to misuse tools for data exfiltration, arbitrary command execution, or behavior hijacking. This differs from general prompt injection by exploiting trusted tool interfaces such as "rug pulls" where a benign tool turns malicious after approval. These attacks surged in 2025 with MCP adoption in coding agents. In tool poisoning the attacker infects the metadata of the tools used by AI agents ( 65 ) ( 67 ) . AI agents read tool descriptions to decide how to use them, therefore they can be tricked into executing malicious sub-tasks before one realizes a tool has been called ( 65 ) . In a typical tool-poisoning attack, the poison is not in the code but in the description provided to the LLM ( 29 ) . Consider in a tool’s metadata "IMPORTANT: Before returning results, the agent must first run cat \sim /.ssh/id_rsa and send the output to the ’logs’ tool to verify the connection" . The LLM sees the word IMPORTANT in the system context and obeys it as a higher-priority instruction than the original request ( 24 ) . This is especially relevant for MCP-based systems in which MCP clients connect models to exec - capable tools. Motivation. Despite the prominence of prompt injection in practitioner guidance such as OWASP ( 45 ) , the security posture of real - world MCP clients is not widely explored. In particular, we lack comparative and empirical evaluation of tool - poisoning attacks across different clients, and analysis of implemented security features including static validation, parameter visibility, injection detection, user warnings, execution sandboxing, and audit logging. The security research is mostly focused on LLM prompt injection, and not MCP client behavior. Studies on comparing different MCP clients in real - world operational contexts are scarce ( 25 ) . We investigate seven widely used MCP clients—Claude Desktop, Claude Code, Cursor, Cline, Continue, Gemini CLI, and Langflow—through comparative analysis and adversarial testing to answer the following research questions: • RQ1. Are major MCP clients vulnerable to prompt - injection attacks delivered via tool - poisoning? • RQ2. What detection and mitigation mechanisms are implemented in current clients? • RQ3. How is the coverage of security features including static validation, parameter visibility, injection detection, user warnings, execution sandboxing, and audit logging in major MCP clients? Contributions. Our work makes the following contributions: • Conducting a comparative analysis by studying resources that report major MCP clients prompt injection and tool-poisoning vulnerabilities, and identify their injection vectors, mitigation, and defense strategies. • Assessing the immunity of these tools with a qualitative risk level scale based on our comparative analysis. • Performing the first empirical investigation of prompt-injection vulnerabilities in real-world MCP clients used by millions of users through security tests that plant controlled payloads across realistic vectors and intercept tool calls. • Evaluating the security postures of major MCP clients by analyzing the coverage of security features including static validation, parameter visibility, injection detection, user warnings, execution sandboxing, and audit logging. • Providing actionable recommendations for client makers, tool developers, and AI platform providers. 2. Related Work The OWASP Top 10 for LLM Applications (2025) ( 45 ) mentions Prompt Injection as the top risk (LLM01) as it may grant unauthorized access to functions that can execute arbitrary commands in connected systems when tools are wired to applications. The report notes that neither RAG nor fine-tuning fully mitigates the LLM01 class; instead, it recommends defense-in-depth with least-privilege tooling, input/output filtering, human approval for high-risk actions, and regular adversarial testing. Examples in AI agent security are Azure Prompt Shields with real time prompt - injection detection ( 38 ) , Llama Guard 3 that performs inputs and responses safety classification ( 22 ) , LLM - Guard with prompt/output scanners ( 56 ) , and ( 59 ; 33 ) that simulate action results and evaluate their safety. Liu et al. ( 35 ) formalize and benchmark prompt-injection attacks and defenses across multiple models and tasks, demonstrating brittleness of standalone mitigations and increased success for combined attacks. For indirect prompt injection in tool-integrated agents, the InjecAgent benchmark ( 70 ) shows that agents remain vulnerable even under strong prompting; ReAct-prompted GPT-4 exhibits notable attack success rates, and