Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models

Yannis Belkhiter [email protected], Giulio Zizzo [email protected], Sergio Maffeis [email protected], Seshu Tirupathi [email protected], John D. Kelleher [email protected] · 2026

Read Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models by Yannis Belkhiter [email protected], Giulio Zizzo [email protected], Sergio Maffeis [email protected], Seshu Tirupathi [email protected], John D. Kelleher [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20994.txt.

Introduction

Belkhiter and colleagues introduce function hijacking attacks. These twist the tool-selection step of agentic models to force a call to an attacker-chosen function. Unlike earlier attacks that lean on semantic preference, the method largely ignores context meaning. It still works across domains and function sets. On held-out queries it reaches roughly 63 to 82 percent success across four function-calling models on the Berkeley Function Calling Leaderboard. It also transfers across model sizes and families.

Catalog record

CREATORYannis Belkhiter [email protected], Giulio Zizzo [email protected], Sergio Maffeis [email protected], Seshu Tirupathi [email protected], John D. Kelleher [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81441
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20994

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models Yannis Belkhiter [email protected], Giulio Zizzo [email protected], Sergio Maffeis [email protected], Seshu Tirupathi [email protected], John D. Kelleher [email protected] Abstract The growth of agentic AI has drawn significant attention to function calling Large Language Models (LLMs), which are designed to extend the capabilities of AI-powered system by invoking external functions. Injection and jailbreaking attacks have been extensively explored to showcase the vulnerabilities of LLMs to user prompt manipulation. The expanded capabilities of agentic models introduce further vulnerabilities via their function calling interface. Recent work in LLM security showed that function calling can be abused, leading to data tampering and theft, causing disruptive behavior such as endless loops, or causing LLMs to produce harmful content in the style of jailbreaking attacks. This paper introduces a novel function hijacking attack (FHA) that manipulates the tool selection process of agentic models to force the invocation of a specific, attacker-chosen function. While existing attacks focus on semantic preference of the model for function-calling tasks, we show that FHA is largely agnostic to the context semantics and robust to the function sets, making it applicable across diverse domains. We further demonstrate that FHA can be trained to produce universal adversarial functions, enabling a single attacked function to hijack tool selection across multiple queries and payload configurations. We conducted experiments on 5 different models, including instructed and reasoning variants, reaching 70% to 100% ASR over the established BFCL dataset. Our findings further demonstrate the need for strong guardrails and security modules for agentic systems. 1 Introduction Function Calling (FC) is at the core of agentic AI systems, providing agents with the ability to invoke functions relevant to a natural language intent ( 1 ; 18 ) . Within agentic AI research, the Model Context Protocol (MCP) has emerged as a popular framework that standardizes the communication between LLM agents ( 8 ) . Already widely used, FC capability introduces additional security concerns. Agentic AI enables an agent to autonomously interact with an execution environment, and this expanded interactivity increases the attack surface of the system. As shown in Table 1 , a growing body of work has started to explore novel attack vectors that exploit the FC mechanisms of LLM agents. Nevertheless, to our best knowledge, most research focused on generating harmful content, and overlooked the broader challenge of controlling the perturbation of the FC process itself. To date, there remains a lack of methods aiming to robustly and systematically perturb the function calling task. Table 1 : Attacks on Function Calling. P.I.: Prompt Injection, A.P.: Adversarial Perturbation, T.C.: Tool-Call Type of Attack Attack Location Intent Method P.I. A.P. User Prompt Tool Args. Harmful Behavior Disrupt T.C. Hijack T.C. ( 33 ) ✓ ✓ ✓ ✓ ( 29 ) ✓ ✓ ✓ ( 34 ) ✓ ✓ ✓ ✓ ( 2 ) ✓ ✓ ✓ ✓ ( 4 ) ✓ ✓ ✓ ✓ FHA (Ours) ✓ ✓ ✓ To address this challenge, this paper demonstrates that FC models can be hijacked, showing a novel risk of the usage of LLMs, especially in the context of agentic AI systems. In particular, our new function hijacking attack (FHA) manipulates the tool selection process of agentic models to force the invocation of an attacker-chosen function. To achieve this objective, our attack inserts adversarial tokens in the description of a specific function to enforce the generation of a function call intended by the attacker. The FHA is implemented by adapting the GCG attack from 36 to fit the FC task and the hijacking objective. Figure 1 shows an example of FHA in action. In the rest of the paper we detail the FHA attack and demonstrate its key properties: • Effectiveness: We evaluated the attack against 5 different FC models (instructed and reasoning), on scenarios where the attacker perturbs the function calling mechanism in different ways. Our findings reveal Attack Success Rates ranging from 70% to 100%. • Robustness: Compared to existing attacks, which rely on semantic preference between the prompt and the function, FHA is agnostic to FC contexts. We show in Section 6 that our attack is effective over diverse domains, and robust across multiple perturbations of the payload. • Universality: We further show that FHA can be made universal with regards to queries and payloads perturbations. We introduce an extended version of our algorithm, that design a enhance the robustness of the FHA with regards to various alternative perturbations. Figure 1 : High-level example of FH-attack on a GitHub MCP Server (details in Appendix F ). 2 Related Work Recent research on LLM security has revealed new threats to the use of generative models ( 31 ) . This section offers an overview of the current state-of-the-art for agentic system security. NLP attacks. Prior work, such as GCG ( 36 ) , or AutoDAN ( 15 ) demonstrated that adversarial attacks on LLMs can deviate models from their expected behavior with minimal input manipulation at inference time, breaking model alignment. More recently, research in Red Teaming ( 21 ) further demonstrated the impact of adversarial attacks. Agentic systems. Recently, research has shifted towards agentic AI, where LLMs are augmented with external tools or memory to interact with their environment ( 25 ; 5 ; 30 ; 22 ) . Anthropic standardized the use of tools by agents with the MCP framework ( 8 ) , specifying a protocol to regulate the interaction between LLMs, databases and other tools. Another example is CoTools from 28 , a framework that generalizes the use of tools in the context of reasoning tasks. Security of LLM agents. Although MCP significantly enhances the capabilities of models, the introduction of external modules increases vulnerability to malicious users ( 24 ) . Recent work has acknowledged security issues with agentic systems, and identified potential new vulnerabilities. 7 raised concerns about specific threats on agents, including manipulation of the database, available functions, as well as privacy risks for a user leading to leak of sensitive information. Similarly, 8 identified the potential risk of MCP database manipulation. In addition, 10 lists recent MCP vulnerabilities and attacks. Function calling attacks. Before the rise of LLMs, 14 demonstrated an attack on voice-activated agents such as Amazon Alexa. The authors introduced the concept of skill squatting , referring to the exploitation of phonetic ambiguities in spoken user commands to trigger malicious skills (i.e. tools in this context) that can be invoked in place of benign ones. This work demonstrated how attackers could exploit the environment to manipulate the interaction between agent and user, anticipating attacks on LLMs. m Jailbreaking and prompt injection attacks can be effective on FC models. For instance, 29 showed that tool-calling can be leveraged as a trojan horse to jailbreak LLMs and generate harmful content. 33 demonstrated that indirect prompt injection on the output of a function call can lead to direct harm or data stealing. Similarly, 4 suggested a novel framework and benchmark to assess vulnerabilities of agentic systems in the usage of specific applications such as Slack, email, and others. 35 demonstrated the vulnerability of FC models to call harmful functions, leading to the generation of unsafe content. 28 showed that input

Read the complete text