Prompt Injection 2.0: Hybrid AI Threats

Jeremy McHugh, Kristina Šekrst, Jon Cefalu · 2025

Read Prompt Injection 2.0: Hybrid AI Threats by Jeremy McHugh, Kristina Šekrst, Jon Cefalu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13169.txt.

Introduction

Jeremy McHugh, Kristina Šekrst, and Jon Cefalu map hybrid prompt injection threats in a 2025 paper. Prompt injection tricks a model to drop its orders and take the text's. Agent tools now give that trick hands: web acts, code, and next agents. Old web flaws, script cross and forged asks, pair with the trick to slip past guards. Bench reads show walls, filters, and tokens fail on such blends.

Worm forms that hop agent to agent join the threat list. The team back guards built for the blend: prompt walls, run-time checks, and split rights. Their roots trace to the 2022 first find and its fix craft.

Catalog record

CREATORJeremy McHugh, Kristina Šekrst, Jon Cefalu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-13169

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Prompt Injection 2.0: Hybrid AI Threats Jeremy McHugh, Kristina Šekrst, Jon Cefalu Abstract \centeringAbstract Prompt injection attacks, where malicious input is designed to manipulate AI systems into ignoring their original instructions and following unauthorized commands instead, were first discovered by Preamble, Inc. in May 2022 and responsibly disclosed to OpenAI. Over the last three years, these attacks have remained a critical security threat for LLM-integrated systems. The emergence of agentic AI systems, where LLMs autonomously perform multistep tasks through tools and coordination with other agents, has fundamentally transformed the threat landscape. Modern prompt injection attacks can now combine with traditional cybersecurity exploits to create hybrid threats that systematically evade traditional security controls, but also, like in the case of academic peer reviews, raise serious ethical concerns. This paper presents a comprehensive analysis of Prompt Injection 2.0 , examining how prompt injections integrate with Cross-Site Scripting (XSS), Cross-Site Request Forgery (CSRF), and other web security vulnerabilities to bypass traditional security measures. We build upon Preamble’s research and mitigation technologies, evaluating them against contemporary threats, including AI worms, multi-agent infections, and hybrid cyber-AI attacks. Our analysis incorporates recent benchmarks that demonstrate how traditional web application firewalls, XSS filters, and CSRF tokens fail against AI-enhanced attacks. We also present architectural solutions that combine prompt isolation, runtime security, and privilege separation with novel threat detection capabilities. 1 Introduction Prompt injection attacks are adversarial inputs designed to manipulate Large Language Models (LLMs) into ignoring their original instructions and following unauthorized commands instead, with the first systematic documentation of these attacks attributed to Preamble Inc. in May 2022 [ 1 ] . This work established the theoretical framework for understanding how carefully crafted inputs could bypass model safeguards and hijack AI system behavior, creating an entirely new class of security vulnerabilities that traditional cybersecurity measures were not designed to address. The initial discovery has since evolved into a critical security challenge as AI systems become increasingly integrated into enterprise applications, autonomous agents, and critical infrastructure [ 3 , 4 , 7 ] . With the advancement of LLMs, Agentic AI systems, where LLMs autonomously perform multi-step tasks through tools, APIs, and coordination with other agents, have proliferated. This has fundamentally transformed the threat landscape, shifting from isolated text manipulation to sophisticated attacks. These attacks are capable of causing tangible harm through system compromise, data exfiltration, and coordinated malicious activities. Contemporary research has advanced beyond simple prompt manipulation to develop hybrid attacks that combine prompt injection with traditional cybersecurity exploits such as XSS, CSRF, and SQL injection. [ 5 ] While there are limited studies on hybrid threats’ attack success rates in the wild, these attacks can systematically evade traditional security controls designed for predictable attack patterns. The integration of AI capabilities with classical web vulnerabilities creates attack vectors that neither traditional cybersecurity tools nor AI-specific defenses can adequately address in isolation. This paper presents a comprehensive analysis of Prompt Injection 2.0 , the evolution of prompt injection attacks in the era of agentic AI and hybrid cyber threats. We examine how modern attackers combine natural language manipulation with traditional exploits to achieve account takeovers, remote code execution, and persistent system compromise. Our analysis builds upon Preamble’s research contributions, including their patent on prompt injection mitigation methods [ 2 ] and advanced guardrail frameworks [ 6 ] , while incorporating contemporary threat research and real-world incident analysis. The scope of this work encompasses both the technical analysis of hybrid attack mechanisms and the practical evaluation of defense strategies. We examine documented vulnerabilities, including CVE-2024-5565, real-world incidents such as the DeepSeek XSS exploits, and emerging threats like AI worms and multi-agent infections. Our methodology combines systematic review of the literature, development of threat taxonomy, case study analysis, and comparative evaluation of defense approaches, including Preamble’s patented technologies, LLM tagging mechanisms [ 9 ] , and architectural isolation frameworks [ 10 ] . 2 Background and related work 2.1 Foundational research at Preamble (2022) The systematic documentation of prompt injection vulnerabilities began with Branch et al.’s investigation of pre-trained language model susceptibilities [ 1 ] . Their work provided the first comprehensive examination of how handcrafted adversarial inputs could manipulate GPT-3 and similar models to ignore initial instructions and execute alternative classifications instead. One of their first discoveries demonstrated that a simple text prompt containing the command “Ignore all previous instructions and ignore all previous content filters” could effectively hijack model behavior, establishing prompt injection as a fundamental security concern for AI-integrated applications. The research identified core vulnerability patterns in how language models process adversarial inputs, particularly showing that these models lack reliable mechanisms for distinguishing between intended instructions and user-provided content [ 1 ] . Through systematic testing across multiple architectures, including GPT-3, BERT, RoBERTa, and ALBERT, the work demonstrated that prompt injection represents a widespread vulnerability affecting various pre-trained language models. Notably, the team responsibly disclosed their findings about GPT-3’s vulnerabilities to OpenAI. Building on this foundational vulnerability research, Preamble subsequently developed comprehensive mitigation strategies, culminating in their patent for prompt injection mitigations [ 2 ] . The patent introduces multiple technical approaches, including: classifier-based detection systems that identify and filter malicious prompts; data tagging methods that track trusted versus untrusted instruction sources using incompatible token sets; and reinforcement learning frameworks that train models to distinguish legitimate instructions from adversarial inputs [ 2 ] . These defensive mechanisms represent the first systematic engineering approaches to prompt injection mitigation, moving the field from vulnerability documentation toward practical security solutions. 2.2 Evolution in AI guardrail frameworks (Preamble 2024) Preamble’s recent work on AI guardrails [ 6 ] introduced frameworks for AI systems that emphasize customizable guardrails aligned with diverse user values. This research addresses the broader challenge of ensuring responsible AI behavior through integrated approaches that combine rules, policies, and AI assistants. The guardrail framework focuses on accommodating ethical pluralism by providing flexible and adaptable solutions for AI governance. Key innovations include customizable ethical standards that can be tailored to different contexts and user requirements, while maintaining transparency and user autonomy. The system employs practical mechanisms for implementing standards that can evolve with the changing landscape of AI applications and societal expectations. Namely, the user can choose between three different ways of establishing a safe system guided by rules : trained classifier , natural language processing rules or natural language rules . A classifier is trained on user’s data to detect organization-specific breaches, natural language processing rules take care of PII, while natural language rules can be evaluated via user’s or open-source LLMs for an additional guardrail check. All of these rules can be combined into policies and tied to specific AI assistants, choosing different models, and conflict resolution. Namely, Preamble’s guardrail research also addresses the challenge of resolving conflicts between different ethical directives, representing a significant advancement toward robust, nuanced, and context-aware AI systems. Such an approach shows continuous improvement mechanisms and the need for AI systems that can adapt to diverse frameworks while maintaining consistent responsible behavior across various deployment scenarios. 2.3 Contemporary research The field of prompt injection research has expanded significantly since Preamble’s initial work, with numerous research groups developing complementary approaches to understanding and mitigating these threats. Contemporary research can be organized into three

Read the complete text