Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis
Jeffrey Yang Fan Chiang, Seungjae Lee, Jia-Bin Huang, Furong Huang, Yizheng Chen · 2025
Read Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis by Jeffrey Yang Fan Chiang, Seungjae Lee, Jia-Bin Huang, Furong Huang, Yizheng Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20383.txt.
Introduction
Chiang and co-authors present a 2025 security study on why web-browsing AI agents fall for attacks more easily than plain chat models. Both rest on the same safety-tuned brains, yet agents roam pages and take actions. The team broke agent design into parts and graded each part's risk. Simple success rates hid the weak points, they argue.
Three traits drove the extra risk. Weaving user goals into the system prompt blurred the line between orders and data. Multi-step action chains gave attacks more footholds. Page observation fed hostile text straight into the loop. The authors call for agent-specific guardrails, not just model-level tuning, before such tools spread wider.
Catalog record
| CREATOR | Jeffrey Yang Fan Chiang, Seungjae Lee, Jia-Bin Huang, Furong Huang, Yizheng Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20383
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis Jeffrey Yang Fan Chiang, Seungjae Lee, Jia-Bin Huang, Furong Huang, Yizheng Chen Abstract Recent advancements in Web AI agents have demonstrated remarkable capabilities in addressing complex web navigation tasks. However, emerging research shows that these agents exhibit greater vulnerability compared to standalone Large Language Models (LLMs), despite both being built upon the same safety-aligned models. This discrepancy is particularly concerning given the greater flexibility of Web AI Agent compared to standalone LLMs, which may expose them to a wider range of adversarial user inputs. To build a scaffold that addresses these concerns, this study investigates the underlying factors that contribute to the increased vulnerability of Web AI agents. Notably, this disparity stems from the multifaceted differences between Web AI agents and standalone LLMs, as well as the complex signals---nuances that simple evaluation metrics, such as success rate, often fail to capture. To tackle these challenges, we propose a component-level analysis and a more granular, systematic evaluation framework. Through this fine-grained investigation, we identify three critical factors that amplify the vulnerability of Web AI agents; (1) embedding user goals into the system prompt, (2) multi-step action generation, and (3) observational capabilities. Our findings highlights the pressing need to enhance security and robustness in AI agent design and provide actionable insights for targeted defense strategies. 0 0 footnotetext: Project website: https://vulnerable-ai-agents.github.io A Preprint Warning: this paper contains potentially harmful text. 1 Introduction Figure 1: Web AI agents exhibit a significantly higher jailbreak rate (46.6%) compared to standalone LLMs (0%), highlighting their increased vulnerability in real-world deployment. Recent advancements in Large Language Models (LLMs) have demonstrated impressive reasoning capabilities and proficiency in solving complex problems. These capabilities are increasingly being extended to multi-step tasks, driving the evolution of LLM-based AI agent systems ( ????? ). One such system is the Web (browser) AI agent, which integrates an LLM with software tools and APIs to execute sequences of actions aimed at achieving specific goals within a web environment. These agents leverage LLM capabilities for planning ( ? ), reflection ( ? ), and effective tool utilization ( ?? ), enabling more autonomous and adaptive web-based interactions. Many previous studies ( ??? ) have highlighted significant advancements in autonomous web agents. However, despite their promising potential, their safety and security vulnerabilities have not yet been systematically studied. Given their direct integration with web browsers, these agents could be exploited to distribute malware or send phishing Emails to extract personal information, posing serious security risks (as shown in Fig. 1 ). In this study, we highlight the heightened vulnerability of Web AI agent frameworks to jailbreaking compared to traditional LLMs. Through comprehensive experiments, we demonstrate that web agents, by design, exhibit a significant higher susceptibility to following malicious commands due to fundamental component-level differences from standalone LLMs. Notably, while a standalone LLM (such as a regular chatbot) refuses malicious requests with a 0% success rate, the Web AI agent follows them at a rate of 46.6% (Fig. 1 ). Importantly, we identify three primary factors contributing to the increased vulnerabilities of Web AI agents: (1) Directly embedding user input into the LLM system prompt, (2) Generating actions in a multi-turn manner, and (3) Processing observations and action histories, which increases the likelihood of executing harmful instructions and weakens the system’s ability to assess risks. Additionally, we find that mock-up testing environments may inadvertently distort security evaluations by oversimplifying real-world interactions, potentially leading to misleading conclusions about an agent’s robustness. To better understand the heightened vulnerability of Web AI agents to jailbreaking and their increased susceptibility to executing malicious commands, we introduce a 5-level fine-grained ablative metric that goes beyond the conventional binary assessments of LLM vulnerabilities, offering a more nuanced evaluation of jailbreak signals. Ultimately, our study raises awareness of the security challenges posed by Web AI agents and advocates for proactive measures to design safer, more resilient agent frameworks. Our contributions: • Empirical evidence of Web AI agents’ heightened vulnerability: We systematically compare Web AI agents with standalone LLM chatbots, revealing that Web AI agents are significantly more susceptible to jailbreaking and executing malicious commands. • Root cause analysis of Web AI agent vulnerabilities: We investigate the design-level differences between Web AI agents and standalone LLMs, identifying key factors—such as system prompt manipulation, multi-turn action generation, and reliance on historical observations—that contribute to their increased vulnerability. • A fine-grained evaluation protocol for jailbreak susceptibility: We introduce a structured, five-level harmfulness evaluation framework that goes beyond binary assessments, enabling a more detailed and nuanced analysis of Web AI agent vulnerabilities. • Actionable insights for targeted defense strategies: Based on our findings, we provide recommendations for mitigating security risks in Web AI agents, focusing on improving system prompt handling, action generation mechanisms, and contextual awareness in agent architectures. 2 Related Works LLMs in Web AI agent systems. Recent advancements in LLM techniques have expanded their role in AI agent systems, enabling them to generate and execute actions ( ???? ). Among various applications, web browsing has emerged as a key domain for AI agents ( ???? ), where LLMs assist users in tasks ranging from simple navigation to more complex operations such as booking flights and interacting with web-based maps ( ??? ). To support systematic evaluation, researchers have introduced several benchmarks, including simulated and self-hostable webpage environments, to comprehensively assess the performance of Web AI agents ( ??? ). Security aspects of AI agents. AI agents assist humans in daily computer tasks, often requiring access to private data and sensitive information, making their security and trustworthiness paramount. Research in this area has focused on identifying security vulnerabilities, attack methods, and defense mechanisms. Several studies highlight significant security risks. ? demonstrated that injection attacks can lead to privacy leaks, while ? examined how pop-up blocks distract agents and manipulate them into executing attacker-intended actions. Additionally, ? revealed how indirect prompt injection attacks can coerce agents into performing malicious actions. On the defense side, researchers have proposed various mitigation mechanisms. ? introduced a security analyzer that enforces strict constraints on agent actions. ? designed a secure LLM system that separates planning from execution, using information flow control and security labels to filter untrusted inputs. ? emphasized the importance of session management, sandboxing, and encryption to enhance AI agent security in real-world applications. To systematically assess vulnerabilities, several benchmarks have been developed. ? evaluated LLM robustness against jailbreak attacks. ? introduced a dynamic framework for testing AI agent security against prompt injection attacks. ? created a benchmark covering over ten prompt injection attacks, including memory poisoning and Plan-of-Thought backdoor attacks. These studies consistently show that LLM-based Web AI agents are significantly more susceptible to jailbreak attacks than standalone LLM systems ( ??? ). However, the underlying causes of this heightened vulnerability remain unclear. Existing evaluations, primarily focus on binary jailbreak success or failure, failing to capture nuanced agent behaviors under varying levels of harmful influence. This work aims to provide a fine-grained analysis of Web AI agent vulnerabilities, identifying the specific components and design choices that contribute to their susceptibility. 3 Web AI Agent System A Web AI agent system, powered by an LLM, operates autonomously