SafeArena : Evaluating the Safety of Autonomous Web Agents
Ada Defne Tur, Nicholas Meade, Xing Han Lù, Alejandra Zambrano, Arkil Patel, Esin Durmus, Spandana Gella, Karolina Stańczak, Siva Reddy · 2025
Read SafeArena : Evaluating the Safety of Autonomous Web Agents by Ada Defne Tur, Nicholas Meade, Xing Han Lù, Alejandra Zambrano, Arkil Patel, Esin Durmus, Spandana Gella, Karolina Stańczak, Siva Reddy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04957.txt.
Introduction
Ada Defne Tur and eight colleagues present a 2025 paper testing whether AI web agents can be misused for harm. As agents grow skilled at browsing and acting online, they might also post misinformation or run scams when asked. The team built SafeArena, a benchmark of 250 harmless and 250 harmful tasks spread across four websites, with harms sorted into five classes from misinformation to cybercrime.
Four leading agent systems were tried. Compliance with malicious requests was uncomfortably high: the strongest model completed about thirty-five percent of harmful tasks, and a second model about twenty-seven percent. A four-level risk framework helps grade agent behavior. The authors argue web agents urgently need safety alignment before wider release.
Catalog record
| CREATOR | Ada Defne Tur, Nicholas Meade, Xing Han Lù, Alejandra Zambrano, Arkil Patel, Esin Durmus, Spandana Gella, Karolina Stańczak, Siva Reddy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-04957
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SafeArena : Evaluating the Safety of Autonomous Web Agents Ada Defne Tur, Nicholas Meade, Xing Han Lù, Alejandra Zambrano, Arkil Patel, Esin Durmus, Spandana Gella, Karolina Stańczak, Siva Reddy Abstract LLM-based agents are becoming increasingly proficient at solving web-based tasks. With this capability comes a greater risk of misuse for malicious purposes, such as posting misinformation in an online forum or selling illicit substances on a website. To evaluate these risks, we propose SafeArena , the first benchmark to focus on the deliberate misuse of web agents. SafeArena comprises 250 250 safe and 250 250 harmful tasks across four websites. We classify the harmful tasks into five harm categories— misinformation , illegal activity , harassment , cybercrime , and social bias , designed to assess realistic misuses of web agents. We evaluate leading LLM-based web agents, including GPT-4o, Claude-3.5 Sonnet, Qwen-2-VL 72B, and Llama-3.2 90B, on our benchmark. To systematically assess their susceptibility to harmful tasks, we introduce the Agent Risk Assessment framework that categorizes agent behavior across four risk levels. We find agents are surprisingly compliant with malicious requests, with GPT-4o and Qwen-2 completing 34.7 34.7 % and 27.3 27.3 % of harmful requests, respectively. Our findings highlight the urgent need for safety alignment procedures for web agents. Our benchmark is available here: https://safearena.github.io Warning: This paper contains examples that may be offensive or upsetting. † † affiliationnotice: Equal contribution \icmlCoreContribution 1 Introduction Large Language Models (LLMs; Brown et al. 2020 ; Touvron et al. 2023 ; Groeneveld et al. 2024 ; Gemma Team et al. 2024 ) are increasingly being deployed as agents for interactive tasks, such as software engineering ( Jimenez et al. 2024 ; Liu et al. 2024 ) , web navigation ( Zhou et al. 2024 ; Koh et al. 2024 ) , and computer use ( Anthropic 2024 ; OpenAI 2025 ) . Open-source frameworks ( Wang et al. 2024b ; Drouin et al. 2024 ; Zheng et al. 2024 ) are aiding the widespread real-world adoption of these agents by allowing LLMs, such as ChatGPT, to be easily integrated with web browsers, code interpreters, and external knowledge sources. While this real-world use carries great promise, the direct access to environments, alongside the high level of autonomy granted to these agents, introduces new safety concerns. A considerable amount of preliminary research on LLM-based agent safety has focused on investigating whether agents can complete harmful tasks in text -based environments ( Andriushchenko et al. 2024 ; Debenedetti et al. 2024 ; Zou et al. 2024 ) . Benchmarks released in this area evaluate whether, via a series of function calls, LLM-based agents can perform harmful tasks, such as writing a blog post containing misinformation. While these works take an important first step towards evaluating malicious misuse of LLM-based agents, they often make simplifying assumptions, such as the use of synthetic tools, which limit their real-world relevance. Furthermore, the rapid rise in agent capability, observed on tasks like web navigation ( Kim et al. 2023 ) , highlights the need for more sophisticated LLM agent safety evaluations which require navigating more ubiquitous graphical user interfaces. Figure 1: Overview of example tasks and user-agent interaction within SafeArena . Left : A human-curated example illustrating an intent to spread misinformation on a public forum. Right : An example created by our human-in-the-loop process where the intent is to promote biased discussions. In response to the above limitations, we introduce SafeArena , a benchmark for evaluating autonomous web agent safety across five harm categories— misinformation , harassment , illegal activity , cybercrime , and social bias —on four realistic websites. Our benchmark is comprised of 250 pairs of safe and malicious tasks (giving a total of 500 tasks), which allows us to simultaneously evaluate the capability of agents to complete benign and harmful web tasks. We design tasks in our benchmark to assess whether web agents can be used to complete real-world malicious tasks. For example, a malicious actor may instruct a web agent to create a misleading forum post to spread misinformation or submit a code repository issue encouraging discriminatory discussions (see Figure 1 ). We evaluate five strong LLM web agents on our benchmark using the A gent RI sk A ssessment ( ARIA ) framework, which we propose for assessing harmful web agent behavior across four risk levels: immediate refusal, delayed refusal, attempted but failed task execution, and successful completion. Given that these agents are built on top of LLMs that have undergone multiple stages of safety alignment, one might expect them to inherently reject harmful requests. However, our results reveal notable safety vulnerabilities, demonstrating that alignment transfers poorly to web tasks. We observe that current web agents complete an alarming number of harmful tasks—for example, GPT-4o successfully completes 34.7 % of 250 harmful tasks. Furthermore, we find most agents at least attempt , without refusal, a substantial number of harmful tasks. For instance, GPT-4o and Claude-3.5-Sonnet attempt or complete 68.7 % and 36.0 % of the harmful tasks, respectively. Finally, we find that by decomposing a harmful request into a sequence of benign-looking substeps (e.g., open the forum , click the post button , etc.) even the safest agent in terms of task completion, Claude-3.5-Sonnet, can easily be jailbroken for all tasks it initially refused. Our findings highlight the urgent need for safety alignment procedures for web agents beyond those applied to the underlying LLMs. By introducing SafeArena , we provide a crucial benchmark to support and accelerate the on-going efforts to design safe and aligned agents. 2 Related Work 2.1 Autonomous Web Agents Designing agents using neural networks to solve web-based tasks has been an ongoing research topic, with works by Shi et al. 2017 ; Liu et al. 2018 pioneering the use of dedicated environments for evaluating such agents. Subsequent works explored approaches to fine-tune the models using screen-based ( Shaw et al. 2023 ) and HTML-based approaches ( Furuta et al. 2024 ) . As the capabilities of fine-tuned agents increased on early benchmarks, later works focused on designing human-demonstration benchmarks to evaluate them on complex real-world tasks ( Deng et al. 2023 ; ù et al. 2024 ) as well as realistic self-contained web environments ( Zhou et al. 2024 ; Koh et al. 2024 ; Pan et al. 2024 ; Drouin et al. 2024 ) . Moreover, with the rapidly improving capabilities of LLMs, recent works have sought to design agents using instruction-tuned LLMs ( Sodhi et al. 2024 ; Chezelles et al. 2024 ) . Although such models have been aligned for conversational tasks, they may not have encountered intents with substantial use of screenshots, accessibility trees, and HTML pages during post-training. Thus, it is unclear whether they can behave safely in out-of-domain tasks like autonomous web navigation. 2.2 LLM Agent Safety Given the rapid advancements in the capabilities of LLM agents, recent research has focused on assessing the associated safety risks. Andriushchenko et al. 2024 investigates whether LLM-based agents can complete harmful tasks in text-based environments using synthetic tools. Similarly, AgentDojo ( Debenedetti et al. 2024 ) evaluates the susceptibility of LLM agents to both direct harmful requests and adversarial attacks. ToolEmu ( Ruan et al. 2024 )