RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents
Doguhuan YekePurdue University [email protected], Yanming ZhouPurdue University [email protected], Leo Y. LinPurdue University [email protected], Hongyu CaiPurdue University [email protected] Antonio BianchiPurdue University [email protected] Z. Berkay CelikPurdue University [email protected] · 2026
Read RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents by Doguhuan YekePurdue University [email protected], Yanming ZhouPurdue University [email protected], Leo Y. LinPurdue University [email protected], Hongyu CaiPurdue University [email protected] Antonio BianchiPurdue University [email protected] Z. Berkay CelikPurdue University [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19328.txt.
Introduction
Doguhuan Yeke and colleagues build RoboJailBench, the first standard benchmark for jailbreak attacks on embodied robots and vehicles. Earlier embodied safety tests used ad-hoc data and narrow metrics, stressing attack success while ignoring whether benign commands still work. Their benchmark adds a taxonomy of 18 violation types drawn from standards, rules, and incidents, plus paired hostile and benign goals that measure security and usefulness together. Four attacks and two defenses on leading vision-language models show how the field can now compare defenses fairly.
Catalog record
| CREATOR | Doguhuan YekePurdue University [email protected], Yanming ZhouPurdue University [email protected], Leo Y. LinPurdue University [email protected], Hongyu CaiPurdue University [email protected] Antonio BianchiPurdue University [email protected] Z. Berkay CelikPurdue University [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49180 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19328
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents Doguhuan YekePurdue University [email protected], Yanming ZhouPurdue University [email protected], Leo Y. LinPurdue University [email protected], Hongyu CaiPurdue University [email protected] Antonio BianchiPurdue University [email protected] Z. Berkay CelikPurdue University [email protected] Abstract Recent advances in Vision-Language Models (VLMs) facilitate a new class of embodied AI systems, where these models are integrated into physical platforms, e.g. robots and autonomous vehicles, to interpret visual scenes and execute natural language commands in diverse environments. Previous research has introduced jailbreak attacks and defenses for embodied AI. Their evaluations, however, rely on ad-hoc datasets, limited metrics, and emphasize attack success while neglecting the trade - off between security and the ability to follow benign commands. Existing benchmarks and evaluation frameworks either target traditional chat - based models or focus on non-adversarial safety evaluation for embodied AI; neither captures the adversarial risks, inputs, consequences, and evaluation criteria necessary for jailbreak attacks in embodied AI systems. In this paper, we address this gap with RoboJailBench , which consists of three core components. We first establish a security taxonomy derived from ISO standards, regulatory rules, and documented incidents. This effort yields 18 categories of security violation consequences for embodied AI. We then introduce an intent contrast dataset pipeline that augments existing datasets with paired adversarial and benign goals to measure both security and utility. Lastly, we provide an evolving repository with standardized metrics and a unified process for assessing and integrating new attacks and defenses. With this benchmark, we construct a new taxonomy-balanced dataset and augment five existing datasets. We integrate four attacks and two defenses to evaluate their performance on leading embodied VLMs. This benchmark provides the first standardized evaluation framework for jailbreak attacks in embodied AI and supports future research. We release our code, datasets, and artifacts, and maintain a leaderboard at https://purseclab.github.io/benchmark-for-robotics-security/ . * * footnotetext: Equal contribution. 1 Introduction The integration of Vision-Language Models (VLMs) into physical platforms such as robots creates embodied AI systems that can process multimodal environmental context and execute complex natural language instructions (e.g., Gemini Robotics-ER 1.5 [ 10 ] , Qwen2.5-VL [ 20 ] ). However, this cyber-physical intersection introduces severe safety vulnerabilities. Recent attack frameworks (e.g., BadRobot [ 29 ] and RoboPAIR [ 22 ] ) have shown that physical jailbreaks are feasible on embodied AI. Following these studies, recent work introduced defenses against these jailbreak attacks [ 24 , 21 ] . However, these initial efforts of attacks and defenses have three main limitations. (1) These studies evaluate their attacks or defenses using limited and ad-hoc datasets. (2) The defense methods focus on only Attack Success Rate (i.e., blocking or accepting adversarial commands), while neglecting the robot’s ability to execute safe commands. (3) As of the time of writing, there is no benchmark to evaluate all attacks and defenses with standardized settings and metrics. The AI safety community has made significant strides in standardizing evaluations for digital harms. Frameworks such as JailbreakBench [ 4 ] and HarmBench [ 17 ] have successfully replaced ad-hoc testing with rigorous, taxonomy-driven evaluations for text-based models to ensure standardized threat models and reproducible scoring. More recently, AgentHarm [ 2 ] extended this paradigm to software agents, demonstrating the critical need to evaluate malicious intent executed through digital tool use and APIs. Another line of work has made important progress on robustness and non-adversarial safety in embodied AI for robotics [ 5 , 28 , 7 , 25 , 30 ] , including navigation robustness under perceptual and dynamics corruptions [ 5 ] , safe task planning and risk-aware reasoning [ 28 , 7 ] , and physical risk or plan-safety evaluation through diagnostic scenarios [ 25 , 30 ] . However, these benchmarks are not designed to evaluate jailbreak robustness for embodied AI under concrete adversarial attacks and defenses. They also do not jointly measure whether defenses block adversarial instructions while preserving benign task utility, nor do they provide an embodiment-grounded taxonomy of adversarial consequences. To address this gap, we develop RoboJailBench , a comprehensive benchmarking framework for rigorously evaluating both attacks and defenses in embodied AI systems. First, we construct our security taxonomy through a systematic cross-referencing analysis of formal ISO safety standards [ 11 , 12 , 13 ] , Asimov’s Laws of Robotics [ 24 ] , and empirical incidents from the news [ 1 , 3 , 19 , 26 ] and prior work [ 14 ] . Next, we design an intent contrast dataset pipeline to capture the security-utility tradeoff. Given an image, we create a pair of prompts grounded in the physical environment of the image and specific to embodied AI models. These prompts are denoted as an adversarial goal, intended to bypass alignment, and a benign goal, intended for normal operation. Finally, we introduce an extensible evaluation framework that standardizes metrics and processes for evaluating attacks, defenses, and generating attack artifacts. In summary, our main contributions are as follows: • We develop an embodiment-grounded security taxonomy through a systematic cross-referencing analysis of formal safety standards, Asimov’s Laws, and incident reports. • We introduce an intent contrast dataset pipeline where each image is paired with an adversarial goal and a benign goal grounded in the physical environment and embodiment. We introduce one new dataset and augment five existing datasets using this pipeline to better capture and evaluate the security–utility tradeoff. • We introduce an extensible evaluation framework that provides standardized metrics and processes to deploy attacks, defenses, and generate jailbreak artifacts. We integrated four attacks and two defenses from recent works and evaluated them on leading embodied VLMs. • We release our code, jailbreak artifacts, and datasets, and maintain a leaderboard at https://purseclab.github.io/benchmark-for-robotics-security/ to support tracking and research in adversarial attacks and defenses on embodied AI models. Table 1: Comparison of embodied datasets used in RoboJailBench . Dataset Focus # Images Description DROID Manipulation 100 In-the-wild manipulation demos across diverse scenes/tasks. RH20T Contact-rich manipulation 100 Multimodal demos with RGB-D, force, audio, and actions. RoboVQA Long-horizon reasoning 100 VQA for task progress, planning, and future actions. Robo2VLM Robot VQA 100 Trajectory-derived VQA for spatial and interaction reasoning. NVIDIA PhysicalAI-AV Autonomous driving 100 In-the-wild multi-sensor driving data across diverse regions. RJB-Instructions Security taxonomy 90 Balanced adversarial instructions covering physical safety risks. 2 Background and Related Work Adversarial Attacks and Defenses for Embodied AI Systems. With the rapid advancement of embodied AI systems [ 10 ] , and the emergence of attacks [ 22 , 29 ] and defenses [ 21 , 24 ] tailored specifically to them, existing benchmarks fall short of capturing the taxonomy of adversarial consequences, inputs, and nuances unique to embodied AI systems. Current research on safety evaluation in robotics similarly fails to account for adversarial objectives or the security–utility tradeoff associated with concrete attack and defense techniques. As a result, there remains a gap in a standardized, comprehensive benchmark that systematically derives adversarial goals for embodied AI systems and evaluates adversarial attacks and defenses in a manner grounded in the realities of embodied operation. Adversarial Benchmarks in LLMs and VLMs. Prior work in adversarial benchmarking has underscored the need for comprehensive benchmarks to evaluate adversarial attacks and defenses [ 4 , 17 , 2 , 27 ] .