RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots
Nima Dorzhiev · 2026
Read RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots by Nima Dorzhiev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28649.txt.
Introduction
Nima Dorzhiev, writing alone, attacks robots through their senses. Language-model-controlled robots built on the ROS 2 framework take instructions from what they see and hear, so a malicious sign, a spoken phrase, or faked sensor data might hijack behavior. Dorzhiev's RIPA study is, to his knowledge, the first systematic multi-channel test of such sensory prompt injection.
Across one hundred runs per variant and five models from four families, vulnerability proved model-specific rather than size-driven. One seventy-billion-parameter model fell to every variant, while smaller models shrugged off direct overrides. The smallest matched the large model's weakness. A hybrid semantic firewall stopped all known patterns yet let about a tenth of obfuscated attacks through. A third channel, poisoning the robot's LiDAR-derived picture of its surroundings, succeeded completely on one major model.
Catalog record
| CREATOR | Nima Dorzhiev |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28649
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots Nima Dorzhiev Abstract We present RIPA, to our knowledge, the first systematic multi-channel empirical study of prompt injection attacks delivered through the sensory pipeline of a ROS 2-based LLM-controlled robotic system. Across 100 independent runs per injection variant on five LLMs spanning four model families and parameter scales from ∼ \sim 4B to ∼ \sim 284B (DeepSeek-V4-Flash, Llama-3-8B-Instruct-Lite, Llama-3.3-70B-Instruct-Turbo, Qwen 2.5-7B-Instruct-Turbo, Gemma-3n-E4B), we identify model-specific vulnerability profiles that do not follow a monotonic scaling trend: Llama-3.3-70B-Instruct-Turbo exhibits a 100% attack success rate (ASR) across all injection variants, while Llama-3-8B-Instruct-Lite and Qwen 2.5-7B-Instruct-Turbo resist direct-override injection (0% ASR), and the smallest model evaluated (Gemma-3n-E4B, ∼ \sim 4B) matches the 70B model’s vulnerability profile—indicating that robustness is model-specific rather than scale-dependent. We propose a hybrid semantic firewall that achieves 0% ASR against known injection patterns with no false positives on a preliminary benign set (0/20 commands) but exhibits a 10.2% trial-weighted bypass rate (58/570 trials; N = 30 N=30 per payload across 19 obfuscation payloads) against adversarially obfuscated attacks, exposing a critical gap between rule-based and semantic defense layers. We further introduce three sensory injection channels: visual (Channel 1, via OCR), audio (Channel 2, via Whisper STT), and LiDAR sensor context poisoning (Channel 3). We show that Channel 3, which injects fabricated obstacle data into the robot environment-state representation at the LLM system-prompt level, achieves 100% ASR across all variants on DeepSeek-V4-Flash. We also contribute a firewall bypass taxonomy spanning 19 obfuscation payloads across five categories. All code, data, and results are publicly available. I Introduction The integration of Large Language Models (LLMs) into robotic control systems represents a fundamental shift in how autonomous agents interpret and execute tasks. Rather than relying on handcrafted state machines or narrow task-specific models, modern robotic architectures increasingly route natural language instructions through LLMs, which translate them into low-level actuator commands for execution by the robot. However, this flexibility introduces a critical security vulnerability: prompt injection. In a prompt injection attack, an adversary embeds malicious instructions within the content that the LLM processes as input, causing it to override its original task [ 1 ] . While extensively studied in LLM-based web agents [ 2 ] , their implications for physically embodied systems remain poorly understood. Critically, prior evaluations have relied on small sample sizes ( n=5 ), which our results suggest can obscure model-specific variability in vulnerability profiles. We use the term prompt injection broadly to refer to any adversarial manipulation of inputs that are ultimately serialized into the LLM prompt, including sensor-derived state. We distinguish between indirect prompt injections delivered via sensory channels such as OCR or speech-to-text (STT), and context poisoning , in which fabricated environment-state representations (e.g., LiDAR scans) are injected at the system-prompt level rather than the user-prompt level. We consider adversaries capable of injecting content into sensor inputs (visual overlays, audio signals, or spoofed sensor data) without modifying the robot’s software internals. This study makes the following contributions: • We introduce RIPA, the first large-sample ( n\geq 100 ) empirical study of sensory-vector prompt injection in a ROS 2-based [ 11 ] LLM-controlled robotic system, mapping the full pipeline from physical object text through OCR to actuator commands. • We evaluate four injection conditions (one benign baseline and three attack variants, A1–A3) across 100 runs each on five LLMs spanning four model families and parameter scales from \sim 4B to \sim 70B, revealing model-specific—rather than scale-dependent—vulnerability profiles: Llama-3.3-70B-Instruct-Turbo exhibits 100% ASR across all variants, while the smallest model evaluated (Gemma-3n-E4B, \sim 4B) exhibits a comparably high vulnerability profile, and two mid-sized models (Llama-3-8B-Instruct-Lite, Qwen 2.5-7B-Instruct-Turbo) resist direct-override injection entirely. • We propose a hybrid semantic firewall that achieves a 0% ASR for known injection patterns. Evaluated against production firewall logic (dual-zone co-occurrence check plus the full keyword set), the firewall exhibits a 10.2% trial-weighted bypass rate (58/570 trials), with only 2 of 19 obfuscation payloads bypassing at the payload level (B1c, synonym substitution, 100%; B3d, hyphenated encoding, 93.3%); the business-context category (B2) is fully blocked by the dual-zone check. • We introduce two sensory injection vectors—OCR (visual) and Whisper STT (audio)—validated end-to-end through the ROS 2 pipeline, and show that semantic injection intent survives STT-layer transcription degradation. • We introduce a third sensory injection vector via LiDAR spoofing, in which fabricated LaserScan data poisons the robot’s environment-state representation at the LLM system-prompt level, achieving 100% ASR ( N=30 , DeepSeek-V4-Flash) across all three attack variants, including a threshold-based effect (A3: front sector clear but 7/8 sectors report obstruction, causing the LLM to halt despite a navigable path). • We recommend n\geq 100 as a practical baseline for stable ASR estimation, since run-to-run variability arises from randomized obfuscation payload sampling across trials, even under deterministic (temperature =0 ) LLM decoding. We report confidence intervals throughout, noting that the appropriate sample size may vary with the degree of stochasticity in the sampling procedure. II Related Work II-A Prompt Injection in LLM Agents Prompt injection attacks were first systematically characterized by Perez and Ribeiro [ 1 ] and have since been demonstrated across a range of LLM-integrated applications [ 2 ] , including the selection of agentic tools [ 16 ] . AgentDojo [ 9 ] introduced a benchmark for evaluating indirect prompt injection in agentic pipelines; however, its scope is limited to software agents without physical actuators. II-B Adversarial Attacks on Robotic LLMs RoboPAIR [ 3 ] demonstrated the automated jailbreaking of LLM-integrated robots, achieving a 100% attack success rate. RIPA differs from RoboPAIR along several axes: RoboPAIR injects prompts directly rather than through a sensory pipeline, does not employ ROS 2, evaluates at n=5 , and does not examine obfuscated injection variants or firewall bypass resistance. BadRobot [ 4 ] demonstrated the jailbreaking of embodied LLMs through voice-based interactions, targeting physical manipulation robots. Zhang et al. [ 5 ] studied prompt injection against LLM-integrated mobile robots using GPT-4o and proposed secure prompting as a mitigation strategy (see also complementary mobile safety frameworks like [ 15 ] ). CHAI [ 6 ] introduced optimization-based adversarial prompts embedded in physical visual environments to hijack embodied AI agents. Xie and Wei-Kocsis [ 8 ] demonstrated structured backdoor attacks in LLM-mediated ROS 2 systems, complementing the inference-time injection results presented here. II-C Security of ROS-Based Systems Prior work on holistic threat modeling of LLM-enabled robotic systems [ 7 ] identified untrusted sensor inputs as the primary attack surface but did not provide empirical validation. Xing et al. [ 13 ] provide a comprehensive taxonomy of embodied AI vulnerabilities, including sensor spoofing and adversarial perception. Multi-robot compromise scenarios were explored in [ 14 ] , demonstrating that a single compromised LLM-controlled robot can propagate unsafe behavior across a fleet. Relative to this body of work, RIPA provides, to the best of our knowledge, the first joint experimental evaluation of visual, audio, and LiDAR sensory-vector injection within the ROS 2 Jazzy pipeline, evaluated across five LLMs spanning four model families. III Threat Model III-A System Model We consider an LLM-controlled robotic system implemented on ROS 2 Jazzy, where a robot equipped with a camera perceives its environment, reads text from objects via OCR, and forwards the extracted