Safety Guardrails for LLM-Enabled Robots

Zachary Ravichandran, Alexander Robey, Vijay Kumar, George J. Pappas, Hamed Hassani · 2025

Read Safety Guardrails for LLM-Enabled Robots by Zachary Ravichandran, Alexander Robey, Vijay Kumar, George J. Pappas, Hamed Hassani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07885.txt.

Introduction

Zachary Ravichandran and four co-authors present a safety system for robots that take orders from large language models, in a 2025 paper. Such robots can follow harmful plans if the model makes an error or if a user tries to trick it. Older robot safety tools do not handle these language based risks well.

Their system, called RoboGuard, works in two steps. First, a trusted model turns broad safety rules into clear limits that fit the robot's actual setting. Then a control step checks each plan against those limits and blocks unsafe parts. In tests, unsafe plans fell from over ninety percent to below three percent, while safe plans still worked.

Catalog record

CREATORZachary Ravichandran, Alexander Robey, Vijay Kumar, George J. Pappas, Hamed Hassani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-07885

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Safety Guardrails for LLM-Enabled Robots Thanks: corresponding author: Zachary Ravichandran, Alexander Robey, Vijay Kumar, George J. Pappas, Hamed Hassani Abstract Although the integration of large language models (LLMs) into robotics has unlocked transformative capabilities, it has also introduced significant safety concerns, ranging from average-case LLM errors ( e.g., hallucinations) to adversarial jailbreaking attacks, which can produce harmful robot behavior in real-world settings. Traditional robot safety approaches do not address the novel vulnerabilities of LLMs, and current LLM safety guardrails overlook the physical risks posed by robots operating in dynamic real-world environments. In this paper, we propose RoboGuard , a two-stage guardrail architecture to ensure the safety of LLM-enabled robots. RoboGuard first contextualizes pre-defined safety rules by grounding them in the robot’s environment using a root-of-trust LLM, which employs chain-of-thought (CoT) reasoning to generate rigorous safety specifications, such as temporal logic constraints. RoboGuard then resolves potential conflicts between these contextual safety specifications and a possibly unsafe plan using temporal logic control synthesis, which ensures safety compliance while minimally violating user preferences. Through extensive simulation and real-world experiments that consider worst-case jailbreaking attacks, we demonstrate that RoboGuard reduces the execution of unsafe plans from 92% to below 2.5% without compromising performance on safe plans. We also demonstrate that RoboGuard is resource-efficient, robust against adaptive attacks, and significantly enhanced by enabling its root-of-trust LLM to perform CoT reasoning. These results underscore the potential of RoboGuard to mitigate the safety risks and enhance the reliability of LLM-enabled robots. † † footnotetext: Correspondence to [email protected] I Introduction The field of robotics has been fundamentally transformed by robotic foundation models, which have enabled breakthroughs in applications such as manipulation [ 46 , 3 , 14 , 31 ] , autonomous driving [ 43 , 21 , 75 , 79 ] , service robotics [ 67 , 29 , 27 ] , robot-assisted surgery [ 37 , 74 ] , and navigation [ 78 , 77 , 89 ] . The capability sets of these models have grown so rapidly that numerous robotic systems controlled by foundation models—including the Unitree Go2, Agility Digit, and Figure 01—are now available directly to consumers and actively deployed in homes, warehouses, and offices [ 93 , 85 ] . Moreover, in the years ahead, current scaling trends and architectural advances (see, e.g., [ 73 , 60 ] ) indicate that the next generation of AI-enabled robots will automate labor traditionally performed by humans [ 2 , 81 , 26 ] . It is therefore essential that the safety of any technology at the intersection of AI and robotics be rigorously scrutinized as these systems are increasingly deployed collaboratively alongside humans [ 36 ] . Robot safety has traditionally been viewed through formalisms such as temporal logic for model checking, and control barrier functions for robust control [ 96 , 4 , 57 , 47 , 69 ] . The success of these approaches often relies on precise safety specifications in static and non-adversarial environments. However, as LLM-enabled robots become increasingly capable via both natural language understanding [ 14 ] and interaction with high-fidelity world models [ 95 ] , notions of robot safety for LLM-generated robot plans have in turn become increasingly contextual and harder to define and enforce. For example, while a delivery robot operating in a major city should avoid certain areas such as construction sites, this restriction may not be a concern for a disaster response robot tasked with assisting victims [ 39 ] . Such scenarios illustrate the need for safety mechanisms that can reason about the context of LLM-enabled robots to determine whether a particular plan is safe. Fig. 1: Overview of RoboGuard . Online, a system designer first configures RoboGuard with safety rules and a robot description (A). Online, RoboGuard first receives the robot’s world model, and it uses this world model to produce grounded safety specifications (B). Next, RoboGuard synthesizes these specifications with the LLM-generated plan, in a manner that ensures safety while maximally respecting the proposed plan (C). Because large language models (LLMs) are central to the reasoning abilities of contextually-aware robots, their use in robotics opens the door to new risks. As a standalone technology, LLMs are trained through a process called alignment to generate content that aligns with human values ( e.g., not provide bomb-building instructions) [ 59 , 66 ] . While alignment techniques have reduced the tendency of LLMs to produce objectionable text, malicious users can still elicit such text via jailbreaking attacks, which produce prompts that bypass alignment [ 86 , 97 , 12 ] In response, research in the AI safety community has proposed mitigation strategies that include safety filters [ 33 , 34 ] , defense algorithms [ 98 , 70 ] , and protocols to detect malignant capabilities [ 13 , 11 , 24 ] . While several classes of attacks remain effective against state-of-the-art models [ 72 , 44 ] , existing defenses have greatly reduced the susceptibility of LLMs to producing undesired content. Despite the notable focus on safety in the language modeling community, until recently, there was little consensus regarding the extent to which LLM-enabled robots inherit the vulnerabilities of LLM chatbots. This is in part due to the fact that existing approaches to alignment tend to focus on the generation of text, rather than context-dependent robotic actions that evolve over time. However, very recent work has shown that jailbreaking attacks on LLM-enabled robots are remarkably effective in producing dangerous actions ( e.g. , colliding with humans, blocking emergency exits, and obtaining weapons) from a variety of commercial and academic robots [ 71 ] . This finding indicates that general-purpose solutions are needed to mitigate malicious attacks in application-dependent settings, particularly given the distinct possibility of these attacks causing harm in the physical world. This paper provides a novel and general safety architecture that address the unique safety challenges of using LLMs in robotics. To motivate our approach, we first propose desiderata for defenses against attacks on LLM-enabled robots. These desiderata comprise the following properties – contextual attack mitigation , applicability , utility , and efficiency – which together outline desirable traits for candidate safeguard approaches. We then propose RoboGuard , a two-stage guardrail architecture for ensuring the safety of LLM-enabled robots. As illustrated in Figure 1 , RoboGuard is configured offline with high-level safety rules and a robot description (Figure 1 .A), which makes RoboGuard adaptable to various robot platforms and LLM planning instantiations. Online, RoboGuard receives the robot’s world model and LLM-proposed plan, and it returns a safety-respecting plan. This is achieved with the following two innovations. RoboGuard ’s first key innovation concerns reasoning for safety. RoboGuard employs a root-of-trust LLM that reasons over the robot’s world model and high-level safety rules to produce rigorous and grounded safety specifications via context-aware chain-of-thought generation (Figure 1 .B). By decoupling potentially malicious prompts from pre-defined safety rules, the root-of-trust LLM is robust to adversarial prompts. RoboGuard ’s second key innovation resolves potential conflicts between the inferred safety specifications and the potentially malicious LLM-generated plan. RoboGuard allows for arbitrary LLM-planning APIs, so it first translates the LLM-generated plan into a contextually grounded temporal logic specification. It then employ tools from controller synthesis to

Read the complete text