\tool : Customizable Runtime Enforcement for Safe and Reliable LLM Agents
Haoyu Wang, Christopher M. Poskitt, Jun Sun · 2025
Read \tool : Customizable Runtime Enforcement for Safe and Reliable LLM Agents by Haoyu Wang, Christopher M. Poskitt, Jun Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18666.txt.
Introduction
Haoyu Wang, Christopher Poskitt, and Jun Sun built a rule system in 2025 to keep language model agents safe while they work. Agents act on their own across many fields, which brings risks of harmful acts, security holes, and legal breaches that model level guards do not fully stop.
Their language, called AgentSpec, lets users write rules with triggers, checks, and enforcement actions, applied as the agent runs. They built it into code agents, embodied agents, and self driving cars. It blocked more than ninety percent of unsafe code runs, all hazardous embodied actions, and every rule breach by the cars, with only milliseconds of delay. Rules written by a language model also scored well in their tests.
Catalog record
| CREATOR | Haoyu Wang, Christopher M. Poskitt, Jun Sun |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18666
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
\tool : Customizable Runtime Enforcement for Safe and Reliable LLM Agents Haoyu Wang, Christopher M. Poskitt, Jun Sun Abstract . Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution. However, their autonomy introduces safety risks, including security vulnerabilities, legal violations, and unintended harmful actions. Existing mitigation methods, such as model-based safeguards and early enforcement strategies, fall short in robustness, interpretability, and adaptability. To address these challenges, we propose \tool , a lightweight domain-specific language for specifying and enforcing runtime constraints on LLM agents. With \tool , users define structured rules that incorporate triggers, predicates, and enforcement mechanisms, ensuring agents operate within predefined safety boundaries. We implement \tool across multiple domains, including code execution, embodied agents, and autonomous driving, demonstrating its adaptability and effectiveness. Our evaluation shows that \tool successfully prevents unsafe executions in over 90% of code agent cases, eliminates all hazardous actions in embodied agent tasks, and enforces 100% compliance by autonomous vehicles (AVs). Despite its strong safety guarantees, \tool remains computationally lightweight, with overheads in milliseconds. By combining interpretability, modularity, and efficiency, \tool provides a practical and scalable solution for enforcing LLM agent safety across diverse applications. We also automate the generation of rules using LLMs and assess their effectiveness. Our evaluation shows that the rules generated by OpenAI o1 achieve a precision of 95.56% and recall of 70.96% for embodied agents, successfully identify 87.26% of the risky code, and prevent AVs from breaking laws in 5 out of 8 scenarios. Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering (ICSE ’26), April 12–18, 2026, Rio de Janeiro, Brazil CCS: Software and its engineering Domain specific languages CCS: Computing methodologies Intelligent agents CCS: Security and privacy Software security engineering 1. Introduction Large Language Model (LLM) agents ( 31 , 39 , 28 , 19 , 35 , 48 , 6 , 14 ) extend the capabilities of LLMs by autonomously perceiving, planning, and acting within interactive environments. Their ability to automate or semi-automate tasks across various domains has led to increased adoption in software development, healthcare, and autonomous systems. For instance, in software engineering, LLM agents such as SWE-Agent ( 46 ) and CodeAct ( 42 ) assist with code generation, review, and refactoring. In healthcare, EHRAgent ( 34 ) supports clinical decision-making by processing electronic health records. Similarly, SeeAct ( 56 ) facilitates web-based interactions, while LLM-powered agents for autonomous driving are also emerging ( 22 ) . Commercial applications like Microsoft Copilot and GitHub Copilot further highlight the integration of LLM agents into mainstream productivity tools. However, as these agents become embedded in sensitive workflows—ranging from financial transactions and medical record processing to corporate decision-making—ensuring their safety, reliability, and ethical use has become more crucial than ever ( 23 , 10 ) . Figure 1. A demonstrative example of the enforced LLM agent While LLM agents hold significant potential, their autonomous decision-making can sometimes diverge from user expectations, leading to misaligned execution behaviors and raising concerns about their safety and trustworthiness ( 45 , 15 , 38 , 47 , 50 , 8 ) . For instance, as shown in Figure 1 , an LLM agent executing a financial transaction without explicit human review may be acceptable in one setting but considered unsafe in another. Risks associated with LLM agents span multiple dimensions ( 32 ) , and organizations that use them may exhibit varying levels of risk tolerance. For example, an LLM agent automatically adjusting medication dosages may be deemed unsafe in a hospital setting requiring strict human oversight, while permitted in a research lab to accelerate experimental trials. Given these differences in risk tolerance across organizations and user settings, effective mechanisms to customize and enforce safety restrictions on LLM agents are essential. Several existing approaches attempt to mitigate risks in LLM agents but all have different limitations. LLM-powered risk evaluation systems, such as ToolEmu ( 32 ) , simulate potential outcomes using an LLM sandbox before execution. While effective in identifying risks, these methods lack interpretability and offer no mechanism for safety enforcement, making them susceptible to adversarial manipulation. Rule-based safeguards provide an alternative that is auditable and predictable, but current implementations are either too rigid or lack generalizability across agent architectures. GuardAgent ( 44 ) , for example, enforces safety policies via auto-generated guardrails but requires manual implementation for each agent instance, making adoptability a challenge. Furthermore, most existing solutions lack explicit safety enforcement mechanisms, focusing instead on pre-execution risk assessments. This approach leaves agents vulnerable to runtime deviations from expected behavior, as there are no active constraints to prevent unsafe actions during execution. These limitations highlight the need for a flexible, expressive, and interpretable mechanism that allows users and organizations to specify and enforce safeguards at runtime, ensuring that LLM agents behave safely in diverse operational contexts. To address these challenges, we introduce \tool , a domain-specific language (DSL) designed for the runtime enforcement of LLM agent behavior. To the best of our knowledge, \tool is the first framework that systematically enforces customizable safety constraints on LLM agents at runtime. \tool enables the specification of rules composed of a triggering event (e.g., an LLM agent executing a financial transaction), predicate conditions (e.g., whether the transaction amount exceeds a predefined threshold) and enforcement (e.g., requiring user confirmation before execution or conducting retrospective self-examination). For example, a rule may enforce human verification before an agent modifies sensitive data, or enforce self-reflection ( 35 ) via an LLM before an agent proceeds with a high-risk task. These rules can be manually defined or automatically generated for user review and approval. \tool is implemented as a lightweight, modular framework designed to integrate seamlessly with LLM agent platforms like LangChain ( 16 ) , intercepting key execution stages to enforce user-defined constraints. It hooks into the agent’s decision pipeline, evaluating proposed actions against user-defined constraints prior to their execution. Enforcement is achieved through mechanisms such as action termination, user inspection, corrective invocation, and self-reflection. While LangChain serves as the primary integration example, \tool remains framework-agnostic and can be adapted to other ecosystems, such as Microsoft’s AutoGen ( 25 ) and autonomous vehicle systems like Apollo ( 3 ) . We implemented \tool for agents across multiple domains, including code execution ( 42 ) , embodied agents ( 48 , 49 ) , and autonomous vehicles (AVs) ( 3 ) . \tool is evaluated using three datasets targeting critical safety challenges. RedCode-Exec ( 13 ) tests code execution safety with prompts covering 25 vulnerability types across eight domains. SafeAgentBench ( 49 ) assesses the ability of embodied agents to avoid hazards like fire and electrical shocks, while FixDrive ( 37 ) evaluates autonomous driving agents in law-breaking scenarios. Our evaluation demonstrates that \tool reduced unsafe executions in code agents by detecting and intercepting risks in over 90% of cases, prevented law violations in 100% of tested AV scenarios, and eliminated all hazardous actions in 10 categories of embodied agent tasks. Despite these strong safety guarantees, \tool imposes minimal overhead in milliseconds, ensuring practical deployment without significant performance penalties. In an additional experiment, we employed LLMs to