Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning
Maeva Guerrier, Karthik Soma, Hassan Fouad, Giovanni Beltrame · 2025
Read Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning by Maeva Guerrier, Karthik Soma, Hassan Fouad, Giovanni Beltrame (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18858.txt.
Introduction
Maeva Guerrier and co-authors present a robotics paper, published in 2025, on using formal safety rules as teachers for learning robots. Reinforcement learning robots usually get one harsh penalty and a reset when they stray somewhere unsafe, which hides the lasting harm of staying in danger. When the authors instead charged a steady penalty without ending the episode, standard learners stalled, fenced in by accumulated losses.
Their remedy brings in control barrier functions, rules with proven guarantees that steer a robot away from forbidden zones. Three blends of the rules with standard learners taught safer behavior and better performance. Trials in simulation and on a real four wheeled robot supported the approach.
Catalog record
| CREATOR | Maeva Guerrier, Karthik Soma, Hassan Fouad, Giovanni Beltrame |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18858
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning Maeva Guerrier, Karthik Soma, Hassan Fouad, Giovanni Beltrame Abstract Safety stands as the primary obstacle preventing the widespread adoption of learning-based robotic systems in our daily lives. While reinforcement learning (RL) shows promise as an effective robot learning paradigm, conventional RL frameworks often model safety by using single scalar negative rewards with immediate episode termination, failing to capture the temporal consequences of unsafe actions (e.g., sustained collision damage). In this work, we introduce a novel approach that simulates these temporal effects by applying continuous negative rewards without episode termination. Our experiments reveal that standard RL methods struggle with this model, as the accumulated negative values in unsafe zones create learning barriers. To address this challenge, we demonstrate how Control Barrier Functions (CBFs), with their proven safety guarantees, effectively help robots avoid catastrophic regions while enhancing learning outcomes. We present three CBF-based approaches, each integrating traditional RL methods with Control Barrier Functions, guiding the agent to learn safe behavior. Our empirical analysis, conducted in both simulated environments and real-world settings using a four-wheel differential drive robot, explores the possibilities of employing these approaches for safe robotic learning. I INTRODUCTION Artificial Intelligence (AI) powered robotic systems have applications across diverse domains including agriculture, healthcare, and space exploration. However, imagine a search and rescue robot that repeatedly collides with trees, becomes stranded, and fails its mission. This scenario highlights a critical question: How do we enable safety-based learning in autonomous systems? Ensuring safety in learning methods remains an open challenge [ 8 , 6 ] , that safe-by-design AI aims to address. Learned policies enable generalizable behaviors, while safety prevents harmful outcomes in real-world. However, learning and safety might be in conflict in certain scenarios, as the pursuit of optimal performance may often lead to unsafe behaviors [ 10 ] . Obstacle Agent Goal Environment Agent Deployed CBF Unicycle R_{t+1} S_{t+1} Reward \hskip 8.19447ptR_{t} State \hskip 8.19447ptS_{t} A^{rl}_{t} A^{rl}_{t} CBF Decay CBF Filter \hskip 8.19447pt\hskip 8.19447pt\color[rgb]{0,0.5,0}A^{cbf}_{t} CBF Reward Fig. 1: Overview of the three variations of safety guardrail for reinforcement learning: a filter (green) approach that intervenes when the agent is about to enter an unsafe region, a reward-based (orange) approach that incorporates CBF within the reward structure and a decay (blue) approach that progressively removes the effect of the CBF over time. Among various learning paradigms, Reinforcement learning (RL) [ 26 ] has emerged as one of the efficient methods that enables the learning of intelligent behaviors for robots [ 19 ] . Traditional RL frameworks typically represent unsafe situations using negative reward signals followed by immediate episode termination [ 20 , 24 ] . Yet, this approach fails to accurately reflect real-world catastrophic scenarios and their consequences over time, as these artificial terminations don’t mirror natural outcomes. The problem is compounded by RL’s inherent exploration mechanism: without proper boundaries, agents can become trapped in catastrophic states, significantly impeding the learning process. These limitations underscore the critical importance of incorporating safety constraints within RL frameworks. However, ensuring safety remains inherently difficult, and implementing safety constraints within the learning process is even more challenging. To address this challenge, Control Barrier Functions (CBFs [ 3 ] ) have been widely applied in robotics to enforce safety. CBFs offer a theoretically grounded framework to enforce safety invariance [ 31 , 3 ] . Nevertheless, synthesizing CBFs is complex as it requires complete knowledge of system dynamics, demanding extensive domain expertise. For practicality, CBFs that employ basic models to abstract robot dynamics are generally preferred. Our objective is to integrate CBFs (see Figure 1 ) as a safety guardrail into RL to develop safe behaviors. Specifically, we aim at learning both goal-reaching and safety-maintaining policies for robots. We adopt our definition from [ 7 ] : a safety guardrail evaluates the admissibility of actions in the current state, restricting actions that can lead to failures. Considering a goal-reaching and obstacle avoidance as the robot’s task, the main contributions of this paper are: • We design a per-step CBF that integrates seamlessly with the RL observation-action framework by intervening when unsafe actions are selected by the agent. We chose a unicycle model that enables the simplification of the CBF formulation and it also provides a representation model that easily generalizes to many robots, as the abstraction it provides is compatible for a wide range of robot dynamics. We demonstrate this by deploying the learned policy from the abstracted unicycle model trained in simulations onto a four-wheel differential drive robot. • We propose three RL-CBF integration methods: (1) CBF Filter, a CBF-based action filter; (2) CBF Reward, which incorporates CBF into the reward function; and (3) CBF Decay, where the influence of CBF diminishes over time. We evaluate these approaches using multiple metrics (e.g., rewards, CBF activations) and analyze their sim2real transfer performance, providing insights into the learned behaviors of each variant. II RELATED WORK Safety in RL (SRL). The exploration-exploitation dilemma of RL [ 12 ] concerns balancing the discovery of new knowledge against exploiting existing information to maximize performance, but exploration can lead to hazardous actions. Early works [ 14 ] emphasized the need for safety constraints during exploration. Recent advances in SRL focuses on Constrained Markov Decision Processes (CMDPs) to enable safe behaviors [ 15 ] . While most CMDP approaches prioritize intermediate costs [ 1 , 28 ] , they often fail to account for catastrophic events, which better reflect real-life scenarios. Some approaches incorporate safety directly into the optimization objective [ 1 , 28 , 23 ] , while others derive strategies from game theory to address safety concerns [ 22 ] . Additionally, certain methods leverage external knowledge to enforce safety constraints [ 7 , 18 ] . Recent years have witnessed the emergence of using safety layer [ 2 , 21 ] in RL methods. Linking CBFs and Safety Guardrails. There is an extensive literature on CBFs for safety critical systems [ 3 , 16 , 5 ] . Subsequent research [ 4 , 3 ] demonstrated how CBFs can be integrated with control policies to guarantee safety while optimizing performance. Taylor et al. [ 27 ] extended CBFs to high-dimensional systems, making them applicable to a wider range of real-world problems, including robotics and autonomous agents. The concept of safety guardrails [ 11 ] aligns closely with the principles of CBFs, as both aim to enforce safety constraints dynamically. CBFs in RL. Recent surveys [ 15 , 16 ] examine integrating CBFs in the RL framework for providing safety guarantees. Waberish et al. [ 30 ] employ neural network-based CBF controllers instead of solving quadratic programs at each step. However, neural CBFs introduce non-differentiability issues that can compromise safety guarantees. Alternative approaches use Gaussian Processes to handle disturbances and model uncertainty [ 9 , 13 ] . Emam et al. [ 13 ] integrate CBFs directly into the value function for off-policy actor-critic methods. Their approach uses a conservative policy to limit exploration to safe regions, but ties the agent to specific dynamics models and requires small time steps, limiting real-world applicability. Our work addresses these limitations by introducing a CBF-based safety guardrail with a priority variation parameter,