Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation
Luca Marzari, Francesco Trotti, Enrico Marchesini, Alessandro Farinelli 1 1 Department of Computer Science, University of Verona, Verona, Italy. 2 2 Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy. 3 3 Massachusetts Institute of Technology, Boston, USA.Contact · 2025
Read Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation by Luca Marzari, Francesco Trotti, Enrico Marchesini, Alessandro Farinelli 1 1 Department of Computer Science, University of Verona, Verona, Italy. 2 2 Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy. 3 3 Massachusetts Institute of Technology, Boston, USA.Contact (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-21643.txt.
Introduction
Luca Marzari, Francesco Trotti, Enrico Marchesini, and Alessandro Farinelli make learning-based robot navigation provably safer with automatically designed barrier functions, in a paper from 2025. Robots in messy real settings cannot rely on a learned policy never straying into danger. A verified safety layer can overrule bad moves.
Their method samples the state space to find unsafe regions, then builds a barrier function that keeps the robot out of them. The safety layer wraps any policy, correcting actions only when needed. Tests ran in simulation and on a real robot, including a fast-changing water monitoring task. Unsafe actions were corrected while navigation stayed efficient.
Catalog record
| CREATOR | Luca Marzari, Francesco Trotti, Enrico Marchesini, Alessandro Farinelli 1 1 Department of Computer Science, University of Verona, Verona, Italy. 2 2 Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy. 3 3 Massachusetts Institute of Technology, Boston, USA.Contact |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-21643
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation Luca Marzari, Francesco Trotti, Enrico Marchesini, Alessandro Farinelli 1 1 Department of Computer Science, University of Verona, Verona, Italy. 2 2 Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy. 3 3 Massachusetts Institute of Technology, Boston, USA.Contact authors: [email protected] , [email protected] Abstract Achieving safe autonomous navigation systems is critical for deploying robots in dynamic and uncertain real-world environments. In this paper, we propose a hierarchical control framework leveraging neural network verification techniques to design control barrier functions (CBFs) and policy correction mechanisms that ensure safe reinforcement learning navigation policies. Our approach relies on probabilistic enumeration to identify unsafe regions of operation, which are then used to construct a safe CBF-based control layer applicable to arbitrary policies. We validate our framework both in simulation and on a real robot, using a standard mobile robot benchmark and a highly dynamic aquatic environmental monitoring task. These experiments demonstrate the ability of the proposed solution to correct unsafe actions while preserving efficient navigation behavior. Our results show the promise of developing hierarchical verification-based systems to enable safe and robust navigation behaviors in complex scenarios. 1 Introduction Achieving safe autonomous navigation is a crucial requirement to deploy robots in the real world, where human interactions and expensive hardware are often involved. For example, robots can assist humans in monitoring and mitigating pollution in a variety of environmental sustainability scenarios (e.g., aquatic ecosystems [ 1 ] ) or perform complex autonomous navigation in precision agriculture tasks [ 2 ] . Traditionally, these applications are based on manual human intervention, which is costly and struggles to capture real-time changes reliably. These are challenges that a drone equipped with a safe autonomous navigation stack can successfully address [ 3 ] . To this end, online model-based learning methods embedding hierarchical control architectures have been used to deal with uncertainty in various navigation tasks [ 4 , 5 , 6 , 7 ] . Nevertheless, these methods rely on extensive domain knowledge, which could be difficult to obtain in practice. To address this issue, deep reinforcement learning (DRL) algorithms have been employed to learn how to navigate in complex and unknown (i.e., mapless) environments [ 8 , 9 , 10 , 11 ] . However, DRL-based policies are modeled as deep neural networks (DNNs), which are known to be vulnerable to adversarial inputs—small perturbations to the input state leading to unexpected and unsafe actions [ 12 , 13 ] . Two main lines of work have been investigated to address such an issue. On the one hand, formal verification (FV) of neural networks can certify safety, providing rigorous theoretical guarantees that a DNN-based navigation policy lies within predefined safety constraints [ 14 ] . However, FV scales poorly and novel probabilistic verification approaches have recently gained traction to address the limits of formal methods [ 15 , 16 ] . In detail, probabilistic enumeration [ 17 ] can identify regions of the state space (i.e., parts of the environment) where a DNN exhibits unsafe behavior with a high degree of confidence, but this information has not been exploited to correct unsafe navigation behaviors. On the other hand, control barrier functions (CBFs) [ 18 ] also provide a principled way to enforce safety constraints by specifying a function that characterizes a safe set, ensuring its forward invariance [ 18 ] . Hence, CBFs can potentially guarantee that navigation trajectories remain within pre-defined safety regions. However, designing effective CBFs for high-dimensional, complex environments remains challenging when dealing with uncertainty and complex dynamics. To mitigate these issues, recent work has explored the use of neural CBFs and their verification [ 19 , 20 , 21 ] . Despite the different lines of research, ensuring the successful deployment of safe policies and correcting their decisions when detecting an unsafe situation remains an open research question. Figure 1: Overview of the proposed approach. We address this gap by leveraging the advantages of probabilistic verification and CBF-based control, introducing a framework that guarantees safe DRL-based navigation policies and correct policy actions in potentially unsafe situations (Fig. 1 ). At a high level, we compute a safe navigation set by first identifying unsafe regions through offline probabilistic enumeration [ 17 ] and then removing them from the state space of the DRL policy. We then design a CBF-based control mechanism to ensure the agent avoids both these pre-identified unsafe regions and any obstacles detected during navigation—ensuring the agent remains within the precomputed safe set. At deployment time, a quadratic programming (QP) optimization incorporating the CBF’s safety constraints evaluates the policy’s action, determining whether the chosen action keeps the agent within the precomputed safe set or not. If the action violates any safety constraints, a low-level controller modifies the action to ensure the agent returns to the safe set while maintaining effective navigation behaviors. A key advantage of our approach is that the enumeration component, which employ an abstract interval representation to identify the (un)safe regions, is agnostic to the specific navigation environment where the agent is deployed, thus enhancing generalization. Furthermore, the proposed framework works on top of arbitrary DRL policies and it is orthogonal to both the training and verification strategies employed by a neural CBF for the execution of safe policies. To confirm the effectiveness of the proposed approach, we first validated our strategy in two different Unity-based simulated navigation scenarios using a mobile Turtlebot3 and an aquatic drone for water monitoring. We then performed a real-world deployment of a trained DRL policy to validate the correctness of our pipeline. Our empirical results demonstrate that the proposed method enables safe and scalable autonomous robotic navigation, achieving zero violations of safety restrictions at deployment while increasing navigation effectiveness. 2 Background and Preliminaries In this section, we cover the main foundations related to the research topics considered by our framework. 2.1 Reinforcement Learning and Constrained Approaches Navigation tasks are typically modeled as a Markov decision process (MDP), represented by a tuple <\mathcal{S},\mathcal{A},R,\mathcal{P},\gamma> where \mathcal{S}\in\mathbb{R}^{d} is the state space, \mathcal{A}\in\mathbb{R}^{2} is the action space—which in this work consists of linear and angular velocities— \mathcal{P}:\mathcal{S}\times\mathcal{A}\to\mathcal{S} is the state transition function and R:\mathcal{S}\times\mathcal{A}\to\mathbb{R} is a reward function. Given a stationary policy \pi\in\prod , an agent aims to maximize J_{r}^{\pi}=\mathbb{E}_{\tau\sim\pi}\left[\sum_{t=0}^{\infty}\gamma^{t}R(s_{t},a_{t})\right] , i.e., the expected discounted return for each trajectory \tau=(s_{0},a_{0},\cdots,s_{n},a_{n}) , where \gamma\in(0,1] is a discount factor used to regulate the impact of future rewards. To consider safety, MDPs are typically extended to constrained MDPs (CMDPs) and incorporate a set of constraints during the training using one or more cost functions C:\mathcal{S}\times\mathcal{A}\to[0,1] . Hence, constrained DRL algorithms aim at maximizing the expected return J_{r}^{\pi} while maintaining costs under hard-coded thresholds \mathbf{t} : \max_{\pi\in\Pi}J_{r}^{\pi}\quad\text{s.t.}\quad J_{C}^{\pi}\leq\mathbf{t} , with J_{C}^{\pi}=\mathbb{E}_{\tau\sim\pi}[\sum_{t=0}^{\infty}\gamma^{t}C(s_{t},a_{t})] . However, CMDPs present non-negligible limitations, such as the necessity of a parametrization of the policy (i.e., they are not applicable to value-based methods) [ 22 ] . To address this issue, recent work [ 23 ] showed how incorporating penalties into the reward effectively influences policy toward safer behaviors while avoiding the limitations introduced by constrained approaches. Nonetheless, these methods only guarantee safety in expectation, making them unsuitable to ensure safe deployment. 2.2 Formal Verification of Neural