Learning State-Action Control Barrier Functions for Model-Free Control under Constraints
Kanghui He, Shengling Shi, Ton van den Boom, Bart De Schutter · 2025
Read Learning State-Action Control Barrier Functions for Model-Free Control under Constraints by Kanghui He, Shengling Shi, Ton van den Boom, Bart De Schutter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15515.txt.
Introduction
Kanghui He, Shengling Shi, Ton van den Boom, and Bart De Schutter present a safety method for learned controllers in a 2025 paper. Controllers trained from data can perform well, but keeping them inside safety limits without a physics model is hard. Classic safety filters lean on such models, which model-free learning lacks.
Their framework picks safe inputs straight from data. Its core is a new safety certificate that judges a state and action pair together, with two ways to build it from experience. Both the goal and the safety rule can be updated by learning as the system runs. The design targets robots and vehicles that learn on the job.
Catalog record
| CREATOR | Kanghui He, Shengling Shi, Ton van den Boom, Bart De Schutter |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15515
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning State-Action Control Barrier Functions for Model-Free Control under Constraints Kanghui He, Shengling Shi, Ton van den Boom, Bart De Schutter Abstract Ensuring constraint satisfaction for learning-based control is a critical challenge, especially in the model-free case. While safety filters address this challenge in the model-based setting, they typically rely on predictive models derived from physics or data. This reliance limits their applicability for advanced model-free learning control methods. To address this gap, we propose a new optimization-based control framework that determines safe control inputs directly from data. The benefit of the framework is that both the objective function and the safety constraint can be updated using model-free learning algorithms, enabling performance optimization while maintaining safety. As a key component, the concept of direct data-driven safety filters (3DSF) is first proposed. The framework employs a novel safety certificate, called the state-action control barrier function (SACBF). We present two approaches for synthesizing SACBFs, including learning from an expert controller and reinforcement learning. We develop a robustness framework that guarantees safety and recursive feasibility for the expert-guided approach under learning errors. The proposed control framework bridges the gap between model-free learning-based control and constrained control, by decoupling performance optimization from safety enforcement. Simulations on vehicle control illustrate the superior performance regarding constraint satisfaction and task achievement compared to model-based methods and reward shaping. I Introduction I-A Background Learning-based control has achieved state-of-the-art performance in addressing complex problems, including applications in transportation systems [ 1 , 2 ] and robotics [ 3 , 4 ] . However, ensuring safety is still a challenging problem, particularly when an explicit model of the system is unavailable. Traditional model-based approaches to safety-critical control, such as model predictive control (MPC) [ 5 ] , struggle with online computational efficiency and require an explicit prediction model, while emerging data-driven methods often lack well-understood safety guarantees. In control problems, safety is typically defined as maintaining state and input variables within given constraints throughout the system’s evolution. A widely adopted approach to enforcing long-term safety is the use of safety filters, which provide a modular framework that can be applied to any control policy, even those without explicit safety considerations [ 6 , 7 , 8 , 2 , 9 , 10 , 11 ] . The basic principle of safety filters is to post-process the input of a given control policy such that the resulting closed-loop system remains forward invariant w.r.t. the specified state and input constraints. The design of safety filters typically consists of two phases: an offline phase, where a safety certificate characterizing safe states is computed, and an online phase, where this certificate is incorporated as a constraint to modify potentially unsafe control actions from the reference controller. Learning-based approaches have increasingly been used to synthesize safety certificates, particularly for complex, nonlinear systems with non-convex state and input constraints. For a detailed overview, we refer the reader to relevant work in the literature [ 12 , 7 , 13 , 14 , 15 ] as well as comprehensive surveys [ 16 , 17 ] . Despite recent progress, most safety filters rely on a mathematical model, which is either exactly derived from physical principles or approximately estimated. In most formulations, model information is required in both the offline and online phases. Recently, there has been a growing number of approaches focusing on offline construction of safety certificates using only state transition data [ 18 ] . However, these approaches still require an explicit (identified) model during online execution. This limitation is mainly due to the inherent property of existing safety certificates, whose safety conditions are defined solely over the state space. To overcome the above limitation, data-driven safety filters have received significant attention. Almost all existing data-driven safety filters belong to indirect data-driven methods [ 19 , 16 , 9 , 14 , 15 , 20 , 11 ] . In indirect approaches, first system identification or disturbance estimation is performed and then a controller is learned based on the obtained model. In direct methods the controller is learned directly from data, bypassing the need for system identification. Indirect data-driven safety filters have one main issue: The errors arising from both model identification and certificate learning will compound, leading to a degradation in the safety performance of the filtered controller. Fig. 1: The comparison between the proposed 3DSF (top) and the existing indirect data-driven counterparts (bottom). In this paper, we focus on designing a direct data-driven (model-free) safe control framework based on our previously proposed concept of state-action control barrier functions (SACBFs) [ 21 ] . Unlike classical safety certificates, which only evaluate safety in the state space, SACBFs enable safety evaluation for each state-action pair, and the safety condition is enforced at the current state and action without propagating the dynamics forward. As a result, the corresponding safety filter can assess the safety of candidate control actions without explicitly querying or simulating the system dynamics. The comparison between the proposed control framework and the existing indirect data-driven counterparts is illustrated in Fig. 1. We next discuss several recently developed direct data-driven approaches for safe control. The authors of [ 22 ] learn discriminating hyper-planes to directly regulate the control inputs. However, this method is limited to linear safety constraints on inputs and lacks formal guarantees regarding constraint satisfaction. [ 23 ] develops CBF-based control without a model by using techniques from funnel control. The considered safety constraint is specifically introduced to keep the tracking error within a prescribed bound. In comparison, our method directly handles general nonlinear state constraints and can optimize a given performance criterion. Similarly, [ 24 ] achieves CBF-based safe control for robotic systems without knowing the dynamics. Its key idea is to exploit the known kinematics to generate a safe velocity and then employ a model-free controller to track this velocity. Consequently, the approach in [ 24 ] relies on the particular structure of robotic systems. In contrast, our proposed method does not rely on such assumptions and applies to general nonlinear systems. I-B Contributions The paper contributes to the state of the art in the following aspects: (1) We propose a safe control framework based on SACBFs that combines the advantages of learning-based control (applicability to model-free cases) and optimization-based control (explicit inclusion of safety constraints). The core mechanism relies on our previously introduced safety certificate called the SACBF [ 21 ] . We extend existing learning CBF approaches, including learning from an expert controller [ 12 ] and RL [ 25 ] , to learning SACBFs. The framework can be trained and implemented using only state transition data, without system identification, and can integrate with various learning-based controller synthesis methods. (2) We develop an expert-guided approach for learning SACBFs directly from transition data. We establish a converse result guaranteeing existence under the expert-policy assumption and develop an error-to-state safety analysis to characterize recursive feasibility and constraint satisfaction under learning errors. As an extension, we further employ RL to synthesize SACBFs when an expert controller is unavailable. (3) Based on the proposed control framework, we present a modular strategy for extending classical unconstrained value-based RL algorithms to constrained ones. This modular construction separates controller synthesis into two components: learning a