Learning Ensembles of Vision-based Safety Control Filters
Ihab Tabbara, Hussein Sibai · 2024
Read Learning Ensembles of Vision-based Safety Control Filters by Ihab Tabbara, Hussein Sibai (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02029.txt.
Introduction
Ihab Tabbara and Hussein Sibai test whether groups of learned safety filters beat single ones in a 2024 paper. A safety filter watches a controller and corrects commands that would break safety rules. Learning such filters from camera images is attractive but hard to verify, so reliability must be earned empirically.
The authors trained ensembles that mix different pre-trained vision backbones, training styles, and ways of combining outputs. On the DeepAccident data set, the ensembles judged safe versus unsafe states and controls more accurately than their own member models and than large single models. Gains were strongest for varied ensembles, and held better on unfamiliar data. The paper frames ensembling as a step toward trustworthy learned safety.
Catalog record
| CREATOR | Ihab Tabbara, Hussein Sibai |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-02029
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Ensembles of Vision-based Safety Control Filters Ihab Tabbara, Hussein Sibai Abstract Safety filters in control systems correct nominal controls that violate safety constraints. Designing such filters as functions of visual observations in uncertain and complex environments is challenging. Several deep learning-based approaches to tackle this challenge have been proposed recently. However, formally verifying that the learned filters satisfy critical properties that enable them to guarantee the safety of the system is currently beyond reach. Instead, in this work, motivated by the success of ensemble methods in reinforcement learning, we empirically investigate the efficacy of ensembles in enhancing the accuracy and the out-of-distribution generalization of such filters, as a step towards more reliable ones. We experiment with diverse pre-trained vision representation models as filter backbones, training approaches, and output aggregation techniques. We compare the performance of ensembles with different configurations against each other, their individual member models, and large single-model baselines in distinguishing between safe and unsafe states and controls in the DeepAccident dataset. Our results show that diverse ensembles have better state and control classification accuracies compared to individual models. keywords Safety filters; Ensembles; Control barrier functions; Pre-trained vision models 1 Introduction Ensuring safety of control systems is a fundamental challenge in various application domains, including autonomous driving ( Betz et al. (2019) ), aerospace ( Breeden and Panagou (2022) ), and robotic surgery ( Haidegger (2019) ). It entails verifying that the trajectories of a system remain in a region of the state space that the user considers safe, or synthesizing controllers that drive the system to remain there. One of the prominent solutions is to design barrier certificates that guarantee the safety of the system. These certificates can guide the selection of controls or modify nominal ones to maintain safety, effectively serving as safety control filters ( Ames et al. (2016) ). Unfortunately, synthesizing such certificates is generally NP-hard ( Clark (2021) ) and accordingly, existing algorithms do not scale beyond few dimensions. Moreover, these algorithms require white-box settings where the dynamics of the system and its environment are known. For many modern systems, e.g., vision-based autonomous navigation, such conditions are not satisfied. Recently, deep learning-based methods have been proposed for designing certificates and controllers, offering an easier and more scalable approach for their design ( Dawson et al. (2023) ; Abdi et al. (2023) ; Tong et al. (2023) ; Xiao et al. (2022) ; Yang and Sibai (2024) ). However, the learned neural certificates are not formal ones. They do not necessarily satisfy the required conditions at every state and control for them to guarantee safety. Algorithms for formally verifying them suffer from similar curse-of-dimensionality limitations as the traditional methods for their design, despite significant progress in neural networks (NN) verification over the past few years (e.g., Katz et al. (2022) ; Wu et al. (2023) ; Albarghouthi (2021) ; Shi et al. (2024) ). Similarly, verifying the safety of systems with NN-based controllers, particularly vision-based ones, suffer from the same scalability challenges and is usually constrained to predefined simple environments or particular images and their local neighborhoods (e.g., Santa Cruz and Shoukry (2022) ; Hsieh et al. (2022) ; Cai et al. (2024) ). In this work, we take an alternative approach to formal verification and investigate using ensemble learning to improve the reliability and accuracy of vision-based safety filters. Ensemble learning has been used for uncertainty quantification ( Rahaman et al. (2021) ), accuracy improvement, and out-of-distribution generalization ( Sagi and Rokach (2018) ), in various machine learning (ML) tasks, but has not been used for safety control filters design before, up to our knowledge. We build the member models of our ensembles using the approach we presented in Yang and Sibai (2024) , which uses pre-trained vision representation models (PVRs), such as CLIP ( Radford et al. (2021) ) and VC1 ( Majumdar et al. (2023) ), as perception backbones for the safety filters, significantly decreasing the sample complexity of learning the filters while improving generalization. We focus on the vision-based collision avoidance task in autonomous driving as the application domain. We use the DeepAccident dataset ( Wang et al. (2024) ), generated using CARLA ( Dosovitskiy et al. (2017) ), to train and evaluate the filters. We experiment with diverse ensembles which have member models with different PVR backbones, training methods, and model aggregation techniques. We analyze the trade-off between performance and complexity. Our results show the advantages of using diverse ensembles instead of individual models, showing their potential as more reliable safety control filters. 2 Related Work Ensembles in reinforcement learning (RL) and control Ensembles have been used to represent and tackle uncertainty in risk-sensitive RL ( Eriksson et al. (2022) ; Hoel et al. (2023) ), for learning from unstable estimations of value functions ( Faußer and Schwenker (2015) ; Anschel et al. (2017) ), for learning value functions more efficiently ( Chen et al. (2021) ), to facilitate optimism for efficient exploration in model-based online deep RL ( Pacchiano et al. (2021) ), to enable pessimism in offline RL ( Ghasemipour et al. (2022) ), to approximate reward functions in inverse RL ( Lin et al. (2020) ), for robust dynamic motion prediction ( Mortlock et al. (2024) ), and for anomaly detection ( Ji et al. (2024) ). Moreover, it has been shown that carefully designed reward functions define Q functions that are equivalent to control barrier functions ( Tan et al. (2023) ), the control version of barrier certificates. This implies that the demonstrated benefits of ensembles in RL can also be potentially obtained in learning safety filters. The results of this paper can be seen as a supporting evidence. Learning safety filters Recent approaches to designing safety filters use deep learning to overcome the scalability challenges inherent in methods based on sum-of-squares optimization and reachability analysis ( Dawson et al. (2022b) ; Dawson et al. (2022a) ) and to account for unknown dynamics ( Qin et al. (2022) ; Lavanakul et al. (2024) ; Castaneda et al. (2023) ) and high-dimensional observations such as images and point clouds ( Tong et al. (2023) ; Abdi et al. (2023) ; Xiao et al. (2023) ). The resulting filters are not guaranteed to satisfy the conditions for them to be valid certificates, unless under restrictive assumptions of known Lispchitz constants of the NNs and corresponding grid-like training datasets that cover the whole domain ( Anand and Zamani (2023) ; Tayal et al. (2024) ). Several works have used NN verification techniques to guarantee these conditions as well as generating counter examples that can be used for retraining ( Wu et al. (2023) ; Hu et al. (2024) ). However, these techniques are not yet scalable enough to verify high-dimensional, observation-based filters across all possible scenarios, e.g., those that can be observed in autonomous driving settings, and are instead constrained to local structured environments ( Abdi et al. (2023) ). Instead, we aim to improve the empirical performance and tackle the epistemic uncertainty of vision-based safety filters through ensemble learning. 3 Preliminaries In this section, we recall the definition of control barrier functions (CBF) and the guarantees they provide. We generally assume that the dynamics of the control