Safe Adaptive Cruise Control Under Perception Uncertainty: A Deep Ensemble and Conformal Tube Model Predictive Control Approach
Xiao Li, Anouck Girard, Ilya Kolmanovsky · 2024
Read Safe Adaptive Cruise Control Under Perception Uncertainty: A Deep Ensemble and Conformal Tube Model Predictive Control Approach by Xiao Li, Anouck Girard, Ilya Kolmanovsky (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-03792.txt.
Introduction
Li, Girard, and Kolmanovsky design a safe cruise control system that accounts for perception uncertainty in a 2024 paper. Self-driving cars lean on perception to read the road. Errors in that reading can threaten safety.
An ensemble of neural networks reads camera images and estimates the traffic state. A conformal method turns the spread of those estimates into honest uncertainty bounds. A tube-based predictive controller then plans speed with those bounds in mind, aiming for safety with stated chance. Tests in a high-fidelity simulator showed good speed tracking and safe gaps. The system also held up when scenes fell outside its training range.
Catalog record
| CREATOR | Xiao Li, Anouck Girard, Ilya Kolmanovsky |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-03792
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Safe Adaptive Cruise Control Under Perception Uncertainty: A Deep Ensemble and Conformal Tube Model Predictive Control Approach Xiao Li, Anouck Girard, Ilya Kolmanovsky Abstract Autonomous driving heavily relies on perception systems to interpret the environment for decision-making. To enhance robustness in these safety critical applications, this paper considers a Deep Ensemble of Deep Neural Network regressors integrated with Conformal Prediction to predict and quantify uncertainties. In the Adaptive Cruise Control setting, the proposed method performs state and uncertainty estimation from RGB images, informing the downstream controller of the DNN perception uncertainties. An adaptive cruise controller using Conformal Tube Model Predictive Control is designed to ensure probabilistic safety. Evaluations with a high-fidelity simulator demonstrate the algorithm’s effectiveness in speed tracking and safe distance maintaining, including in Out-Of-Distribution scenarios. \coltauthor University of Michigan, Ann Arbor, MI, USA keywords Perception Uncertainty, Deep Neural Network, Conformal Prediction, Model Predictive Control 1 Introduction Advances in Deep Neural Networks (DNNs) have significantly enhanced visual-based perception capabilities in autonomous driving. However, as black-box models, DNNs lack interpretability and operate in a high-dimensional image space vulnerable to pixel-level attacks ( Goodfellow et al. 2016 ) , and to Out-Of-Distribution (OOD) observations, which are unseen in training data ( Yang et al. 2024 ) . The integration of these models can impact the autonomous driving systems’ safety through other system modules, such as control and decision-making ( Zablocki et al. 2022 ) . Researchers have been investigating Bayesian frameworks to model inference uncertainty in DNNs ( Neal 2012 ; MacKay 1992 ) , despite their computational intensity. Bayesian DNNs are attractive as they offer theoretical guarantees useful in uncertainty quantification by assuming prior distributions on model parameters or architectures. To address computational demands, the Monte Carlo Dropout method ( Gal and Ghahramani 2016 ) has been proposed that approximates prior distributions through empirical dropout sampling. Similarly, Deep Ensembles ( Lakshminarayanan et al. 2017 ) leverage a diverse set of DNN architectures, enabling a voting mechanism that enhances inference robustness against OOD and adversarial attacks at the pixel level. Other approaches, such as Laplace Approximation ( Ritter et al. 2018 ) and Variational Neural Networks ( Graves 2011 ) , have also been developed. A comprehensive review is available in Abdar et al. 2021 . While these methods are easy to implement, they may lack theoretical guarantees. In contrast, Conformal Prediction ( Vovk et al. 2005 ) offers distribution-free uncertainty quantification, relying on exchangeability assumptions. This work combines Deep Ensembles with Conformal Prediction for robust uncertainty quantification with statistical guarantees, and it demonstrates how such guarantees can be integrated into Model Predictive Control (MPC) using Adaptive Cruise Control (ACC) as an application. Additionally, methods have been explored to control robots using DNN-based perception and prediction ( Tong et al. 2023 ) integrated with Control Barrier Functions, though they do not address the uncertainties inherent in DNN perception. In autonomous driving, researchers have developed DNN architectures to quantify classification uncertainties ( Liu et al. 2022 ) and trajectory prediction uncertainties ( Ding et al. 2021 ; Tang et al. 2022 ) . However, these methods only provide heuristic uncertainty estimates without theoretical support. Control co-design approaches have also been investigated, assuming bounded DNN errors to achieve trajectory tracking ( Dean et al. 2020 ) and ensure system-level safety ( Li et al. 2024a ) . These approaches, however, are limited to in-distribution scenarios ( Dean et al. 2020 ) . Previous studies ( Li et al. 2024b ) have applied Gaussian assumptions to model perception uncertainties in DNNs, though this assumption may lack practical validity in complex environments. Conformal Prediction has been previously integrated into MPC design in Lindemann et al. 2023 . Differently, in this paper, we consider perception uncertainties; their handling requires us to propagate the uncertainty of Conformal Prediction results through the system dynamics. More specifically, in this paper, we integrate Deep Ensembles with the Conformal Prediction to quantify uncertainty associated with image-based sensory information. We illustrate the use of this uncertainty quantification in a Conformal Prediction Tube MPC to control vehicle acceleration in an ACC application. The contributions of this paper are summarized as follows: • We develop a Deep Ensemble that assembles a heterogeneous set of DNN architectures to enhance robustness in state estimation from images and provide reliable uncertainty measures. The DNNs are optimized through iterative pruning and fine-tuning to reduce memory demand. • We integrate the Deep Ensemble with the Conformal Prediction, transforming heuristic uncertainty estimates into statistically guaranteed uncertainty measures under exchangeabilty assumptions. This integration demonstrates reliable uncertainty quantification, even in the presence of pixel-level adversarial attacks and OOD observations. • We design a Conformal Tube MPC that leverages the uncertainty estimates for uncertainty-aware trajectory prediction as a tube of Conformal Prediction sets. The proposed MPC is formulated as a Quadratic Program, ensuring recursive feasibility with a probabilistic lower-bound guarantee of system safety. This paper is organized as follows: In Sec. 2 , we introduce the ACC problem, along with model assumptions and design objectives. In Sec. 3 , we present our development of a Deep Ensemble and Conformal Prediction approach for state estimation, which subsequently informs a Conformal Tube MPC to control the acceleration of the ego vehicle. In Sec. 4 , we demonstrate that our method provides robust state estimation and quantifies uncertainty in the presence of adversarial attacks and OOD observations. Additionally, we validate the proposed adaptive cruise controller using a high-fidelity simulator. Finally, Sec. 5 presents a summary and a direction for future work. 2 Problem Formulation Figure 1 : A schematic diagram of the Adaptive Cruise Control implementation. In this study, we investigate an ACC scenario where the ego vehicle follows a lead vehicle (see Fig. 1 ). Our focus is on integrating visual-based perception and control mechanisms. The ego vehicle uses cameras to capture RGB images, which are processed to estimate system states. The Adaptive Cruise Controller then commands the ego vehicle to accelerate or brake to maintain a safe distance while adhering to a user-specified speed. This scenario presents challenges in both perception and control aspects. For perception, we assume two cameras are mounted on the left and right sides at the front of the ego vehicle to emulate stereo vision. The perception is modeled using the following mapping, x_{k}=y_{k}\left(\left\{I_{l,k-i},I_{r,k-i}\right\}_{i=0}^{M-1}\right)+e_{k}\left(\left\{I_{l,k-i},I_{r,k-i}\right\}_{i=0}^{M-1}\right), (1) where x_{k}=[d_{k}~\Delta v_{k}~v_{k}]^{T}\in\mathbb{R}^{3} is the state vector; d_{k}\in\mathbb{R}^{\geq 0} is the distance headway defined as the bumper-to-bumper distance in meters between the ego and the lead vehicle; \Delta v_{k}=v_{k}^{(l)}-v_{k}\in\mathbb{R} in m/s is the difference between the lead vehicle speed v_{k}^{(l)} and ego vehicle speed v_{k} ; I_{l,k},I_{r,k}\in\mathcal{I}\subset\mathbb{R}^{3\times 224\times 224} are RGB images of 3 color channels and of size 224\times 224 acquired from the left and right cameras, respectively; y_{k}\in\mathbb{R}^{3} denotes the state estimation derived from a historical image buffer \left\{(I_{l,k-i},I_{r,k-i})\right\}_{i=0}^{M-1} of length M ; e_{k}\in\mathbb{R}^{3} is the unknown state estimation error and follows an unknown distribution. Image-based perception has been focused on enhancing estimation accuracy by minimizing e_{k} using Convolutional Neural Networks (CNNs). However, e_{k} is influenced by inputs residing in a high-dimensional image space \mathcal{I} , making it highly unpredictable when subjected to disturbances at the pixel level, such as adversarial attacks in Goodfellow et al. 2016 . For enhanced safety, our perception