Learning Safe Autonomous Driving Policies Using Predictive Safety Representations
Mahesh Keswani, Raunak Bhattacharyya · 2025
Read Learning Safe Autonomous Driving Policies Using Predictive Safety Representations by Mahesh Keswani, Raunak Bhattacharyya (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17586.txt.
Introduction
Mahesh Keswani and Raunak Bhattacharyya test whether a safety method built for controlled settings survives contact with real driving data. The method, called SRPL, gives a learning agent a model that predicts future safety violations, so the agent can weigh reward against risk before it acts.
On the Waymo Open Motion Dataset and on NuPlan, SRPL improved the balance between reward and safety, with large gains in success rate and lower costs, both statistically significant. The benefit depends on the optimizer beneath it and on the data mix, and the method also held up better when sensor input grew noisy. Agents trained with it transferred to a fresh dataset without retraining better than agents without it, a sign the approach may carry toward real roads.
Catalog record
| CREATOR | Mahesh Keswani, Raunak Bhattacharyya |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-17586
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Safe Autonomous Driving Policies Using Predictive Safety Representations Mahesh Keswani, Raunak Bhattacharyya Abstract Safe reinforcement learning (SafeRL) is a prominent paradigm for autonomous driving, where agents are required to optimize performance under strict safety requirements. This dual objective creates a fundamental tension, as overly conservative policies limit driving efficiency while aggressive exploration risks safety violations. The Safety Representations for Safer Policy Learning (SRPL) framework addresses this challenge by equipping agents with a predictive model of future constraint violations and has shown promise in controlled environments. This paper investigates whether SRPL extends to real-world autonomous driving scenarios. Systematic experiments on the Waymo Open Motion Dataset (WOMD) and NuPlan demonstrate that SRPL can improve the reward–safety tradeoff, achieving statistically significant improvements in success rate (effect sizes r = 0.65 − 0.86 r=0.65-0.86 ) and cost reduction (effect sizes r = 0.70 − 0.83 r=0.70-0.83 ), with p < 0.05 p<0.05 for observed improvements. However, its effectiveness depends on the underlying policy optimizer and the dataset distribution. The results further show that predictive safety representations play a critical role in improving robustness to observation noise. Additionally, in zero-shot cross-dataset evaluation, SRPL-augmented agents demonstrate improved generalization compared to non-SRPL methods. These findings collectively demonstrate the potential of predictive safety representations to strengthen SafeRL for autonomous driving. I INTRODUCTION The development of autonomous driving technology presents an opportunity to improve transportation safety and efficiency by mitigating risks associated with human error [ 31 ] . While traditional rule-based control systems have been foundational, their inherent brittleness struggles to address the complexity of real-world driving scenarios [ 3 ] . This limitation has motivated a shift towards learning-based approaches, with reinforcement learning emerging as a promising paradigm. Reinforcement learning (RL) enables agents to learn optimal driving policies and adapt to novel situations through experience rather than manual programming [ 16 ] . Although deep RL has shown considerable success in continuous control tasks [ 10 , 12 ] , standard RL formulations optimize for reward maximization without explicit safety constraints, which can lead to dangerous behaviors in safety-critical applications [ 30 ] . Safe reinforcement learning (SafeRL) addresses this by formulating tasks as Constrained Markov Decision Processes (CMDPs), seeking policies that optimize performance while satisfying constraints on undesirable outcomes such as collisions [ 4 ] . Fig. 1 : Integration of SRPL with SafeRL algorithms: Raw Waymo Open Motion Dataset (WOMD) scenarios are converted to MetaDrive-compatible format using ScenarioNet, generating bird’s-eye view driving environments. The baseline SafeRL approach (blue) uses raw state observations as direct input to policy \pi_{\theta} . The SRPL framework (green) augments decision-making by concatenating raw states with predictive safety information from the Steps-to-Cost model S_{\nu}(s) before policy execution. Pink arrows indicate the ego-agent’s position in each scenario. Despite its theoretical advantages, the practical application of SafeRL in autonomous driving has revealed significant challenges. A recognized issue is the tendency of SafeRL agents to develop conservative behaviors, a phenomenon partly attributed to primacy bias, where initial encounters with trajectories violating constraints can persistently hinder agents’ willingness to explore their environment [ 22 ] . To encourage safety during training, SafeRL algorithms employ approaches such as failure penalties and explicit safety constraints during exploration [ 32 , 1 ] . Although these methods effectively minimize unsafe behaviors, they often lead to conservative policies that sacrifice performance for constraint satisfaction [ 21 ] . Beyond the exploration-safety balance, real-world autonomous driving deployment faces additional challenges such as sensor noise and diverse traffic patterns that differ from controlled training environments. These factors can reduce policy performance and compromise the safety guarantees established during training. Therefore, the primary difficulty in applying SafeRL in autonomous driving is developing agents that can achieve high task performance without violating safety constraints while remaining robust to environmental variations. To address these challenges, an alternative approach is to augment agents’ decision-making capabilities with predictive models that anticipate future constraint violations. This enables agents to make better-informed decisions by anticipating potential constraint violations during exploration. The Safety Representations for Safer Policy Learning (SRPL) framework [ 21 ] implements this concept by augmenting the state with information from a learned Steps-to-Cost (S2C) model, which provides a probabilistic distribution of potential constraint violations over a future horizon. By incorporating these predictive safety representations, agents can make more nuanced decisions about when to explore and exercise caution. Investigating the real-world viability of the SRPL framework in autonomous driving, therefore, requires a comprehensive evaluation under conditions that mimic real-world deployment challenges such as sensor noise and domain variations. Such a comprehensive evaluation includes not only measuring standard performance metrics, but also systematically assessing the robustness of learned policies to sensor noise and their ability to generalize across diverse driving domains. In this paper, we make the following contributions: • We provide a systematic evaluation of SRPL’s effectiveness in real-world autonomous driving scenarios, demonstrating statistically significant improvements in success rates and cost reduction across multiple SafeRL algorithms on Waymo Open Motion Dataset (WOMD) and NuPlan datasets, while revealing algorithm-specific and context-dependent performance patterns. • We show through robustness analysis that these safety representations improve resilience to observational noise and provide insights into the policy output stabilization effect underlying this robustness benefit. • We identify asymmetric cross-dataset transfer, where agents trained on the diverse WOMD dataset generalize better to NuPlan than the reverse. In addition, we empirically demonstrate that SRPL augmentation improves domain generalization compared to non-SRPL methods. • We establish practical guidance for SafeRL algorithm selection through comprehensive cross-dataset and robustness evaluations, identifying which algorithms benefit from SRPL augmentation under different deployment scenarios. II RELATED WORK The field of learning-based autonomous driving has recently shifted its focus from procedurally generated scenarios [ 9 , 8 ] to large-scale, real-world datasets like WOMD [ 11 ] and NuPlan [ 6 ] . While these datasets have been valuable for motion forecasting and planning [ 27 , 26 , 6 ] , decision-making and control in autonomous driving remain challenging. Representation learning has become crucial to address these decision-making and control challenges, where compact representations capturing task-relevant information improve performance and sample efficiency. This is often achieved using auxiliary training signals or predicting future latent states [ 14 , 19 , 13 ] . A promising direction within this area is to develop representations that explicitly encode safety-relevant information by learning to predict proximity to future constraint violation, so agents can make more informed decisions [ 23 , 25 , 21 ] . Control policies for autonomous driving can be learned through two main paradigms: Imitation Learning (IL), which is sample-efficient but struggles with out-of-distribution scenarios, and RL, which discovers more robust policies but faces sample efficiency and safety challenges [ 20 ] . While recent work explores hybrid IL-RL methods [ 20 , 5 ] , this paper focuses specifically on SafeRL algorithms augmented with predictive safety representations to understand their isolated effectiveness in real-world scenarios. Concurrent work by [ 7 ] introduces V-Max, a comprehensive RL framework built on Waymax that provides standardized tools for training and evaluating various standard RL and IL algorithms in autonomous driving scenarios. While V-Max focuses on benchmarking different algorithmic approaches and system components (observation shaping, network architectures, reward shaping), our work