DriveSafer: End-to-End Autonomous Driving with Safety Guidance
Shounak Sural, Ragunathan (Raj) Rajkumar · 2026
Read DriveSafer: End-to-End Autonomous Driving with Safety Guidance by Shounak Sural, Ragunathan (Raj) Rajkumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16737.txt.
Introduction
Sural and Rajkumar propose DriveSafer, a safety system for end-to-end self-driving planners that learns from failures. Generative planners still fail badly in safety-critical scenes, often by breaking physical limits and safety rules. DriveSafer steers planners toward safe behavior with safety limits in training and safety guidance at run time. On the NAVSIM test set, it cuts catastrophic failures against the DiffusionDrive model by 48%, with over 65% fewer drivable-area compliance failures.
Catalog record
| CREATOR | Shounak Sural, Ragunathan (Raj) Rajkumar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19045 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16737
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DriveSafer: End-to-End Autonomous Driving with Safety Guidance Shounak Sural, Ragunathan (Raj) Rajkumar Abstract End-to-End (E2E) autonomous driving models have shown growing capability in recent years, with performance improving on increasingly challenging benchmarks. However, modern generative E2E planners still suffer from a substantial number of catastrophic failures in safety-critical scenarios. We find that many such failures arise from violations of physical constraints and safety requirements, leading to unsafe behavior. Motivated by this finding, in this paper, we focus on improving safety outcomes in generative end-to-end driving with a targeted reduction of catastrophic planning failures, instead of enhancing average planning quality. Towards this end, we propose DriveSafer , a failure-aware safety framework for end-to-end planners. DriveSafer explicitly steers generative planners towards safe behaviors leveraging both training-time safety constraints and inference-time safety guidance. Compared to the state-of-the-art DiffusionDrive model, on the NAVSIM benchmark, DriveSafer reduces the number of catastrophic failures ( PDMS=0 ) by 48 % 48\% , with over 65 % 65\% reduction in drivable-area compliance failures. 1 Introduction End-to-End (E2E) autonomous driving models have recently become increasingly popular. Compared to traditional modular and rule-based architectures, these models show promise in addressing more complex scenarios with a large number of interacting agents, and in long-tailed scenarios aided by the fusion of vision and language with internet-scale datasets. While these E2E models handle many situations well, they can also be surprisingly brittle in many commonly seen situations which are straightforward for humans and even rule-based systems to handle. These misbehaviors typically show up in the long tail in end-to-end autonomous driving benchmarks such as NuPlan 2 and NAVSIM 8 and long-tail-focused datasets such as KIT-E2E 30 and WOD-E2E 37 . To identify many such failure cases, some benchmarks also employ CARLA 9 simulation allowing closed-loop testing with benchmarks such as Bench2Drive 14 and Longest6 6 . Importantly, strong average benchmark performance for models, however, still masks a long tail of catastrophic failures. While E2E models built using popular benchmarks have improved planning over the years, even the best-performing models such as DiffusionDrive 23 still suffer from catastrophic failures that simple rule-based planners can easily avoid. To reduce the failure modes of existing E2E models, a careful evaluation of safety is necessary. Many existing E2E models are benchmarked using the L2 distance error metric from human-driven ground truth trajectories as the primary metric on datasets such as NuScenes. While imitation learning minimizing the L2 error from ground truth offers useful insights, such metrics can be surprisingly easy to imitate without adding actual functional value. Methods such as AD-MLP 39 and Ego-MLP 22 have shown that a surprisingly simple MLP model which does not even consume sensor inputs can outperform state-of-the-art methods on the NuScenes benchmark which optimizes only for trajectory error. Such “blind” models that achieve state-of-the-art performance clearly underscore the need for better benchmarks and different evaluation metrics. NAVSIM 8 is one such benchmark that has standardized evaluation metrics which are aligned with driving safety. Their proposed Predictive Driver Model Score (PDMS) metric comprises collision checks, drivable area compliance, comfort, time-to-collision and driving direction compliance. These metrics are much better indicators for a safe planner. NAVSIMv2 3 introduces the Extended PDMS metric that further incorporates traffic-light compliance, lane-keeping and extended comfort metrics for a comprehensive safety-conscious evaluation. Earlier proposed “blind” MLP models which can exhibit unsafe behaviors do not perform favorably under these extended evaluation metrics. Many modern state-of-the-art methods report their performance on NAVSIM in terms of the PDMS score, but there is less emphasis on analyzing catastrophic failure cases in these models. In this paper, we specifically consider a PDMS score of zero in the NAVSIM E2E planning benchmark as a catastrophic failure, since its occurrences can potentially be hazardous and life-threatening in real-world driving. A PDMS score of 0 is assigned in the following cases: (i) the ego vehicle collides with another moving object such as a vehicle or a pedestrian, (ii) the ego vehicle drives off the permitted drivable region possibly onto a sidewalk or a solid divider, and (iii) the ego vehicle motion differs significantly from the intended driving direction, such as going straight when explicitly directed to turn right. Thus, we take a failure-centric view of safe driving behavior and focus on reducing these catastrophic failure cases as opposed to average benchmark performance. We carefully evaluate such failure cases and propose a framework called DriveSafer that strategically reduces these failures. Correspondingly, our contributions are as follows: • We propose DriveSafer , a failure-centric safety enforcement framework for E2E planning that directly targets reducing catastrophic failures using synergistic training- and inference-time safety enforcement. • We introduce a safety-aware training strategy with specialized physics-informed and safety-aware losses along with lightweight and integrated forecasting and planning. • We develop an inference-time safety guidance method that rejects infeasible trajectories and steers generated trajectories towards safety-compliant ones. • DriveSafer reduces catastrophic failures ( PDMS=0 ) on the NAVSIM benchmark by 48\% over the state-of-the-art DiffusionDrive method while also improving overall PDMS. Figure 1 : Overview of our DriveSafer framework 2 Related Work End-to-End Autonomous Driving: Early E2E studies focused on generating driving commands such as steering angles and acceleration/brake commands directly from raw sensor inputs through imitation and reinforcement learning 7 ; 4 . However, these methods do not scale across different platforms due to varying vehicle dynamics. It is now well understood that low-level actuation commands are best generated through specialized controllers for each vehicle given a local motion plan. Hence, recent end-to-end autonomous driving literature focuses on generating local motion plans from sensor data. E2E methods use strategies such as learning scene representations in the BEV space 11 ; 6 ; 29 ; 34 , learning vectorized architectures 15 ; 5 , vision-language action model-based planning 32 ; 25 ; 40 ; 44 ; 10 , world model-based planners 18 ; 33 ; 42 ; 31 and diffusion-based generative planners 43 ; 23 . Datasets for Safety in Autonomous Driving: Another direction of research focuses on enforcing safety in autonomous driving via datasets and simulation that critically evaluates safety, especially in rare and long-tailed scenarios. For example, NuPlan evaluates full-scale E2E behavior using closed-loop simulation. NAVSIMv2 3 uses neural reconstruction to generate alternate scenarios complete with generated sensor data for pseudo simulation. WOD-E2E 37 uses millions of miles of Waymo driving data to mine long-tailed scenarios that occur rarely in daily life ( <0.03\% of cases). Bench2Drive 14 models challenging interactions and harsh weather conditions in CARLA 9 for safety evaluation of models. Methods for Safe Autonomous Driving: Another branch of work aims to make driving policies safer through methods such as CAT 41 which turns simple real-world scenarios into alternate plausible safety-critical ones to perform closed-loop adversarial training. DriveAdapter 13 injects hand-crafted safety rules into a teacher-student paradigm for training. Control Barrier Functions are sometimes used to enforce safety in planning 36 ; 1 ; 16 . LEAD 24 narrows the sim2real gap by minimizing unrealistic privileged information available to planning experts in the CARLA simulator 9 . Similar to LLM safety research, Reinforcement Learning-based alignment ideas have been applied to E2E autonomous driving with methods such as TrajHF 17 , Carl 12 and DriveDPO 26 . Unlike prior