Robust Planning for Autonomous Vehicles with Diffusion-Based Failure Samplers

Juanran Wang, Marc R. Schlichting, Mykel J. Kochenderfer · 2025

Read Robust Planning for Autonomous Vehicles with Diffusion-Based Failure Samplers by Juanran Wang, Marc R. Schlichting, Mykel J. Kochenderfer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-11991.txt.

Introduction

Juanran Wang and co-authors use a generative model in 2025 to make self-driving cars plan more safely at crossings. Intersections are a top site for crashes. The team trained a diffusion model to invent sensor noise patterns that would push a car toward a crash. Inputs were the position and speed of an intruder car at a four-way crossing.

They then squeezed the slow model into a fast one-step version with like quality. A planner samples the fast model's failure cases live and steers to avoid them. In simulations the planner cut both failure and delay rates versus a standard controller. The work shows how imagined failures can sharpen real decisions.

Catalog record

CREATORJuanran Wang, Marc R. Schlichting, Mykel J. Kochenderfer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-11991

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Robust Planning for Autonomous Vehicles with Diffusion-Based Failure Samplers Juanran Wang, Marc R. Schlichting, Mykel J. Kochenderfer Abstract High-risk traffic zones such as intersections are a major cause of collisions. This study leverages deep generative models to enhance the safety of autonomous vehicles in an intersection context. We train a 1000-step denoising diffusion probabilistic model to generate collision-causing sensor noise sequences for an autonomous vehicle navigating a four-way intersection based on the current relative position and velocity of an intruder. Using the generative adversarial architecture, the 1000-step model is distilled into a single-step denoising diffusion model which demonstrates fast inference speed while maintaining similar sampling quality. We demonstrate one possible application of the single-step model in building a robust planner for the autonomous vehicle. The planner uses the single-step model to efficiently sample potential failure cases based on the currently measured traffic state to inform its decision-making. Through simulation experiments, the robust planner demonstrates significantly lower failure rate and delay rate compared with the baseline Intelligent Driver Model controller. \renewrobustcmd \renewrobustcmd \newrobustcmd \B I Introduction Traffic intersections have remained a significant safety bottleneck. In 2022, almost 30% of traffic fatalities involve an intersection, and in 2023, over 50% of crashes resulting in fatalities or injury occur at or near intersections [ 1 , 2 ] . Due to the need to constantly monitor approaching vehicles and predict their intentions accurately, it is extremely challenging to safely navigate an intersection. In recent years, autonomous vehicle (AV) technology has made substantial progress in terms of safety and reliability. However, recent statistics still reveal that intersections are responsible for 55% of driverless car accidents, and there has been growing interest in developing robust autonomous driving systems that effectively navigate these high-risk areas [ 3 ] . This study develops a robust planning framework for an autonomous vehicle navigating a four-way intersection in the presence of an intruder vehicle. Our planning framework is “robust” in the sense that it is aware of the potential failures given the current state. Recent AV robust planning methods seek to mitigate failure-causing vulnerabilities including unseen environments as well as observation and tracking inaccuracies [ 4 , 5 , 6 ] . To the best of our knowledge, few of the frameworks proposed so far comprehensively account for the overall distribution of potential failures given the current relative positions and velocities of intruder vehicles. Inspired by recent safety validation frameworks, we use a denoising diffusion model to represent the broader distribution of potential failures of the autonomous vehicle. We train diffusion models to sample collision-causing observation error sequences based on the current state [ 7 , 8 ] . We perform knowledge distillation for the diffusion models to accelerate inference, enabling real-time generation of failure samples. The robust planning framework uses the distilled diffusion model to efficiently draw potential failure samples based on the real-time traffic situation at each decision step and considers a variety of potential failures during decision-making. In particular, the planner computes a trajectory to its destination that avoids the potential collisions. Our code is available on GitHub. 1 1 1 https://github.com/sisl/AV-Robust-Planning-Diffusion-Failure-Sampler Our main contributions include: • We build a real-time failure sampler for an autonomous vehicle at an intersection, which rapidly generates potential failure trajectories based on the current relative position and velocity of an intruder vehicle. • We use the real-time failure sampler to construct a robust planner which accounts for the overall distribution of potential failures for reliable decision-making. Fig. 1 : Our robust planner uses a distilled denoising diffusion model to sample potential failure cases based on the currently measured traffic state. It plans a path to the destination which avoids the potential failure cases. Section II provides an overview of current research in AV robust planning. Section III discusses the training of the failure sampler and the design of the robust planner. Section IV presents the experimental results for the real-time failure sampler and the robust planner. II Related Work Recent studies have combined techniques from both safety validation and trajectory planning to address a variety of uncertainties in autonomous driving. To adapt the autonomous vehicle to complex environments, Jaafra et al. [ 9 ] trained a deep reinforcement learning (RL) agent with an actor-critic architecture and multi-step returns, ensuring robustness against realistic urban driving conditions. Sun et al. [ 10 ] developed a Cartesian-to-Frenet projector that enables Frenet-based lane keeping on high-curvature roads. To better address unseen traffic scenarios during real-world operations, monitoring controllers that prioritize safety under special circumstances have been designed. Zeng et al . [ 5 ] developed a controller that defaults to a monitoring trajectory that ensures collision avoidance when a lane changing trajectory cannot be computed. Yang et al. [ 11 ] proposed a hybrid controller in which a rule-based policy takes over when the RL policy encounters a scenario not seen during training. To enhance robustness against tracking deviations, Kalaria et al. [ 12 ] designed a framework using robust tube MPC and and adaptive Kalman Filter to deal with actuator delays, Yu et al . [ 4 ] proposed a feedback control technique that bounds the tracking error of the actual trajectory to a geometric tube, and Zhang et al . [ 13 ] developed a gain-scheduling approach for robust lateral tracking. There has been growing interest in improving the robustness of AV planners by considering observation errors, especially those that are likely to cause failures. He et al . proposed a framework in which a Bayesian Optimization-based adversarial attacker introduces worst-case observation perturbations in an attempt to produce a collision, which helps an RL agent learn an optimal policy that is robust against risky observation errors [ 14 , 15 ] . Unal et al . proposed a genetic algorithm that introduces noise to the training data for object classifiers onboard an AV, making the classifiers more robust against data perturbations [ 6 ] . Nevertheless, these approaches account for relatively small sets of worst-case perturbations most likely to lead to failures. Few studies to date have comprehensively modeled the full distribution of potentially failure-causing perturbations for the purpose of robust planning. We believe the latter approach is advantageous because it informs the autonomous vehicle of a wider range of potential failures based on the current traffic state, enabling more robust decision-making in real time. Recent safety validation frameworks have demonstrated the ability of diffusion models to model the overall distribution of failure-causing observation perturbations for many cyber-physical systems [ 8 , 7 ] . In our study, we decide to develop a robust planning framework for an autonomous vehicle in a traffic intersection environment that uses a distilled diffusion model to obtain knowledge about a variety of likely failures based on the current traffic state. III Methods III-A Conditional Diffusion-Based Failure Sampling First, we build a generative model which synthesizes collision-causing observation disturbance sequences for the autonomous vehicle. Consider an autonomous vehicle navigating an intersection 2 2 2 https://github.com/Farama-Foundation/HighwayEnv with four road branches: south (bottom), north (top), west (left), and east (right). Each has two lanes for the two directions of travel. The ego vehicle must travel from the south branch to one of the other branches while avoiding an intruder.

Read the complete text