Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function

Satyesh Shanker Awasthi, Mohammed Irshadh Ismaaeel Sathyamangalam Imran, Stefano Arrigoni, Francesco Braghin · 2025

Read Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function by Satyesh Shanker Awasthi, Mohammed Irshadh Ismaaeel Sathyamangalam Imran, Stefano Arrigoni, Francesco Braghin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-22769.txt.

Introduction

Satyesh Shanker Awasthi and three co-authors hunt for the rare driving scenarios that break self-driving software, using an optimizer that learns where to look, in this 2025 paper. Validating an autonomous driving function means exploring an enormous space of possible situations, and brute-force testing burns simulation budgets on endless uneventful miles. The failures hide in corners nobody enumerates.

Their framework applies Bayesian optimization, which builds a statistical guess about where danger lives and spends each new simulation where the guess is most uncertain or most alarming. Against an MPC-based motion planner, it surfaced hazardous events, including driving off the road, using orders of magnitude fewer simulations than a structured design-of-experiments sweep. The study also probes whether the method scales as scenarios gain dimensions. And it asks whether the method can find several distinct critical regions, not just the first one it trips over.

Catalog record

CREATORSatyesh Shanker Awasthi, Mohammed Irshadh Ismaaeel Sathyamangalam Imran, Stefano Arrigoni, Francesco Braghin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-22769

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bayesian Optimization applied for accelerated Virtual Validation of the Autonomous Driving Function Satyesh Shanker Awasthi, Mohammed Irshadh Ismaaeel Sathyamangalam Imran, Stefano Arrigoni, Francesco Braghin Abstract Rigorous VV ( VV ) of ADF is paramount for ensuring the safety and public acceptance of AV . Current validation relies heavily on simulation to achieve sufficient test coverage within the ODD ( ODD ) of a vehicle, but exhaustively exploring the vast parameter space of possible scenarios is computationally expensive and time-consuming. This work introduces a framework based on BO ( BO ) to accelerate the discovery of critical scenarios. We demonstrate the effectiveness of the framework on an MPC ( MPC )-based motion planner, showing that it identifies hazardous situations, such as off-road events, using orders of magnitude fewer simulations than brute-force DoE ( DoE ) methods. Furthermore, this study investigates the scalability of the framework in higher-dimensional parameter spaces and its ability to identify multiple, distinct critical regions within the ODD of the motion planner used as the case study . I Introduction I-A Why virtual testing and validation is necessary? AV ( AV ) promise to revolutionize urban and highway mobility throughout the world. However, the reliability of autonomous driving needs to be proven sufficiently and in little time as possible. On-road testing of HAV ( HAV ) alone is not sufficient to prove the reliability of ADS ( ADS ), as high-risk driving situations are unlikely to be encountered randomly in real-world miles obtained from naturalistic driving [ 1 , 2 ] . The space of possible scenarios is effectively infinite, making exhaustive testing infeasible [ 3 ] . Moreover, experimental testing might not even be possible in the early VV phase of ADS or might be time-consuming and costly [ 4 ] . Thus, simulation-based virtual testing is essential for AV safety validation [ 5 , 6 ] . I-B Regulations regarding virtual validation In recent years, international and regional regulations have recognized the reasons stated above and have initiated accommodating and standardizing, testing and certification of such vehicles. The World Forum for Harmonization of Vehicle Regulations (WP.29) drafts define “virtual testing” and allow limited use of simulation in type approval [ 7 ] . The UNECE-R157 regulation for ALKS ( ALKS ) already permit certain SiL ( SiL )/ HiL ( HiL ) tests for variant approval [ 7 ] . ASAM ( ASAM ) developed a holistic best practice blueprint to exemplify the diverse testing strategies [ 8 ] . In practice, companies like TÜV SÜD advocate a modular validation pipeline by mixing MiL ( MiL ), SiL , and HiL tests and systematic scenario catalogs for type approval [ 9 , 10 , 11 ] . I-C Fallbacks of virtual testing However, even simulation-based testing can be computationally intensive. Brute-force methods like combinatorial testing and Monte Carlo simulation typically require a prohibitively large number of runs for confidence [ 12 , 13 ] . Advanced uncertainty quantification techniques account for variations in environmental conditions, sensor noise, and behavioral uncertainties, providing robust statistical foundations for safety claims [ 14 , 15 ] . Sufficient scenario coverage can lead to a significant effort in simulation time and cost and is directly proportional to the number of parameters chosen. To overcome these limitations, generally, two solutions have been explored. The first strategy involves increasing the computational power by implementing massive parallelization of the simulations [ 16 , 17 ] . The second and more efficient option reduces the number of simulation by using scenario-based frameworks for expediting the scenario search for higher risk scenarios [ 12 , 3 ] . These tests result in the critical scenarios which could then be retested on the proving grounds for higher fidelity [ 11 ] . While testing only the most critical situations is sufficient for ’type approving’ established systems, a different approach is needed for self-driving systems that are still in development. For these developing systems, safety and performance verification must involve continuous and rapid simulations of both everyday driving situations and high-risk edge cases. These solutions are discussed more in detail in Section II . I-D Contributions This paper presents a virtual testing and homologation framework in which the critical scenario discovery is formulated as a BO ( BO ) problem. This general validation workflow can be applied to any logical scenario with a given number of input dimensions within a defined Operational Design Domain (ODD). The performance of the framework was further demonstrated in a case study on an autonomous driving motion planner previously developed by the authors [ 18 ] . The main contributions of this paper are as follows: 1. BO ( BO ) framework: We formulate critical scenario discovery as a BO ( BO ) problem to find scenario parameters that maximize a safety-violation metric. The framework is agnostic to the scenario specification, allowing any logical scenario with continuous parameters within an ODD to be directly integrated into the Bayesian Optimization loop. 2. Application on a developed algorithm: The works present in the literature either focus on early detection of criticality from standard traffic simulations or they are based on naturalistic datasets. To the best knowledge of the authors, this is the first work demonstrating the use of such a framework on a developed ADF . 3. Scalability and multiple critical cluster identification: As the number of input dimensions increases, the method consistently identifies failure clusters using one to two orders of magnitude fewer simulations than the equivalent brute-force, full-factorial baseline. Furthermore, the framework successfully locates failures across different types of critical zones that were initially identified through the comprehensive baseline analysis in the applied case study. I-E Arrangement of the paper The rest of this paper is organized as follows. Section II reviews work related to acceleration of virtual testing techniques. Section III discusses the developed framework in detail, including the optimization problem formulation. Section IV describes the co-simulation methodology as well as the autonomous driving function used to validate the framework. Section V outlines the outcomes and results of the simulations. Finally, Section VI concludes and suggests future work. II Related Work A VV ( VV ) approach using scenario-based testing for HAV within a highway ODD was defined in the PEGASUS project [ 19 ] . Scenarios within this ODD were classified into functional, logical, and concrete levels to build a comprehensive simulation catalog [ 20 ] . This methodology uses extensive simulation to identify critical parameter regions, which are subsequently validated on test tracks [ 21 ] . Follow-on projects, such as V&V-Method and SETLevel, have since expanded the ODD to include urban intersection scenarios [ 22 , 23 ] . Standard random or combinatorial testing is often inefficient at identifying rare failure events. Given that critical scenarios are rare, prioritizing their discovery is essential for reducing testing redundancy. Consequently, recent research has increasingly focused on learning-based and optimization-driven methods to rapidly converge on these safety-critical scenarios. Several prominent approaches from recent literature are discussed below. One prominent and obvious strategy involves parallel and distributed simulation. In the field of mobility testing, solutions to simulate thousands of scenarios simultaneously, achieving significant improvements in computational speed have been explored. For instance, [

Read the complete text