An LSTM-based Test Selection Method for Self-Driving Cars
Ali Güllü, Faiz Ali Shah, Dietmer Pfahl · 2025
Read An LSTM-based Test Selection Method for Self-Driving Cars by Ali Güllü, Faiz Ali Shah, Dietmer Pfahl (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03881.txt.
Introduction
Ali Gullu and co-authors tackle test choice for self driving cars, where full testing takes much time and cost. Their 2025 study centers on lane keeping systems. The goal is to skip easy tests and spend effort on the hard ones that may show faults.
Road traits such as bend angles and segment lengths are read as sequences. A long short-term memory model then sorts each test case as safe or unsafe. In a simulation setting, the model beat plain machine learning pickers on accuracy and precision. Its recall and F1 scores were on par with theirs. The authors offer the method as a new deep learning route to road test choice.
Catalog record
| CREATOR | Ali Güllü, Faiz Ali Shah, Dietmer Pfahl |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-03881
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An LSTM-based Test Selection Method for Self-Driving Cars Ali Güllü, Faiz Ali Shah, Dietmer Pfahl Abstract Self-driving cars require extensive testing, which can be costly in terms of time. To optimize this process, simple and straightforward tests should be excluded, focusing on challenging tests instead. This study addresses the test selection problem for lane-keeping systems for self-driving cars. Road segment features, such as angles and lengths, were extracted and treated as sequences, enabling classification of the test cases as “safe” or “unsafe” using a long short-term memory (LSTM) model. The proposed model is compared against machine learning-based test selectors. Results demonstrated that the LSTM-based method outperformed machine learning-based methods in accuracy and precision metrics while exhibiting comparable performance in recall and F1 scores. This work introduces a novel deep learning-based approach to the road classification problem, providing an effective solution for self-driving car test selection using a simulation environment. I Introduction Self-driving cars (SDCs), as key components of Cyber-Physical Systems, should be tested in simulation environments before being deployed in the real world in order to identify potential accidents, prevent financial losses, and not endanger human and animal lives [ 8 ] , [ 9 ] . Various simulation platforms, such as MATLAB/Simulink 1 1 1 MATLAB/Simulink , BeamNG.tech 2 2 2 BeamNG.tech , Gazebo 3 3 3 Gazebo , CarSim 4 4 4 CarSim , SUMO 5 5 5 SUMO , PreScan 6 6 6 PreScan , CARLA 7 7 7 CARLA , LGSVL 8 8 8 LGSVL , and AirSim 9 9 9 AirSim , have been developed for testing SDCs in the simulation world [ 10 ] , [ 12 ] . Despite the benefits of simulation platforms, testing vehicles in these simulation environments is costly in terms of time. To minimize the testing costs, it would be helpful to identify and execute only those tests that are likely to cause failures and exclude overly simple tests from the test suite. To address this problem, we developed a test selection model for lane-keeping scenarios in SDCs. The proposed model, named I ntelligent T est S elector for S elf- D riving C ars ( ITS4SDC ), is based on a long short-term memory (LSTM) recurrent neural network [ 23 ] that focuses on identifying challenging driving scenarios to improve test efficiency by selecting failure-triggering roads as test cases. The developed model is also used in the “Self-Driving Car Testing Tool Competition” , organized as part of the 18th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2025 10 10 10 ICST 2025 . To generate roads with diverse characteristics, the Frenetic road generation algorithm is used [ 1 ] . Frenetic is designed to create challenging roads that make it difficult for vehicles to stay within their lanes. It is one of the tools used in the SBST 2021 conference 11 11 11 SBST 2021 to produce road types where vehicles most frequently went off-road. The Frenetic algorithm has also been utilized in other contexts, such as [ 4 ] , [ 5 ] , [ 18 ] . To label generated roads as “PASS” (SAFE) or “FAIL” (UNSAFE), they are evaluated using the BeamNG.tech simulation platform, which offers realistic soft-body physics-based SDC testing. Its realistic physics, including friction forces and deformation during collisions, make it one of the most realistic simulation platforms for SDCs. A part of the labeled roads is then used as training data for models that aim at correctly classifying previously unseen roads as safe or unsafe. The rest of the labeled roads serve as the test set. The evaluation results demonstrate that our LSTM-based model outperforms a set of state-of-the-art machine learning models in accuracy and precision while showing comparable performance in recall and F1 metrics. The main contributions of this study are as follows: • Proposing a novel LSTM-based framework for efficient test selection for SDCs [ 24 ] . • Providing a comprehensive comparison between LSTM and machine learning models for road classification. • Preparation and public release of an open dataset for regression testing for SDCs [ 21 ] . This paper is structured as follows. Section II presents the background to the research presented in this paper. Section III presents related work on testing SDCs for lane-keeping scenarios. Section IV covers the generation and labeling of test cases, as well as dataset preparation and model training. Section VI compares the proposed ITS4SDC model with SDC-Scissor, offering a detailed performance analysis. Section VII summarizes the threats to validity. Section VIII highlights the advantages of the proposed model and outlines recommendations for future work. II Background II-A SDC Scenarios A scenario for SDCs represents a specific situation that the vehicle might encounter. An SDC scenario typically includes the vehicle itself, its environment, and surrounding factors such as roads, buildings, other vehicles or obstacles, and weather conditions [ 2 ] . These scenarios are generated in simulation environments to evaluate the behavior and performance of SDCs. Among the fundamental scenarios for SDCs is lane-keeping, which has been extensively studied in previous studies [ 3 ] - [ 5 ] , [ 14 ] , [ 18 ] - [ 20 ] . This scenario focuses on one of the most basic safety functions of SDCs: preventing the vehicle from leaving the road while considering the geometry of the road. To ensure that the SDCs can adapt to various road geometries, it is essential to thoroughly test the lane-keeping system. For this purpose, the vehicle must be tested on roads with diverse characteristics, such as varying geometries, features, and lengths, and its behavior must be closely examined. However, it is impractical to test every possible road configuration due to the infinite number of potential features and geometries. Therefore, selecting or prioritizing challenging tests that are likely to cause the SDCs to fail or leave the road is crucial. II-B SDC-Scissor Classification Approach Simulation tests are also time-consuming, and evaluating overly simple scenarios where the vehicle is almost guaranteed to succeed can be inefficient. To address the problem of selection “unsafe” test cases, the SDC-Scissor tool was proposed [ 4 ] . By analyzing road geometries and extracting relevant features, SDC-Scissor classifies roads as either “SAFE” or “UNSAFE” . Another significant challenge in testing SDCs is generating roads that adequately test the limits of their lane-keeping systems. To address this, the SBST Tool Competition was organized, aiming to encourage the development of road generation tools designed to create challenging road scenarios for SDCs [ 17 ] . II-C BeamNG.tech Simulation Environment The BeamNG.tech simulator was employed to test the lane-keeping system of SDCs. This simulator stands out among others due to its realistic physics engine, which closely models the unique dynamics of each vehicle, including skidding, traction, and braking behavior. These realistic physics provide a more accurate assessment of the performance of SDCs under diverse road conditions. II-D Road Generation for SDC Testing The first step in testing the lane-keeping systems of SDCs is generating realistic roads. The (x, y) two-dimensional road coordinates are used to create road textures by extending the coordinates laterally on both sides to form a realistic asphalt surface in the simulation environment. Each test starts with