TRACE: Topology-aware Reconstruction of Accidents in CARLA for AV Evaluation Conference: 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering; July 05–09, 2026; Montreal, QC, Canada 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE Companion ’26), July 05–09, 2026, Montreal, QC, Canada DOI: 10.1145/3803437.3806426 ISBN: 979-8-4007-2636-1/2026/07 CCS: Software and its engineering Software safety CCS: Computer systems organization Robotics CCS: Software and its engineering Software verification

Nahian Salsabil, Sebastian Elbaum · 2026

Read TRACE: Topology-aware Reconstruction of Accidents in CARLA for AV Evaluation Conference: 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering; July 05–09, 2026; Montreal, QC, Canada 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE Companion ’26), July 05–09, 2026, Montreal, QC, Canada DOI: 10.1145/3803437.3806426 ISBN: 979-8-4007-2636-1/2026/07 CCS: Software and its engineering Software safety CCS: Computer systems organization Robotics CCS: Software and its engineering Software verification by Nahian Salsabil, Sebastian Elbaum (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22068.txt.

Introduction

Salsabil and Elbaum present TRACE, a pipeline redoing NHTSA crash reports as CARLA simulations. It pulls street-map data to keep true road topology. It uses language models to guess vehicles' start spots from road shapes and pre-crash moves, and makes sim trajectories from report data. With this tool, the authors built a set of 52 crash scenarios with crash kinds, road topologies, and pre-crash moves. The set gives an open tool for testing autonomous cars against real crashes.

Catalog record

CREATORNahian Salsabil, Sebastian Elbaum
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22243
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22068

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TRACE: Topology-aware Reconstruction of Accidents in CARLA for AV Evaluation Conference: 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering; July 05–09, 2026; Montreal, QC, Canada 34th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE Companion ’26), July 05–09, 2026, Montreal, QC, Canada DOI: 10.1145/3803437.3806426 ISBN: 979-8-4007-2636-1/2026/07 CCS: Software and its engineering Software safety CCS: Computer systems organization Robotics CCS: Software and its engineering Software verification Nahian Salsabil, Sebastian Elbaum Abstract . Validating Autonomous Vehicles (AVs) requires exposure to rare, safety-critical scenarios, infrequent in routine driving data. Existing benchmarks address this by generating synthetic conflicts or mapping accident descriptions to abstract road geometries, failing to capture the topological complexity of real-world crashes. We introduce TRACE , a pipeline that automates the reconstruction of NHTSA crash reports into high-fidelity CARLA simulations by (1) retrieving site-specific OpenStreetMap data to preserve exact road topology, (2) leveraging Large Language Models to infer vehicles’ initial state from road geometry and pre-crash maneuvers, and (3) generating simulation trajectories from semi-structured report data. Using this pipeline, we curated a benchmark of 52 diverse accident scenarios covering varied collision types, road topologies, and pre-crash maneuvers, providing a challenging open-source resource for testing AV systems against real-world failures. † † cc-license: by 1. Introduction The successful and responsible deployment of Autonomous Vehicles (AVs) relies on a robust evaluation process. Benchmarks are fundamental to this effort: they standardize evaluation, enable comparisons, and reveal system weaknesses. For AVs, this systematic evaluation is essential to ensure system dependability prior to widespread public adoption. This necessity has led to numerous evaluation suites ( 9 ; 25 ; 12 ; 27 ) and tools ( 1 ; 21 ; 26 ; 11 ; 10 ; 29 ) to create such benchmarks utilizing high-fidelity simulators like CARLA ( 6 ) . While early benchmarks primarily assess routine driving competencies, recent efforts have proposed to include safety-critical scenarios generated from accident reports. However, the few open-source specialized frameworks available oversimplify accidents, rendering limited reproduction fidelity in terms of road topologies, vehicle trajectories, and collision types. This limits the strength of software engineering methods towards evaluation and validation. To address these limitations in evaluation realism and critical scenario validation coverage, we have developed TRACE , a pipeline that systematically derives simulation scenarios from the NHTSA Crash Data ( 16 ) . This pipeline involves transformations of real-world maps to fit CARLA’s format and of analytical and statistical analyses to reconstruct the conditions present in crash reports. More specifically, it estimates the precise initial states (position and orientation) of the involved vehicles, and generates the specific trajectories that lead to the collision described in a crash report. In parallel, the pipeline retrieves the real map where the accident occurred. It then automatically instantiates the map and spawns the vehicles involved in the crash to reproduce the crash within the CARLA simulator. Our pipeline seeks the generation of scenarios that exhibit three key properties: (1) retain the road topology and signage of the accident site and (2) spawn the vehicles with the state and planned trajectories that lead to the reported accident sequence. Using this pipeline, we have constructed and manually curated a benchmark of 52 scenarios that exhibit diverse topology, vehicle trajectory, and crash type as defined by the FARS (Fatality Analysis Reporting System) Manual ( 15 ) . Each scenario package includes the original accident report, a summary of the crash, its corresponding map as retrieved by our pipeline, and the generated CARLA simulation scenario, which includes vehicles’ starting position, direction, speed, and trajectory waypoints. By providing this collection of contextually rich, accident-derived scenarios, this benchmark is intended to offer a more challenging and realistic resource for testing the robustness of autonomous driving systems. The TRACE benchmark and pipeline used in this paper are available at https://doi.org/10.5281/zenodo.19431924 , with the latest updates at https://github.com/NahianSalsabil/carla-benchmark . We envision TRACE serving three use cases: 1) Testing AVs collision avoidance. where researchers can replace vehicle control systems with their own to assess collision avoidance capabilities. 2) Stress testing perception and prediction. Developers can focus on the identified crash-prone topologies that have specific intersection geometries or visual occlusions to evaluate how they affect their agent’s behavior or their ability to anticipate the erratic trajectories of other vehicles. 3) Automated Test-Case Generation . TRACE facilitates the generation of diverse safety-critical training scenarios to address the scarcity of long-tail data required to train robust learned components. (a) At a T-intersection. (b) At a 4-way intersection. (c) While a vehicle stopped. (d) Changing traffic ways. (e) At an Angle. (f) Curve in Front-to-Front. Figure 1 . Sample scenarios generated by TRACE showing the diversity of topologies, collisions, and vehicle trajectories. 2. Background Our work relates to approaches generating safety-critical scenarios to test AVs and more specifically, to those aiming to increase the scenario realism by incorporating data from crash reports. Techniques for autonomous driving simulation are generally classified into three categories: (1) adversarial generation ( 1 ; 19 ; 28 ; 23 ) , which creates challenging environments through malicious agent behaviors but often lacks computational efficiency and diversity; (2) knowledge-based generation ( 2 ; 5 ; 20 ) , which utilizes predefined rules and constraints yet struggles with implementation complexity and the inability to capture the full spectrum of high-risk, non-compliant behaviors; and (3) data-driven generation ( 7 ; 25 ; 26 ; 24 ) , which leverages real-world data but is frequently limited by the scarcity of safety-critical events in standard driving datasets. Our work extends the data-driven approach by generating scenarios directly from real-world crash reports, thereby bypassing the “long-tail" distribution problem to focus on confirmed collision events. Figure 2 . TRACE pipeline. The Extractor parses NHTSA crash reports. The Map Reconstructor retrieves and converts OSM data to OpenDRIVE. The Scene Reconstructor uses an LLM-based State Estimator to infer vehicle trajectories for the Launcher to execute in simulation. Scenario generation approaches that incorporate elements from crash reports can add realism. Numerous datasets of crash reports are available, such as the NHTSA Crash Investigation System ( 17 ) , the California Autonomous Vehicle Collision Reports ( 3 ) , and the European Road Safety Observatory ( 8 ) . Existing techniques for generating scenarios from crash reports vary significantly in fidelity and accessibility. Early tools like AC3R ( 11 ) and SoVAR ( 10 ) are publicly available but rely on abstract geometries or generic map-matching, which fail to capture precise real-world topologies; additionally, AC3R uses ad-hoc mechanism for data extraction and only focuses on the impact, while SoVAR’s template-based extraction limits its ability to handle diverse accident descriptions. More recent frameworks like CrashAgent ( 13 ) and AccidentSim ( 29 ) utilize MLLMs and high-fidelity simulators like CARLA and LGSVL, yet their validation is often restricted to synthetic layouts, and neither their code nor datasets are available for verification. SAFE ( 14 ) is available and similar in its use of MLLMs, but it relies on a domain-specific language (DSL) to synthesize road environments, which again lacks the geospatial target fidelity. Our approach overcomes these limitations by (1) reconstructing exact road networks via OpenStreetMap, (2) employing LLM-supported state

Read the complete text