Generating Traffic Scenarios via In-Context Learning to Learn Better Motion Planner

Aizierjiang Aiersilan · 2024

Read Generating Traffic Scenarios via In-Context Learning to Learn Better Motion Planner by Aizierjiang Aiersilan (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18086.txt.

Introduction

Aizierjiang Aiersilan proposes a cheap way in 2024 to make rare, risky traffic scenes for training self-driving planners. Planners learn from curated data, but rare danger cases are scarce in real records. Hand-building scenes in a simulator costs much human time. His method writes scenes as scripts and lets a large language model draft them from plain text prompts.

The scripts run in a simulator to produce the scenes. Planners trained on a mix of this fake data and real data did far better than planners trained on real data alone. The gain held across existing planner designs. The work offers a low-cost path to safer driving systems.

Catalog record

CREATORAizierjiang Aiersilan
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-18086

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generating Traffic Scenarios via In-Context Learning to Learn Better Motion Planner Aizierjiang Aiersilan Abstract Motion planning is a crucial component in autonomous driving. State-of-the-art motion planners are trained on meticulously curated datasets, which are not only expensive to annotate but also insufficient in capturing rarely seen critical scenarios. Failing to account for such scenarios poses a significant risk to motion planners and may lead to incidents during testing. An intuitive solution is to manually compose such scenarios by programming and executing a simulator (e.g., CARLA). However, this approach incurs substantial human costs. Motivated by this, we propose an inexpensive method for generating diverse critical traffic scenarios to train more robust motion planners. First, we represent traffic scenarios as scripts, which are then used by the simulator to generate traffic scenarios. Next, we develop a method that accepts user-specified text descriptions, which a Large Language Model translates into scripts using in-context learning. The output scripts are sent to the simulator that produces the corresponding traffic scenarios. As our method can generate abundant safety-critical traffic scenarios, we use them as synthetic training data for motion planners. To demonstrate the value of generated scenarios, we train existing motion planners on our synthetic data, real-world datasets, and a combination of both. Our experiments show that motion planners trained with our data significantly outperform those trained solely on real-world data, showing the usefulness of our synthetic data and the effectiveness of our data generation method. University of Macau [email protected] Code — https://ezharjan.github.io/AutoSceneGen Introduction Efficiently evaluating autonomous vehicles (AVs) under diverse real-world challenges on a limited budget is crucial for ensuring their safety and sustaining long-term growth in the AV industry. While existing works on vehicle or pedestrian trajectory prediction, such as TrafficPredict ( Ma et al. 2019 ) , Pishgu ( Alinezhad Noghre et al. 2023 ) , TraPHic ( Chandra et al. 2019 ) , and MSRL ( Wu et al. 2023 ) , rely on real-world datasets like ApolloScapes ( Huang et al. 2018 ) , NGSIM ( Kovvali, Alexiadis, and Zhang PE 2007 ) , UCY ( Lerner, Chrysanthou, and Lischinski 2007 ) , and ETH ( Pellegrini et al. 2009 ) , which may not fully capture the complexities of modern trajectory prediction scenarios involving both AVs and non-AVs, our custom virtual dataset, generated using our framework “AutoSceneGen” , produces improved prediction results. Most importantly, generating and synthesizing data using our prompt-based, configurable, and AI-driven AutoSceneGen framework is more cost-effective and efficient than collecting real-world datasets such as Argoverse2 ( Wilson et al. 2023 ) , Waymo ( Ettinger et al. 2021 ) , Round ( Krajewski et al. 2020 ) , Ind ( Bock et al. 2020 ) , nuScenes ( Caesar et al. 2020 ) , Argoverse ( Chang et al. 2019 ) , Highd ( Krajewski et al. 2018 ) , and ApolloScapes. Our framework automatically generates datasets with real-world features and provides easy control over dataset heterogeneity through scenario descriptions, especially for safety-critical scenarios. Status quo. Despite the challenges and inefficiencies in the testing phase of AVs, significant progress has been made in optimizing the testing of individual modules, such as motion planners. However, most efforts have focused on evaluating trajectory prediction models in simulation environments ( Shet et al. 2023 ; Li et al. 2023 ) . While simulations reduce testing costs related to budget and safety, creating realistic and logically specific scenarios often requires substantial human effort, though still less than testing AVs on real roads. Additionally, safety evaluation is typically performed module-by-module, demanding considerable resources to create safety-critical scenarios within simulators. Scenario-based evaluation methods have largely focused on coverage, unsafe scenarios, and indicator-estimation tests ( Jesenski et al. 2019 ; Amersbach and Winner 2019 ) . Recent advancements in scenario generation prioritize data-driven methods ( Sun et al. 2024 ) , with generative models trained on domain-specific data for traffic scenario generation ( Tan et al. 2023 ) . However, these methods struggle with low-resource scenarios, such as corner cases and accidents, due to limited data availability. There is a clear need for a universal, general, and budget-conscious framework that enhances traffic scenario heterogeneity automatically through scenario descriptions. This would expedite the simulation and testing process. While advanced AI tools, like large language models (LLMs), show promise, they often require extensive post-processing to correct inaccuracies or generate overly generic results. Our framework addresses these issues by automating complex scenario generation in a cost-effective manner, overcoming the limitations of low-resource scenarios and optimizing the use of simulator functionalities to ensure scenarios are diverse, realistic, and reflective of challenging real-world situations, as highlighted by Bogdoll et al. (2021) . Contributions. Scenario generation has traditionally been a manual process requiring significant human effort. However, with the advancement of LLMs, there is now an opportunity to leverage AI to efficiently generate specific traffic scenarios. Building on prior research, the main contributions of this study are as follows: • A universal, general, and cost-effective framework, “AutoSceneGen” , is proposed to automatically enhance the heterogeneity of traffic scenarios through scenario descriptions, thereby accelerating the simulation and testing process. • AutoSceneGen leverages in-context learning (ICL) of LLMs, eliminating the need for training or fine-tuning generative models for scenario generation tasks. • The scenarios generated by AutoSceneGen were demonstrated to produce better datasets, leading to improved training results for motion planners. • AutoSceneGen automatically categorizes the generated scenarios based on their descriptions, eliminating the need for downstream annotation and facilitating the training of motion planners in open-world scenarios. • AutoSceneGen is a modular framework with dynamic components, allowing easy substitution of its generative model and simulation engine used for scenario generation and data collection. Related Work Scenario Generation and Gaps Existing works, such as LCTGen ( Tan et al. 2023 ) , CSG ( Xinxin, Fei, and Xiangbin 2020 ) , and TrafficGen ( Feng et al. 2023 ) , focus on generating specific traffic scenarios but often rely on real-world data or are limited by the functionalities of toolkits like ScenarioNet ( Li et al. 2024a ) . In contrast, AutoSceneGen bridges this gap by enabling users to efficiently collect AI-generated scenarios without relying on real-world data. The concept of a “scenario” revolves around the “logic” governing objects within a scene. Go and Carroll (2004) defined a scenario as a detailed account involving actors, background details, objectives, and sequences of actions. In Autonomous Driving (AD), testing traditionally focuses on perception, motion planning, and decision-making modules ( Huang et al. 2016 ) . Simulators such as CARLA ( Dosovitskiy et al. 2017 ) and AirSim ( Shah et al. 2018 ) have been developed to support these tests. Goyal et al. (2023) proposed a method to configure a scenario and automatically generate similar variants, while König et al. (2024) introduced a formal model for test case generation and simulation verification. However, these approaches rely on abstract models, making them less scalable than prompt-based generation methods. Several studies have explored augmenting traffic data and generating rare objects using ChatGPT ( Alam 2020 ; Xinxin, Fei, and Xiangbin 2020 ) . In this context, automatic scenario generation refers to creating scenarios from user prompts without detailed manual intervention. Zhang et al. (2023) reviewed the

Read the complete text