ML-SceGen: A Multi-level Scenario Generation Framework
Yicheng Xiao, Yangyang Sun, Yicheng Lin · 2025
Read ML-SceGen: A Multi-level Scenario Generation Framework by Yicheng Xiao, Yangyang Sun, Yicheng Lin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10782.txt.
Introduction
Yicheng Xiao, Yangyang Sun, and Yicheng Lin build a three-stage framework for generating driving test scenarios in a paper from 2025. Language models can draft scenarios, but users lose control and danger factors are hit or miss. The first stage turns a user's description into a functional scenario skeleton. Language model agents handle that translation. A logic solver then fills in comprehensive traffic inside uncontrolled intersections.
In the last stage, a language model tunes parameters to raise the scenario's criticality. The pipeline aims at scenarios that are both complete and controllably dangerous. Intersections without signals are the test setting throughout. The authors argue staged generation restores user control. Autonomous driving testers are the intended audience.
Catalog record
| CREATOR | Yicheng Xiao, Yangyang Sun, Yicheng Lin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-10782
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ML-SceGen: A Multi-level Scenario Generation Framework Yicheng Xiao, Yangyang Sun, Yicheng Lin Abstract Current scientific research witnesses various attempts at applying Large Language Models for scenario generation but is inclined only to comprehensive or dangerous scenarios. In this paper, we seek to build a three-stage framework that not only lets users regain controllability over the generated scenarios but also generates comprehensive scenarios containing danger factors in uncontrolled intersection settings. In the first stage, LLM agents will contribute to translating the key components of the description of the expected scenarios into Functional Scenarios. For the second stage, we use Answer Set Programming (ASP) solver Clingo to help us generate comprehensive logical traffic within intersections. During the last stage, we use LLM to update relevant parameters to increase the critical level of the concrete scenario. I INTRODUCTION Over the years, the development of autonomous driving has soared. Society witnessed the gradual advent of driverless cars, as the notion no longer stays online but can be captured by our own eyes. However, as the technology of autonomous systems is evolving, from ontology-based methods to deep learning methods [ 1 , 2 ] , it is always hard to generate scenarios that can ensure both comprehensiveness and critical levels. Comprehensiveness requires a thorough search in the action space while evaluating critical levels sometimes requires crash data that are somehow unpatterned. Current research in autonomous driving systems heavily relies on Machine Learning [ 3 , 4 ] , Deep Learning, or Reinforcement Learning methods, which leads to a lack of interpretability. The outcome usually fails to attain the expected controllability of the users. Also, these methods rely heavily on real-world data, which might be biased and expensive to achieve. Using simulators and generating scenarios within the simulators are becoming an affordable and effective alternate solution. Currently, there are three main types of simulators: • Render Engine-Based Simulator : These simulators, such as CARLA [ 5 ] , require manual modeling of scenes and objects. However, they face two key challenges: a domain gap, as the simulated environments often differ from reality, and the difficulty of constructing realistic and complex scenes. The scenario realism is restricted by asset modeling and rendering qualities, and adding digital assets to the scene is difficult to get started due to the complexity of the simulation system. • Video Generation-Based Simulators : These simulators utilize techniques like Diffusion Models to generate environments based on real-world data. While the results closely resemble reality, the process introduces significant randomness and lacks control, leading to occasional abrupt changes in the generated data. They struggle to maintain view consistency and face challenges in importing external digital assets due to the absence of 3D spatial modeling [ 6 ] . • Scene Reconstruction-Based Simulators : Examples like UniSim [ 7 ] leverage neural rendering methods such as NeRF (Neural Radiance Fields) [ 8 ] to reconstruct realistic scenes. A notable limitation is that these simulators struggle to incorporate external digital assets. From the above descriptions, we can discover the problems of the current generation methods in simulators: lack of controllability, bad scalability, and distance from reality [ 9 ] . We therefore hope to devise a scalable framework that supports the generation of scenarios from Functional to Logical, from Logical to Concrete. Our contributions are as follows: • We identify ASP solver to be a core part of our logical scenario generation framework to achieve comprehensiveness. • We leverage the ability of Large Language Model to help apprehend the possible danger factors and incorporate them in the generated scenarios to increase the critical levels. • We have conducted relevant experiments to demonstrate the controllability of our framework. II RELATED WORK II-A Chatscene and TTSG ChatScene is a novel large language model (LLM)-based agent designed to generate safety-critical scenarios for autonomous vehicles. Its primary function is to translate textual descriptions of traffic scenarios into executable simulations within the CARLA simulation environment. The combination of textual generation, scenario decomposition, knowledge retrieval from a pre-constructed database, and execution of simulation scripts supports ChatScene’s effectiveness in generating safety-critical scenarios. However, the language ChatScene chose to generate is Scenic, which is a less popular language than OpenDrive, OpenScenario format, and a less controllable domain-specific language for it only supports a limited range of control actions. Also, the generation process of ChatScene is unguided, which leads to a high error rate and requires post-processing of the generated result. [ 6 ] Text-to-traffic scene generation (TTSG) framework is to autonomously generate diverse and customizable traffic scenarios from natural language descriptions, enhancing the training of autonomous vehicles. The framework aims to overcome the limitations of traditional methods, which often restrict scenario diversity by relying on predefined paths and fixed conditions. Users can specify various parameters such as weather conditions, vehicle types, and road signals, while the system dynamically selects starting points and generates scenes from scratch, facilitating the representation of both critical and routine traffic situations. However, the problem with the method is that it cannot automatically generate comprehensive scenario under a given circumstances. [ 10 ] II-B Specification, Validation, and Verification of Autonomous System Marius and Joseph presented a multilevel semantic framework designed for the specification and validation of Autonomous Driving Systems (ADS) in 2021. [ 11 ] It addresses the complexities of integrating these systems with non-explainable AI technologies by formalizing physical environments as directed metric graphs. The framework introduces three logics: Metric Configuration Logic (MCL) for mapping parameters, Mobile Metric Configuration Logic (M2CL) for representing dynamic object distributions, and Temporal M2CL for time-related properties [ 12 , 13 ] . This systematic approach facilitates the creation of detailed traffic rules and dynamic scenarios while tackling validation challenges like runtime verification and satisfiability. For scenario generation, using the logical framework can help define logical scenarios and thus provide proper guidance towards concrete scenario generation. Li et al. presented a comprehensive framework for validating autonomous driving systems (ADS) by integrating an industrial simulator with a scenario generator and a runtime verification monitor based on the framework proposed above. [ 14 ] The proposed methodology focuses on systematically generating scenarios that explore high-risk situations, particularly at complex junctions, to uncover defects that standard random exploration might miss. II-C Answer Set Programming Solver Answer Set Programming (ASP) offers a simple and powerful modeling language to describe combinatorial problems as logic programs. The clingo system then takes such a logic program and computes answer sets representing solutions to the given problem. Clingo is part of the Potassco project for Answer Set Programming (ASP), combining a grounder (Gringo) and a solver (Clasp) into a single tool. Problems are expressed in terms of rules, facts, and constraints , which specify the relationships and conditions to be satisfied. Solutions, called answer sets, represent possible scenarios that satisfy all rules in the program . Usually, after formulating the problems into .lp file, the grounder will ground (Gringo) the program to generate a propositional formula, and the solver (Clasp) will solve the formula to find answer sets that satisfy constraints. An example of a calling program is shown as follows. Listing 1: Example Hanoi Program using ASP ⬇ % Facts peg ( a ; b ; c ). disk (1..6). init_on