Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RL
Ahmed Abouelazm, Johannes Ratz, Philip Schoerner, J. Marius Zöllner · 2025
Read Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RL by Ahmed Abouelazm, Johannes Ratz, Philip Schoerner, J. Marius Zöllner (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19146.txt.
Introduction
Ahmed Abouelazm and co-authors teach self-driving cars with a teacher that invents traffic, in a 2025 paper. Driving agents trained by reinforcement learning usually practice against scripted cars that behave predictably, so they flounder in the odd situations that cause real crashes. Focusing only on dangerous scenes is also wrong, since most driving is routine.
Their student-teacher setup lets a graph-based teacher, itself a learning system managing many agents, generate traffic behavior at many difficulty levels. The teacher watches the student driver's progress and raises or lowers the challenge to keep lessons in the sweet spot. The curriculum grows from behavior, not just scenery. The authors argue this balance of the mundane and the critical is what produces drivers that stay safe when real traffic does something strange.
Catalog record
| CREATOR | Ahmed Abouelazm, Johannes Ratz, Philip Schoerner, J. Marius Zöllner |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-19146
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RL Ahmed Abouelazm, Johannes Ratz, Philip Schoerner, J. Marius Zöllner Abstract Autonomous driving faces challenges in navigating complex real-world traffic, requiring safe handling of both common and critical scenarios. Reinforcement learning (RL), a prominent method in end-to-end driving, enables agents to learn through trial and error in simulation. However, RL training often relies on rule-based traffic scenarios, limiting generalization. Additionally, current scenario generation methods focus heavily on critical scenarios, neglecting a balance with routine driving behaviors. Curriculum learning, which progressively trains agents on increasingly complex tasks, is a promising approach to improving the robustness and coverage of RL driving policies. However, existing research mainly emphasizes manually designed curricula, focusing on scenery and actor placement rather than traffic behavior dynamics. This work introduces a novel student-teacher framework for automatic curriculum learning. The teacher, a graph-based multi-agent RL component, adaptively generates traffic behaviors across diverse difficulty levels. An adaptive mechanism adjusts task difficulty based on student performance, ensuring exposure to behaviors ranging from common to critical. The student, though exchangeable, is realized as a deep RL agent with partial observability, reflecting real-world perception constraints. Results demonstrate the teacher’s ability to generate diverse traffic behaviors. The student, trained with automatic curricula, outperformed agents trained on rule-based traffic, achieving higher rewards and exhibiting balanced, assertive driving. I Introduction Car accidents cause 1.19 million deaths annually, with road injuries the leading cause of death for ages 5–29 [ 1 ] , prompting research into safe and reliable autonomous vehicles. Traditional autonomous systems employ a modular structure with distinct modules for tasks like perception, localization, and planning. However, this approach often leads to error propagation and requires significant engineering effort. Additionally, as each module is optimized independently, the overall system may lack alignment toward a unified goal, such as ensuring safe and efficient driving [ 2 ] . End-to-end (E2E) driving systems aim to address these challenges by enabling joint learning across tasks, often mapping raw sensor data to control actions or intermediate representations. These systems enable joint optimization of perception, prediction, and planning, making them well-suited for complex scenarios and reacting to hazards [ 3 ] . Reinforcement learning (RL) has been explored as an E2E approach [ 4 ] , enabling systems to learn optimal driving policies through trial and error. Agents receive rewards based on task performance, such as maintaining lane position or avoiding collisions, allowing continuous refinement of their driving policy. RL agents are typically trained in simulations, enabling safe exploration without real-world risks [ 3 ] . Research Gap. RL agents learn through interaction with the environment, enabling them to handle traffic participant behaviors similar to those encountered during training. Simulation environments must encompass a wide range of diverse, common, and safety-critical behaviors to achieve robustness and generalization. However, most simulations rely on fixed non-player characters (NPCs) behaviors with predefined rules, maintaining constant speed or distance [ 5 ] . This limits the agent’s ability to generalize to unseen situations [ 6 ] . Similarly, prerecorded human driving behaviors [ 7 ] offer realism but are non-adaptive and rarely include safety-critical situations, contributing to the long tail problem [ 8 ] , where critical events are underrepresented. Recent work targets safety-critical behaviors by generating scenarios that provoke the self-driving vehicle (SDV) collisions or off-road events [ 8 , 9 ] . While effective for robustness against safety-critical situations, they neglect more frequent traffic patterns, potentially limiting generalization to everyday driving contexts. To overcome this limitation, more generalizable approaches such as multi-agent reinforcement learning have been explored. Multi-agent RL (MARL) methods [ 10 ] allow NPCs to make independent or cooperative decisions but primarily focus on traffic coordination rather than training SDV agents. To develop driving skills more comprehensively, some works adopt curriculum learning [ 11 ] , a technique inspired by education that trains agents on simpler tasks before progressively advancing to more complex ones. This approach improves learning efficiency and robustness. However, in autonomous driving, current research relies on manually crafted task sequences [ 12 ] , which fail to encompass a diverse range of NPC behaviors. Contribution. This work introduces a student-teacher framework for adaptively generating a behavior curriculum for surrounding NPCs that dynamically adapts to the student’s performance. By integrating MARL and curriculum learning, the framework enhances the generalization and robustness of the student, enabling it to handle a diverse spectrum of traffic behaviors. The key contributions of this work are: • Teacher Design : A novel MARL-based teacher capable of generating traffic behaviors with varying difficulty levels with a graph-based network and a novel reward. • Automatic Curriculum Algorithm : A method for automatically orchestrating the concurrent training of student and teacher components, creating an adaptive behavior curriculum. II related work This section reviews prior research on traffic simulation, with an emphasis on behavior generation techniques. Additionally, curriculum learning is discussed, highlighting advancements in automatic curriculum generation. Fig. 1 : Overview of the proposed behavior curriculum framework. The student (blue) and teacher (red) interact concurrently in the driving environment. The student learns a driving policy from sensor data, guided by a standard reward. The teacher dynamically orchestrates NPC behavior based on a fully observable state representation, the student’s performance, and an auxiliary input \lambda . II-A Scenario Generation Traditional Simulators like CARLA [ 5 ] generate rule-based NPC behavior, ensuring compliance with traffic laws, lane-keeping, and collision avoidance. However, they lack flexibility and require manual adjustments to parameters such as speed limits. In addition, they fail to simulate unpredictable behavior or dynamically adapt to agents’ actions. TrafficGen [ 13 ] addresses these limitations with a generative model trained on real-world scenarios to produce realistic vehicle placements and trajectories. It generates scenarios autoregressively, predicting long-term motion. However, it relies heavily on large datasets, struggles with out-of-distribution road topologies, and lacks adaptability to agent actions. TrafficSim [ 14 ] enhances the learning objective with a common-sense loss and predicts multi-modal trajectories to capture the uncertainty in agent interactions. However, the inclusion of a common-sense loss requires a differentiable simulator, making the approach computationally intensive. CoPO [ 10 ] eliminates the need for differentiable simulators by introducing a MARL algorithm that coordinates traffic flow, allowing NPCs to exhibit cooperative and competitive behaviors. While the aforementioned approaches effectively generate a wide range of traffic behaviors, they are not tailored to capture safety-critical situations. In contrast, AdvSim [ 6 ] generates safety-critical scenarios by modifying real-world traffic through physically plausible perturbations of agents’ trajectories. It employs black-box optimization (BBO) [ 15 ] to identify failure-inducing perturbations. However, BBO is computationally intensive in high-dimensional spaces [ 8 ] . KING [ 8 ] employs a gradient-based procedure to optimize trajectory perturbations iteratively, eliminating the need for BBO. However, this approach requires differentiable approximations of simulator aspects, such as vehicle dynamics, and loss components, such as collision. Liu et al. [ 9 ] use RL to edit scenarios by adding agents or perturbing their trajectories. However, pre-setting trajectories before scenarios unfold limits their ability to capture the impact of agents’ behaviors and interactions at specific time steps on the SDV. Conversely, FailMaker-AdvRL [ 16 ] leverages MARL to coordinate adversarial agents,