Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving
Ahmed Abouelazm, Jonas Michel, Helen Gremmelmaier, Tim Joseph, Philip Schörner, J. Marius Zöllner · 2025
Read Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving by Ahmed Abouelazm, Jonas Michel, Helen Gremmelmaier, Tim Joseph, Philip Schörner, J. Marius Zöllner (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06737.txt.
Introduction
Ahmed Abouelazm and co-authors wrote a 2025 paper on teaching self driving cars with reward based learning. Such systems learn by trial and error, guided by a reward score. Poorly built rewards have known traps, and safety has mostly been scored as a flat penalty for a crash.
That flat view skips the risk in moves that lead up to a crash. The team lays out driving goals in a ranked stack and scales each goal so its weight is clear. They add a fresh risk goal based on an oval risk field and on known safe gap rules. Tests at unsignalized crossings with varied traffic showed fewer crashes with the new reward.
Catalog record
| CREATOR | Ahmed Abouelazm, Jonas Michel, Helen Gremmelmaier, Tim Joseph, Philip Schörner, J. Marius Zöllner |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-06737
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving Ahmed Abouelazm, Jonas Michel, Helen Gremmelmaier, Tim Joseph, Philip Schörner, J. Marius Zöllner Abstract Reinforcement Learning (RL) is a promising approach for achieving autonomous driving due to robust decision-making capabilities. RL learns a driving policy through trial and error in traffic scenarios, guided by a reward function that combines the driving objectives. The design of such reward function has received insufficient attention, yielding ill-defined rewards with various pitfalls. Safety, in particular, has long been regarded only as a penalty for collisions. This leaves the risks associated with actions leading up to a collision unaddressed, limiting the applicability of RL in real-world scenarios. To address these shortcomings, our work focuses on enhancing the reward formulation by defining a set of driving objectives and structuring them hierarchically. Furthermore, we discuss the formulation of these objectives in a normalized manner to transparently determine their contribution to the overall reward. Additionally, we introduce a novel risk-aware objective for various driving interactions based on a two-dimensional ellipsoid function and an extension of Responsibility-Sensitive Safety (RSS) concepts. We evaluate the efficacy of our proposed reward in unsignalized intersection scenarios with varying traffic densities. The approach decreases collision rates by 21% on average compared to baseline rewards and consistently surpasses them in route progress and cumulative reward, demonstrating its capability to promote safer driving behaviors while maintaining high-performance levels. ML Machine Learning CNN Convolutional Neural Network DL Deep Learning AD Autonomous Driving I Introduction End-to-End (E2E) approaches for autonomous driving have gained significant attention in recent years [ 1 ] . Unlike modular frameworks, which divide the driving task into distinct components such as perception, prediction, and planning — each requiring significant manual design, E2E approaches directly learn decision-making from raw sensor inputs. This enables neural networks to learn efficient representations of their interactions with the environment, minimizing reliance on human-engineered modules. As a result, they achieve greater flexibility and adaptability by dynamically learning from real-world experiences [ 2 ] . E2E driving can be realized through Reinforcement Learning (RL). RL offers a flexible and adaptable learning framework where agents learn through direct interactions with the environment [ 3 ] . The reward function guides the learning process, reinforcing actions that yield desirable outcomes and penalizing ineffective and dangerous behaviors [ 4 ] . By maximizing cumulative reward, RL agents improve their decision-making, enabling them to navigate complex environments and achieve their goals [ 5 ] . Research Gap. Autonomous driving is a complex domain characterized by multiple objectives, often conflicting in nature, such as balancing progress toward the goal with ensuring safety. Designing a reward function that effectively captures these complexities is a significant challenge [ 6 ] . Inadequately designed reward functions can result in suboptimal driving policies and potentially hazardous behaviors, particularly in critical situations. Despite their importance, recent RL research has often underemphasized reward design, focusing instead on learning algorithms or handling different state and action spaces [ 7 ] . In this paper, we aim to address this oversight by improving the reward function to better capture the complexities of autonomous driving. Fig. 1 : Example of irrational behavior in RL agents caused by sparse safety objective formulation and poor handling of the conflict between safety and progress objectives [ 6 ] . This research is particularly driven by insights from [ 6 ] , which identifies different failure cases arising from poorly designed rewards commonly employed in autonomous driving. Fig. 1 demonstrates a common misdesign in the reward function that leads to catastrophic outcomes, even in a simple scenario where a single static obstacle blocks the agent’s path to its goal. Unlike a human driver, who would wait indefinitely in such a scenario while the obstacle persists, the agent can opt to collide with the obstacle rather than remain stationary. This seemingly irrational behavior arises because the accumulated progress penalty associated with extended waiting can outweigh the collision penalty. This insufficient handling of the conflict between safety and progress stems from the sparsity of the safety objective, only penalizing collision without considering the associated risk of preceding actions. Adjusting the weights assigned to the safety and progress objectives may mitigate this issue. Still, it fails to resolve the underlying problem fully. Contribution. This work enhances the driving policies of RL agents by analyzing and extending the formulation of reward objectives, with a particular emphasis on risk awareness. The key contributions of this work are: • Hierarchical Structuring of Objectives : Introducing a structured reward function that organizes driving objectives hierarchically as a directed graph. • Enhanced Objective Formulation : Proposing refined and normalized formulations for each driving objective to improve interoperability and comparability. • Advanced Risk-Aware Objective : Developing a driving risk objective that integrates geometric and dynamic risk factors through a two-dimensional ellipsoid model. II Related Work In this section, we review related work on the use of RL in autonomous driving, covering different decision-making levels, sensor configurations, and action spaces, as well as recent advancements in reward function design. II-A RL for Autonomous Driving RL has been utilized in autonomous driving at various decision-making levels. In works such as [ 8 ] , RL is applied as a high-level behavioral planner for lane-assistance systems and turning maneuvers. In such a setting, RL impacts only the agent’s behavior, and an additional trajectory planner is required to align agent motion with desired behavior [ 9 ] . Moreover, RL can be employed for trajectory planning, where the agent estimates a sequence of waypoints over a planning horizon. The task of trajectory planning has been tackled in Cartesian frame [ 10 ] as well as Frenet space [ 11 ] . Finally, RL can directly estimate control inputs such as steering and acceleration, eliminating the necessity for additional components [ 12 ] . In addition to spanning several decision-making levels, RL supports a diverse range of input modalities. In multiple works, object-level information serves as an input observation, with objects typically described by their location, velocity, and acceleration in either a global frame or relative to the self-driving vehicle [ 10 ] . Alternative papers instead learn driving policies directly from raw sensor data. Multiple approaches utilize either camera images [ 11 ] or LiDAR sensors [ 13 ] to perceive the current environment state. Recent advancements in RL incorporate multiple sensor modalities to realize the driving task [ 14 ] . II-B Reward Functions for Autonomous Driving Driving rewards should encapsulate the conflicting objectives of the driving task into a single numerical value to guide RL agent decision-making. Building on the review established in [ 7 ] , this research breaks down driving rewards into key objectives such as safety, progress, comfort, and conformance to traffic rules. The safety objective encourages the agent to avoid collision with other road users and static obstacles. This objective is commonly a sparse penalty at collision [ 15 ] . Only a few studies attempted to improve the safety objective by introducing a penalty for near-misses [ 16 ] , as well as simplified calculations of time to collision (TTC) [ 17 ] and headway [ 18 ] . These studies have