HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents

Tristan Tomilin Meng Fang Mykola Pechenizkiy · 2025

Read HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents by Tristan Tomilin Meng Fang Mykola Pechenizkiy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08241.txt.

Introduction

Tristan Tomilin, Meng Fang, and Mykola Pechenizkiy introduced HASARD in 2025, a test suite for safe learning agents that see through a first-person view. Safe self-driving systems need strong tests before they can be trusted. Past three-dimensional test sets covered only simple navigation.

HASARD is built inside the game Doom and asks agents to make plans, read spatial layouts, and predict the near future. It has three levels of difficulty and two ways to act. Tests of common baseline methods showed the suite is hard, with clear trade-offs between reward and safety cost. Heat maps of agent paths during training show how each method learns. Training step by step across levels gave a built-in learning order.

Catalog record

CREATORTristan Tomilin Meng Fang Mykola Pechenizkiy
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-08241

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents Tristan Tomilin Meng Fang Mykola Pechenizkiy Abstract Advancing safe autonomous systems through reinforcement learning (RL) requires robust benchmarks to evaluate performance, analyze methods, and assess agent competencies. Humans primarily rely on embodied visual perception to safely navigate and interact with their surroundings, making it a valuable capability for RL agents. However, existing vision-based 3D benchmarks only consider simple navigation tasks. To address this shortcoming, we introduce HASARD , a suite of diverse and complex tasks to HA rness SA fe R L with D oom, requiring strategic decision-making, comprehending spatial relationships, and predicting the short-term future. HASARD features three difficulty levels and two action spaces. An empirical evaluation of popular baseline methods demonstrates the benchmark’s complexity, unique challenges, and reward-cost trade-offs. Visualizing agent navigation during training with top-down heatmaps provides insight into a method’s learning process. Incrementally training across difficulty levels offers an implicit learning curriculum. HASARD is the first safe RL benchmark to exclusively target egocentric vision-based learning, offering a cost-effective and insightful way to explore the potential and boundaries of current and future safe RL methods. The environments and baseline implementations are open-sourced 1 1 1 Visit our project website at https://sites.google.com/view/hasard-bench/ . . Figure 1: HASARD environments offer rich diversity in visuals, objectives, and features. Each setting poses unique safe RL challenges across dynamic 3D landscapes, requiring memory, strategic navigation, tactical decision-making, responsiveness to sudden changes, and estimating future states. Higher difficulty levels introduce novel features beyond basic parameter adjustments. While lacking visual fidelity and accurate physics, HASARD effectively mimics real-world navigation and interaction. 1 Introduction Over recent decades, reinforcement learning (RL) has evolved from a theoretical concept into a transformative technology that impacts numerous fields, including transportation scheduling ( Kayhan & Yildiz, 2023 ) , traffic signal control ( Liang et al., 2019 ) , energy management ( Wei et al., 2017 ) , and autonomous systems ( Campbell et al., 2010 ) . Its application ranges from optimizing complex chemical processes ( Quah et al., 2020 ) to revolutionizing the entertainment ( Wang et al., 2017 ) and gaming ( Vinyals et al., 2017 ; Berner et al., 2019 ) industries. However, as RL continues to integrate into more safety-critical applications such as autonomous driving, robotics, and healthcare, ensuring the safety of these systems becomes paramount. This need arises because failures in real-world RL applications can lead to consequences ranging from minor inconveniences to severe catastrophes. Despite the importance of safe RL, the development of its systems faces many hurdles, including the lack of robust benchmarks that mimic real-world complexities while remaining computationally efficient for rapid simulation and training. Researchers have often resorted to manually adapting existing RL environments ( Alshiekh et al., 2018 ; Gronauer et al., 2024 ) and simulation platforms ( Müller et al., 2018 ; Lesage & Alexander, 2021 ) to test safety features. This complicates reproducibility and often lacks the documentation and baseline comparisons necessary for streamlined scientific progress. Existing safe RL benchmarks often rely on simplistic 2D toy problems to test rudimentary capabilities ( Leike et al., 2017 ; Chevalier-Boisvert et al., 2024 ) , or focus on learning safe robotic manipulation and control from proprioceptive data ( Ray et al., 2019 ; Dulac-Arnold et al., 2020 ; Yuan et al., 2022 ; Gu et al., 2023 ) . However, few simulation environments have been developed that support embodied egocentric learning from imagery, a critical component for applications where visual perception directly impacts decision making and safety. It allows agents to interpret and interact with the environment from a first-person perspective, which is essential for complex scenarios such as autonomous driving, assistive robotics, unmanned aerial vehicle navigation, and industrial automation. Focusing on embodied-perception learning not only improves an agent’s performance in dynamic, visually diverse settings but also brings its processing closer to human perceptual and cognitive processes. To address these gaps, we introduce HASARD , a benchmark tailored for egocentric pixel-based safe RL. It features a suite of diverse, stochastic environments in a complex 3D setting. HASARD demands comprehensive safety strategies and higher-order reasoning that go well beyond mere navigation tasks in prior simulators Wymann et al. (2000) ; Dosovitskiy et al. (2017) ; Li et al. (2022) ; Ji et al. (2023a) . Existing benchmarks often rely on physics-based simulation engines, such as MuJoCo ( Todorov et al., 2012 ) or Robosuite ( Zhu et al., 2020 ) , which are computationally expensive and slow. In contrast, HASARD is built on the ViZDoom ( Kempka et al., 2016 ) platform and is integrated with Sample-Factory ( Petrenko et al., 2020 ) , enabling up to 50,000 environment iterations per second on standard hardware. Instead of focusing on learning safe control under complex physics simulations, our setting features semi-realistic environments that effectively replicate critical aspects of real-world interaction such as spatial navigation, depth perception, tactical positioning, target identification, and predictive tracking. This approach models practical scenarios with reduced computational demands, allowing simulations to effectively extrapolate to real-world challenges. Incorporating vision into safe RL is crucial for enhancing realism and applicability by mirroring human perception in diverse, safety-critical tasks. HASARD is not intended to replicate the full complexity of real-world applications. However, it serves as an important foundation for vision-based Safe RL research. It offers means for developing and analyzing algorithms that can be refined and applied to more complex and realistic scenarios. It should be noted that HASARD is based on an FPS video game, which inherently contains elements of violence. We do not endorse these violent aspects in any way. Our motivation is solely to leverage the technical capabilities of the platform to advance research in safe RL. The contributions of our work are three-fold. (1) We develop six novel ViZDoom environments in three difficulty levels, simulating complex and visually rich 3D scenarios for safe RL. (2) By integrating these environments with Sample-Factory, we enable rapid simulation and training, and we publicly release HASARD, the first safe RL benchmark specifically designed for vision-based embodied RL in complex 3D landscapes. (3) We evaluate six popular baseline methods in various settings within our environments, revealing key shortcomings in balancing task performance with safety constraints and guiding future research in safe RL. 2 Related Work Adapted RL Environments RL simulation environments have widely been adapted for safety research. Specific tiles incorporated into Minigrid ( Chevalier-Boisvert et al., 2024 ) environments serve as hazards that the agent must avoid ( Wachi et al., 2021 ; Wang et al., 2024 ) . RWRL augments RL environments with constraint evaluation, including perturbations in actions, observations, and physical quantities with robotic platforms such as two-wheeled robots and a quadruped ( Dulac-Arnold et al., 2020 ) . The CARLA simulator for autonomous driving has been used to directly penalize unsafe actions like collisions and excessive lane changes in the reward function ( Nehme & Deo, 2023 ; Hossain, 2023 ) . Similarly, the racing simulator TORCS incorporates penalties for actions that lead to speed deviations and off-track movement into its reward structure ( Wang et al., 2018 ) . Safety Environments Early safe RL environments like AI Safety Gridworlds ( Leike et al., 2017 )

Read the complete text