JAXenstein: Accelerated Benchmarking for First-Person Environments

Ruo Yu Tao, George Konidaris · 2026

Read JAXenstein: Accelerated Benchmarking for First-Person Environments by Ruo Yu Tao, George Konidaris (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19926.txt.

Introduction

Tao and Konidaris build a fast test ground for first person sight tasks in machine learning. Such tasks test search and coping with half seen scenes, yet fast JAX test grounds for them did not exist. Their open tool reworks the Wolfenstein 3D engine in JAX for quick large scale trials, running several times faster than like sight based test grounds.

Catalog record

CREATORRuo Yu Tao, George Konidaris
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17263
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19926

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

JAXenstein: Accelerated Benchmarking for First-Person Environments Ruo Yu Tao, George Konidaris Abstract The progression of reinforcement learning algorithms have been driven by challenging benchmarks. The rate in which a researcher can iterate on a problem setting directly impacts the speed of algorithm development. Modern machine learning has produced tools that allow for fast and scalable algorithm development like the JAX library. With the availability of these tools, a serious bottleneck in algorithm development is the availability of large and complex domains for experimentation. Most notably, the JAX reinforcement learning ecosystem does not have any benchmarks that test visual first-person tasks; these domains are crucial for testing both exploration and an agent’s ability to overcome partial observability. We introduce JAXenstein 1 1 1 Code: https://github.com/taodav/jaxenstein Baselines: https://github.com/taodav/jaxenstein_baselines/ an open-source JAX-based benchmark that implements the Wolfenstein 3D rendering engine for fast and scalable experimentation in visual first-person tasks. JAXenstein is several times faster than comparable vision-based benchmarks, and is easily extensible to more complex first-person domains. 1 Introduction The development of reinforcement learning algorithms have been driven by challenging benchmarks. Problem settings have preceded the development of algorithms, as they act as a foundation for algorithmic improvement. Most famously, the arcade learning environment ( 3 ) gave rise to deep Q-networks ( 12 ) and discrete, deep value-based control, while the Mujoco benchmark ( 19 ) was the testbed for deep continuous control and policy-gradient methods ( 18 ; 9 ; 17 ) . The speed of benchmarks directly affect the rate of experimentation and hence the speed of algorithmic development. A reinforcement learning research project will usually start with a faster, more lightweight benchmark, such as Cart Pole ( 1 ) or Mountain Car ( 13 ) , to develop an algorithm before moving on to larger domains which require more time to train on. Historically, these larger domains requires much time to train algorithms. With the introduction of multiprocessing ( 11 ) and GPU accelerated ( 7 ) environments, the speed, efficiency and parallelizability of environments has made iterating on larger domains more accessible, albeit at the cost of more computation. With the explosion in modern deep learning tools, a key bottleneck for algorithm development is the need for scalable, larger, more complex domains. Reinforcement learning experimentation has seen large speed and parallelization gains due to the adoption of the JAX ( 4 ) library, which can run experiments several times faster than one trained with traditional frameworks ( 10 ) . This requires both the algorithm and environment be entirely implemented in this framework, which means benchmarks not implemented in JAX are unavailable or costly. Notably, first-person domains ( 2 ; 21 ) that were originally introduced as candidate challenge problems have not been widely adopted in JAX frameworks due to the complexity in reimplementing rendering engines. These benchmarks are crucial for testing decision making in visual domains that both need to overcome partial observability and require good exploration. We introduce JAXenstein: a lightweight JAX-based ( 4 ) implementation of the Wolfeinstein 3D engine for accelerated first-person vision-based environments. JAXenstein environments are simplified versions of first-person vision-based environments used in reinforcement learning benchmarking ( 21 ; 2 ) , foregoing visual rendering assets for speed. JAXenstein is entirely implemented in JAX with just-in-time ( jit ) compilation and vectorized mapping ( vmap ) support, and runs at \sim 6\times the speed of the ViZDoom simulator on a single CPU core. Together with a purely end-to-end JAX pipeline, JAXenstein allows for fast, GPU-accelerated training many times faster than a comparable implementation with PyTorch. (a) my-way-home (b) key-door (c) dmlab-static-03 Figure 1 : Example environments in the JAXenstein benchmark, entirely implemented in end-to-end JAX. Objects are rendered as fixed sprites, like in the original Wolfenstein 3D engine. 2 A simple JAX-native rendering engine JAXenstein runs on a simple rendering engine that uses ray casting, first introduced in a production game in Wolfenstein 3D ( 8 ) . JAXenstein uses this simple 3D rendering engine because its speed: ray casting is extremely fast in rendering frames for simple 3D environments based on tile maps, allowing for fast and scalable environment interactions. We show a few examples of this rendering in Figure 1 . We use this front-end as a rendering engine to reimplement light and fast versions of existing first-person reinforcement learning benchmarks. 2.1 Ray casting Ray casting produces the scene of an image from an observer by calculating, from each column in an image, the intersect of a ray cast from the observer to an intersection in the scene (e.g. a wall). We can use an appropriate shading color to give depth to the rasterization based on the distance of this intersection. We calculate this intersection with the digital differential analyzer (DDA) algorithm ( 20 ) . The DDA algorithm leverages the fact that we are in a tile-based world, and finds the intersection for a particular ray by “marching” along the tile boundaries that intersect with the ray. This requires far fewer steps in the ray casting algorithm as compared to marching along the ray in a set, small interval, and results in significantly fewer intersection checks. 2.2 Performance trade-offs While ray casting with DDA is an extremely fast and simple algorithm for rendering, it does incur trade-offs. First and most importantly, it requires that the environment is a tile-based map. While fitting for many simple first-person environments, anything with more complex environment objects needs more complex algorithms that would necessarily slow down the simulator. This also implies that stairs, jumping, height differences are not possible to render with this engine. While extensions are possible for more complex objects ( 14 ) , this comes at a computational cost that will affect per-timestep performance. Nonetheless, this rendering engine is the first to introduce first-person environments into the JAX reinforcement learning ecosystem. Using this renderer, we port existing first-person environments used in reinforcement learning benchmarks for extremely fast and scalable experimentation. 3 JAXenstein: a benchmark suite for accelerated first-person environments (a) Steps-per-second comparison. JAXenstein is approximately 6\times the speed of MiniWorld and ViZDoom. (b) Training speed comparison between ViZDoom and JAXenstein. ViZDoom crashes after trying to parallelize > 64 environments. Figure 2 : Speed comparisons between JAXenstein and similar benchmarks. We introduce JAXenstein: a purely JAX-based benchmark for first-person environments based on the Wolfenstein 3D engine. This benchmark includes simplified reimplementations of popular first-person environments used throughout reinforcement learning research, such as ViZDoom ( 21 ) and DeepMind Lab ( 2 ) , as well as completely new environments. 3.1 Runtime comparison We conduct experiments to test the computational efficiency of JAXenstein compared to other benchmarks. We first compare JAXenstein run times to MiniWorld ( 6 ) and ViZDoom in Figure 2(a) . JAXenstein achieves around a 6\times speed up in steps per second. Details of these runs are given in Section 5 . One reason JAXenstein achieves comparably faster speeds is due to the availability of native rendering at lower resolutions. Since images are usually downsampled to a smaller size in deep reinforcement learning algorithms, rendering at this smaller size allows for massive per-time-step speed ups, without the per-timestep cost of rescaling images. We also compare a training time comparison

Read the complete text