Jasmine: A Simple, Performant and Scalable JAX-based World Modeling Codebase

Mihir Mahajan *1,2 , Alfred Nguyen *1,2 , Franz Srambical *1,2 Stefan Bauer 2 * Contributed equally 1 p(doom), 2 TUM · 2025

Read Jasmine: A Simple, Performant and Scalable JAX-based World Modeling Codebase by Mihir Mahajan *1,2 , Alfred Nguyen *1,2 , Franz Srambical *1,2 Stefan Bauer 2 * Contributed equally 1 p(doom), 2 TUM (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27002.txt.

Introduction

Mihir Mahajan, Alfred Nguyen, Franz Srambical, and Stefan Bauer release Jasmine, an open codebase for training world models. A world model learns to predict how an environment changes, which can ease data scarcity in fields like robotics. Training tools for such models are still young, so the team built a fast, reusable base in the JAX framework.

Jasmine scales from one machine to hundreds of accelerators with little code change. Speed work across data loading, training, and checkpointing reproduces a standard CoinRun case about ten times faster than prior open code. Training stays fully reproducible and supports many sharding layouts. Paired with curated datasets, it gives researchers shared ground for fair model comparisons.

Catalog record

CREATORMihir Mahajan *1,2 , Alfred Nguyen *1,2 , Franz Srambical *1,2 Stefan Bauer 2 * Contributed equally 1 p(doom), 2 TUM
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27002

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Jasmine: A Simple, Performant and Scalable JAX-based World Modeling Codebase Mihir Mahajan *1,2 , Alfred Nguyen *1,2 , Franz Srambical *1,2 Stefan Bauer 2 * Contributed equally 1 p(doom), 2 TUM Abstract While world models are increasingly positioned as a pathway to overcoming data scarcity in domains such as robotics, open training infrastructure for world modeling remains nascent. We introduce Jasmine, a performant JAX-based world modeling codebase that scales from single hosts to hundreds of accelerators with minimal code changes. Jasmine achieves an order-of-magnitude faster reproduction of the CoinRun case study compared to prior open implementations, enabled by performance optimizations across data loading, training and checkpointing. The codebase guarantees fully reproducible training and supports diverse sharding configurations. By pairing Jasmine with curated large-scale datasets, we establish infrastructure for rigorous benchmarking pipelines across model families and architectural ablations. \reportnumber 001 keywords World Modeling, JAX, Reinforcement Learning 1 Introduction Over the past decades, the field of deep learning has increasingly been shaped by methods that leverage vast data troves ( Jozefowicz et al. 2016 ; Radford et al. 2018 ; Radford et al. 2019 ; Chowdhery et al. 2023 ; Raffel et al. 2020 ; Fedus et al. 2022 ; Deng et al. 2009 ) , and paradigms that unlock new ones ( Guo et al. 2025a ; Christiano et al. 2017 ; Radford et al. 2021 ; Srambical and Mahajan 2025 ; Silver et al. 2016 ; Berner et al. 2019 ) . Internet-scale pre-training, preference modeling, and reinforcement learning using verification signals offer a compelling pathway for language models to attain human-level performance ( Luong and Lockhart 2025 ; Lin and Cheng 2025 ) , yet data is increasingly bottlenecking progress from spiky towards general intelligence. While some domains can leverage user feedback from deployed products for iterative model improvement ( Cursor 2025 ) , domains like robotics cannot afford such a privilege. One paradigm proposed by the research community to overcome the data scarcity in those domains is that of world models ( Ha and Schmidhuber 2018 ) . World models can aid frontier model development in numerous ways, but one particularly ambitious goal of the community is to train a world model to act as a simulation of the real world ( Bruce et al. 2024 ; Parker-Holder et al. 2022 ; Ball et al. 2025 ; Agarwal et al. 2025 ) , in order to train agents in that simulation ( Hafner et al. 2025b ) , via an adaptive curriculum ( Parker-Holder et al. 2022 ) , or otherwise. This regime requires the compounding error of the world model to be orders of magnitude smaller than when solely used for short-term look-ahead. The feasibility of such a world model in its truest sense is entirely understudied. Jasmine provides the foundational infrastructure for future empirical investigation of how compute and data requirements scale with environment complexity for downstream agent training. Figure 1: Autoregressive sampling of Jafar ( Willi et al. 2024 ) (middle row) and Jasmine (bottom row) on the CoinRun case study with four conditioning frames (conditioning frames not shown). The top row shows the ground-truth sequence. 2 Jasmine Our contributions in this work are threefold: i) we introduce Jasmine, a highly optimized and scalable JAX-based codebase for world modeling, which we use to reproduce Genie’s CoinRun case study ( Bruce et al. 2024 ; Cobbe et al. 2020 ) an order of magnitude faster than prior work ( Willi et al. 2024 ) . This speedup is the result of infrastructure optimizations, including a fully reproducible, scalable training and data pipeline built on the JAX ( Bradbury et al. 2018 ) ecosystem. ii) We find that a critical modification to the original Genie architecture, prepending latent actions instead of adding them to video embeddings, is required for the world model to yield generations that faithfully reproduce the CoinRun environment. iii) Finally, we openly release the Jasmine codebase, along with pretrained checkpoints, curated datasets, model inspection notebooks, and a dataset of dense IDE interactions captured during Jasmine’s development, providing the first openly published dataset of months-long software engineering. Jasmine implements the Genie ( Bruce et al. 2024 ) architecture, enabling training of interactive environments from unlabeled videos. The architecture includes a video tokenizer, which encodes videos into tokens, a latent action model (LAM) that extracts latent actions between video frames, and a dynamics model that predicts the tokens of the next frame based on the previous tokens and corresponding latent actions. At sampling time, the LAM is discarded and replaced by input from the user. All modules use an ST-Transformer ( Ho et al. 2019 ) backbone which approximates full attention by performing intra-frame (spatial) followed by inter-frame (temporal) attention, thus reducing the attention sequence length. The tokenizer uses a VQ-VAE ( Van Den Oord et al. 2017 ) to encode image patches using reconstruction, vector-quantization, and commitment losses. To train an action-conditioned video-generation model from unlabeled videos, Genie learns latent actions ( Schmidt and Jiang 2024 ) . Like the tokenizer, the LAM uses a VQ-VAE, with its codebook representing the latent actions. The model learns to distill information from future frames into this bottlenecked codebook: Frames x_{0:t} are encoded, producing latent actions a_{0:t} , which the decoder receives along with past frames x_{0:t-1} to predict the next frame x_{t} . A temporal causal mask allows the entire sequence to be processed in a single forward pass. The dynamics model is a decoder-only transformer that predicts future frames conditioned on past frames and corresponding latent actions. Genie uses MaskGIT ( Chang et al. 2022 ) , which masks input-tokens at training time, similar in spirit to BERT ( Devlin et al. 2019 ) . Unlike MaskGIT, Genie masks with probability p\sim U(0.5,1) (refer to Appendix F for details about extending MaskGIT to videos). Figure 2: Validation metrics of the CoinRun case study (patch size 4). While the loss (left) is similar between the default Genie configuration and our minimal modification, rollout metrics (middle and right, refer to Appendix G ) differ substantially. Building upon prior work ( Willi et al. 2024 ) that openly published a reimplementation of the Genie ( Bruce et al. 2024 ) architecture, we release a highly optimized JAX-based world modeling codebase amenable to scale. Jasmine implements a range of baselines, including MaskGIT-based ( Chang et al. 2022 ) , fully causal ( Srambical 2024 ) , and diffusion-based approaches (Appendix C ). The codebase depends solely on battle-tested libraries from the Google ecosystem (JAX, NNX, Grain ( Ritter et al. 2023 ) , Orbax, Optax, Treescope ( Johnson 2024 ) , ArrayRecord ( Google 2024 ) ), and scales from single hosts to hundreds of accelerators using XLA. Jasmine supports complex sharding configurations in a few lines of code through Shardy ( OpenXLA Project 2025 ) . It provides asynchronous distributed checkpointing with configurable policies, process-parallel dataloading, and checkpointing of model, optimizer, and data loader states. Training runs are bitwise deterministic, yielding identical loss curves under identical seeds (Appendix D ). To enable efficient large-scale experimentation, Jasmine integrates mixed-precision, FlashAttention via cuDNN SDPA ( NVIDIA Corporation 2025 ) , activation checkpointing, host memory offloading, and index-shuffling during data loading. The

Read the complete text