Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization

Frank Röder Jan Benad Manfred Eppe Pradeep Kr. Banerjee · 2025

Read Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization by Frank Röder Jan Benad Manfred Eppe Pradeep Kr. Banerjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20294.txt.

Introduction

A robot trained in one setting often fails when friction or gravity changes in a new one. Frank Röder, Jan Benad, Manfred Eppe, and Pradeep Kr. Banerjee address that gap in a 2025 paper on learning agents that adapt without retraining.

Their method, called DALI, sits inside the Dreamer world model and learns to read hidden context from how the agent's actions play out. The agent can then imagine how a scene would change if, say, gravity were stronger, and those imagined rollouts stay physically sensible. On hard test tasks, the method beat agents that were told nothing about context, and often matched or beat agents that were told the context outright.

Catalog record

CREATORFrank Röder Jan Benad Manfred Eppe Pradeep Kr. Banerjee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-20294

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization Frank Röder Jan Benad Manfred Eppe Pradeep Kr. Banerjee Abstract Real-world reinforcement learning demands adaptation to unseen environmental conditions without costly retraining. Contextual Markov Decision Processes (cMDP) model this challenge, but existing methods often require explicit context variables (e.g., friction, gravity), limiting their use when contexts are latent or hard to measure. We introduce Dynamics-Aligned Latent Imagination (DALI), a framework integrated within the Dreamer architecture that infers latent context representations from agent-environment interactions. By training a self-supervised encoder to predict forward dynamics, DALI generates actionable representations conditioning the world model and policy, bridging perception and control. We theoretically prove this encoder is essential for efficient context inference and robust generalization. DALI’s latent space enables counterfactual consistency: Perturbing a gravity-encoding dimension alters imagined rollouts in physically plausible ways. On challenging cMDP benchmarks, DALI achieves significant gains over context-unaware baselines, often surpassing context-aware baselines in extrapolation tasks, enabling zero-shot generalization to unseen contextual variations. 1 Introduction The ability to generalize across diverse environmental conditions is a fundamental challenge in reinforcement learning (RL). Contextual Markov Decision Processes (cMDPs) formalize this challenge by modeling task variations through latent parameters, such as friction, gravity, or object mass that govern dynamics ( Hallak et al. 2015 ) . However, real-world agents rarely have direct access to these parameters. Consider a robotic control task where a legged agent must adapt to different surfaces, such as smooth tiles, rough gravel, or slippery ice. If the agent were explicitly provided with surface friction coefficients, adaptation would be straightforward. In practice, though, such ground-truth annotations are often unavailable or prohibitively expensive to obtain. Instead, the agent must infer these variations implicitly by observing how its actions influence the environment, using proprioceptive feedback and gait dynamics. Existing approaches to contextual RL often rely on explicit context conditioning, where models are trained with domain-specific instrumentation or carefully crafted architectures ( Seyed Ghasemipour et al. 2019 ; Ball et al. 2021 ; Eghbal-zadeh et al. 2021 ; Mu et al. 2022 ; Beukman et al. 2023 ; Benjamins et al. 2023 ; Prasanna et al. 2024 ) . While effective in controlled settings, such approaches scale poorly and break down in unstructured environments where contexts are latent or difficult to measure. The challenge is to develop RL agents that infer and adapt to hidden contexts in a self-supervised manner, enabling robust generalization without direct supervision. To address this challenge, we propose Dynamics-Aligned Latent Imagination (DALI), a framework integrated within the DreamerV3 architecture ( Hafner et al. 2025 ) that enables zero-shot generalization by inferring latent contexts directly from interaction histories. Unlike DreamerV3, which struggles to generalize across diverse latent contexts due to its limited ability to retain critical environmental information ( Prasanna et al. 2024 ) , DALI overcomes these limitations through a self-supervised context encoder. This encoder learns to predict forward dynamics, capturing the relationship between actions, states, and their consequences. For example, by analyzing how applied forces influence motion trajectories, the encoder distills contextual factors (e.g., gravitational pull or surface friction) into compact, actionable embeddings. These embeddings condition the world model and policy, enabling the agent to reason about hidden dynamics (e.g., how increased inertia affects motion stability) and adapt its control strategies accordingly. Our theoretical analysis demonstrates that DALI’s context encoder enables robust zero-shot generalization by efficiently capturing latent environmental variations, overcoming information bottlenecks and sample inefficiencies in learning diverse contexts ( Prasanna et al. 2024 ; Hafner et al. 2025 ) . By inferring latent contexts from interactions, DALI enhances DreamerV3’s learning framework, achieving robust zero-shot generalization in partially observable environments. Our work makes the following key contributions: • Theoretical foundations : We establish that a dedicated context encoder is essential for robust generalization, proving that it infers latent contexts from short interaction windows with near-optimal sample complexity and mitigates information bottlenecks in partially observable settings. • Strong zero-shot generalization : On challenging cMDP benchmarks, DALI achieves up to +96.4\% gains over context-unaware baselines, often surpassing ground-truth context-aware baselines in extrapolation tasks. • Physically consistent counterfactuals : We show that the learned latent space exhibits counterfactual consistency: perturbing a dimension encoding gravity, for instance, results in imagined rollouts where objects fall faster or slower, faithfully mirroring real-world physics. 2 Related Work Contextual RL for Zero-Shot Generalization. Contextual RL has been studied in various forms, from cMDPs to domain randomization and meta-RL ( Hallak et al. 2015 ; Modi et al. 2018 ) . A recent survey ( Kirk et al. 2023 ) highlights its relevance for zero-shot generalization, emphasizing how clear context sets for training and evaluation enable systematic analysis. One research direction assumes context is explicitly observed as privileged information and integrates it into learning ( Chen et al. 2018 ; Seyed Ghasemipour et al. 2019 ; Ball et al. 2021 ; Eghbal-zadeh et al. 2021 ; Sodhani et al. 2021 ; Mu et al. 2022 ; Benjamins et al. 2023 ; Prasanna et al. 2024 ) . In contrast, we follow recent work that assumes context is latent and must be inferred ( Chen et al. 2018 ; Xu et al. 2019 ; Lee et al. 2020 ; Seo et al. 2020 ; Xian et al. 2021 ; Sodhani et al. 2022 ; Melo 2022 ; Evans et al. 2022 ) , focusing on self-supervised context inference through forward dynamics alignment. Model-Based RL. Model-based RL improves sample efficiency by learning predictive environment models for planning and imagination. Approaches like Dreamer ( Hafner et al. 2019 ; Hafner et al. 2020 ; Hafner et al. 2021 ; Hafner et al. 2025 ) and TD-MPC ( Hansen et al. 2024 ) achieve strong performance by learning compact latent representations of environment dynamics. Recent work has also explored general-purpose model-free RL that leverages model-based representations for broader applicability ( Fujimoto et al. 2025 ) . While Dreamer has been explored for zero-shot generalization with explicit context conditioning ( Prasanna et al. 2024 ) , we build on it with a dynamics-aligned inference mechanism to enhance generalization without context supervision. Meta-RL. Meta-RL trains agents to adapt rapidly to new tasks with minimal experience ( Beck et al. 2023 ) , typically by learning adaptive policies that infer task-specific information from past interactions. However, most meta-RL methods require fine-tuning on new tasks ( Duan et al. 2016 ; Finn et al. 2017 ; Nagabandi et al. 2018 ; Rakelly et al. 2019 ; Zintgraf et al. 2019 ; Melo 2022 ) . Our approach, in contrast, aims for zero-shot generalization by leveraging structured latent representations that transfer across environments. 3 Preliminaries To investigate zero-shot generalization in partially observable environments, we adopt a contextual Markov Decision Process (cMDP) framework, following Hallak et al. 2015 ; Modi et al. 2018 ; Kirk et al. 2023 . A cMDP is defined as a tuple \mathcal{M}=(\mathcal{S},\mathcal{A},\mathcal{O},\mathcal{C},\mathcal{R},\mathcal{P},\mathcal{E},\mu,p_{\mathcal{C}}) , where \mathcal{S} , \mathcal{A} , and \mathcal{O} are resp. the state, action, and observation spaces, \mathcal{C}\subseteq\mathbb{R}^{d} is the context space with contexts c\in\mathcal{C} drawn i.i.d. from a distribution p_{\mathcal{C}} , \mathcal{R}:\mathcal{S}\times\mathcal{A}\times\mathcal{C}\to\mathbb{R} is the reward function, \mathcal{P}:\mathcal{S}\times\mathcal{A}\times\mathcal{C}\to\Delta(\mathcal{S}) is the stochastic transition function specifying the probability distribution over next states given the current state, action, and context, \mathcal{E}:\mathcal{S}\times\mathcal{C}\to\Delta(\mathcal{O}) is the observation

Read the complete text