Learning Abstract World Models with a Group-Structured Latent Space

Thomas Delliaux, Nguyen-Khanh Vu, Vincent François-Lavet, Elise van der Pol, Emmanuel Rachelson · 2025

Read Learning Abstract World Models with a Group-Structured Latent Space by Thomas Delliaux, Nguyen-Khanh Vu, Vincent François-Lavet, Elise van der Pol, Emmanuel Rachelson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01529.txt.

Introduction

Thomas Delliaux and co-authors teach abstract world models whose inner space carries a known symmetry in a 2025 paper. Agents that plan by a learned model of their world do better when that model generalizes from little data. Past models used a plain inner space with no built in structure, even when the world itself has clear symmetries, such as turning or sliding.

The team shape the inner space so group actions on it encode those symmetries, while extra room holds other facts. Tests in worlds with turning and sliding traits, including first person views of 3D scenes, gave better forecasts of the next state and better scores on later tasks. The inner forms were also simpler and cleaner.

Catalog record

CREATORThomas Delliaux, Nguyen-Khanh Vu, Vincent François-Lavet, Elise van der Pol, Emmanuel Rachelson
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01529

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Abstract World Models with a Group-Structured Latent Space Thomas Delliaux, Nguyen-Khanh Vu, Vincent François-Lavet, Elise van der Pol, Emmanuel Rachelson Abstract Learning meaningful abstract models of Markov Decision Processes (MDPs) is crucial for improving generalization from limited data. In this work, we show how geometric priors can be imposed on the low-dimensional representation manifold of a learned transition model. We incorporate known symmetric structures via appropriate choices of the latent space and the associated group actions, which encode prior knowledge about invariances in the environment. In addition, our framework allows the embedding of additional unstructured information alongside these symmetries. We show experimentally that this leads to better predictions of the latent transition model than fully unstructured approaches, as well as better learning on downstream RL tasks, in environments with rotational and translational features, including in first-person views of 3D environments. Additionally, our experiments show that this leads to simpler and more disentangled representations. The full code is available on GitHub 1 1 1 https://github.com/khanhvu207/world-models-group-latents to ensure reproducibility. 1 Introduction Figure 1 : Illustration of an MDP dynamics with some symmetries that we want to use as a prior knowledge in the abstract space geometry. In recent years, abstract world models have emerged as an important foundation for tackling complex reinforcement learning (RL) problems. World model learning can capture meaningful representations by embedding complex, high-dimensional data into lower-dimensional abstract spaces ( Ha and Schmidhuber, 2018 ; Francois-Lavet et al., 2019 ; Hafner et al., 2019 ; Schrittwieser et al., 2020 ; Kipf et al., 2019 ; van der Pol et al., 2020a ) . While the goal of world model learning is improved sample efficiency, state-of-the-art methods ( Ye et al., 2021 ; Hafner et al., 2023 ) still demand a large number of training samples to achieve superhuman performance. The agent’s performance depends on how the representation space models the environment’s underlying dynamics. The representation should be minimal yet sufficient to ensure generalization to unseen scenarios. Prior knowledge can also be leveraged. For instance, it is possible to enforce equivariances to improve generalization in RL ( van der Pol et al., 2020b ; Wang et al., 2022c ; Park et al., 2022 ) . In decision-making settings, geometric priors allow agents to represent states and actions in a way that respects the underlying symmetry of the dynamics—for instance, an agent that rotates in place eventually returns to its original state. Such priors can enable faster learning and better generalization when the environment exhibits known geometric structure. Consider the simple MDP shown in Figure 1 , which contains both symmetric and non-symmetric features. Repeated applications of action a_{1} bring the agent back to the same state. We propose encoding such symmetries by choosing a latent space and corresponding group action that naturally reflect this structure and the rest is learned (e.g. how much an agent rotates with a given action as well as unstructured features). Our approach employs a coordinate system tailored to the symmetry group, and performs a change of coordinates to recover a Euclidean representation, as illustrated in Figure 2 . The paper is organized as follows: Section 2 provides the background and introduces notations. Section 3 provides the methodology for integrating prior knowledge in the learning of abstract representations. Section 4 reports experiments on different settings including first-person view games in 3D environments. The experiments show the superior performance when generalizing to unseen scenarios and in the low-data regime. Section 5 provides further discussion on the related work. Section 6 provides a conclusion that highlights and discusses the main contributions. Figure 2 : Illustration of the approach. ( Left ) A grid-world with periodic boundary condition where the agent will appear on the opposite side when it attempts to move outside the grid. ( Middle ) The abstract state space {\mathcal{Z}} is modeled as a product space {\mathbb{R}}/2\pi{\mathbb{Z}}\times{\mathbb{R}}/2\pi{\mathbb{Z}} . ( Right ) Elements of {\mathcal{Z}} can be mapped to \mathbb{R}^{3} using a homeomorphism (x,y)\mapsto((\alpha+\beta\cos(x))\cos(y),(\alpha+\beta\cos(y))\cos(x),\beta\sin(y)) for \alpha,\beta\in\mathbb{R}_{+} specifying the major and minor radius of a torus. 2 Background 2.1 Group Theory This section introduces several essential definitions from group theory and abstract spaces. Groups A group G is defined by a pair ({\textnormal{G}},*) , consisting of a set G and a binary operator *:{\textnormal{G}}\times{\textnormal{G}}\rightarrow{\textnormal{G}} , such that: (i) there is an identity element e\in{\textnormal{G}} satisfying e*g=g*e=g for any g\in{\textnormal{G}} , (ii) the operator * is associative , i.e., (g_{1}*g_{2})*g_{3}=g_{1}*(g_{2}*g_{3}) for any g_{1},g_{2},g_{3}\in{\textnormal{G}} , and (iii) every element has an inverse , meaning that for any g\in{\textnormal{G}} , there exists h\in{\textnormal{G}} such that g*h=h*g=e . An example of a group is the cyclic group ({\mathbb{Z}}/n{\mathbb{Z}},+) , where each element is an integer in \{0,\ldots,n-1\} , and for every u,v\in{\mathbb{Z}}/n{\mathbb{Z}} , we have u+v:=(u+v)\mod n . Group actions. Groups can also be used to define transformations on another set. In the context of reinforcement learning, applying an action moves the agent from the current state s_{t}\in{\mathcal{S}} to the next state s_{t+1}\in{\mathcal{S}} . In this setting, the transition dynamics T defines a group action acting on the state space {\mathcal{S}} , which we use as prior knowledge. Formally, a group action \cdot:{\textnormal{G}}\times{\mathcal{S}}\rightarrow{\mathcal{S}} is a binary operation that satisfies two axioms: e\cdot s=s and (g*g)\cdot s=g\cdot(g\cdot s) for any g\in{\textnormal{G}} and s\in{\mathcal{S}} . An orbit of a state s\in{\mathcal{S}} under the group action is a set of all states reachable from s via elements of G , i.e., \mathrm{Orb}(s):=\{g\cdot s\mid g\in{\textnormal{G}}\} . For example, given a state s , all the 90-degree rotations of that state form an orbit of s under the group of 90-degree rotations. The group action G can partition the state space {\mathcal{S}} into multiple orbits. In Figure 1 , the action a_{1} consistently generates an orbit of 4 states, a pattern that remains predictable across different initial states. For a toy MDP with a single orbit generated by a single action (illustrated by a single directed cycle in Figure 1 ), we may represent the state space as {\mathcal{S}}:=\{z\mid z\in{\mathbb{C}},|z|=1\} , a set of unit complex numbers. The group action on {\mathcal{S}} is defined as g\cdot s=\exp\left(\frac{2\pi i}{n}\right)\times s , where g is the n^{\text{th}} root of unity. Composing multiple actions generates the set of n^{\text{th}} roots of unity, which forms {\mathbb{Z}}/n{\mathbb{Z}} under complex multiplication. Alternatively, when {\mathcal{S}} is a vector space \mathbb{R}^{n} , the group action on {\mathcal{S}} is performed via invertible matrices from \textnormal{SO}(n) ( Quessard et al., 2020 ) . Equivalence classes and Quotient spaces. An equivalence relation groups together elements of a set that share a property. If two elements are considered equivalent under the relation, they belong to the same equivalence class . An equivalence relation \sim must satisfy three simple rules: (i) Reflexivity : every element is equivalent to itself, s\sim s , (ii) Symmetry : if s_{x}\sim s_{y} , then s_{y}\sim s_{x} , and (iii) Transitivity : if s_{x}\sim s_{y} and s_{y}\sim s_{z} , then s_{x}\sim s_{z} . A quotient space arises naturally when we identify equivalent elements within a set. For example, consider the real numbers \mathbb{R} with an equivalence relation defined by translation by multiples of a constant k\in\mathbb{R} . In this case, two real numbers x and y are equivalent if their difference is a multiple of

Read the complete text