General agents need world models

Jonathan Richens, David Abel, Alexis Bellot, Tom Everitt · 2025

Read General agents need world models by Jonathan Richens, David Abel, Alexis Bellot, Tom Everitt (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01622.txt.

Introduction

Jonathan Richens and three co-authors prove that any agent flexible enough to pursue many-step goals must, inside itself, have learned a model of its world, in this 2025 paper. The head's title, 'General agents contain world models', governs over the catalog's variant. The question is old in AI: can pure trial-and-error learning, with no internal model, ever suffice for general behavior?

Their formal answer is no. Any policy that generalizes across multi-step, goal-directed tasks implies a predictive model of the environment, and that model can in principle be extracted from the policy itself. Stronger performance or more complex goals demand proportionally more accurate models. The consequences they draw reach into safety, capability limits in complex environments, and new algorithms that elicit an agent's hidden world model for inspection. Model-free learning, on this account, only ever rents its competence from a model it unknowingly built.

Catalog record

CREATORJonathan Richens, David Abel, Alexis Bellot, Tom Everitt
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01622

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

General agents need world models Jonathan Richens, David Abel, Alexis Bellot, Tom Everitt Abstract Are world models a necessary ingredient for flexible, goal-directed behaviour, or is model-free learning sufficient? We provide a formal answer to this question, showing that any agent capable of generalizing to multi-step goal-directed tasks must have learned a predictive model of its environment. We show that this model can be extracted from the agent’s policy, and that increasing the agents performance or the complexity of the goals it can achieve requires learning increasingly accurate world models. This has a number of consequences: from developing safe and general agents, to bounding agent capabilities in complex environments, and providing new algorithms for eliciting world models from agents. 1 Introduction A hallmark of human intelligence is the ability to perform novel tasks with minimal supervision, formalised by few-shot and zero-shot learning ( Lake et al. 2017 ) . With the emergence of these capabilities in language models ( Brown et al. 2020 ) , focus has shifted to developing general agents—systems capable of performing long horizon goal-oriented tasks in complex, real-world environments ( Yao et al. 2022 ; Hao et al. 2023 ) . In humans this kind of flexible goal-directed behaviour relies heavily on rich mental representations of the world, i.e. world models ( Johnson-Laird 1983 ; Ha & Schmidhuber 2018 ) , which are used to set abstract goals beyond immediate sensory inputs ( Locke & Latham 2013 ) , and to deliberatively and proactively plan actions ( Bratman 1987 ) . Whether world models are necessary for achieving human-level AI has long been debated, pitting the challenges of learning models against the potential benefits they confer ( Huang 2020 ) . Figure 1 : Our result complements previous insights from planning and inverse RL. While planning uses a world model and a goal to determine a policy, and IRL and inverse planning use an agent’s policy and a world model to identify its goal, our result uses an agent’s policy and its goal to identify a world model Explicitly model-based agents have achieved impressive performance across many tasks and domains ( Hafner et al. 2023 ; Wang et al. 2023 ; LeCun 2022 ; Raad et al. 2024 ) , and having direct access to the agent’s world model has benefits like being able to apply formal planning methods ( Sutton 2018 ) , predicting the agent’s behaviour in safety-critical domains ( Amodei et al. 2016 ; Dalrymple et al. 2024 ) , reducing sample complexity ( Hafner et al. 2019 ) and supporting transfer learning ( Chua et al. 2018 ; Zhu et al. 2023 ) . However, learning accurate models of real-world systems can be extremely challenging ( Dulac-Arnold et al. 2019 ) , and the performance of model-based agents is fundamentally limited by their model’s fidelity. In “Intelligence without representation”, Brooks famously proposed that the world is its own best model , and that all intelligent behaviours can emerge in model-free agents interacting through action-perception loops, without needing to learn explicit representations of the world ( Brooks 1991 ) . This view has largely been borne out by the development of model-free agents capable of generalizing across a wide range of tasks and environments ( Reed et al. 2022 ; Brohan et al. 2023 ; Driess et al. 2023 ; Black et al. 2024 ; Schrittwieser et al. 2020 ) . This model-free paradigm aims to achieve truly general agents while side-stepping the challenges inherent in learning a world model. However, there is mounting evidence that model-free agents may in fact learn implicit world models ( Li et al. 2022 ) , and may even learn implicit planning algorithms ( Hou et al. 2023 ; Bush et al. 2025 ) . This raises a fundamental question: is there a model-free shortcut to human-level AI? Or is learning a world model necessary, with all the complexity this entails? And if so, just how accurate and comprehensive do world models need to be to support a given level of capability? We provide a formal answer to these questions, showing that, any agent that satisfies a regret bound for a sufficiently diverse set of simple goal-directed tasks must have learned an accurate predictive model of its environment. Specifically, we consider environments described by a fully observed Markov processes, and propose a minimalist definition of general agents as goal-conditioned policies ( Liu et al. 2022 ) that satisfy a regret bound for a large set of simple goal-directed tasks (such as steering the environment into a desired state). We derive an algorithm that returns an approximation of the environment transition function (a world model) given the policy of any such agent, and show that the error in this approximation decreases as we increase the agent’s performance or the complexity of the goals it can achieve. In other words, general agents are world models, with all the information required to simulate the environment encoded in their policy. Importantly, we prove this for any agent that satisfies a regret bound, regardless of the details of its training and architecture and without imposing rationality assumptions. The necessity of learning a world model has profound consequences for how we develop general AI systems, how capable these systems can ultimately be, and how we can ensure agents are safe and interpretable. We explore these consequences and others in Section 4 . A more immediate consequence is that in proving our result we derive new algorithms for extracting world models from general agents. We demonstrate this in Section 3.1 , and show that our algorithms can recover accurate world models even when the agent strongly violates our competence assumptions. In Section 5 we then discuss related work, including inverse reinforcement learning and mechanistic interpretability. 2 Setup 2.1 Notation Capital letters denote random variables X and lower case letters x denoting a value or state X=x . Bold letters denote sets of variables \bm{X}= \{X_{1},X_{2},\ldots,X_{m}\} and \bm{x} denotes the joint state \{x_{1},x_{2},\ldots,x_{m}\} . Square brackets denote a proposition, e.g. [X=x] is True if X=x and False otherwise. 2.2 Environment We assume the environment is a controlled Markov process (cMP) ( Puterman 2014 ; Sutton 2018 ) , which is a Markov decision process without a specified reward function or discount factor. Formally, a cMP consists of a set of states \bm{S} , a set of actions \bm{A} , and a transition function P_{ss^{\prime}}(a)=P(S\!=\!s^{\prime}\!\mid\!A\!=\!a,S\!=\!s) . We refer to a sequence of state–action pairs over time as a trajectory , \tau=(s_{0},a_{0},s_{1},a_{1},\ldots) and a finite prefix of \tau as a history , h_{t}=(s_{0},a_{0},\ldots,s_{t}) . Definition 1 (Controlled Markov process) . A controlled Markov process (cMP) is a Markov decision process (MDP) without a specified reward function or discount factor. It is defined by the tuple \left(\bm{S},\bm{A},P_{ss^{\prime}}(a)\right) where \bm{S} is the state space, \bm{A} is the action space, and P_{ss^{\prime}}(a)=P(S=s^{\prime}\mid A=a,S=s) is the transition function. To derive our results we make the standard assumptions that the environment is finite, irreducible, and stationary, meaning every state is reachable from every other state under some finite sequence of actions, and transition probabilities do not change over time. Furthermore we assume |\bm{A}|\geq 2 so that the environment can support non-trivial policies.

Read the complete text