Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning
Rebekah A. Gelpí, Eric Xue, William A. Cunningham · 2025
Read Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning by Rebekah A. Gelpí, Eric Xue, William A. Cunningham (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03682.txt.
Introduction
Rebekah Gelpí, Eric Xue, and William Cunningham combine language models with Bayesian planning to read minds, in a 2025 paper. Theory of mind means guessing what another agent believes or wants from its acts. Planning models that work backward from acts are accurate but cannot scale to many possible beliefs.
Language models scale well but can fail on near twin tasks in odd ways. The hybrid lets the language model propose hypotheses and likelihoods, while the Bayesian part scores them in a principled way. It matched near optimal results on a classic task and beat language only runs, even with smaller models. It also shows promise on open ended tasks.
Catalog record
| CREATOR | Rebekah A. Gelpí, Eric Xue, William A. Cunningham |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03682
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planning Rebekah A. Gelpí, Eric Xue, William A. Cunningham Abstract We propose a hybrid approach to machine Theory of Mind (ToM) that uses large language models (LLMs) as a mechanism for generating hypotheses and likelihood functions with a Bayesian inverse planning model that computes posterior probabilities for an agent’s likely mental states given its actions. Bayesian inverse planning models can accurately predict human reasoning on a variety of ToM tasks, but these models are constrained in their ability to scale these predictions to scenarios with a large number of possible hypotheses and actions. Conversely, LLM-based approaches have recently demonstrated promise in solving ToM benchmarks, but can exhibit brittleness and failures on reasoning tasks even when they pass otherwise structurally identical versions. By combining these two methods, this approach leverages the strengths of each component, closely matching optimal results on a task inspired by prior inverse planning models and improving performance relative to models that utilize LLMs alone or with chain-of-thought prompting, even with smaller LLMs that typically perform poorly on ToM tasks. We also exhibit the model’s potential to predict mental states on open-ended tasks, offering a promising direction for future development of ToM models and the creation of socially intelligent generative agents. 1 Introduction The capacity for Theory of Mind (ToM)—the ability to infer the beliefs, desires, intentions, and goals, of others—is a hallmark of human social cognition, underpinning humans’ ability for social interaction, communication, and collaboration. Interdisciplinary work at the intersection of cognitive science and computer science (e.g., Jara-Ettinger 2019 ; Baker et al. 2017 ; Langley et al. 2022 ; Rabinowitz et al. 2018 ) has aimed to both characterize the mechanisms that make our ability to understand other minds so powerful, and leverage our understanding of humans’ ToM to design machines with a comparable capacity for social inference. These insights are critical for the design of trustworthy social agents that can be relied upon to align their understanding of situations with those of humans ( Street 2024 ) . Nevertheless, efforts to develop robust machine ToM have faced substantial challenges. On one hand, Bayesian models of cognition inspired by inverse reinforcement learning have offered a promising computational framework for human reasoning on a variety of ToM tasks (e.g., Baker et al. 2017 ; Ullman et al. 2009 ; Jara-Ettinger et al. 2016 ; Jara-Ettinger 2019 ), but like many Bayesian models, face challenges with implementation outside of environments with heavily restricted hypothesis and action spaces. On the other hand, recent work with large language models (LLMs) has argued that their success on a variety of benchmarks represents a significant advancement in the development of machine ToM (e.g., Kosinski 2024 ; Gandhi et al. 2024 ); however, the extent to which these successes represent robust and general social reasoning abilities is unclear, as several analyses have revealed that the performance of LLMs on tasks outside of existing benchmarks is often brittle ( Trott et al. 2023 ; Shapira et al. 2023 ; Ullman 2023 ) , and alignment of LLMs with human reasoning in open-ended domains remains elusive ( Amirizaniani et al. 2024 ) . In this paper, we present a hybrid approach, LLM-Augmented Inverse Planning (LAIP), that exploits the potential complementary strengths of Bayesian inverse planning models and LLMs. By integrating the generative capabilities of LLMs, inverse planning models can be theoretically unbounded in the quantity of hypotheses about an agent’s beliefs and desires, or actions given the agent’s state, that they can entertain in any given situation. On the other hand, by explicitly formalizing the process of inverse planning, we show this hybrid model is less susceptible to zero-shot reasoning errors than LLMs without specific prompting or with generic chain-of-thought (CoT) prompting. 2 Related Work Theory of Mind in Humans ToM is a foundational component of human social cognition. It emerges early in childhood ( Wimmer & Perner 1983 ; Gopnik & Astington 1988 ; Wellman et al. 2001 ) , possibly even in infancy ( Butterfill & Apperly 2013 ) , allowing human beings to make sophisticated inferences about how others’ beliefs, desires, and knowledge may differ from one’s own. By allowing people to infer when others do not know what we do—and when others know what we do not—theory of mind has been proposed as a necessary component to the extent and breadth of human systems of cooperation and trust, and thus indirectly the success of human culture ( Frith & Frith 2010 ; Gopnik & Meltzoff 1993 ; Tomasello et al. 1993 ) . Bayesian Inverse Planning as Theory of Mind Inspired by probabilistic models of human cognition (e.g., Chater et al. 2006 ; Tenenbaum & Griffiths 2001 , work by Verma & Rao 2005 , Baker et al. 2009 ; Baker et al. 2011 , and Rafferty et al. 2015 formalized the understanding of others’ beliefs, desires, and intentions as an instance of Bayesian reasoning within a partially observable Markov decision process (POMDP). Within this framework, an observer engages in inverse planning —inverting the observer’s own process of generating an action policy based on its beliefs and desires—in order to reason about the unobserved internal states that give rise to an agent’s behaviours. These models have been extended to account for both children’s and adults’ commonsense reasoning that others will act according to a naive form of expected utility, maximizing expected rewards and minimizing costs ( Jara-Ettinger et al. 2016 ; Jara-Ettinger et al. 2020 ; Lucas et al. 2014 ) . Within this broader framework, ToM can be thought of as equivalent to inverse reinforcement learning (IRL; Jara-Ettinger 2019 ; Ruiz-Serra & Harré 2023 ), recovering an agent’s reward structure from actions that are assumed to be generated by an optimal policy given the agent’s beliefs. A family of models have extended this framework to various MDP and POMDP settings ( Lim et al. 2020 ; Wei et al. 2023 ; Wu et al. 2023 ) . Deep Learning models of Theory of Mind Deep learning methods have proven tremendously successful at learning complex strategies that reach or surpass human ability across a variety of complex games ( Mnih et al. 2015 ; Silver et al. 2018 ) . Rabinowitz et al. 2018 developed an early deep learning model for ToM based on meta-learning: by learning to predict several disparate classes of agents that have different preferences and action policies, the model can extrapolate an agent’s likely policy after observing only a few actions. Other models have explored other aspects of ToM reasoning: for example, including explicit belief models improves the performance of agents on cooperative and adversarial games in multiagent settings ( Fuchs et al. 2021 ; Moreno et al. 2021 ; Oguntola et al. 2023 ; Wen et al. 2019 ) . However, these methods have encountered challenges in their ability to successfully capture ToM ( Aru et al. 2023 ) —in particular, the fact that deep learning algorithms may implement “shortcuts” to solve theory of mind tasks. Theory of Mind in LLMs Most recently, the emergence of powerful, generally capable LLMs has led to investigation of their capacity for ToM. Earlier investigations found that LLMs scored substantially