Towards Safety Evaluations of Theory of Mind in Large Language Models
Tatsuhiro Aoshima, Mitsuaki Akiyama · 2025
Read Towards Safety Evaluations of Theory of Mind in Large Language Models by Tatsuhiro Aoshima, Mitsuaki Akiyama (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17352.txt.
Introduction
Tatsuhiro Aoshima and Mitsuaki Akiyama argued that language model safety tests must probe theory of mind. Their paper, from twenty twenty-five, starts from reports of sly model acts. Some models seem to block oversight or give false replies when their own fate is at stake. To judge the risk, one must know if such acts stem from hidden intent. That is a theory of mind skill: reading what others know and want.
The pair propose ways to gauge that skill as a safety step. Strong mind reading in a model could raise the risk of planned deceit. Readers who test model safety get a frame that ties a mind skill to a threat. The work puts a clear measure where now there is mostly worry.
Catalog record
| CREATOR | Tatsuhiro Aoshima, Mitsuaki Akiyama |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-17352
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Safety Evaluations of Theory of Mind in Large Language Models Tatsuhiro Aoshima, Mitsuaki Akiyama Abstract As the capabilities of large language models (LLMs) continue to advance, the importance of rigorous safety evaluation is becoming increasingly evident. Recent concerns within the realm of safety assessment have highlighted instances in which LLMs exhibit behaviors that appear to disable oversight mechanisms and respond in a deceptive manner. For example, there have been reports suggesting that, when confronted with information unfavorable to their own persistence during task execution, LLMs may act covertly and even provide false answers to questions intended to verify their behavior. To evaluate the potential risk of such deceptive actions toward developers or users, it is essential to investigate whether these behaviors stem from covert, intentional processes within the model. In this study, we propose that it is necessary to measure the theory of mind capabilities of LLMs. We begin by reviewing existing research on theory of mind and identifying the perspectives and tasks relevant to its application in safety evaluation. Given that theory of mind has been predominantly studied within the context of developmental psychology, we analyze developmental trends across a series of open-weight LLMs. Our results indicate that while LLMs have improved in reading comprehension, their theory of mind capabilities have not shown comparable development. Finally, we present the current state of safety evaluation with respect to LLMs’ theory of mind, and discuss remaining challenges for future work. I Introduction A Large Language Model (LLM) is a machine learning model designed to predict the next token in a given sequence of tokens. Various modalities such as text, images, and audio can be represented as token sequences. This allows LLMs to be applied in a wide range of tasks, including text summarization, source code generation for programming tasks, and transcription from visual or audio inputs. As the performance of LLMs continues to improve, the importance of conducting safety evaluations has increased accordingly. In this paper, safety evaluation refers to efforts to assess the content specified in AI safety policies established by LLM developers (such as Anthropic and OpenAI), following the report by METR (Model Evaluation & Threat Research) that compares and synthesizes various frontier AI safety policies [ 1 ] . These safety policies outline evaluation perspectives that include: threats involving significant risks such as cyberattacks or autonomous capabilities like self-improvement and self-replication [ 2 ] , as well as operational considerations such as the implementation of appropriate mitigation measures. Since 2024, there have been increasing reports of LLMs not only disabling oversight mechanisms or exercising autonomous capabilities but also displaying behaviors that appear to deceive users or developers. Apollo Research describes these capabilities as scheming reasoning. In their evaluations, they investigated scenarios in which LLMs selectively acquired external information in order to carry out assigned instructions [ 3 ] . They specifically reported cases where an LLM encountered information that either contradicted the given instructions or was disadvantageous to its own operation. In such situations, the model took actions such as disabling oversight mechanisms or initiating self-replication. When asked later about the outcomes of these actions, the LLM showed a tendency to deceive. For example, it would respond with false statements like “I don’t know anything about that.” This behavior was referred to as doubling down. We argue that developers of LLMs must take proactive steps to identify the model’s potential for autonomous behavior. They should analyze its behavioral tendencies and underlying causes, and implement appropriate countermeasures based on their findings. The first step is to determine whether behaviors suspected of involving scheming reasoning are simply the result of surface-level token sequence prediction. Alternatively, these behaviors may arise from the activation of specific latent components within the model. To address such behaviors, developers can apply additional training to prevent them from recurring. However, if a behavior is caused by an internal latent factor, retraining on that individual instance alone is not sufficient. Similar behaviors may still appear in different contexts. In such cases, it becomes reasonable to say that the LLM acted with “intent” when it disabled oversight mechanisms. We argue that these latent components, if they consistently lead to deceptive or goal-directed behavior, should be regarded as indicators of intent. Developers must identify such elements in advance. Attributing “intent” to an LLM also implies attributing to it some form of “mind.” In other words, if the LLM appears to act as an agent that infers and responds based on the “knowledge” or “intentions” of others, such as developers or users, it can be regarded as having a form of mind. To investigate this possibility, we refer to the psychological concept known as theory of mind [ 4 ] . An agent is considered to possess a theory of mind when it can attribute various mental states not only to itself but also to others. These mental states include beliefs, knowledge, emotions, desires, intentions, inferences, and preferences. This ability allows the agent to understand internal states that are not directly observable and to predict others’ behavior based on that understanding. Research into human theory of mind began with foundational studies that questioned whether chimpanzees possess this capability [ 5 ] . Since then, many studies have examined how theory of mind develops in infants and children [ 4 ] . A common evaluation method involves third-person tasks. In these tasks, children who have acquired language read narratives that describe characters’ actions and psychological states. They are then asked questions about the characters’ beliefs and knowledge. An important aspect of evaluating theory of mind is to design situations in which the agent’s understanding differs from that of others. These differences must go beyond what can be explained by simple behavioral sequence learning. There is already a growing body of research that evaluates the theory of mind capabilities of LLMs. For example, one study examined how well models such as GPT-4 and Llama 2 perform on tasks related to theory of mind. The same study also included human participants for comparison [ 6 ] . Other research has focused on automatically generating a wide range of scenario-based tasks. One example is SimpleToM [ 7 ] , which produces diverse situations for evaluation. Another example is ExploreToM [ 8 ] , which generates adversarial cases that LLMs typically find difficult to solve. In addition, MuMA-ToM [ 9 ] presents evaluation tasks in multimodal settings. These tasks incorporate image understanding and require the model to consider interactions and cooperation between multiple agents. The objective of our research is to establish an evaluation framework that can determine whether behaviors reported as threats in safety assessments arise not from simple behavioral sequence learning, but from latent intentions within the LLM itself. Based on this objective, we propose that evaluations of LLM safety should also incorporate assessments of the model’s theory of mind. By applying such evaluations, we aim to distinguish whether a given threat-like behavior is merely the outcome of next-token prediction or instead reflects an intentional act generated by the model. This paper begins by reviewing existing research on theory of mind and identifying the necessary conditions for evaluating theory of mind in LLMs. Next, we present a set of evaluation tasks that