ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind
Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno · 2025
Read ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind by Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08838.txt.
Introduction
Kazutoshi Shinoda and a team of co-authors build ToMATO, a benchmark for theory of mind in role-playing language models. Their paper from twenty twenty-five faults older tests on three counts. They cover few mental states, skimp on false beliefs, and ignore character traits. ToMATO is a set of multiple-choice questions over staged conversations.
Two models chat with lopsided knowledge: each speaks its thoughts aloud first, but the other cannot hear them. Those spoken thoughts give graded answers on belief, intent, desire, emotion, and knowledge, at first and second order. Hidden thoughts breed false beliefs by design. Set personality patterns vary the cast further. Nine models were tested. Even a strong small flagship lagged people, most on false beliefs. Trait shifts also shook scores, which the authors flag as a robustness gap.
Catalog record
| CREATOR | Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-08838
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Saki Mizuno Abstract Existing Theory of Mind (ToM) benchmarks diverge from real-world scenarios in three aspects: 1) they assess a limited range of mental states such as beliefs, 2) false beliefs are not comprehensively explored, and 3) the diverse personality traits of characters are overlooked. To address these challenges, we introduce ToMATO, a new ToM benchmark formulated as multiple-choice QA over conversations. ToMATO is generated via LLM-LLM conversations featuring information asymmetry. By employing a prompting method that requires role-playing LLMs to verbalize their thoughts before each utterance, we capture both first- and second-order mental states across five categories: belief, intention, desire, emotion, and knowledge. These verbalized thoughts serve as answers to questions designed to assess the mental states of characters within conversations. Furthermore, the information asymmetry introduced by hiding thoughts from others induces the generation of false beliefs about various mental states. Assigning distinct personality traits to LLMs further diversifies both utterances and thoughts. ToMATO consists of 5.4k questions, 753 conversations, and 15 personality trait patterns. Our analysis shows that this dataset construction approach frequently generates false beliefs due to the information asymmetry between role-playing LLMs, and effectively reflects diverse personalities. We evaluate nine LLMs on ToMATO and find that even GPT-4o mini lags behind human performance, especially in understanding false beliefs, and lacks robustness to various personality traits. NTT Corporation [email protected] 1 Introduction Theory of Mind (ToM) is the cognitive ability to infer unobservable mental states such as beliefs, intentions, and desires of others ( Premack and Woodruff 1978 ) . The ToM reasoning capability is thought to be the cornerstone of human social intelligence ( Fan et al. 2022 ) , and indispensable to interact with others ( Baron-Cohen, Leslie, and Frith 1985 ) . To investigate whether large language models (LLMs) possess human-like ToM, researchers have used various benchmarks. However, existing ToM benchmarks are not aligned well with real-world scenarios in the following three aspects. (1) Despite various categories of mental states that can be inferred by ToM as studied in psychology ( Beaudoin et al. 2020 ) , only limited types of mental states such as beliefs have been assessed ( Ma et al. 2023 ) , especially for second-order ToM. (2) False beliefs about beliefs or world states have been the main focus of previous studies ( Le, Boureau, and Nickel 2019 ; Kim et al. 2023b ) . However, false beliefs about other types of mental states have not been explored. Understanding false beliefs about a range of mental states should be crucial for LLMs to facilitate effective social interaction in real-world scenarios. (3) The behaviors and mental states of the characters in most benchmarks do not depend on their personality traits, even though they do in the real world ( Costa and McCrae 1980 ; Izard et al. 1993 ; Mehl, Gosling, and Pennebaker 2006 ) . Figure 1: (a) Conversation between two role-playing LLMs with information asymmetry. Before speaking to the other, our Inner Speech prompting (e.g., I feel, or I think that he/she/they feels) promptes each agent to verbalize their first- and second-order mental states as thoughts. The verbalized thoughts are used as the answers to the questions in ToMATO. (b) To detect false beliefs, both GPT4o mini and human annotators judge whether character B misunderstands A’s mental state at each turn. To address the above issues, we introduce ToMATO, a new T heory- o f- M ind d AT aset generated via Inner Speech pr O mpting. 1 1 1 Our dataset and codes are available at https://github.com/nttmdlab-nlp/ToMATO . Firstly, ToMATO comprehensively evaluates first- and second-order ToM for five categories of mental states: beliefs, intentions, desires, emotions, and knowledge. Secondly, we provide ToMATO-FB, a subset of ToMATO for evaluating understanding of false beliefs about the five mental states of others, e.g., understanding Bob thinks that Alice feels relieved , while Alice feels frustrated (Figure 1 ). In this regard, ToMATO is the most comprehensive benchmark compared to the existing ones. Lastly, ToMATO can evaluate the robustness of LLMs to the diverse personality traits of characters as seen in the real world. See Table 1 for the detailed comparison. Collecting human conversations and mental states of participants with self-report can be challenging in terms of cost, privacy, and accuracy ( Nisbett and Wilson 1977 ) . As recent LLMs have been shown to role-play assigned personalities ( Jiang et al. 2023 ; Jiang et al. 2024 ) and engage in conversations ( Zhou et al. 2024b ) , we generate ToMATO with a newly designed LLM-LLM interaction. Namely, we design Inner Speech prompting (Table 2 ), which promotes role-playing LLMs to verbalize their mental states as thoughts in conversations with another LLM. This idea is inspired by the debate in Dillion et al. (2023) about the feasibility of LLMs to simulate human participants in psychological science. Moreover, we hypothesize that ensuring information asymmetry about thoughts is a crucial factor in having false beliefs about the mental states of others as shown in Figure 1 . In addition, we assign big five personality traits to LLMs to diversify utterances and thoughts. This design enables ToMATO to evaluate robustness to diverse personality traits. The effects of these approaches are verified with analyses in § 6 . We evaluate nine LLMs including local and proprietary ones on ToMATO. Our experiments show that even the most advanced LLM, GPT-4o mini, under-performs the human performance in ToMATO. In addition, we show that the ToM performance of LLMs drops for the false belief subset, ToMATO-FB. Furthermore, we find that LLMs lack robustness to diverse personality traits. These results suggest that ToM in LLMs is still far from deployable to real-world applications. Table 1: Comparison of ToMATO to existing ToM benchmarks. ToMi ( Le, Boureau, and Nickel 2019 ) , Hi-ToM ( Wu et al. 2023 ) , BigToM ( Gandhi et al. 2023 ) , FauxPas-EAI ( Shapira, Zwirn, and Goldberg 2023 ) , FANToM ( Kim et al. 2023b ) , OpenToM ( Xu et al. 2024 ) , ToMBench ( Chen et al. 2024 ) . B: belief, I: intention, D: desire, E: emotion, K: knowledge, FB: false beliefs, W: world state, #P: the number of personality trait patterns. Assessable Mental State Input Context Benchmark First-order Second-order B about FB about #P Context Generator ToMi B B W,B - Narrative Template Hi-ToM B B - - Narrative Template BigToM B - W - Narrative Template+LLM FauxPas-EAI B - - - Narrative Psychological Test FANToM B B B - Conversation Single LLM OpenToM B,E B - 3 Narrative Single LLM ToMBench B,I,D,E,K B B - Narrative Human ToMATO (ours) B,I,D,E,K B,I,D,E,K B,I,D,E,K 15 Conversation LLM-LLM Conversation 2 Preliminaries Theory of Mind (ToM) has been studied for decades ( Premack and Woodruff 1978 ) , with various definitions and conceptions proposed. In this section, we clarify the scope addressed in this paper. Mental states. Following Ma et al. (2023) , we define the scope of ToM in this study by focusing on specific mental state categories identified by Beaudoin et al. (2020) : beliefs, intentions, desires, emotions, and