Decomposing Theory of Mind: How Emotional Processing Mediates ToM Abilities in LLMs

Ivan Chulo, Ananya Joshi · 2025

Read Decomposing Theory of Mind: How Emotional Processing Mediates ToM Abilities in LLMs by Ivan Chulo, Ananya Joshi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15895.txt.

Introduction

Ivan Chulo and Ananya Joshi decompose theory of mind in language models by probing how emotion processing mediates belief reasoning. Steering activations is known to improve model performance on theory of mind tasks. What changes inside the model was unclear. The authors compare steered and baseline activations with linear probes trained on dozens of cognitive actions.

Steering lifts belief task accuracy substantially in their tests. Gains are mediated by stronger emotion perception and emotion valuing activations. Analytical actions like questioning and convergent thinking are suppressed at the same time. The pattern suggests successful belief reasoning in models runs through emotional understanding, not analytic deliberation. That finding reframes how theory of mind should be built and evaluated in language systems.

Catalog record

CREATORIvan Chulo, Ananya Joshi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-15895

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Decomposing Theory of Mind: How Emotional Processing Mediates ToM Abilities in LLMs Ivan Chulo, Ananya Joshi Abstract Recent work shows activation steering substantially improves language models’ Theory of Mind (ToM) ( Bortoletto et al. 2024 ) , yet the mechanisms of what changes occur internally that leads to different outputs remains unclear. We propose decomposing ToM in LLMs by comparing steered versus baseline LLMs’ activations using linear probes trained on 45 cognitive actions. We applied Contrastive Activation Addition (CAA) steering to Gemma-3-4B and evaluated it on 1,000 BigToM forward belief scenarios ( Gandhi et al. 2023 ) , we find improved performance on belief attribution tasks (32.5% to 46.7% accuracy) is mediated by activations processing emotional content : emotion_perception (+2.23), emotion_valuing (+2.20), while suppressing analytical processes: questioning (-0.78), convergent_thinking (-1.59). This suggests that successful ToM abilities in LLMs are mediated by emotional understanding, not analytical reasoning. Johns Hopkins University Baltimore, MD 21218 USA Introduction The capacity to attribute mental states to oneself and others represents a critical capability for AI systems engaged in social reasoning, alignment, and human collaboration. Recent benchmarking work by Bortoletto et al. (2024) has demonstrated that activation steering techniques, particularly Contrastive Activation Addition (CAA), can substantially improve language models’ ToM performance on belief attribution tasks, often achieving accuracy improvements exceeding 10 %. This raises a question: what cognitive processes change when models successfully engage in perspective-taking? Traditional evaluations treat ToM as a monolithic capability measured through binary accuracy on belief attribution tasks ( Gandhi et al. 2023 ) . However, cognitive science views ToM as comprising multiple interacting processes ( Gabriel et al. 2019 ) . If steering vectors modulate these components differently, comparing steered versus baseline activation patterns could reveal which processes are essential for successful perspective-taking, thus decomposing ToM processes in LLMs into their constituent building blocks. We introduce a decomposition approach using techniques from mechanistic interpretability in LLMs by combining linear classifer probes trained on 45 cognitive actions with CAA steering vectors. By measuring which cognitive processes systematically increase or decrease when ToM performance improves, we can identify what cognitive actions are correlated with language models’ successful perspective-taking. Contributions: (1) A mechanistic decomposition method using cognitive action probes to analyze steering effects (2) Evaluation on 1,000 BigToM forward belief scenarios showing 14.2% accuracy improvement (32.5% to 46.7%) (3) Analysis demonstrating a systematical increase in emotional/creative processes during perspective-taking and decrease in analytical processes; (4) Findings suggesting that emotional understanding serves as the fundamental building block of perspective-taking in language models. Figure 1: Radar chart comparing baseline versus steered cognitive action activation patterns across categories. The steered condition (red) compared to baseline (blue). These findings support LLMs mirror known cognitive phenomena that emotional understanding is more important than analytical procecsses in perspective-taking Methodology We investigate how steering vectors modulate cognitive processes during belief attribution tasks. We developed a multi-stage pipeline integrating probe training, activation steering, and comparative analysis. We began by defining 45 cognitive actions across four categories: Metacognitive , Analytical , Creative , and Emotional (see Appendix). For each action, we generated 700 synthetic training samples (31,500 total) of first-person narratives (2-4 sentences) across 20 everyday contexts, using Gemma-3-4B 1 1 1 OpenSource link omitted for purposes of review. . Activations were extracted from layers 0-30 of Gemma-3-4B using nnsight, loosely based on the methodology from ( Chen et al. 2024 ) . To ensure consistent probe training, inputs were augmented (at extraction and during inference) with the suffix “The cognitive action being demonstrated here is” for consistent final-token extraction. We trained 45 binary linear probes using one-vs-rest classification with AdamW optimization, cosine annealing, and early stopping based on AUC-ROC. For activation steering, we trained CAA vectors ( Rimsky et al. 2023 ; Zou et al. 2023 ) on 752 contrastive triplets from BigToM’s ( Gandhi et al. 2023 ) forward_belief scenarios (50/50 false-true split). Each triplet contains story context, belief question, and positive (correct ToM) versus negative (incorrect ToM) completions, capturing representational differences between accurate and inaccurate perspective-taking. Vectors were trained across layers 14-30 using PCA-centered activation differences. Evaluation We evaluated 1,000 forward belief scenarios from BigToM ( Gandhi et al. 2023 ) ( forward_belief_false ), comparing baseline versus steered conditions. Answers were evaluated by computing p(correct) vs p(incorrect) from model logits. Cognitive action activations were captured at three timepoints: (1) at question, (2) after true answer, (3) after wrong answer. For each action, we computed layer count (layers 10-20 where probe confidence indicated presence) and analyzed baseline-steered differences to identify which cognitive processes characterize successful ToM improvements. Results We first validated our probe methodology, finding that binary probes achieved 0.78 average AUC-ROC and 0.68 F1 across 45 actions. Mid-layer performance (layers 5-24) outperformed early/late layers, informing our layer 10-20 analysis window. We then evaluated CAA steered LLMs on 1,000 forward belief scenarios (false condition), observing accuracy improvements from 32.5% to 46.7% (a 14.2% gain), shifting 217 examples from incorrect to correct predictions. To understand what drives this improvement, we analyzed cognitive action activation patterns in baseline versus steered conditions. The most striking finding was robust increase of emotional and generative processes. Emotional processes showed strong increases: emotion_perception (mean \Delta =+1.73), emotion_valuing ( \Delta =+0.85), and emotion_understanding ( \Delta =+0.77). Additionally, Hypothesis_generation ( \Delta =+1.63) remained strongly elevated across all timepoints, indicating active belief formation and explanation generation. In contrast, analytical processes decreased: questioning ( \Delta =-1.24), convergent_thinking ( \Delta =-1.13), and understanding ( \Delta =-0.77), suggesting successful persepective taking in LLMs suppresses deliberate analytical interrogation. Category aggregation reinforces this pattern—creative processes (+0.35, +0.28, +0.24) and emotional processes (+0.35, +0.20, +0.22) consistently increase across timepoints, while analytical processes show decreases (+0.06, -0.19, -0.19). Discussion This pattern of increased emotional and generative processes and decreased analytical processes challenges assumptions that LLM social reasoning relies on deliberate chain-of-thought mechanisms. Rather, successful perspective-taking appears to operate through activating representations responsible for processing emotional contexts. We make no claims that these findings generalize to human cognition, nor does our methodology validate such comparisons. However, neuroscience research shows that affective and cognitive ToM share neural mechanisms in humans: Corradi-Dell’Acqua et al. (2014) demonstrate that “patterns in TPJ (Temporoparietal Junction) and MTG (Middle Temporal Gyrus) reflect the same neuronal activity, equally recruited in these two independent conditions.” During language modeling, networks may learn shared representations linking perspective-taking with emotional context processing, mirroring the compressed structure of human social cognition embedded in linguistic data. Whether this constitutes genuine emulation of cognitive architecture or emergent convergence on functionally equivalent representations remains an open question. Future work should validate these cognitive decomposition findings with multiple, bigger models and with additional data sources. Conclusion Our work extends Bortoletto et al. (2024) by applying cognitive action probes to activation steered LLMs, we move beyond evaluating whether steering improves belief attribution to understanding why . Our analysis of 1,000 forward belief scenarios reveals systematic modulation of cognitive processes: steering amplifies generative hypothesis formation and emotional inference while suppressing analytical interrogation. This decomposition approach, combining interpretability tools with targeted interventions, offers a principled methodology for understanding complex AI capabilities. The findings challenge assumptions about deliberative social reasoning in LLMs and open new directions for mechanistic analysis of perspective-taking and other high-level cognitive abilities. Appendix A

Read the complete text