Individualized Cognitive Simulation in Large Language Models: Evaluating Different Cognitive Representation Methods

Tianyi Zhang, Xiaolin Zhou, Yunzhe Wang, Erik Cambria, David Traum, Rui Mao · 2025

Read Individualized Cognitive Simulation in Large Language Models: Evaluating Different Cognitive Representation Methods by Tianyi Zhang, Xiaolin Zhou, Yunzhe Wang, Erik Cambria, David Traum, Rui Mao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20252.txt.

Introduction

Tianyi Zhang and five co-authors test how well large language models can copy the thought style of one named person. They call this task individualized cognitive simulation. Models can mimic surface acts like role-play with ease. It is far less clear that they catch deep thought habits. Their test set uses new novels that came out after the models were built.

Seven stock models face 11 test states. The states vary how a mind is shown: by speech traits, by idea maps, or by a plain profile. Speech traits plus idea maps work best as a pair. Plain profiles do worse. Models copy speech style far more than story shape, which marks a clear bound on deep thought mimic work.

Catalog record

CREATORTianyi Zhang, Xiaolin Zhou, Yunzhe Wang, Erik Cambria, David Traum, Rui Mao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-20252

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Individualized Cognitive Simulation in Large Language Models: Evaluating Different Cognitive Representation Methods Tianyi Zhang, Xiaolin Zhou, Yunzhe Wang, Erik Cambria, David Traum, Rui Mao Abstract Individualized cognitive simulation (ICS) aims to build computational models that approximate the thought processes of specific individuals. While large language models (LLMs) convincingly mimic surface-level human behavior such as role-play, their ability to simulate deeper individualized cognitive processes remains poorly understood. To address this gap, we introduce a novel task that evaluates different cognitive representation methods in ICS. We construct a dataset from recently published novels (later than the release date of the tested LLMs) and propose an 11-condition cognitive evaluation framework to benchmark seven off-the-shelf LLMs in the context of authorial style emulation. We hypothesize that effective cognitive representations can help LLMs generate storytelling that better mirrors the original author. Thus, we test different cognitive representations, e.g., linguistic features, concept mappings, and profile-based information. Results show that combining conceptual and linguistic features is particularly effective in ICS, outperforming static profile-based cues in overall evaluation. Importantly, LLMs are more effective at mimicking linguistic style than narrative structure, underscoring their limits in deeper cognitive simulation. These findings provide a foundation for developing AI systems that adapt to individual ways of thinking and expression, advancing more personalized and human-aligned creative technologies. 1 Introduction Artificial intelligence (AI) was initially conceived as an attempt to replicate human intelligence, drawing inspiration from the mechanisms of human thought and reasoning ( Cambria et al. 2023 ) . Recently, the rapid development of large language models (LLMs) has marked a qualitative leap in the processing capabilities of AI systems, enabling machines to perform complex linguistic, analytical, and generative tasks with unprecedented fluency and scale. Yet, the broader aspiration of modeling human cognition remains only partially realized. Conventional role-play techniques within LLMs have shown promise in imitating professional roles, social identities, and behavioral archetypes ( Shanahan et al. 2023 ) , whereas these simulations are inherently constrained: they reproduce externalized patterns of behavior rather than capturing the subtle, dynamic, and individualized nature of cognition. Extending such methods to represent individual cognitive profiles is difficult because our theoretical and computational understanding of how to represent individual cognition, as distinct from generalized occupational categories such as lawyers, doctors, or researchers, is still limited. The significance of cognitive representations for individuals lies in their potential to uncover the unique ways in which individuals perceive, reason, and act across diverse contexts. It is critical in designing systems that move beyond generic behavioral templates to support adaptive, personalized interactions, whether in education, healthcare, decision-making support, or social simulation ( Mao et al. 2025a ) . In this work, we aim to test the utility of different cognitive representation methods in the context of individualized cognitive simulation (ICS). Objectively evaluating cognitive patterns is difficult because cognition itself is multifaceted, context-dependent, and often expressed implicitly through behavior, language, and decision-making rather than in directly observable forms. In light of this, we propose a novel evaluation task that measures how well LLM continuations match the authorial style of original texts after injecting different cognitive representations. Our hypothesis is that prompting LLMs with effective cognitive representations can lead to greater stylistic consistency in storytelling, making their outputs more closely resemble the original authors. This hypothesis is in line with “ Bayesian Models of Cognition ” from Griffiths et al. 2001 , where human behavior is characterized as a conditional distribution: P(\text{response}\mid\text{stimulus},\text{cognitive model}) This formulation links observable responses to external stimuli through the mediating role of internal cognitive structures. By analogy, LLM outputs can be viewed as responses conditioned not only on direct context stimuli but also on injected cognitive representations. Thus, we construct a dataset from recently published novels and propose an 11-condition cognitive evaluation framework to benchmark seven off-the-shelf LLMs in the context of authorial style emulation. To prevent the risk of data leakage ( Wu et al. 2025 ) , we source novels that were published later than the release date of the tested LLMs. The evaluated cognitive representation methods include linguistic features, concept mappings, and author profile information (persona, background, Big Five/OCEAN personality). Linguistic features reflect the direct writing habits of an author; concept mappings capture personalized understandings of abstract concepts and thinking frameworks; and profile information represents prior experiences that may shape unique cognitive, emotional, and stylistic patterns. Together, these dimensions provide complementary perspectives on how cognition manifests in text. We prioritize these representations because of their theoretical grounding and non-invasive accessibility. They are readily derived from text or public reports, making them practical to implement for uncovering the cognitive signatures of individual authors in real-world applications. By testing on the continuous writing tasks from five authors, we find that the combination of concept mappings and linguistic features achieves the highest overall performance, significantly outperforming both single-feature and other multi-feature settings. This synergy suggests that while linguistic cues capture surface-level stylistic habits, concept mappings provide deeper insight into how authors conceptualize and structure meaning across narrative contexts, and together they offer a complementary pathway to more faithful style emulation. In contrast, profile-based features such as persona, background, or personality traits yield limited improvements and sometimes even degrade performance when combined, highlighting the challenges of translating high-level biographical information into effective narrative generation signals. The contribution of this work can be summarized as follows: 1) Individualized Cognitive Simulation: A new task using computational models to approximate individual cognitive and stylistic processes. 2) Cognitive Evaluation Framework: An 11-condition framework that integrates linguistic, conceptual, and profile-based representations into narrative continuation, combining LLM-based automatic metrics and human evaluation to assess the utilities of different cognitive representations. 3) Empirical Research Findings: The combination of concept mappings and linguistic features is the most effective signal for ICS, achieving the strongest overall performance; LLMs mimic linguistic style more effectively than narrative structure, revealing the limits of their deeper cognitive simulation. 2 Related Work Research on ICS draws from computational linguistics, cognitive modeling, and narrative generation. Stylometry and authorship attribution showed how linguistic patterns reveal individual traits ( Stamatatos 2009 ) , later extended to style transfer and personalized generation with neural models ( Shen et al. 2017 ; Fu et al. 2018 ) . Recent LLM studies explored author imitation and persona-driven writing ( Zhang et al. 2018 ; Huang et al. 2023 ; Bhandarkar et al. 2024 ) , which we extend by integrating stylistic, conceptual, and profile signals. Role-play has emerged as another common approach for guiding LLMs to simulate personas or identities. This is typically realized through a short role preamble that assigns the model an identity (e.g., “You are a doctor”) ( de Araujo & Roth 2024 ) , in-context exemplars that script how the role should sound or behave ( Rupprecht et al. 2025 ; Zhang et al. 2025 ) , and conversational updating that incrementally refines the role as interaction unfolds across turns ( Xu et al. 2022 ; Wang et al. 2025 ) . However, role-play remains limited to surface behaviors: it can mimic style but struggles to represent the dynamic and individualized cognitive traits of real authors. Recent studies confirm this limitation, with Hu & Collier 2024 showing that current persona effects in LLMs are shallow and

Read the complete text