TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation

Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu · 2025

Read TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation by Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25536.txt.

Introduction

Bangde Du led ten co-authors in presenting TwinVoice. Minghao Guo, Songming He, and Ziyi Ye helped. Xi Zhu, Weihang Su, and Shuqi Zhu joined. Yujia Zhou, Yongfeng Zhang, and Qingyao Ai joined. Yiqun Liu closes the eleven-author team. Their 2025 paper benchmarks persona run with language models.

Models increasingly simulate single contact style and personality. Current tests rely on synthetic dialogues mostly. Systematic approaches and capability study are lacking. TwinVoice assesses persona run across real-world contexts. Three dimensions form the benchmark design. Social persona covers public social effects. Interpersonal persona covers private dialogues between people. Narrative persona covers role-based expression tasks. Six basic capabilities decompose model results further. They include opinion consistency and memory recall. Logical thinking, lexical fidelity, tone, and style are measured. Advanced models reach moderate persona run quality. They fall short on syntactic style and memory. Average results remains considerably below human rival.

Catalog record

CREATORBangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-25536

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu Abstract Large Language Models (LLMs) are exhibiting emergent human-like abilities and are increasingly envisioned as the foundation for simulating an individual’s communication style, behavioral tendencies, and personality traits. However, current evaluations of LLM-based persona simulation remain limited: most rely on synthetic dialogues, lack systematic frameworks, and lack analysis of the capability requirement. To address these limitations, we introduce TwinVoice, a comprehensive benchmark for assessing persona simulation across diverse real-world contexts. TwinVoice encompasses three dimensions: Social Persona (public social interactions), Interpersonal Persona (private dialogues), and Narrative Persona (role-based expression). It further decomposes the evaluation of LLM performance into six fundamental capabilities, including opinion consistency, memory recall, logical reasoning, lexical fidelity, persona tone, and syntactic style. Experimental results reveal that while advanced models achieve moderate accuracy in persona simulation, they still fall short of capabilities such as syntactic style and memory recall. Consequently, the average performance achieved by LLMs remains considerably below the human baseline. † † affiliation: Tsinghua University † † affiliation: Rutgers University † † affiliation: Fudan University † † contribution: Equal Contribution † † contribution: Corresponding authors † † \faGlobe\spaceProject Page: https://twinvoice.github.io † † \faGithub\spaceCode: https://github.com/TwinVoice/TwinBench † † \faSmileBeam\spaceLeaderboard: https://huggingface.co/spaces/bangdedadi/TwinVoice-Leaderboard † † \faDatabase\spaceDataset: https://huggingface.co/datasets/bangdedadi/TwinVoice † † footnotetext: Authors’ email addresses: Bangde Du( [email protected] ), Minghao Guo( [email protected] ), Songming He( [email protected] ), Ziyi Ye( [email protected] ), Xi Zhu( [email protected] ), Weihang Su( [email protected] ), Shuqi Zhu( [email protected] ), Yujia Zhou( [email protected] ), Yongfeng Zhang( [email protected] ), Qingyao Ai( [email protected] ), Yiqun Liu( [email protected] ). 1 Introduction Large Language Models (LLMs) are rapidly evolving from basic text generators into human-like agents ( Bubeck et al. 2023 ; Wei et al. 2022 ; Chang et al. 2024 ) . Existing studies have shown that the most advanced LLMs are capable of producing text indistinguishable from human writing ( Jones and Bergen 2025 ; Jones et al. 2025 ; Jones and Bergen 2024 ) . Consequently, the research focus is shifting toward a highly specific challenge: Can we construct “digital twins” of specific individuals that are indistinguishable from themselves? To address this challenge, existing studies have relied on LLM-based persona simulation, which replicates a person’s unique style of talking, behavior, and personality ( Shanahan et al. 2023 ; Park et al. 2023 ) based on their data. LLM-based persona simulation is supposed to unlock a series of applications, including highly personalized assistants ( Ma et al. 2023 ; Li et al. 2025a ) , social simulations ( Li et al. 2023 ; Ran et al. 2025 ; Rossetti et al. 2024 ) , healthcare ( Barricelli et al. 2020 ) , and marketing ( Hornik and Rachamim 2025 ) . Despite growing interest in creating digital twins with LLM-based persona simulation, its current ability remains unexplored due to the lack of systematic evaluation ( Toubia et al. 2025 ; Zhou et al. 2025 ) . Figure 1 : The conceptual framework of TwinVoice: (Left) The evaluation is structured across three core dimensions that represent distinct aspects of identity expression: Social Persona (public facing), Interpersonal Persona (private interaction), and Narrative Persona (fictional scenarios). The LLMs are prompted with a person’s historical context to simulate their behavior. The LLM’s ability for persona simulation is categorized into six fundamental capabilities . (Right) Experimental results averaged over three dimensions are presented. To address this issue, current evaluations have attempted to test LLM’s ability in imitating and predicting human behaviors. For example, BehaviorChain ( Li et al. 2025b ) evaluates continuous persona-based behavior by requiring models to iteratively predict the next action given persona profile and history, with performance degrading as chains lengthen. Human Simulacra ( Xie et al. 2025 ) and PersoBench ( Afzoon et al. 2024 ) assess human-likeness and personalized response quality, while other studies probe persona-driven decision making, counterfactual adherence, and large-scale dynamic profiling ( Xu et al. 2024 ; Kumar et al. 2025 ; Jiang et al. 2025 ) . However, those evaluation benchmarks face limitations in both their scope and granularity. On the one hand, the predominant reliance on synthetic dialogues ( Shen et al. 2023 ; Tu et al. 2024 ) prevents benchmarks from capturing the rich expression of human identity across diverse real-world contexts ( Scope Limitation ). On the other hand, current benchmarks are often evaluated based on an LLM’s accuracy in predicting human behavior, leaving a critical gap in understanding the fundamental capabilities—such as memory, reasoning, and lexical fidelity—that a model must possess for authentic simulation ( Granularity Limitation ). To bridge the gap between the vision of digital twins and the current capabilities of persona simulation, we introduce TwinVoice , a comprehensive benchmark designed for realistic and fine-grained persona evaluation (see Table 1 for a comparison with prior persona‑simulation benchmarks; “persona size” denotes the number of distinct, independent personas per benchmark). To address the scope limitation, TwinVoice is grounded in both real-world and synthetic data across three complementary dimensions in persona simulation (see Figure 1 ): Social Persona , Interpersonal Persona , Narrative Persona . The Social Persona dimension leverages real-world social media data to evaluate a public-facing identity, while the Interpersonal Persona dimension utilizes multi-session dialogue data to assess a more private, relational self. While these two dimensions are grounded in authentic digital footprints, the Narrative Persona is designed to complement such data with fictional scenarios to test behaviors and narrative consistency in more diverse contexts. Addressing the granularity limitations of holistic accuracy evaluations, we shift from end-to-end scoring to capability-level assessment. Building on psycholinguistic evidence that language conveys both what people say and how they say it ( Pennebaker et al. 2003 ) , we group persona fidelity into Mindset Coherence and Linguistic Expression. Mindset Coherence assesses the logical and factual consistency of the content, including Opinion Consistency ( Zaller 1992 ) , Memory Recall ( Clark and Brennan 1991 ) , and Logical Reasoning ( Kahneman 2011 ) . Linguistic Expression evaluates the language’s stylistic form, encompassing Lexical Fidelity ( Mehl et al. 2006 ; Koppel et al. 2009 ) , Persona Tone ( Brown 1987 ) , and Syntactic Style ( Biber 1995 ) . Digital twins face two practical interaction modes: constrained selection among plausible replies and open-ended generation under realistic stimuli. Accordingly, TwinVoice instantiates both modes in its tasks to balance objectivity and ecological validity. We later evaluate the tasks with a multiple-choice protocol and a judge-based protocol, and include a human consistency check (see Sections 5.3 and 5.4). Table 1 : A comparison of TwinVoice with Prior LLM Persona‑Simulation Benchmarks. Benchmark Persona Size Real-World Sourcing Multiple Dimensions Multi‑Paradigm Evaluation Human Baseline Fine-Grained Capabilities Multilingual Coverage Human Simulacra ( Xie et al. 2025 ) 11 ✗ ✗ ✗ ✓ ✗ ✗ BehaviorChain ( Li et al. 2025b ) 1,001 ✓ ✗ ✓ ✗ ✗ ✗ PersonaEval ( Zhang et al. ) 130 ✓ ✓ ✗ ✓ ✓ ✗ PERSONAMEM ( Jiang et al. 2025 ) 20 ✗ ✓ ✓ ✗ ✓ ✗ TwinVoice OURS 4,553 ✓ ✓ ✓ ✓ ✓ ✓ We evaluate a series of state-of-the-art

Read the complete text