Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions

Minwoo Kang, Suhong Moon, Seung Hyeong Lee, Ayush Raj, Joseph Suh, David M. Chan, John Canny · 2025

Read Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions by Minwoo Kang, Suhong Moon, Seung Hyeong Lee, Ayush Raj, Joseph Suh, David M. Chan, John Canny (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-11673.txt.

Introduction

Minwoo Kang and six co-authors ask whether language models truly think like group members in a 2025 study. Models are often used to stand in for people in polls and surveys. The team calls a model deep if it answers as a true in-group member would. They call it shallow if it answers as an outsider thinks a member would.

To test this, they build virtual personas from long, made-up life stories told as interview transcripts. The idea rests on the view that identity grows out of self-narratives. Personas built this way match real human response patterns far more closely. The gain reaches 87 percent on one distance measure, and group bias effects match the source studies well.

Catalog record

CREATORMinwoo Kang, Suhong Moon, Seung Hyeong Lee, Ayush Raj, Joseph Suh, David M. Chan, John Canny
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE108 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-11673

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deep Binding of Language Model Virtual Personas: a Study on Approximating Political Partisan Misperceptions Minwoo Kang, Suhong Moon, Seung Hyeong Lee, Ayush Raj, Joseph Suh, David M. Chan, John Canny Abstract Large language models (LLMs) are increasingly capable of simulating human behavior, offering cost-effective ways to estimate user responses to various surveys and polls. However, the questions in these surveys usually reflect socially understood attitudes: the patterns of attitudes of old/young, liberal/conservative, as understood by both members and non-members of those groups. It is not clear whether the LLM binding is deep , meaning the LLM answers as a member of a particular in-group would, or shallow , meaning the LLM responds as an out-group member believes an in-group member would. To explore this difference, we use questions that expose known in-group/out-group biases. This level of fidelity is critical for applying LLMs to various political science studies, including timely topics on polarization dynamics, inter-group conflict, and democratic backsliding. To this end, we propose a novel methodology for constructing virtual personas with synthetic user “backstories” generated as extended, multi-turn interview transcripts. This approach is justified by the theory of narrative identity which argues that personality at the highest level is constructed from self-narratives. Our generated backstories are longer, rich in detail, and consistent in authentically describing a singular individual, compared to previous methods. We show that virtual personas conditioned on our backstories closely replicate human response distributions (up to an 87% improvement as measured by Wasserstein Distance) and produce effect sizes that closely match those observed in the original studies of in-group/out-group biases. Altogether, our work extends the applicability of LLMs beyond estimating socially understood responses, enabling their use in a broader range of human studies. † † *Equal contribution. Co-first authors listed in alphabetical order. † † Code available at: CannyLab/alterity and data available at: HuggingFace/alterity_backstory † † No Google authors used or analyzed data from Llama models. 1 Introduction Human identity is intrinsically relational , intertwined with how one perceives others both in relation and in contrast to oneself ( Cooley, 1902 ; Tajfel & Turner, 1979 ; Tajfel & Turner, 1986 ) . As documented across the social sciences, psychology, and philosophy, individual identities cannot be meaningfully examined outside their social contexts ( Chen & Li, 2009 ; Benjamin et al., 2010 ; Charness & Chen, 2020 ; Shayo, 2020 ) . Importantly, the way individuals perceive group norms to form collective social judgment and engage in inter-group interactions is central to understanding various social phenomena ( Chambers et al., 2006 ; Westfall et al., 2015 ; Lees & Cikara, 2020 ; Saguy & Kteily, 2011 ; Waytz et al., 2014 ) . While recent large language models (LLMs) have been shown to simulate human behavior expressed in natural language ( Dillion et al., 2023 ; Korinek, 2023 ; Bail, 2024 ; Moon et al., 2024 ; Park et al., 2023 ; Park et al., 2024a ) , prior analysis has overlooked the interplay between individual opinions and social identities. For example, prior work consider questions eliciting self-opinions of respondents ( Li et al., 2023 ; Santurkar et al., 2023 ; He et al., 2024 ) , as shown in the first example in Figure 1 : “Would you support using political violence?” Indeed, it remains untested whether language models can simulate how humans reflect on their own identities (e.g. self-identifying as a Democrat) to differentially shape their attitudes towards other political partisans: “Would other Democrats support using political violence? What about Republicans ? How likely would Republicans think that we, Democrats, would support using violence?” Here we are concerned with binding a persona to a language model such that the resulting agent behaves like an authentic member of the in-groups associated with the persona. To achieve this, we rely on a widely-used model in personality psychology: McAdams’ theory of narrative identity ( McAdams, 1995 ; McAdams, 2001 ; McAdams, 2011 ; McAdams & McLean, 2013 ) . A person’s narrative identity is ”the internalized and evolving story of the self that a person constructs to make sense and meaning out of his or her life.” Narrative identity asserts that a person does not just construct a life story that explains themself, the self is in fact constructed by such a story. Figure 1: Towards Deep Binding of LLMs. Prior work on LLM virtual personas use a variety of techniques to bind the LLM to various social groups, including demographic and political. They do not distinguish whether the model response is similar to an authentic in-group member (deep binding, repesented by the identity above the robot icon) vs an out-group member (shallow binding). There are well known in-group/out-group biases for certain questions that can be used to test the strength of the LLM binding, such as the above. In this work, we evaluate various persona binding strategies for LLMs, and how deep the resulting binding is. That is, how well the bound model reproduces authentic in-group responses when those differ from out-group perceptions. In particular, we focus on domains such as political polarization, intergroup conflict, and democratic backsliding ( Pew Research Center, 2022 ; Braley et al., 2023 ; Moore-Berg et al., 2020 ) , where political attitudes are shaped not only by individual beliefs but also by one’s identity as a member of a social group ( Iyengar et al., 2012 ; Mason, 2018 ; Ahler & Sood, 2018 ) . Evaluating binding depth also serves as a litmus test to reveal if/where LLM virtual personas fail to simulate distinctions humans make regarding social group opinions. We find that existing methods of conditioning virtual subjects ( Park et al., 2024a ; Moon et al., 2024 ) yield only shallow conditioning that fall short of emulating the differences between perceived group opinions ( Section 4 ). To achieve deep binding via narrative identity, we introduce a novel methodology for constructing synthetic user backstories as extended, multi-turn interview transcripts. Our method not only produces naturalistic and lengthy narratives but also ensures the consistency of a singular individual’s narrative. Our experimental findings show that virtual subjects constructed via our approach present closer replication of human response distributions and better align effect sizes with empirical data on partisan misperception and exaggerated meta-perceptions ( Moore-Berg et al., 2020 ; Pew Research Center, 2022 ; Braley et al., 2023 ) . Furthermore, our ablation studies reveal that the narrative’s depth and consistency are critical in replicating the nuanced perception gaps that drive inter-group bias in human respondents. In short, we present the following contributions: • We introduce a novel problem context for LLM simulation of behavioral studies that highlights the differences between perception and meta-perception of different social groups, through which we expand the scope of studies considered in the existing literature. • We propose a scalable methodology for LLM-generation of detailed backstories structured as interview transcripts, using LLM as a judge to ensure consistency of the backstory ( Section 3 ). • We show that LLMs conditioned on our backstories achieves deep binding to target personas enabling a 87% improvement in matching the human responses to survey questions on outgroup hostility ( Section 4.2 ), democratic backsliding ( Section 4.2 ), and

Read the complete text