Examining Identity Drift in Conversations of LLM Agents
Junhyuk Choi, Yeseon Hong, Minju Kim, Bugeun Kim · 2024
Read Examining Identity Drift in Conversations of LLM Agents by Junhyuk Choi, Yeseon Hong, Minju Kim, Bugeun Kim (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00804.txt.
Introduction
Junhyuk Choi, Yeseon Hong, Minju Kim, and Bugeun Kim examine, in 2024, whether chatbots keep a stable identity over long talks. Large language models can chat well, yet their style or persona sometimes drifts as a conversation goes on. The team tested nine models in multi-turn chats on personal themes, scoring drift in two ways.
Three findings stand out. Larger models drifted more, not less. Differences between model families mattered, but less than model size. Giving a model a persona did not reliably hold it in place. The results point to design work needed for agents that must stay in character over long-term use.
Catalog record
| CREATOR | Junhyuk Choi, Yeseon Hong, Minju Kim, Bugeun Kim |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-00804
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Examining Identity Drift in Conversations of LLM Agents Junhyuk Choi, Yeseon Hong, Minju Kim, Bugeun Kim Abstract Large Language Models (LLMs) show impressive conversational abilities but sometimes show identity drift problems, where their interaction patterns or styles change over time. As the problem has not been thoroughly examined yet, this study examines identity consistency across nine LLMs. Specifically, we (1) investigate whether LLMs could maintain consistent patterns (or identity) and (2) analyze the effect of the model family, parameter sizes, and provided persona types. Our experiments involve multi-turn conversations on personal themes, analyzed in qualitative and quantitative ways. Experimental results indicate three findings. (1) Larger models experience greater identity drift. (2) Model differences exist, but their effect is not stronger than parameter sizes. (3) Assigning a persona may not help to maintain identity. We hope these three findings can help to improve persona stability in AI-driven dialogue systems, particularly in long-term conversations. 1 Introduction Recent research has actively explored the utilization of Large Language Models (LLMs) as chatbot systems by assigning them specific personas ( Samuel et al., 2024 ; Nandkumar and Peternel, 2024 ; Tseng et al., 2024 ) . To enhance user satisfaction in such systems, maintaining the consistency of the persona assigned to the LLM is critical. If the persona of an LLM loses its consistency, it may fail to deliver the user experience expected by the users, leading to usability issues ( Tanprasert et al., 2024 ) . So, researchers recently focused on investigating whether LLMs can preserve persona during a conversation, focusing on two aspects of persona: (1) memory that avoids conflict in conversation and (2) identity 1 1 1 Here, we refer to the term ’identity’ as factors that influence LLMs responses, such as behavioral patterns or talking style. This differs from psychological identity or consciousness, which we believe LLMs do not have. that maintains talking style or response patterns. Among the two aspects, we focus on whether LLMs can retain the given identity. Regarding the identity of persona, existing studies focused on LLMs’ identity ( Huang et al., 2023 ; Wang et al., 2024 ; Zhang et al., 2024a ; Frisch and Giulianelli, 2024 ) without any conversation. Mainly, most researchers examined which identity LLMs exhibit in a specific isolated situation. Though existing work revealed LLMs have a stable identity without any interaction, it is questionable whether LLMs can retain such identity throughout a long conversation. As many reports suggest that LLMs are very sensitive to contextual changes ( Sclar et al., 2024 ) , so having a conversation may make an ‘identity drift’ of LLMs during the interaction. A single case study on GPT ( Frisch and Giulianelli, 2024 ) supports this claim: identity can be changed only with a few agent interactions. Despite the case study, the result cannot be easily generalized to other models due to the difference in model families and parameter sizes. Therefore, we need a study to identify model-specific effects on identity drift. Thus, this paper compares the patterns of identity drift across nine LLMs and attempts to reveal the cause of such drifts. Especially, as our motivation begins with the persona of chatbots, we wanted to know whether LLMs suffer identity drifts during a conversation. In the experiment, we asked two LLM agents to discuss 36 themes that are related to one’s life, emotions, values, and feelings. We borrowed these themes from human study ( Aron et al., 1997 ) since they make agents discuss their virtual identity. After collecting conversational logs, we analyze identity drift patterns with the following two questions. RQ1. How do structural differences among LLMs affect identity drift? This research question focuses on the effect of model structure. As parameter sizes and model families may affect the performance and behavior of LLMs, we also suspect that such differences can cause changes in identity drifts. Thus, we employ a systematic comparison of identity patterns. Using topic modeling and PsychoBench ( Huang et al., 2023 ) , we successfully identified a relationship between model structure and identity drift. Here, we decided not to provide a persona as input because the persona may introduce unwanted effects. RQ2. How does the provided persona affect identity drift? We pose another research question to observe the effect of persona. Specifically, we provide two kinds of personas to LLMs regarding how much the prompt asks LLMs to be influenced by the conversational partner: low and high. As instruction-tuned LLMs try to follow inputs as instruction, we suspect that low-influence persona may show a lower identity drift than the others. So, we used LLMs, which showed strong drifts in RQ1, to test whether the effect of persona is larger than that of the model. 2 Related Work Researchers have been examining two factors that affect consistency in conversations: memory and identity. Because people generally expect consistency throughout a dialogue, researchers first started by examining memory consistency, which can easily form a task. A large body of existing research has focused on how memory is retained, largely verifying whether an LLM continues to remember certain information during conversation ( Tseng et al., 2024 ; Chen et al., 2023 ; Maharana et al., 2024 ; Zhang et al., 2024b ; Afzoon et al., 2024 ) . For instance, Chen et al. (2023) analyzed how consistently an LLM can uphold a given memory. Meanwhile, Maharana et al. (2024) created the LoCoMo dataset to investigate how well they remember information over prolonged conversations. However, memory is not the only factor that affects task performance or the naturalness of a dialogue; identity should be provided ( Wu et al., 2023 ; Li et al., 2023 ; Abbasiantaeb et al., 2024 ; Zhang et al., 2024a ) . For example, Zhang et al. (2024a) assessed LLMs’ ability to engage in cooperative interactions based on Society of Mind theory ( Minsky, 1988 ) in a multi-agent environment. Similarly, Abbasiantaeb et al. (2024) reported that it is possible to model a conversational question-answering task as a virtual interaction between a teacher agent and a student agent using an LLM. By qualitatively assessing the quality of the interaction, they found that providing two identities could improve the interaction process in a more human-like manner. Also, Li et al. (2023) simulated a job fair scenario with two agents: a job seeker and an employer. They explored how their cooperative interaction affects task performance. However, all of these studies assume that the identity remains unchanged when a conversation progresses. Considering that the memory of a persona changes during a conversation, the identity could also be changed. Hence, recently, researchers attempted to quantify the identity of persona before measuring its consistency. Some researchers designed benchmarks measuring the identity of LLM ( Huang et al., 2023 ; Wang et al., 2024 ; Zhang et al., 2024a ; Frisch and Giulianelli, 2024 ) . For example, Huang et al. (2023) assessed the identity of LLMs using fourteen types of questionnaires. Though they found that different LLMs exhibit different identities, they did not let LLMs converse before measuring the identity. However, impact of conversation is crucial because accumulated chat histories can introduce unexpected effects, as memory-related studies suggested. Frisch and Giulianelli (2024) supports this claim.