Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generation

Yonghyun Jun, Hwanhee Lee · 2025

Read Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generation by Yonghyun Jun, Hwanhee Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11423.txt.

Introduction

Yonghyun Jun and Hwanhee Lee studied in 2025 how the mood of a user persona steers chatbot conversation. Personalized dialogue systems feed a user profile into a language model, but the profile's emotional tilt had gone unexamined. The team generated dialogues across a range of polarized profiles. Strongly negative personas made the model lean too hard on profile traits, crowding the exchange.

Positive profiles produced smoother chats that used persona details selectively. Weak or neutral personas gave the weakest dialogues of all. The authors answer with a generation method that plans for polarity: it builds replies turn by turn, orders profile facts deliberately, and prompts with sentiment in mind. Their analysis gives designers a new dial to watch. Persona sentiment, they show, is not decoration; it shapes quality.

Catalog record

CREATORYonghyun Jun, Hwanhee Lee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11423

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generation Yonghyun Jun, Hwanhee Lee Abstract Personalized dialogue systems have advanced considerably with the integration of user-specific personas into large language models (LLMs). However, while LLMs can effectively generate personalized responses, the influence of persona sentiment on dialogue quality remains underexplored. In this work, we conduct a large-scale analysis of dialogues generated using a range of polarized user profiles. Our experiments reveal that dialogues involving negatively polarized users tend to overemphasize persona attributes. In contrast, positively polarized profiles yield dialogues that selectively incorporate persona information, resulting in smoother interactions. Furthermore, we find that personas with weak or neutral sentiment generally produce lower-quality dialogues. Motivated by these findings, we propose a dialogue generation approach that explicitly accounts for persona polarity by combining a turn-based generation strategy with a profile ordering mechanism and sentiment-aware prompting. Our study provides new insights into the sensitivity of LLMs to persona sentiment and offers guidance for developing more robust and nuanced personalized dialogue systems. * * * Our implementation is publicly available at https://github.com/imsongpasimin/PesonaSensitivity 1 Introduction Personalized dialogue systems have evolved through advances in dataset construction Li et al. (2016) ; Zhang (2018) , training methodologies Kingma (2013) ; Bowman et al. (2015) , and studies on integrating user-specific preferences to enhance conversational abilities Zhang (2019) ; Roller (2020) . With the advent of Large Language Models (LLMs), embedding personas via system prompts Yang et al. (2023) ; Wang et al. (2023b) has become effective. This progress has enabled applications ranging from personalized chatbots Shuster et al. (2022) ; Lee et al. (2023) to enhanced performance in downstream tasks through inter-LLM conversations Park et al. (2023) and synthetic dataset generation Jandaghi et al. (2023) . These developments underscore the growing importance of personalization in dialogue systems. Figure 1: Examples of dialogues generated by polarized users. We indicate Positive personas in blue text and Negative personas in red text. Matching bullet points denote where user persona attributes appear in the utterance, and we highlight them in yellow if they contradict. Incoherent segments of the dialogue are marked with a red background. However, despite these advances, recent studies have shown that LLMs are highly sensitive to contextual sentiment polarity Liu et al. (2024) ; Wu et al. (2024b) . In contrast, the impact of assigned persona sentiment on dialogue quality remains less explored. For example, Figure 1 illustrates dialogues generated by users with distinctly positive and negative personas. In dialogues driven by positive personas, the integration of persona attributes occurs in a manner that supports coherence and aligns naturally with the flow of conversation. Conversely, dialogues featuring negative personas often display contradictions and inconsistencies, as the persona attributes may be overemphasized or misaligned with the dialogue context. These findings highlight the necessity of systematically examining how variations in persona polarity influence LLM behavior. In this paper, we construct and analyze personalized dialogues generated across a spectrum of polarized profiles. (§ 3 ) Our extensive analysis of personalized dialogues generated across a diverse spectrum of polarized profiles reveals several key insights: 1) Dialogues between negatively polarized users tend to overemphasize persona attributes , which leads to increased contradictions and reduced overall coherence. 2) Dialogues between positively polarized users selectively incorporate persona information , resulting in interactions that are both coherent and contextually appropriate. 3) Profiles with weak or neutral sentiment generally yield lower-quality dialogues , as evidenced by significant differences in various metrics. These insights underscore the need for methods that explicitly account for persona polarity in the design of personalized dialogue systems. To address this challenge, we propose a dialogue generation framework that combines a turn - based strategy, a profile ordering mechanism, and a sentiment - aware prompting. Specifically, personas exhibiting neutral or weak sentiment are placed earlier in the conversation, whereas more positive personas are positioned later. A concise prompt further instructs the model to attend carefully to negative or neutral sentiments when generating responses. These approaches promote more consistent and coherent interactions (§ 4 ). Our findings shed light on LLM sensitivity to persona sentiment and offer valuable guidance for developing more nuanced and effective personalized dialogue systems. 2 Study Design and Experimental Setup This section explains the overall basic setup for our study. We first introduce the dataset and describe how we construct polarized user profiles. Next, we present our research questions, detail the dialogue generation strategies, and describe the metrics used to assess the dialogues. We use four RTX 4090 GPUs to generate and evaluate all the dialogues. 2.1 Dataset Early research on personalized dialogue systems typically used two-party conversations paired with user profiles. These profiles are commonly represented in three ways Chen et al. (2024) : • Descriptive sentences: Each persona is described by a few sentences (often five) that together form a user profile Zhang (2018) ; Dinan et al. (2020) . • Sparse key-value attributes: Personas are specified using concise attributes Qian et al. (2017) ; Gao et al. (2023) . • User history: Each speaker is identified by an ID that can be used to retrieve external persona information Salemi et al. (2023) . Although several datasets exist, most trace back to PersonaChat Zhang (2018) and ConvAI2 Dinan et al. (2020) . In our experiments, we adopt the ConvAI2 dataset and define user personas primarily through descriptive sentences . 2.2 Polarized User Profile Construction We extract polarized user personas from the ConvAI2 dataset using a sentiment classifier and then build coherent profiles by combining multiple personas while avoiding contradictions. Polarity-Aware Persona Sampling Each persona in the ConvAI2 spans various content domains and carries a distinct sentiment polarity. We use the distilbert-base-uncased-finetuned-sst-2-english model HF Canonical Model Maintainers (2022) to classify the sentiment of each sentence, retaining only those with a confidence score above 0.99 as polarized personas . Consequently, as shown in Figure 2 , 2,691 are classified as positive, 1,006 as negative, and 2,429 as neutral (weak), indicating that polarized personas constitute the majority (around 60% of all personas). Notably, approximately 17% of the dataset’s personas are negatively polarized—a higher proportion than one might intuitively expect. These figures suggest that polarized personas are not merely reflect a contrived scenario fabricated for the experiment. For detailed statistics, see Appendix F . Figure 2: Distribution of persona sentences by polarity in the ConvAI2 dataset. We only map predictions with a confidence score of at least 0.99 to their respective polarity, classifing all remaining cases as neutral (weak). Synthesizing User Profiles To create a user profile, we combine K polarized personas, where K is the number of personas we include in each profile. To avoid internal contradictions, we follow the procedure in Jandaghi et al. (2023) : starting with an initial persona, we iteratively add additional personas only if the nli-deberta-v3-large He et al. (2021) detects no conflicts with the existing set. This approach yields three profile types: • Negative Profile: only negative personas. • Positive Profile: only positive personas. • Mixed Profile: a random mix of positive and negative personas. We generate 10K unique profiles for each type, which serve as the foundation for our dialogue generation experiments. 2.3 Research Questions Building on the established user-polarized profiles, we structure our study around several inquiries.

Read the complete text