From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment
Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong · 2025
Read From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment by Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16610.txt.
Introduction
Jing Ye, Lu Xiang, Yaping Zhang, and Chengqing Zong published a 2025 paper on making AI emotional support personal. Large language models can sound caring, but their comfort is often generic. Good support depends on the person: their character, their mood, and their situation.
The authors propose a two-stage framework in which the model improves itself. First it learns basic support skills from a small set of conversations. Then it reflects on its own answers, rewrites them to fit the user better, and trains on the improved versions. Repeated rounds steadily sharpen the fit to each user’s unspoken preferences. Tests showed fewer unhelpful replies and answers that matched users more closely.
Catalog record
| CREATOR | Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16610
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong Abstract Effective emotional support hinges on understanding users’ emotions and needs to provide meaningful comfort during multi-turn interactions. Large Language Models (LLMs) show great potential for expressing empathy; however, they often deliver generic and one-size-fits-all responses that fail to address users’ specific needs. To tackle this issue, we propose a self-evolution framework designed to help LLMs improve their responses to better align with users’ implicit preferences concerning user profiles (personalities), emotional states, and specific situations. Our framework consists of two distinct phases: (1) Emotional Support Experience Acquisition , where LLMs are fine-tuned on limited emotional support conversation data to provide basic support, and (2) Self-Improvement for Personalized Emotional Support , where LLMs leverage self-reflection and self-refinement to generate personalized responses. Through iterative direct preference optimization between the pre- and post-refined responses, our model generates responses that reflect a better understanding of the user’s implicit preferences. Extensive experiments and evaluations demonstrate that our method significantly enhances the model’s performance in emotional support, reducing unhelpful responses and minimizing discrepancies between user preferences and model outputs. Corresponding Author 1 Introduction Figure 1: Example responses. Direct prompting of LLaMA results in verbose and formulaic outputs. Task-Specific SFT is empathetic but often lacks depth and variety, giving it a perceived "AI-like" quality. In contrast, self-reflection on user preferences provides a pathway to more specific and engaging responses. Emotional support conversation (ESC) systems require a deep understanding of users’ emotions and need to provide meaningful comfort and assistance during multi-turn interactions Peng et al. (2022) ; Rains et al. (2020) , which are vital in practical applications such as mental health care, emotional companionship, and customer service. Given that each user has unique emotional needs and experiences Rogers (2013) , delivering personalized and contextually appropriate emotional support is essential for ensuring practical assistance Campos et al. (2018) ; Cheng et al. (2023) . Despite the promising potential of LLMs for generating empathetic responses Touvron et al. (2023) ; Yang et al. (2024) ; Achiam et al. (2023) , they often struggle to provide diverse and contextually appropriate support Wang et al. (2024a) . As illustrated in Figure 1 , direct prompting LLMs often results in superficial empathy, verbosity, and formulaic structures . A simple yet effective approach is supervised fine-tuning (SFT) LLMs on ESC corpora Zheng et al. (2024b) ; Zheng et al. (2023) ; Qiu et al. (2023) ; Zhang et al. (2024) . However, SFT relies on substantial, high-quality ESC data, which is often scarce and difficult to acquire. Moreover, over-reliance on SFT can lead to repetitive responses that express empathy overtly but lack depth and variety Irvine et al. (2023) . As demonstrated in Figure 1 and 6 , SFT models can fall into predictable patterns, frequently using phrases like " It sounds like… " or " I’m sorry to hear that… ". Recent insights highlight that LLMs can self-improve their performance through self-reflection and self-refinement guided by human-designed principles Lu et al. (2024b) ; Madaan et al. (2023) ; Ye et al. (2023) ; Yasunaga et al. (2024) . Inspired by these findings, we pose the intriguing question: C an LLMs be taught to consider what kind of responses are genuinely needed by users, and can this reflective process lead to refined and more personalized responses? This work seeks to bridge the gap between generic empathetic responses and truly user-centered personalized emotional support by incorporating self-reflection and self-refinement into automated systems. Effective ES systems require an iterative approach that continuously reflects ongoing dialogue to refresh user understanding and refine responses, ultimately delivering targeted empathy and tailored solutions. The empirical evidence presented in Figure 1 and Table 3 demonstrates that instructing LLMs to summarize user situations, infer emotions and causes, and choose appropriate support strategies leads to a significant improvement in response quality. To this end, we introduce a self-evolution framework for user preference alignment. As depicted in Figure 2 , our self-evolution framework comprises two steps: (1) Emotional Support Experience Acquisition : we first fine-tune LLMs on limited ESC data, enabling them to provide essential emotional support. (2) Self-Improvement for Personalized Emotional Support : Subsequently, we leverage LLMs’ inherent self-reflection and self-refinement capabilities to generate responses that consider the implicit user preference, including profile , situation , and emotions . The pre- and post-refined responses are considered the preference data. Through direct preference optimization, the model generates responses that reflect an understanding of the user’s implicit preferences during interactions, thereby eliminating the need for explicit reflection and refinement steps. Experimental results and extensive human evaluations indicate that our generated responses are more diverse and better aligned with user input. These improved responses effectively reduce ineffective empathy and preference misalignment, facilitating more productive multi-turn interactions. Our main contributions can be summarized as follows: • We reveal the limitations of the current Emotional Support Chatbot, which is notably deficient in understanding users’ implicit preferences, resulting in repetitive and superficial expressions of empathy. • We present a simple yet effective self-evolution framework for personalized emotional support without explicit reflection and refinement steps. • Experimental results and comprehensive human evaluations demonstrate that our method effectively minimizes unhelpful responses and discrepancies in personalized preferences. 2 Method Figure 2: The overview of our self-evolution framework, which enhances personalized emotional support capabilities through a two-stage learning phase: (1) Emotional Support Experience Acquisition: We fine-tune LLMs on minimal human-annotated ESC data, equipping them with basic emotional support capability. (2) Self-Improvement for Personalized Emotional Support: We utilize the LLMs’ self-reflection abilities to tailor responses to the user’s personality, situation, and emotions. The pre- and post-refined responses are natural synthetic preference data. The process involves iterative preference optimization for generating responses that align with the user’s implicit preferences, eliminating the need for explicit reflection steps. Inspired by recent insights highlighting that LLMs can self-improve through language feedback Lu et al. (2024b) ; Madaan et al. (2023) ; Ye et al. (2023) ; Yasunaga et al. (2024) , we present a self-evolution framework designed to enable LLMs to provide personalized emotional support. This framework operates in two phases: Emotional Support Experience Acquisition (Section 2.1 ) and Self-Improvement for Personalized Emotional Support (Section 2.2 ). 2.1 Emotional Support Experience Acquisition 2.1.1 Task Definition Emotional Support (ES) involves understanding the user’s situation and choosing appropriate supportive strategies to alleviate their distress. Formally, we can denote the ES model as \mathcal{M} , and represent the current dialogue context as \mathcal{C}_{n}=(q_{1},r_{1},...,q_{i-1},r_{i-1},...q_{n}) . In this representation, q_{i} and r_{i} correspond to the i -th utterance from the user and the model, respectively. Given the task and strategy description prompt \mathcal{P}_{task} , the goal of the ES model is to generate an emotional supportive response r_{n} , which can be represented as: r_{n}=\mathcal{M}(\mathcal{P}_{task}\ ||\ \mathcal{C}_{n}) (1) 2.1.2 Task Learning We equip the model with emotional support capability by fine-tuning the backbone on the manually annotated ESConv dataset Liu et al. (2021) . To preserve the general abilities, we employ Low-Rank Adaptation Hu et al. (2021) , fine-tuning only the LoRA adapter parameters. We further incorporate a replay mechanism by incorporating some instruction-following data Wang et al. (2024c)