Cupid : Evaluating Personalized and Contextualized Alignment of LLMs from Interactions

Tae Soo Kim ♡ Yoonjoo Lee ♡ Yoonah Park ♠ Jiho Kim ♢ · 2025

Read Cupid : Evaluating Personalized and Contextualized Alignment of LLMs from Interactions by Tae Soo Kim ♡ Yoonjoo Lee ♡ Yoonah Park ♠ Jiho Kim ♢ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01674.txt.

Introduction

Tae Soo Kim and five co-authors test whether chatbots can learn what each user likes from past chats. Their paper appeared in two thousand twenty-five. Personalization often assumes fixed tastes across all tasks. Real preferences shift with context. Users reveal those shifts through feedback over many turns. A good model should infer and reuse them later.

Their CUPID benchmark holds hundreds of curated interaction histories. Each session ties requests to a context and expressed preferences. A new request then tests whether the model applies the right past lesson. Ten open and closed models faced the benchmark. Results show how far contextual alignment still has to go. Personal assistants gain a sharper yardstick.

Catalog record

CREATORTae Soo Kim ♡ Yoonjoo Lee ♡ Yoonah Park ♠ Jiho Kim ♢
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01674

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cupid : Evaluating Personalized and Contextualized Alignment of LLMs from Interactions Tae Soo Kim ♡ Yoonjoo Lee ♡ Yoonah Park ♠ Jiho Kim ♢ Abstract Personalization of Large Language Models (LLMs) often assumes users hold static preferences that reflect globally in all tasks. In reality, humans hold dynamic preferences that change depending on the context. As users interact with an LLM in various contexts, they naturally reveal their contextual preferences, which a model must infer and apply in future contexts to ensure alignment. To assess this, we introduce Cupid , a benchmark of 756 human-curated interaction session histories between users and LLM-based chat assistants. In each interaction session, the user provides a request in a specific context and expresses their preference through multi-turn feedback. Given a new user request and prior interaction sessions, our benchmark assesses whether LLMs can infer the preference relevant to this request and generate a response that satisfies this preference. With Cupid , we evaluated 10 open and proprietary LLMs, revealing that state-of-the-art LLMs struggle to infer preferences from multi-turn interactions and fail to discern what previous context is relevant to a new request—under 50% precision and 65% recall. Our work highlights the need to advance LLM capabilities for more contextually personalized interactions and proposes Cupid as a resource to drive these improvements. https://github.com/kixlab/CUPID Figure 1 : Example instance in Cupid illustrates a user that holds distinct preferences in different contexts due to personal experiences, where the preferences are only revealed to the LLM through the user’s feedback in prior interactions. 1 Introduction Large Language Models (LLMs) have shown remarkable capabilities across various tasks ( Nakano et al. 2021 ; Achiam et al. 2023 ) , benefiting users through diverse applications and conversational assistants ( OpenAI 2022 ; Anthropic 2023 ; Microsoft 2023 ) . Aligning these models with human values and preferences is crucial as they are increasingly integrated into user experiences ( Ji et al. 2023 ) . Initial efforts focused on aligning LLMs on broad, general values (e.g., helpfulness, harmlessness, honest) ( Bai et al. 2022a ; Bai et al. 2022b ) or the aggregated preferences of diverse users ( Ouyang et al. 2022 ; Köpf et al. 2024 ) . Recognizing that these approaches overlook users’ diverse expectations, more recent work focuses on personalized alignment , where LLMs are aligned with the users’ individual preferences ( Wu et al. 2024b ; Jang et al. 2023 ; Lee et al. 2024 ) . Despite this progress, existing work largely assumes that users have static preferences (i.e., preferences are global and do not vary over time) ( Wu et al. 2024b ; Lee et al. 2024 ) or focus on stable characteristics (e.g., sociodemographics) to characterize preferences ( Kirk et al. 2024 ) . In reality, human preferences are context-dependent ( Warren et al. 2011 ; Baker et al. 2009 ; Malaviya et al. 2024 ) : an individual’s intents and expectations shift depending on their situation. For example, in Figure 1 , a researcher consults an LLM to refine their paper’s writing but their preference varies by collaborator: focus on classical methods with Dr. Chen due to prior disagreements while embracing computational methods with Dr. Park who advocated for them. These contextual variability means that models must understand what values and preferences a user holds in distinct contexts. Interactions between a user and an LLM may indirectly reveal these shifting preferences as users provide feedback to the model in diverse situations ( Wu et al. 2024b ; Kim et al. 2024c ) . However, it is unclear whether current LLMs possess the capability to identify a user’s preference from their feedback, infer the relevant context, and proactively apply this knowledge in future interactions. In this work, we present Cupid 1 1 1 https://huggingface.co/datasets/kixlab/CUPID ( C ontextual U ser P reference I nference D ataset), a challenging and scalable benchmark that is designed to assess LLMs’ ability to infer users’ preferences tied to diverse contexts from user-LLM interaction histories. Cupid consists of 756 human-curated interaction histories, where each history presents a series of interaction sessions or dialogues between a simulated user and an LLM. Each interaction session presents a task-oriented dialogue where a user asks a request explicitly stating the context, and then gradually reveals their contextual preference through multi-turn feedback to the LLM. Given a new request and a history of previous interaction sessions, the benchmark evaluates whether LLMs can (1) infer the user’s preference related to the context of the new request, and (2) generate a response to the request that will satisfy this preference. To construct Cupid , we designed a pipeline that generates diverse and rich interaction sessions. After generating a persona pool, for each persona, the pipeline generates a list of context factors (i.e., significant people, objects, locations, etc. in the persona’s world that influences their expectations) and contextual preference (i.e., value, principle, or criterion associated with each factor). For each persona, the pipeline then generates a series of interaction sessions that involve these factors and preferences. We combine the concepts of LLM-as-a-Judge ( Zheng et al. 2023 ; Ye et al. 2023 ) and LLM-simulated users ( Wu et al. 2024a ; Wu et al. 2024b ) to create multi-turn dialogues where a simulated user evaluates and provides feedback to LLM’s responses, gradually disclosing its contextual preferences. All the data was verified with human annotators and \sim 9% of the data was manually edited. With Cupid , we assess 10 open and proprietary LLMs. We find that all models struggle to infer users’ preferences based on past interaction sessions, with no model exceeding 50% precision and 65% recall. Models failed at recognizing relevant contexts in prior interactions and extracting preferences from multi-turn conversations. We find a strong correlation (r=0.764) between performance in inferring preferences and generating responses that satisfy them, suggesting that the capability to infer context-dependent preferences underpins personalization capabilities. Finally, we present a finetuned metric, PrefMatcher-7B 2 2 2 https://huggingface.co/kixlab/prefmatcher-7b , to reduce the cost of evaluation on our benchmark and a larger unverified dataset 3 3 3 https://huggingface.co/datasets/kixlab/CUPID-Unverified to support training and further research. 2 Cupid Benchmark Cupid evaluates LLMs’ ability to infer a user’s distinct preference in different contexts from prior interactions between the user and the LLM. 2.1 Definitions Our benchmark is composed of interaction sessions S_{i}=(c_{i},p_{i},D_{i}) . Context Factor, c_{i} Represents an element (e.g., person, location, tool, activity) in a user’s environment or world, which plays a role in and influences that interaction session. We assume that each session’s context is only defined and influenced by a single factor. Contextual Preference, p_{i} Represents a value, principle, criterion, requirement, or constraint that the user holds when the context factor c_{i} is involved in the situation. While prior work focused on more simple and concrete lifestyle preferences (e.g., ”I cannot eat spicy food” ) ( Wu et al. 2024b ; Zhao et al. 2025 ) , we focus on more complex, abstract, and task-oriented preferences (e.g., ”Instructions must break down complex procedures into numbered micro-steps with verification points after each major section” )—more examples in Table 6 . Preferences in our dataset often require multiple rounds of feedback to be satisfied. Dialogue, D_{i}=(u_{i,1},m_{i,1},...,m_{i,L_{i}-1},u_{i,L_{i}}) Represents

Read the complete text