Preference-Aware Rubric Learning for Personalized Evaluation
Yilun Qiu Xiaoyan Zhao Yang Zhang Yuxin Chen Cilin Yan, Jiayin Cai Xiaolong Jiang Yao Hu Yoko Yamakata Tat-Seng Chua · 2026
Read Preference-Aware Rubric Learning for Personalized Evaluation by Yilun Qiu Xiaoyan Zhao Yang Zhang Yuxin Chen Cilin Yan, Jiayin Cai Xiaolong Jiang Yao Hu Yoko Yamakata Tat-Seng Chua (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31545.txt.
Introduction
Qiu and team present PARL, a system that learns preference-aware rubrics for judging personalized text generation. As models shift to user-centred agents, judging personalized alignment has become a key bottleneck. Old methods miss subjective tastes embedded in long interaction histories. PARL induces rubrics straight from raw user histories and self-checks them for consistency with user tastes. Tests on real personalized generation tasks show the rubrics spot user-aligned responses well and carry across users and tasks.
Catalog record
| CREATOR | Yilun Qiu Xiaoyan Zhao Yang Zhang Yuxin Chen Cilin Yan, Jiayin Cai Xiaolong Jiang Yao Hu Yoko Yamakata Tat-Seng Chua |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 122767 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.31545
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Preference-Aware Rubric Learning for Personalized Evaluation Yilun Qiu Xiaoyan Zhao Yang Zhang Yuxin Chen Cilin Yan, Jiayin Cai Xiaolong Jiang Yao Hu Yoko Yamakata Tat-Seng Chua Abstract As Large Language Models (LLMs) evolve from general-purpose assistants to user-centric agents, personalization has become central to aligning model behavior with individual preferences, making the evaluation of personalized alignment a critical bottleneck. Existing evaluation methods—ranging from automatic metrics to LLM-as-a-judge approaches—fail to capture subjective, user-specific preferences embedded in long-term interaction histories. We identify three essential principles for reliable and effective personalized evaluation: Representativeness , User-Consistency , and Discriminativeness . To address these principles, we introduce Personalized Evaluation as Learning , a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment. Under this paradigm, we propose PARL ( P reference- A ware R ubric L earning), a framework that learns to induce preference-aware evaluation rubrics directly from raw user histories and performs a self-validation mechanism to ensure consistency with the user’s preferences. PARL integrates rubric induction with a discriminative reinforcement learning objective that contrasts user-authored responses against competitive personalized model outputs, enabling the learned rubrics to capture precise, user-specific decision boundaries. Experiments on real-world personalized text generation tasks show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users and tasks, while capturing stable stylistic preferences and fine-grained evaluative patterns. To ensure reproducibility, our code is available at https://github.com/SnowCharmQ/PARL . 1 1 footnotetext: Corresponding Authors 1 Introduction Large Language Models (LLMs) ( 1 ; 14 ; 64 ) have significantly advanced general-purpose intelligence and are now widely applied to reasoning, content generation, and decision-making tasks. However, their prevailing “one-size-fits-all” paradigm limits their ability to accommodate diverse and user-specific preferences. This limitation has motivated LLM Personalization ( 7 ; 69 ) , which aims to adapt model behavior to individual users across personalized tasks such as writing assistance and conversational interaction. Currently, LLM personalization has become an important and rapidly growing research direction, attracting increasing attention from both academia ( 72 ; 42 ; 53 ) and industry ( 21 ; 70 ; 10 ) . Existing research on LLM personalization has primarily focused on aligning model behavior with individual user preferences through improvements to the generation process , leading to a range of approaches such as user-specific context injection ( 39 ; 45 ; 75 ) and fine-tuning on historical user contexts ( 51 ; 29 ; 72 ) . In contrast, the evaluation of LLM personalization remains largely underexplored, despite being a critical bottleneck for personalization method design and verification. This gap stems from a fundamental mismatch in evaluation assumptions: conventional evaluation methods rely on universal and objective ground truths, whereas personalized generation is inherently governed by subjective, user-specific criteria that are usually unobservable and only implicitly reflected in users’ historical behaviors. Existing evaluation methods for LLM personalization can be broadly grouped into three categories: (1) Automatic metrics , like ROUGE ( 28 ) and BERTScore ( 63 ) . These metrics generally assume a single objective reference and prioritize surface-level lexical overlap to the reference. As a result, they are inherently inadequate for personalized evaluation, failing to capture subjective, user-specific preferences and alternative but equally valid personalized responses. (2) Human evaluation , which is often regarded as the gold standard in general scenarios. Yet, it suffers from an inferential gap in personalization settings: external annotators lack direct access to a user’s true preferences, making it difficult to faithfully assess alignment within the user. (3) LLM-as-a-judge approaches ( 31 ; 11 ) , which partially alleviate these issues via LLMs’ semantic capabilities. However, they rely on static judges and handcrafted prompts, restricting adaptation to individual users and task-specific preferences, and they typically yield coarse, holistic scores without fine-grained grounding, limiting the interpretability and verifiability of their judgments. Given the limitations of existing approaches, we propose a shift toward a Personalized Evaluation as Learning paradigm for LLM personalization. Rather than relying on static and generic evaluation rules, this paradigm aims to adaptively and systematically learn user-specific evaluation criteria aligned with individual preferences. To realize this paradigm, we build upon the LLM-as-judge framework but depart from existing practices by instantiating it through rubric-based evaluation ( 23 ; 16 ; 35 ) , which decomposes holistic judgments into explicit, multi-dimensional criteria, referred to as rubrics. Such a formulation provides a natural foundation for learning-based personalized evaluation: rubrics can be adaptively induced from individual user historical contexts and applied across diverse scenarios. The core challenge, thereafter, is to learn how to drive rubrics that faithfully capture individual user-specific decision boundaries. To answer this, we identify and formalize three essential principles that any rigorous and reliable rubric-based framework for personalized evaluation must satisfy: (i) Representativeness : Rubrics should faithfully distill the diverse and latent preferences embedded in a user’s historical contexts, capturing the key dimensions that define their evaluative standards. (ii) User-Consistency : Rubrics should reflect stable, user-specific preferences that persist across different historical contexts, rather than artifacts arising from isolated tasks or individual samples. (iii) Discriminativeness : Rubrics should enable fine-grained distinction between authentic user-authored responses and generic yet high-quality model outputs, isolating true preference awareness from universal quality signals. Guided by the three principles, we propose PARL ( P reference- A ware R ubric L earning), a framework that learns to induce structured, multi-dimensional, preference-aware rubrics from raw user historical behaviors to support personalized LLM evaluation. PARL distills semantically rich, user-specific preferences from users’ historical contexts into explicit rubric dimensions, ensuring representativeness . The induced rubrics are then validated across diverse historical contexts to enforce user-consistency . To further enhance discriminativeness , PARL incorporates a reinforcement learning stage that contrasts user-authored ground-truth responses with competitive personalized model outputs, encouraging learned rubrics to capture precise, user-specific decision boundaries. We conduct comprehensive experiments on real-world personalized text generation tasks to systematically assess the learned rubrics. Empirical results demonstrate that PARL consistently induces rubrics that are representative, user-consistent, and discriminative. Further analysis reveals that the learned rubrics capture stable stylistic preferences and fine-grained criteria, reflecting distinct evaluative signatures. Moreover, the trained rubric generator generalizes well across contexts, inducing rubrics that are semantically rich, diverse, and tailored to individual users. Remarkably, these rubrics constitute a novel, reliable, and high-fidelity set of personalized evaluation metrics. Our main contributions are summarized as follows: • We introduce Personalized Evaluation as Learning paradigm, which formalizes personalized LLM evaluation as a learnable process grounded in three essential criteria: Representativeness , User-Consistency , and Discriminativeness . • We propose PARL , a learning-based framework that induces preference-aware evaluation rubrics from user histories, optimized with a discriminative margin objective via RL. • We conduct extensive experiments and analysis across real-world personalized generation tasks, demonstrating that PARL produces high-fidelity, reusable rubrics and offering new insights into the structure of user-specific evaluative preferences. 2 Related Work In this section, we review related research across two dimensions: the development of LLM personalization and the evaluation of personalized text generation. 2.1 LLM Personalization The widespread use of LLMs across diverse domains ( 17 ; 9 ; 65 ; 68 ) has catalyzed a paradigm shift from “one-size-fits-all” alignment toward personalized outputs tailored to individual preferences. Current research in LLM personalization primarily