PREF: Reference‑Free Evaluation of Personalised Text Generation in LLMs
Xiao Fu, Hossein A. Rahmani, Bin Wu, Jerome Ramos, Emine Yilmaz, Aldo Lipani · 2025
Read PREF: Reference‑Free Evaluation of Personalised Text Generation in LLMs by Xiao Fu, Hossein A. Rahmani, Bin Wu, Jerome Ramos, Emine Yilmaz, Aldo Lipani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10028.txt.
Introduction
Xiao Fu and co-authors grade personalized writing without a gold answer to copy. Personalized generation serves one user's taste, so standard scoring misses the point. Their 2025 framework, PREF, runs in three stages. First a large model drafts a query-specific guideline covering universal traits like factuality and coherence. Then the target user's profile and stated or inferred tastes re-rank and extend those criteria into a personal rubric.
Finally a judge model scores candidates against that rubric. Splitting coverage from preference keeps the process transparent and reusable. On the PrefEval benchmark, including hidden-preference tasks, PREF beats strong baselines in accuracy and human agreement. Small models using PREF approach big models' personal quality. The team pitches it as infrastructure for user-centric systems.
Catalog record
| CREATOR | Xiao Fu, Hossein A. Rahmani, Bin Wu, Jerome Ramos, Emine Yilmaz, Aldo Lipani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-10028
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PREF: Reference‑Free Evaluation of Personalised Text Generation in LLMs Xiao Fu, Hossein A. Rahmani, Bin Wu, Jerome Ramos, Emine Yilmaz, Aldo Lipani Abstract Personalised text generation is essential for user-centric information systems, yet most evaluation methods overlook the individuality of users. We introduce PREF , a P ersonalised R eference-free E valuation F ramework that jointly measures general output quality and user-specific alignment without requiring gold personalised references. PREF operates in a three-step pipeline: (1) a coverage stage uses a large language model (LLM) to generate a comprehensive, query-specific guideline covering universal criteria such as factuality, coherence, and completeness; (2) a preference stage re-ranks and selectively augments these factors using the target user’s profile, stated or inferred preferences, and context, producing a personalised evaluation rubric; and (3) a scoring stage applies an LLM judge to rate candidate answers against this rubric, ensuring baseline adequacy while capturing subjective priorities. This separation of coverage from preference improves robustness, transparency, and reusability, and allows smaller models to approximate the personalised quality of larger ones. Experiments on the PrefEval benchmark, including implicit preference-following tasks, show that PREF achieves higher accuracy, better calibration, and closer alignment with human judgments than strong baselines. By enabling scalable, interpretable, and user-aligned evaluation, PREF lays the groundwork for more reliable assessment and development of personalised language generation systems. Introduction Large language models (LLMs) such as GPT-3 ( Brown et al. 2020 ) and ChatGPT ( Achiam et al. 2023 ) have propelled open-ended text generation to new heights, enabling high-quality dialogue, code synthesis, and data-to-text narration at scale. Despite these successes, evaluating the outputs of such models remains an open problem. Traditional reference-based metrics, including BLEU and ROUGE ( Papineni et al. 2002 ; Lin 2004 ) , which count n -gram overlap with one or more gold references, correlate weakly with human judgements on tasks that admit many equally valid answers (e.g., creative writing, recommendation, and advice). Embedding-based alternatives such as BERTScore ( Zhang et al. 2020 ) mitigate surface mismatch by measuring semantic similarity in a latent space, yet they still require high-quality reference texts and ultimately reflect generic rather than user-specific desiderata. A parallel line of research bypasses references altogether by treating a strong LLM as an automatic judge. Benchmarks such as MT-Bench ( Zheng et al. 2023 ) and AlpacaEval ( Li et al. 2023 ) ask GPT-4 to rate or rank candidate answers and report impressive agreement with crowd workers. Although this LLM-as-a-judge paradigm scales cheaply and reproducibly, its rubric is fundamentally universal—“a good answer is a good answer for everyone”—and therefore blind to individual preferences, constraints, or prior interactions that might radically alter a user’s perception of quality. Modern applications increasingly expose LLMs directly to end users who expect outputs tailored to their tastes, goals, and contexts. Recent work has shown that LLMs can indeed adapt to user profiles, feedback signals, and stylistic norms ( Zhang et al. 2024 ) . However, evaluating such personalised generation is especially tricky because quality is now user-dependent, i.e., an answer that delights one person might frustrate another . Generic references or model-centric rubrics cannot capture this subjective dimension. Human evaluation remains the gold standard but is expensive, time-consuming, and prone to annotator variance and bias—even when detailed guidelines are provided ( Van Der Lee et al. 2019 ) . Collecting user-aligned ratings for every system update or user cohort is therefore impractical, hindering rapid iteration on personalised experiences. To fill this evaluation gap, we propose PREF , a P ersonalised, R eference-free E valuation F ramework that scores generated text without gold personalised answers. PREF evaluates in two stages: 1. General-quality stage. A general guideline enumerates the salient factors to check—truthfulness, coherence, completeness, etc.— explicitly ignoring user preferences to ensure baseline adequacy. 2. User-alignment stage. A personalised guideline is synthesised from user information (profile attributes, past dialogue, stated preferences) and used to weight or re-rank the general factors, yielding a customised rubric that reflects what this user cares about. The LLM then judges candidate answers against the composite rubric, producing a scalar score without ever consulting ground-truth references. In this way, PREF combines universal quality control with user-centric alignment while remaining scalable and reproducible. Our contribution can be summarised as: • We formulate a reference-free evaluation protocol for personalised generation that jointly considers task adequacy and user alignment. • We instantiate this protocol with automatic rubric construction and LLM-based scoring, eliminating the need for gold references or repeated human ratings. • Through extensive experiments on the PrefEval benchmark, we demonstrate that PREF’s scores track human judgements of personalised quality more faithfully than existing baselines. • We show that integrating PREF during development helps smaller models (e.g., LLaMA-3 8B) close much of the performance gap to larger counterparts, facilitating cost-effective deployment. Taken together, our findings position PREF as a practical and effective foundation for evaluating—and ultimately improving—user-aligned language generation systems. Related Work Foundations of Personalisation The idea of personalisation comes from two related areas of research: Explicit user modelling. Adaptive hypermedia systems demonstrated that maintaining a fine-grained learner or reader model enables real-time adaptation of content, sequencing, and navigation ( Brusilovsky 2001 ) . These systems relied on explicitly encoded attributes—background knowledge, goals, learning style—to decide what to show next. Implicit preference inference. Collaborative filtering techniques such as GroupLens showed that preferences can be inferred from behavioural signals : user–item ratings aggregated at scale allow “people who agreed in the past to agree again” ( Resnick et al. 1994 ) . Modern LLM personalisation strategies still trace back to this dichotomy: explicit profile conditioning versus implicit signal mining (e.g. clicks, dwell time, edits). Personalised Text Generation with LLMs Early neural persona modelling. The Persona-Chat framework appended vector-based speaker embeddings to seq-to-seq dialogue models, boosting persona consistency and opening the door to persona-aware generation ( Li et al. 2016 ) . Attribute and style control. Plug-and-Play Language Models (PPLM) pioneered gradient-based steering of frozen GPT-2 towards user-specified attributes without expensive full fine-tuning ( Dathathri et al. 2020 ) . Subsequent parameter-efficient methods such as prefix-tuning, adapters, and LoRA attach small trainable modules, enabled per-user or per-task specialisation with a few thousand parameters ( Li and Liang 2021 ) . Personalised RLHF and PEFT. Reinforcement Learning from Human Feedback (RLHF) has recently been adapted to heterogeneous user groups: Personalised-RLHF trains multiple reward heads or preference models, each aligned with a user cluster ( Li et al. 2024 ) . In parallel, lightweight PEFT modules distribute personalisation across thousands of users at commodity cost ( Tan et al. 2024 ) . Prompt- and profile-based conditioning. Instead of opaque embeddings, several works propose editable natural-language profiles that the model can quote, justify, or critique, improving scrutability in recommendation and dialogue ( Ramos et al. 2024 ) . Soft-prompt methods such as PeaPOD blend collaborative and individual signals into one prompt, achieving strong personalisation with no model updates ( Ramos, Wu, and Lipani 2024 ) . Evaluating Personalisation in LLMs Limitations of reference metrics. Reference-based scores (BLEU, ROUGE) capture lexical overlap, and even semantic metrics like BERTScore remain insensitive to who the answer is for. Consequently, they correlate poorly with user-perceived quality in personalised settings. LLM-as-judge approaches. Automatic judges such as AuPEL