Improving RAG for Personalization with Author Features and Contrastive Examples

Mert Yazan, Suzan Verberne, Frederik Situmeang · 2025

Read Improving RAG for Personalization with Author Features and Contrastive Examples by Mert Yazan, Suzan Verberne, Frederik Situmeang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08745.txt.

Introduction

A writing assistant that fetches an author's past work still often misses what makes that author sound like themselves. Mert Yazan, Suzan Verberne, and Frederik Situmeang add two enrichments in a 2025 paper. First, they hand the model a short profile of the author's habits: typical mood, pet words, favorite names, and sentence patterns. Second, they fetch contrastive examples, texts by other authors, so the model can see what sets this one apart.

Experiments show the profile alone improves personalization, and adding the contrastive examples lifts results further, about 15 percent over plain retrieval. The authors release their code and call the contrast trick a new direction for the field.

Catalog record

CREATORMert Yazan, Suzan Verberne, Frederik Situmeang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-08745

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving RAG for Personalization with Author Features and Contrastive Examples Mert Yazan, Suzan Verberne, Frederik Situmeang Abstract Personalization with retrieval-augmented generation (RAG) often fails to capture fine-grained features of authors, making it hard to identify their unique traits. To enrich the RAG context, we propose providing Large Language Models (LLMs) with author-specific features, such as average sentiment polarity and frequently used words, in addition to past samples from the author’s profile. We introduce a new feature called Contrastive Examples : documents from other authors are retrieved to help LLM identify what makes an author’s style unique in comparison to others. Our experiments show that adding a couple of sentences about the named entities, dependency patterns, and words a person uses frequently significantly improves personalized text generation. Combining features with contrastive examples boosts the performance further, achieving a relative 15% improvement over baseline RAG while outperforming the benchmarks. Our results show the value of fine-grained features for better personalization, while opening a new research dimension for including contrastive examples as a complement with RAG. We release our code publicly. 1 1 1 https://github.com/myazann/AP-Bots/tree/main 1 Introduction Personalized text generation has many application areas, from recommendation [ 25 ] to personalized content creation [ 1 , 15 ] . Its implementation with LLMs falls into two groups: via fine-tuning [ 18 , 19 , 27 ] , or via Retrieval-Augmented Generation (RAG) [ 13 , 15 , 24 ] . In fine-tuning-based approaches, the language model is individually adapted for each user [ 19 ] , which remains computationally costly even with parameter-efficient fine-tuning (PEFT) [ 7 ] . RAG-based personalization works by retrieving samples from the author profile, then instructing the LLM to generate personalized content given the samples [ 15 ] . RAG eliminates the need to fine-tune for each author and outperforms PEFT-based methods when the amount of user data is scarce [ 16 , 26 ] . The key challenge with RAG is to capture fine-grained author features: retrieving separate, standalone samples from the author’s history is not enough to recognize global patterns [ 18 ] . When presented with previous documents of the author, LLMs tend to memorize certain words and n-grams that are frequently used, instead of understanding the author’s style [ 26 ] . To address this problem, we derive features from the author’s profile. Features enhance the traditional RAG approach by providing global context about the author, nudging the LLM to do more than repeating words and n-grams. We experiment with different types of features, such as the average sentiment polarity, or the words and named entities the person uses the most. In addition, we introduce a new feature called Contrastive Examples (CE) which retrieves samples from other authors to better highlight the differences between individual styles. Figure 1 gives an overview of our framework. Our results show that adding only a handful of author features alongside samples from the author’s profile significantly improves personalization. Furthermore, retrieving contrastive examples helps LLM better understand an author’s style. A combination of CE and a couple of author features improve the RAG performance by up to 15% . Additionally, our approach does not add any computational overhead and can easily be implemented and deployed. We contribute to the literature by showing that: (a) LLMs can personalize better when presented with author features, (b) contrastive examples helps LLMs distinguish authors, and (c) retrieving contrastive examples can be a tool to differentiate between relevant and irrelevant contexts. 2 Background and Related Work RAG for personalization is initially implemented by retrieving similar documents from the user’s profile, and instructing the LLM to personalize given the documents [ 15 ] . Later work focuses on different parts of the RAG pipeline, such as optimizing the retriever [ 13 , 14 ] or summarizing and synthesizing the documents to improve the reading comprehension of the LLM [ 10 ] . Alhafni et al. [ 1 ] experiment with fine-grained features, such as lexical and syntactic attributes. The crucial difference with our method is that they train smaller models ( \leq 1B) by prepending the attribute vectors to the inputs, instead of RAG. In addition, they do not include features such as the most commonly used words of the author. Even though personalization by its nature is user-specific, finetuning with multiple users before personalizing to one can aid in learning shared features [ 18 , 27 ] . Tan et al. [ 18 ] introduce Personalized Pieces, where clusters of users share PEFT parameters, and HYDRA framework proposes an approach to obtain the shared general knowledge between all authors [ 27 ] . What makes our approach unique is we focus on the differences between authors by retrieving contrastive examples, whereas previous work focused on capturing the similarities. The novelty of our idea is not limited to personalized text generation, but can be seen as a new paradigm for retrieval in RAG. Traditionally, RAG is based on retrieving the most relevant documents, and how contrastive examples would work is an open question. A contrastive chain-of-thought approach with both valid and invalid demonstrations performs better than regular chain-of-thought prompts [ 3 ] and shows that LLMs are capable of making use of contrastive examples. In the context of RAG, retrieving random documents can help with question-answering, while high-scored documents that are not directly relevant hurts the performance [ 4 ] . Since contrastive examples do not fit into either of these categories, their impact on RAG remains unclear. Contrastive learning has previously been studied in Information Retrieval (IR) for training retrievers to differentiate between relevant and irrelevant documents [ 9 ] . In contrast, our method incorporates both types of documents directly into the LLM’s prompt, without modifying the retriever. 3 Methodology Figure 1 : An overview of our approach. The language model receives three inputs: (1) most similar samples from the author profile, given the author’s input, (2) features derived from the author profile, (3) contrastive examples gathered from other users. Input (1) denotes the baseline RAG approach, while (2) and (3) are our additions to enrich RAG context. Data We used 3 datasets offered by the LaMP (When Large Language Models Meet Personalization) benchmark [ 15 ] : LaMP-4, LaMP-5, and LaMP-7. Since we are focusing on text generation, we excluded the datasets with classification tasks, as well as LaMP-6 because it wasn’t accessible. We chose the user-based splits for evaluation. The task in LaMP-4 is to generate a title in the style of the news editor based on a news article given as the input. The author profile consists of previous article-title pairs. The task in LaMP-5 is to generate a title, given the abstract as the input, and the author profile consists of the previous abstract-title pairs of the scholar. Finally, in LaMP-7, the input is a tweet and the task is to paraphrase it given the author’s past tweets. Models Given its success in the LaMP benchmark [ 15 , 26 ] , we used a pre-trained Contriever [ 8 ] as the retriever. The average length of the samples differs between datasets, where LaMP-5U has longer documents [ 15 ] . Therefore, we retrieved 50 documents from author profiles

Read the complete text