Prefer : Personalized Review Summarization with Online Preference Learning
Millend Roy, Agostino Capponi, Vineet Goyal · 2026
Read Prefer : Personalized Review Summarization with Online Preference Learning by Millend Roy, Agostino Capponi, Vineet Goyal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05911.txt.
Introduction
Millend Roy, Agostino Capponi, and Vineet Goyal tackle a common shopping problem: too many product reviews to read. Standard review summaries are generic and fixed, yet shoppers care about different traits, and their tastes can shift as they browse. A summary that ignores those latent preferences serves most users poorly.
Their system, PREFER, learns each user's preferences online from feedback on the summaries it produces. Over time it refines its picture of what a given shopper values and reshapes the summary to match. A case study on the Amazon Reviews 2023 data set used controlled simulations. The results show better alignment with target user interests while summary quality held steady.
Catalog record
| CREATOR | Millend Roy, Agostino Capponi, Vineet Goyal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 121 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.05911
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Prefer : Personalized Review Summarization with Online Preference Learning Millend Roy, Agostino Capponi, Vineet Goyal Abstract Product reviews significantly influence purchasing decisions on e-commerce platforms. However, the sheer volume of reviews can overwhelm users, obscuring the information most relevant to their specific needs. Current e-commerce summarization systems typically produce generic, static summaries that fail to account for the fact that (i) different users care about different product characteristics, and (ii) these preferences may evolve with interactions. To address the challenge of unknown latent preferences, we propose an online learning framework that generates personalized summaries for each user. Our system iteratively refines its understanding of user preferences by incorporating feedback directly from the generated summaries over time. We provide a case study using the Amazon Reviews’23 dataset, showing in controlled simulations that online preference learning improves alignment with target user interests while maintaining summary quality. 1 Introduction Online product reviews have a significant impact on the purchase decisions of customers on e-commerce platforms [ 12 ] . Before buying, users often read reviews to learn about other customers’ experiences with the product, including its quality, reliability, ease of use, and overall value for money [ 27 ] . A larger number of reviews can increase confidence by providing more evidence about the product, but it can also make the decision-making process slower and harder [ 17 ] , especially when users are looking for specific information [ 39 ] . Review summarization has emerged as one way to address this problem by condensing many reviews into a shorter and easier-to-read summary [ 21 , 30 ] . This idea has become especially relevant in practice with the introduction of AI-generated review summaries on e-commerce platforms such as Amazon [ 36 ] . Such summaries aim to help users understand products more quickly by highlighting common themes from customer feedback [ 32 ] . On the other hand, recent work [ 14 ] has pointed out that generated summaries may miss important details, underscoring the need to study not only whether summaries are concise but also whether they are useful for decision-making. Most existing review summarization systems are designed to produce a single generic summary for each product [ 21 , 30 ] . This is a strong assumption. Different users often care about different product attributes; for example, one user may prioritize durability, another may prioritize value-for-money, and another may prioritize ease of use. Therefore, the same product may be evaluated differently by different users. This suggests that a single summary for all users may not be appropriate in helping with purchase decisions. This becomes even more challenging when preferences are not stationary [ 20 ] . A user’s interests may vary across products and may also evolve over time, requiring the system to adapt to preference drift rather than assume a fixed user-profile [ 25 ] . To address this dynamics, we formulate personalized review summarization as an online feedback-adaptive decision problem : the system must generate useful summaries while simultaneously learning which kinds of reviews are most useful for a given user from lightweight scalar feedback. In this work, we present Prefer 1 1 1 An anonymized version of the code repo is at: https://anonymous.4open.science/r/prefer-037D/README.md . , a modular framework for operationalizing this feedback-adaptive view. Prefer needs to balance the fundamental tradeoff between exploration and exploitation of learning user preferences from their feedback and generating a good personalized summary for the user. We assume that the users provide a (scalar) feedback for each summary. Prefer uses these feedback to refine its estimate of that user’s preferences and improve future summaries. We evaluate Prefer on Amazon Reviews [ 16 ] against generic summarization and static personalized baselines. Our results show that, in controlled simulations, incorporating online preference learning improves alignment with target user interests while maintaining coherence and overall summary quality. More broadly, our findings suggest that review summarization should move beyond producing a single generic summary for all users and instead adapt to the information each user finds most useful. 2 Related Work Research on review summarization goes back to 18 , who extracted product features using part-of-speech patterns and grouped positive/negative opinion sentences around them. Subsequent works improved aspect extraction and sentiment modeling: using linguistic patterns and lexicons [ 45 , 8 , e.g.,] , aspect-expression clustering [ 43 ] , and topic-modeling with Bayesian approaches [ 31 ] . Neural methods further reduced reliance on hand-crafted rules: 15 learn latent aspects directly from review text, while 3 infer fine-grained segment-level sentiment from document labels. Related tagging and folksonomy methods, such as 34 and 37 , use explicit user-generated tags for recommendation. By contrast, Prefer does not rely on such explicit tags/keyword labels: it discovers latent sentence-level aspects from review text. Prior work also studies representative content selection once such sentiment representations are available: 28 select review subsets that cover a collection of micro-reviews, while 22 formulate review selection as a combinatorial optimization problem. Most of these systems produce product-level summaries, including recent AI-generated review summaries on e-commerce platforms [ 36 ] , rather than personalizing content to an individual user’s evolving preferences. Other studies have have addressed the question of whether review information should be personalized when presented to users. 4 show that personalized review presentation can improve decision-making efficiency, but does not learn an adaptive summarizer from feedback. Related work on information filtering models user interests through latent content representations; for example, 38 combine collaborative filtering with probabilistic topic modeling for article recommendation. More generally, social-learning models with heterogeneous preferences by 23 show that preference diversity can either help or hurt learning depending on network density, highlighting that the same information need not be equally useful to users with different preferences. A separate line of work treats “review" and “recommendation" systems as interactive feedback processes rather than static prediction tasks. In online reviews, 1 show that observed review signals are endogenous: users decide whether to purchase and review a product based on previously revealed information, creating selection effects in review-based learning. More closely related works, such as the linear-bandit formulation of 10 , explicitly model the exploration-exploitation trade-off between learning user preferences and making useful recommendations. The work of 19 , 25 , 41 on recommender systems study evolving preferences, concept drift, and adaptation to user feedback, respectively. Text-based interactive recommendation further incorporates natural-language feedback; for example, 44 use constraint-augmented reinforcement learning (RL) to avoid recommendations that violate users’ historical preferences. 2 2 2 For surveys of RL-based and interactive recommender systems, see 2 and 11 . Recent work on LLM-agent personalization, such as AdaPA-Agent by 29 , similarly models dynamic preference strengths from interaction history and uses them to steer generation. However, it operates at the level of preference-weighted next-token distributions rather than structured review-evidence selection. These works share our feedback-adaptive view, but typically choose among atomic actions such as items, arms, or resources. In contrast, Prefer acts over a structured summarization pipeline: (i) it selects review evidence and (ii) rewrites them into a personalized summary. Thus, adaptation in our setting must operate over both preference learning and compositional summary generation. In summary, our work lies at the intersection of these three strands. We inherit