PreferThinker: Reasoning-based Personalized Image Preference Assessment

Shengqi Xu ♠ , Xinpeng Zhou ♣ , Yabo Zhang ♣ , Ming Liu ♣ , Tao Liang ⧫ , Tianyu Zhang ⧫ Yalong Bai ⧫ , Zuxuan Wu ♠ , Wangmeng Zuo ♣, 🖂 ♠ Fudan University ♣ Harbin Institute of Technology ⧫ iN2X Project Page · 2025

Read PreferThinker: Reasoning-based Personalized Image Preference Assessment by Shengqi Xu ♠ , Xinpeng Zhou ♣ , Yabo Zhang ♣ , Ming Liu ♣ , Tao Liang ⧫ , Tianyu Zhang ⧫ Yalong Bai ⧫ , Zuxuan Wu ♠ , Wangmeng Zuo ♣, 🖂 ♠ Fudan University ♣ Harbin Institute of Technology ⧫ iN2X Project Page (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00609.txt.

Introduction

Shengqi Xu and eight co-authors tackle personalized image preference: judging what one specific user will like, given only a few reference images. General preference models train on large data for well-defined tasks, but personal taste is scarce in data, diverse, and complex. Their bridge is a common preference profile, a shared description format that lets large-scale user data train a profile predictor usable for individuals.

The framework follows a predict-then-assess pattern. First it predicts a user's profile from the reference images, then it scores candidate images across several dimensions with interpretable reasoning. A large chain-of-thought dataset with diverse profiles supervises the reasoning, and two-stage training, supervised fine-tuning followed by reinforcement learning, sharpens it further. A similarity-aware reward improves profile prediction, and experiments show the method beating rivals.

Catalog record

CREATORShengqi Xu ♠ , Xinpeng Zhou ♣ , Yabo Zhang ♣ , Ming Liu ♣ , Tao Liang ⧫ , Tianyu Zhang ⧫ Yalong Bai ⧫ , Zuxuan Wu ♠ , Wangmeng Zuo ♣, 🖂 ♠ Fudan University ♣ Harbin Institute of Technology ⧫ iN2X Project Page
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00609

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PreferThinker: Reasoning-based Personalized Image Preference Assessment Shengqi Xu ♠ , Xinpeng Zhou ♣ , Yabo Zhang ♣ , Ming Liu ♣ , Tao Liang ⧫ , Tianyu Zhang ⧫ Yalong Bai ⧫ , Zuxuan Wu ♠ , Wangmeng Zuo ♣, 🖂 ♠ Fudan University ♣ Harbin Institute of Technology ⧫ iN2X Project Page Abstract Personalized image preference assessment aims to evaluate an individual user’s image preferences by relying only on a small set of reference images as prior information. Existing methods mainly focus on general preference assessment, training models with large-scale data to tackle well-defined tasks such as text-image alignment. However, these approaches struggle to handle personalized preference because user-specific data are scarce and not easily scalable, and individual tastes are often diverse and complex. To overcome these challenges, we introduce a common preference profile that serves as a bridge across users, allowing large-scale user data to be leveraged for training profile prediction and capturing complex personalized preferences. Building on this idea, we propose a reasoning-based personalized image preference assessment framework that follows a predict-then-assess paradigm: it first predicts a user’s preference profile from reference images, and then provides interpretable, multi-dimensional scores and assessments of candidate images based on the predicted profile. To support this, we first construct a large-scale Chain-of-Thought (CoT)-style personalized assessment dataset annotated with diverse user preference profiles and high-quality CoT-style reasoning, enabling explicit supervision of structured reasoning. Next, we adopt a two-stage training strategy: a cold-start supervised fine-tuning phase to empower the model with structured reasoning capabilities, followed by reinforcement learning to incentivize the model to explore more reasonable assessment paths and enhance generalization. Furthermore, we propose a similarity-aware prediction reward to encourage better prediction of the user’s preference profile, which facilitates more reasonable assessments exploration. Extensive experiments demonstrate the superiority of the proposed method. Our code and dataset will be publicly released. 1 Introduction Image preference assessment is essential for both evaluating and aligning generative models with human preferences, playing a key role in applications like content recommendation. However, most existing works ( 16 ; 43 ) focus on assessing general preferences, such as text-image alignment and aesthetics, relying on large-scale general preference data. The practical but challenging task of personalized preference assessment remains largely underexplored, where only a limited amount of personalized data is available for each user as prior information to assist in the assessment. As shown in Fig. 1 (a), personalized preference assessment is particularly challenging for two main reasons. First, unlike easily scalable general preference data, where most users share similar assessment criteria, each user’s personalized data is typically limited and distinct from others, making large-scale, user-specific training difficult. Second, personalized preferences are often complex and diverse , covering multiple dimensions such as art style, color, and art medium, which are more challenging than clear general human preferences ( e.g. text-image alignment and aesthetics). Current research on image preference assessment mainly falls into two categories: CLIP-based methods and Multimodal Large Language Model (MLLM)-based methods. CLIP-based methods ( 41 ; 40 ) typically fine-tune the CLIP model using large-scale text-image pairs with similarity to assess general preferences. However, they struggle with personalized preferences since the amount of personalized data is typically limited for large-scale training. Besides, simple similarity fails to capture complex individual preferences and lacks interpretability, as it outputs only a numerical score. MLLM-based methods ( 37 ; 42 ) achieve interpretable preference assessment by fine-tuning the MLLM on large-scale visual question-answering (VQA) pairs. However, the scarcity of personalized images makes it difficult to obtain sufficient VQA pairs for large-scale training, which limits their effectiveness for personalized preference assessment. Overall, most existing methods are designed for general preferences and therefore struggle to handle personalized preference. Recently, an MLLM-based method named ViPer ( 30 ) was proposed for personalized preference assessments. It directly feeds both personalized reference images and candidate images into an MLLM and fine-tunes the model using a supervised score regression strategy. Nevertheless, it only implicitly utilizes reference images to assist in evaluating the candidate images, failing to fully leverage the critical prior information they contain. Moreover, it lacks interpretable reasoning steps that explain how to produce its final score based on the individual reference images. To address the above issues, we propose a reasoning-based personalized preference assessment system with preference profile prediction, named PreferThinker . Our key insight is to bridge various users by introducing a preference profile comprising multiple visual elements ( e.g. color, art style), as shown in Fig 1 (b). Although each user’s personalized preferences are unique, the fundamental elements that form them are common and can serve as a bridge across users . This profile offers three main advantages. First, the element-level profile allows us to characterize personalized preferences, which alleviates the challenge of complex individual tastes. Second, since these visual elements can be shared and learned across users, we can leverage large-scale user samples for training to effectively predict each user’s visual preference profile, thereby mitigating the issue of limited personalized data. Third, it provides a solid foundation for subsequent interpretable and multi-dimensional assessments. Specifically, given a user’s personalized reference images (preferred and non-preferred) and two candidate images, where reference images may reflect multiple aspects of user’s tastes. PreferThinker follows a predict-then-assess CoT-style structure for personalized preference assessment. As shown in Fig. 2 , the process consists of two stages: 1) Profile Prediction: PreferThinker first predicts the user’s visual preference and non-preference profiles comprising multiple visual elements based on the reference images. 2) Multi-dimensional and Interpretable Assessment: Using the predicted profile, PreferThinker provides interpretable scores for the candidate images across multiple dimensions and produces a final result based on the total score. This predict-then-assess framework not only enables accurate assessment, but also achieves interpretable scoring grounded in the user’s reference images. Figure 1: Illustration of challenges and motivation. (a) The general preference data is easily scalable since users share common assessment criteria, while personalized preference data for each user is typically limited and unscalable, as each user’s preferences are distinct. Besides, general preferences are often clear ( e.g. text-image alignment and aesthetics), while personalized preferences are typically complex and diverse. (b) We propose a preference profile comprising multiple common visual elements, based on the observation that although each user’s personalized preferences are unique, the key visual elements that shape them are shared and can therefore serve as a bridge to connect users. To this end, we first construct a large-scale CoT-style personalized image preference assessment dataset, named PreferImg-CoT , which contains 60,000 user samples, annotated with preference profiles and high-quality CoT-style assessments, enabling explicit supervision of structured reasoning. Next, we adopt a two-stage training strategy: a cold-start supervised fine-tuning (SFT) phase to elicit the model structured reasoning, followed by reinforcement fine-tuning (RFT) based on Group Relative Policy Optimization (GRPO) to guide model toward exploring more reasonable assessments and enhancing generalization. Moreover, we find that accurate preference profile prediction facilitates exploring reasonable assessments, which motivated us to propose a similarity-aware prediction reward to better predict user’s preference profile. Overall, our main contributions are summarized as follows: • We propose PreferThinker, a reasoning-based personalized image assessment system with preference profile prediction, which achieves accurate assessment along with interpretable and multi-dimensional scoring based on the predicted profile.

Read the complete text