PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation

Linhai Zhang, Jialong Wu, Deyu Zhou, Yulan He · 2025

Read PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation by Linhai Zhang, Jialong Wu, Deyu Zhou, Yulan He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01303.txt.

Introduction

Linhai Zhang, Jialong Wu, Deyu Zhou, and Yulan He present PROPER, in a 2025 preprint, a staged way to personalize large language models. Fine-tuning an add-on per user is the usual trick. It falters because each user's data is thin. The authors borrow a meso-level idea from social science: people cluster into groups with shared tastes. Their framework adapts a model to the crowd first, then to groups, then to the person. A mixture-of-experts design with low-rank add-ons carries the stages.

Two routers steer the process. One assigns each user to a fitting group. The other blends the person's add-on with the group's. Across tasks, PROPER beat rival systems by clear margins. The staged path lets scarce personal data lean on group patterns. The authors offer it as a route to assistants that feel personal without huge per-user training sets.

Catalog record

CREATORLinhai Zhang, Jialong Wu, Deyu Zhou, Yulan He
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01303

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation Linhai Zhang, Jialong Wu, Deyu Zhou, Yulan He Abstract Personalized large language models (LLMs) aim to tailor their outputs to user preferences. Recent advances in parameter-efficient fine-tuning (PEFT) methods have highlighted the effectiveness of adapting population-level LLMs to personalized LLMs by fine-tuning user-specific parameters with user history. However, user data is typically sparse, making it challenging to adapt LLMs to specific user patterns. To address this challenge, we propose PROgressive PERsonalization (PROPER), a novel progressive learning framework inspired by meso-level theory in social science. PROPER bridges population-level and user-level models by grouping users based on preferences and adapting LLMs in stages. It combines a Mixture-of-Experts (MoE) structure with Low Ranked Adaptation (LoRA), using a user-aware router to assign users to appropriate groups automatically. Additionally, a LoRA-aware router is proposed to facilitate the integration of individual user LoRAs with group-level LoRAs. Experimental results show that PROPER significantly outperforms SOTA models across multiple tasks, demonstrating the effectiveness of our approach. 1 1 1 Our code is available at https://anonymous.4open.science/r/PROPER-63E5/ . 1 Introduction Though large language models (LLMs) have demonstrated superior performance across various tasks Zhao et al. 2023 ; Chen et al. 2024 , they primarily offer a “ one-size-fits-all ” service, which falls short of adapting to individual user preferences. Personalized LLMs, aimed at tailoring their outputs to user-specific preferences, have therefore become a hot research topic Salemi et al. 2024 ; Mysore et al. 2024 ; Tan et al. 2024b . Figure 1: The comparison between different paradigms of LLM personalization, the solid line represents the current paradigms, which adapt the population-level LLM directly to the user-level LLM, while the dashed line illustrates the proposed paradigms, which adapt progressively through a group-level LLM using meso-level data as a bridge. Early efforts to personalizing LLMs focused on incorporating user history into prompts using in-context learning Dai et al. 2023 , retrieval-augmented generation Mysore et al. 2024 , and profile-augmented generation Richardson et al. 2023 . However, these prompt-based methods struggle with ensuring user data privacy and have limited generalization capabilities Tan et al. 2024b . Recent research has shifted towards fine-tuning personalized LLMs, where the base LLMs are fine-tuned on user history to better capture individual preferences. Tan et al. 2024b first proposed to store user-specific preferences and behavior patterns in personalized Parameter-Efficient Fine-Tuning (PEFT) parameters ( e.g. , LoRA Hu et al. 2022 ) to enable computationally efficient adaptation from population-level LLMs to user-specific models. Further research has explored training LoRAs based on representative users and integrating them into ensembles for target users, enhancing both time and space efficiency Tan et al. 2024a . However, due to the difficulty in collecting user data, data scarcity remains a significant issue. For instance, in the LaMP benchmark Salemi et al. 2024 , the average number of task-adaptation data tokens exceeds 1,000k, while the average number of tokens per user is only 48k, resulting in extreme data sparsity. Additionally, user data distribution follows the Pareto principle Backhaus 1980 , with the top 10% of users contributing 85% of the data, further exacerbating this sparsity. This sparsity makes it difficult for fine-tuning-based methods to learn complex user behavior patterns effectively. Social science research suggests that the meso level , bridging macro-level (population-level) and micro-level (user-level) analysis, is crucial for understanding the interplay between these two levels McConnell et al. 2002 ; Fine 2012 ; Faist 2021 . Inspired by this, as shown in Figure 1 , we propose using group-level LLMs as an intermediary (meso-level) layer between macro- and micro-level LLMs. Users with similar preferences and backgrounds can be grouped together Wood 1989 , allowing group-level LLMs to capture common patterns from a richer dataset. Data-sparse users can then benefit from group-level knowledge, enhancing their personalized models. In this paper, we propose PROPER (PROgressive PERsonalization), a novel personalized LLM framework that incorporates a group-level LLM and gradually adapts to users via progressive learning Fayek et al. 2020 . Our framework is also inspired by residual learning , where group-level preferences are modeled as a residual shift from population-level preferences, while user-level preferences are further captured by individual residuals beyond the group-level model. Rather than learning user preferences directly from individual user data, we treat personalization as a hierarchical refinement process. In this framework, the base model learned from population-level adaptation remains fixed, while subsequent group-level adaptation captures only the residual preferences that the population-level model fails to encode, and similarly for user-level adaptation. PROPER thus decomposes LLM personalization into three stages: population-level adaptation, group-level adaptation, and user-level adaptation. All adaptation stages employ LoRA to improve computational efficiency. To construct the group-level LLM, we employ a Mixture-of-Experts (MoE) structure Dou et al. 2024 , where each expert represents a user group, and users are automatically assigned to appropriate experts by a user-aware router. During user-level adaptation, we further introduce a LoRA-aware router that integrates group-level and user-level LoRAs by selecting the most relevant group-level LoRA based on user-level LoRA knowledge. Experimental results on the LaMP benchmark show that PROPER significantly outperforms all prior fine-tuning-based baselines. In conclusion, our contributions are three-fold: • New Framework : We are the first to propose a personalized LLM method that introduces a group-level LLM between population-level and user-level LLMs and integrates it into a progressive learning framework. • Group-level LLM Construction : we enable automatic user grouping via LoRAMoE and user-aware routers, while effectively integrating user and group-level knowledge through a LoRA-aware router. • Empirical Performance : Our method achieves state-of-the-art results, outperforming all existing fine-tuning-based baselines across all tasks in the LaMP benchmark. 2 Method Figure 2: Overview of the training process of PROPER, which consists of three steps: (1) Population-level adaptation , where task information is learned via regular LoRA training; (2) Group-level adaptation , where group-level preferences are learned by LoRAMoE; (3) User-level adaptation , where user preference is learned into user-specific LoRA. The LoRAs are applied to the FFN layers while other components of the Transformer blocks are omitted for simplicity. Task Formulation Following previous studies Tan et al. 2024b ; Tan et al. 2024a , we define the task objective of personalized LLM as generating a user-specific response r_{u} for a given user u at time t , based on the user query q_{u} and the user history H_{u} : r_{u}=\text{LLM}(q_{u}|H_{u}) , where H_{u}=\{h_{u}\} and each history entry can take one of two forms: either a task-specific query-response pair h_{u}=(q_{u},r_{u}) or plain text h_{u}=t_{u} . For fine-tuning-based personalized LLMs, the goal is to capture user preferences through user-specific parameter \Theta_{u} , refining the model’s response generation as follows: r_{u}=\text{LLM}(q_{u}|\Theta_{u}). (1) Overall Framework As shown in Figure 2 , PROPER is based on progressive learning that consists of three stages: • Stage 1 (Population-Level Adaptation): The model learns population-level preference for specific tasks using standard LoRA. • Stage 2 (Group-Level Adaptation): The population-level LoRA from Stage 1 is kept fixed, while group-level preferences are learned using a combination of LoRA and MoE. • Stage 3 (User-Level Adaptation): The LoRAs from the previous two stages remain unchanged, and user-specific LoRA is trained to capture individual user preferences. 2.1 Population-Level Adaptation Though LLMs are

Read the complete text