ComMer: a Framework for Compressing and Merging User Data for Personalization

Yoel Zeldes, Amir Zait, Ilia Labzovsky, Danny Karmon, Efrat Farkash · 2025

Read ComMer: a Framework for Compressing and Merging User Data for Personalization by Yoel Zeldes, Amir Zait, Ilia Labzovsky, Danny Karmon, Efrat Farkash (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03276.txt.

Introduction

Yoel Zeldes and co-authors built ComMer in 2025, a way to personalize big language models on a budget. Feeding a user's documents into every prompt is capped by context size and costs at each call. Fine-tuning the model per user costs even more and ages fast.

ComMer instead squeezes each user's documents into a compact code. Codes from many sources merge into one, which feeds a frozen model. The squeeze happens once, offline, and the merged code rides along cheaply. Personal answers then draw on the user's own material without retraining. The frame suits services with millions of users and little compute to spare.

Catalog record

CREATORYoel Zeldes, Amir Zait, Ilia Labzovsky, Danny Karmon, Efrat Farkash
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-03276

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ComMer: a Framework for Compressing and Merging User Data for Personalization Yoel Zeldes, Amir Zait, Ilia Labzovsky, Danny Karmon, Efrat Farkash Abstract Large Language Models (LLMs) excel at a wide range of tasks, but adapting them to new data, particularly for personalized applications, poses significant challenges due to resource and computational constraints. Existing methods either rely on exposing fresh data to the model through the prompt, which is limited by context size and computationally expensive at inference time, or fine-tuning, which incurs substantial training and update costs. In this paper, we introduce ComMer - Compress and Merge - a novel framework that efficiently personalizes LLMs by compressing users’ documents into compact representations, which are then merged and fed into a frozen LLM. We evaluate ComMer on two types of personalization tasks - personalized skill learning, using the tweet paraphrasing dataset and the personalized news headline generation dataset from the LaMP benchmark, and knowledge-intensive, using the PerLTQA dataset. Our experiments demonstrate that in constrained inference budget scenarios ComMer achieves superior quality in skill learning tasks, while highlighting limitations in knowledge-intensive settings due to the loss of detailed information. These results offer insights into trade-offs and potential optimizations in multi-document compression for personalization. 1 Introduction LLMs have demonstrated remarkable capabilities across a wide range of natural language processing tasks. In many tasks, a simple prompt suffices to guide the model toward the desired behavior, while leveraging any required knowledge acquired during its pretraining ( Brown et al., 2020 ) . However, in some cases, such as personalizing LLMs to users’ data, new data becomes available only after the model’s initial training. Adapting LLMs to accommodate this new data introduces additional resource requirements and computational overhead, which is significant when scaling to a large number of users. Figure 1 : Approaches for adapting LLMs to new data include integrating it through the prompt or modifying the model by updating its existing weights or introducing new trainable weights. Both methods have advantages and drawbacks, while our proposed method, ComMer, combines the benefits of both approaches. The two prevalent approaches for adapting LLMs to new data, as illustrated in Figure 1 , are (1) prompt engineering, where new data is incorporated into the model through the prompt ( Salemi & Zamani, 2024 ) , and (2) full or parameter-efficient fine-tuning (PEFT) ( Wang et al., 2024 ) . The first approach faces limitations due to the model’s supported context window. While recent advancements allow LLMs to process millions of tokens ( Team et al., 2024a ) , this becomes problematic as the number and length of documents increase — particularly in data-rich modalities like video and audio. Even when the context window is sufficient, LLMs often exhibit positional bias, processing documents differently depending on their placement within the prompt, which can affect output quality ( Liu et al., 2024b ) . Additionally, regardless of quality, both latency and computational costs scale with the length of the prompt ( Duman Keles et al., 2023 ) , making this approach less feasible for large-scale applications. In the second approach, the LLM is adapted to new data by modifying its weights — or a subset of them — to incorporate the information from the new data. While this addresses the limitations of prompt engineering, it introduces new challenges. Specifically, training a separate model for each user leads to substantial computational costs. Furthermore, updating personalized models as new user data becomes available is a resource-intensive and complex process, often necessitating offline updates that compromise the model’s freshness. Finally, training personalized models for users with limited data can result in low-quality models ( Zhuang et al., 2024 ) . In this paper, we propose ComMer - Compress and Merge - a novel approach designed to address these challenges by learning to compress user data into a compact representation that efficiently adapts the LLM. The cost of training ComMer is amortized across all users, eliminating the need for individualized training processes. This approach offers several advantages, including reduced computational costs, streamlined and cost-effective adaptation to new data, and improved quality for users with limited data. We evaluate ComMer on two types of personalization tasks: personalized skill learning, using the tweet paraphrasing dataset and the personalized news headline generation dataset from the LaMP benchmark ( Salemi et al., 2024b ) , and knowledge-intensive question answering, using the PerLTQA dataset ( Du et al., 2024 ) . Our experiments demonstrate that ComMer strikes an effective balance between quality and computational cost in personalized skill learning. It efficiently processes multiple documents under a constrained token budget, with quality improving as the number of documents grows. However, in knowledge-intensive tasks, ComMer struggles to represent all the information from the documents, resulting in quality degradation as the document count increases. Additionally, we examine the effects of pretraining, generalization to out-of-distribution data, and alternative merging strategies, offering insights into the trade-offs and potential optimizations for ComMer’s architecture. 2 Related Work 2.1 Personalization Personalizing LLMs spans a spectrum from treating user data as a knowledge base to personalized skill learning. ( Du et al., 2024 ) introduced PerLTQA, focusing on knowledge base personalization by combining synthetic semantic and episodic memories for Q&A tasks. In contrast, ( Salemi et al., 2024b ) explored personalized skill learning, and introduced the LaMP benchmark consisting of seven personalized tasks. Different methods have been devised to tackle the personalization challenge: ( Salemi & Zamani, 2024 ) evaluated Retrieval-Augmented Generation (RAG) and PEFT on the LaMP benchmark; ( Salemi et al., 2024a ) optimized RAG pipelines for retrieval; and ( Zhuang et al., 2024 ) proposed HYDRA, a model factorization framework with user-specific rerankers and adapters. All these methods incur high inference or training costs. Concurrent to our work, ( Liu et al., 2024a ) introduced PPlug, which compresses users’ documents into pluggable prompts using a frozen encoder. Their approach relies on a query-dependent weighted average of document embeddings, which adds computational and storage overhead since it cannot be precomputed. Furthermore, PPlug’s frozen encoder and small embedding dimension limit its capacity, causing embedding interference. In contrast, ComMer uses trainable embeddings with mean pooling, enabling it to co-embed multiple documents in the same latent space while mitigating interference. 2.2 Efficiency and Compression Despite advances in context window sizes ( Team et al., 2024a ) , processing long contexts remains resource-intensive. Compression methods include latent representations ( Mu et al., 2023 ; Chevalier et al., 2023 ; Ge et al., 2024 ) and natural language compressions ( Zhou et al., 2023 ; Jiang et al., 2023 ; Jiang et al., 2024 ) . Our approach extends latent representation methods to the multi-document setting by merging document representations, yielding further compression gains. 2.3 Model Merging Merging models enhances their collective performance across tasks, with methods ranging from simple weight averaging ( Wortsman et al., 2022 ; Li et al., 2022 ) to sophisticated techniques like spherical linear interpolation ( Goddard et al., 2024 ) and sparse-aware merging ( Yu et al., 2024 ) . Some works focus on merging entire models, while others address merging PEFT models ( Chitale et al., 2023 ; Wu et al., 2024 ; Lin et al., 2024 ; Ostapenko et al., 2024

Read the complete text