InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing
Shuaiyi Li, Zhisong Zhang, Yang Deng, Chenlong Deng, Tianqing Fang, Hongming Zhang, Haitao Mi, Dong Yu, Wai Lam · 2025
Read InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing by Shuaiyi Li, Zhisong Zhang, Yang Deng, Chenlong Deng, Tianqing Fang, Hongming Zhang, Haitao Mi, Dong Yu, Wai Lam (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22156.txt.
Introduction
Shuaiyi Li and co-authors proposed in 2025 a way to edit what a large language model knows without retraining it, by feeding edits as context. Context-based editing fails as edits pile up, because the context window fills and quality drops. Their framework, called InComeS, compresses each edit into a compact cache entry tied to a special token.
Extra attention modules then pick the most relevant compressed edits for each new question. The model can juggle many edits without being bound by window size. Tests across several editing benchmarks and formats showed the method works well and stays efficient. The authors present it as a flexible route to keeping deployed models factually current.
Catalog record
| CREATOR | Shuaiyi Li, Zhisong Zhang, Yang Deng, Chenlong Deng, Tianqing Fang, Hongming Zhang, Haitao Mi, Dong Yu, Wai Lam |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22156
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing Thanks: This work is partially supported by CityU Startup Grant (Grant No. 9382014) and the Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1 grant (Proposal ID: 24-SIS-SMU-002). Zhisong Zhang is supported by CityU Startup Grant (Grant No. 9382014). Yang Deng is support by the Lee Kong Chian Fellowship awarded by Singapore Management University. Shuaiyi Li, Zhisong Zhang, Yang Deng, Chenlong Deng, Tianqing Fang, Hongming Zhang, Haitao Mi, Dong Yu, Wai Lam Abstract Although existing model editing methods perform well in recalling exact edit facts, they often struggle in complex scenarios that require deeper semantic understanding rather than mere knowledge regurgitation. Leveraging the strong contextual reasoning abilities of large language models (LLMs), in-context learning (ICL) becomes a promising editing method by comprehending edit information through context encoding. However, this method is constrained by the limited context window of LLMs, leading to degraded performance and efficiency as the number of edits increases. To overcome this limitation, we propose InComeS, a flexible framework that enhances LLMs’ ability to process editing contexts through explicit compression and selection mechanisms. Specifically, InComeS compresses each editing context into the key-value (KV) cache of a special token, enabling efficient handling of multiple edits without being restricted by the model’s context window. Furthermore, specialized cross-attention modules are added to dynamically select the most relevant information from the pool of special tokens, enabling adaptive and effective utilization of edit information. We conduct experiments on diverse model editing benchmarks with various editing formats, and the results demonstrate the effectiveness and efficiency of our method. † † † Corresponding author. 1 Introduction Model editing, also known as knowledge editing, has seen rapid progress in recent years Fang et al. (2025) ; Li et al. (2024) ; Wang et al. (2024) ; Zhang et al. (2024a) . Its primary goal is to precisely integrate updated knowledge into a model, enabling targeted behavioral modifications while maintaining performance on unrelated tasks. Existing techniques have demonstrated strong performance in accurately recalling edited facts Yao et al. (2023) ; Zhang et al. (2024a) ; Zhang et al. (2025) . However, they often struggle in more complex editing scenarios, such as multi-hop editing composition Zhong et al. (2023) ; Zhang et al. (2025) , natural language editing Akyürek et al. (2023) , and editing tasks that require reasoning and generalization Cohen et al. (2024) ; Zhang et al. (2024a) . Moreover, recent studies Zhang et al. (2025) show that previous editing methods are prone to overfitting: they may assign excessively high probabilities to edited targets, which can distort the model’s responses to more complex or nuanced queries. Leveraging the in-context learning (ICL) abilities of large language models (LLMs) provides a promising direction for addressing these problems. As LLMs continue to grow in size and capability, their ability to understand and utilize contextual information continues to improve. By incorporating all the editing information into the prefix contexts, ICL enables a simple, powerful, and flexible approach for employing updated knowledge in complex scenarios. However, this approach faces significant challenges as the number of edits increases. First, the finite context window restricts the maximum number of edits that can be included, and the computational cost of self-attention over long contexts leads to a sharp decline in efficiency . Moreover, the effectiveness of ICL is constrained by the model’s ability to process extended contexts, and the retrieval accuracy of the most relevant editing information also tends to decrease as the editing context grows. To address these challenges, we introduce InComeS 1 1 1 https://github.com/Syon-Li/InComeS ( In tegrating Com pr e ssion and S election Mechanisms), a novel framework for efficient and scalable model editing. InComeS adopts context compression techniques to condense the representation of each edit into the KV cache of special gist tokens, which can be cached and reused for computational efficiency. While gisting Mu et al. (2023) was originally developed to compress single-input prompts, we extend this approach to handle multiple edits by further introducing a specialized selection mechanism. We further augment the model with cross-attention modules that allow each input token to attend to the compressed gist representations of edits, enabling fine-grained and adaptive selection of the most relevant information. Since each edit is compressed in parallel, our framework overcomes the limitations imposed by the context window, and the specialized selection modules can be learned to enhance retrieval accuracy. We conduct experiments across a range of complex model editing settings, including multi-hop editing, natural language editing, and tasks requiring complicated reasoning. Experimental results demonstrate that InComeS outperforms existing editing methods, effectively handling diverse editing scenarios while offering efficiency gains. 2 Preliminary Model editing Yao et al. (2023) ; Mitchell et al. (2022a) aims to adjust a base model \psi to a post-edited model \psi^{\prime} according to a set of editing information \mathcal{T}=\{t_{1},\dots,t_{n}\} : \psi^{\prime}=\text{Edit}(\psi,\{t_{1},\dots,t_{n}\}) . Here, “ Edit ” indicates the model editing method, while \{t_{1},\dots,t_{n}\} represents the knowledge pieces to be integrated. A typical example of editing information is query-label pair t=(x,y) , where the goal is for the edited model to produce y in response to input x , even if the original model does not: \psi(x)\neq y,\psi^{\prime}(x)=y . When the editing set contains only a single piece of information ( |\mathcal{T}|=1 ), this is known as single-instance editing. In contrast, batch editing refers to the scenario where multiple pieces of knowledge are updated simultaneously ( |\mathcal{T}|>1 ). Batch editing is particularly practical in real-world applications, where simultaneously updating several edits is often required. In these scenarios, it will be more efficient to integrate them into the model in a single operation. In practice, editing information can take various forms beyond simple query-label pairs. For instance, multiple related edits can be combined to enable multihop editing, or updated knowledge may be provided as a paragraph of natural language text. In such scenarios, many traditional editing methods may struggle to produce the desired outcomes, since they are not designed to handle these diverse types of editing information. In contrast, in-context learning (ICL) approaches, where editing information is simply concatenated as contextual prefixes, offer a straightforward yet powerful solution: \text{Edit}_{\text{ICL}}(\psi,\{t_{1},\dots,t_{n}\})(x)=\psi(t_{1},\dots,t_{n},x) . By leveraging the LLM’s ability to understand and reason over context, ICL can naturally accommodate a wide range of editing scenarios. Nevertheless, ICL is constrained by the context window of LLMs, and its accuracy and efficiency tend to decline when processing larger batches of edits. 3 Method Figure 1: An overview of InComeS . At the compression stage, each edit is individually condensed into KV cache representations of a gist token. These representations are integrated into the model via selection through the cross-attention modules. A special zero-gist token is included alongside the cached gists from actual edits, allowing the model to have the option to “select nothing.” Note that the compression and integration steps are performed separately, but both use the same underlying model. In this work, we aim to enhance the ICL-based editing approach to better understand multiple edits and accurately extract relevant information from the edit batch. Given a batch of editing information set \mathcal{T}=\{t_{1},\dots,t_{n}\} , an input query x , and the subset of its related 2 2 2 We