Dynamic Retriever for In-Context Knowledge Editing via Policy Optimization

Mahmud Wasif Nafee, Maiqi Jiang, Haipeng Chen, Yanfu Zhang · 2025

Read Dynamic Retriever for In-Context Knowledge Editing via Policy Optimization by Mahmud Wasif Nafee, Maiqi Jiang, Haipeng Chen, Yanfu Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21059.txt.

Introduction

Mahmud Wasif Nafee leads three co-authors on DR-IKE, a dynamic retriever for editing knowledge inside language models through in context examples, with no gradient updates. Models keep repeating stale facts, and black box services rule out weight editing, so example based editing is attractive.

Static example sets chosen by surface similarity face a quantity quality trade off and cannot adapt to edit difficulty. DR-IKE trains a small BERT retriever with policy optimization to pick demonstrations by their actual utility for each edit. Lightweight and adaptive selection aims at edits that stick without flooding the prompt with weak examples.

Catalog record

CREATORMahmud Wasif Nafee, Maiqi Jiang, Haipeng Chen, Yanfu Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21059

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dynamic Retriever for In-Context Knowledge Editing via Policy Optimization Mahmud Wasif Nafee, Maiqi Jiang, Haipeng Chen, Yanfu Zhang Abstract Large language models (LLMs) excel at factual recall yet still propagate stale or incorrect knowledge. In - context knowledge editing offers a gradient - free remedy suitable for black - box APIs, but current editors rely on static demonstration sets chosen by surface - level similarity, leading to two persistent obstacles: ( i ) a quantity–quality trade - off, and ( ii ) lack of adaptivity to task difficulty. We address these issues by dynamically selecting supporting demonstrations according to their utility for the edit. We propose D ynamic R etriever for I n-Context K nowledge E diting ( DR-IKE ), a lightweight framework that (1) trains a BERT retriever with REINFORCE to rank demonstrations by editing the reward, and (2) employs a learnable threshold to prune low - value examples, shortening the prompt when the edit is easy and expanding it when the task is hard. DR - IKE performs editing without modifying model weights, relying solely on forward passes for compatibility with black - box LLMs. On the CounterFact benchmark, it improves edit success by up to 17.1%, reduces latency by 41.6%, and preserves accuracy on unrelated queries, demonstrating scalable and adaptive knowledge editing. 1 Introduction Large Language Models (LLMs) have demonstrated remarkable capabilities in memorizing vast amounts of knowledge and applying it across various applications, such as question answering Min et al. 2024 , dialogue systems Feng et al. 2023 , and code generation He et al. 2024 . Nevertheless, because these models are trained on a fixed corpus Touvron et al. 2023 ; Brown et al. 2020 , their knowledge often lags behind real-world developments—either due to erroneous data in the training set Cao et al. 2021a or the inherent delay between data collection and model deployment Dhingra et al. 2022 ; Gu et al. 2024b . For instance, during a conversation a user might ask, “ What is the newest iPhone right now? ”, and a model trained before 2024 could confidently reply, “ The iPhone 15 is the latest model, ” even though Apple introduced the iPhone 16 in September 2024. Such mismatches propagate to end tasks that depend on up-to-date factual grounding, including temporal reasoning Zhu et al. 2025 , fact verification Mousavi et al. 2024 , and personalized recommendation Bao et al. 2024 . Retraining a full-scale LLM with fresh data would in principle close this gap, but the computational and financial costs are prohibitive DeepSeek-AI et al. 2024 ; as a result, methods for Knowledge Editing —which aim to surgically modify the original model in order to incorporate new or corrected facts Mitchell et al. 2022a —have emerged as an attractive alternative due to their efficiency and targeted updates. However, existing practical knowledge editing methods still face three major obstacles. (1) Computation cost . Gradient-based editors such as ROME Meng et al. 2022 , MEND Mitchell et al. 2022a , MEMIT Meng et al. 2023a , and SERAC Cao et al. 2021b first calculate the gradient with respect to the parameters that most influence the prediction, and then apply a low-rank update to the implicated weight matrices, so that the model produces the desired answer while leaving other behaviours unchanged. Although one edit on GPT-J-6B takes only seconds Meng et al. 2023a , buffering intermediate activations for T5-11B already consumes \sim 60 GB of GPU memory Mitchell et al. 2022a , and both memory and compute scale linearly with model size and the number of edits, rendering thousands of real-time updates on 70-B models infeasible for most labs or start-ups. Moreover, these algorithms assume full read–write access to the weights, an assumption violated by the growing number of commercial LLM APIs. Consequently, in-context knowledge editing (IKE) Zheng et al. 2023 has been proposed: it injects the corrected facts as demonstrations in the prompt, thereby achieving competitive success rates without any parameter updates or gradient computations. Because most production LLMs are exposed only through black-box APIs, we confine our study to this realistic black-box-only scenario and ask how far a purely in-context approach can be pushed. Even for in-context-based editors Madaan et al. 2022 ; Zheng et al. 2023 ; Zhong et al. 2023 , two further challenges remain. (1) Trade-off between example quantity and quality . A series of recent in-context-learning studies reveals that accuracy rises sharply when the prompt includes the first few highly relevant examples, but the marginal benefit of each additional example soon saturates and may even reverse once redundancy or noise creeps in Chen et al. 2023 ; Agarwal et al. 2024 ; Zhang et al. 2025 ; Liu et al. 2023 . In knowledge editing, this fragility is amplified: irrelevant, conflicting, or poorly ordered supporting facts not only fail to help but also propagate errors to otherwise unrelated predictions Yu et al. 2025 , producing “ripple” side-effects Cohen et al. 2024 . Consequently, practical editors must retrieve and present a minimal set of maximally informative, non-overlapping demonstrations rather than indiscriminately lengthening the prompt. (2) Adaptivity to task difficulty . Not all edits are equally tractable: recent work shows that success rates vary by knowledge type. Edits involving abstract categories Ge et al. 2024a , commonsense reasoning Wu et al. 2024 , temporal references Ge et al. 2024b , reasoning-heavy logic Hua et al. 2024 , or popular entities Cohen et al. 2024 are consistently more error-prone. In addition, editing multiple related facts increases the risk of contradiction or unintended side-effects Li et al. 2024 . These findings highlight the need for adjust the editing methods based on the structure and complexity of each task. To address the above challenges, we propose D ynamic R etriever for I n-Context K nowledge E diting ( DR-IKE ), an adaptive example retrieval framework optimized via policy gradients. Our method augments a frozen LLM with a trainable BERT-based retriever that selects and ranks auxiliary factual examples to best facilitate accurate edits. Rather than relying on heuristic retrieval or static prompts, the retriever learns to balance informativeness and factual consistency through interaction: it is rewarded for example selections that lead the LLM to produce the correct, updated output. This approach mitigates the cost and rigidity of gradient-based editors, while also adapting to the difficulty of individual edits through a learnable thresholding mechanism that filters harmful or redundant context. In doing so, our framework offers a scalable, model-agnostic alternative for real-time factual updates, especially in settings with limited access to model internals. Our main contributions are as follows: • We design a BERT-based retriever trained with policy gradients that selects and ranks auxiliary facts without touching model weights . This light-weight, inference-time solution directly avoids the memory and compute overheads for gradient-based editors, making it viable in commercial API settings. • By learning to surface only the most informative, non-overlapping examples, our method curbs prompt length and minimises redundancy, resolving the quantity–quality trade-off identified in in-context knowledge editing. • We introduce a dynamic threshold that tightens or relaxes retrieval based on the predicted hardness of each edit, furnishing extra support for abstract, temporal, or popular-entity updates while avoiding over-prompting on easier cases—thereby addressing

Read the complete text