Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, Kang Liu · 2025
Read Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement by Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, Kang Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23895.txt.
Introduction
Yuqiao Tan, Shizhu He, Huanxuan Liao, and two colleagues make retrieval-augmented language models absorb documents into their parameters at test time, dynamically, in a paper published in two thousand twenty-five. Classic retrieval pastes documents into the prompt, which inflates inference cost and can trigger hallucinations when the model lacks the background to use the text.
An earlier parametric approach baked documents into weights offline, but training and storing a model per document set is costly and rigid. The new framework, DyPRAG, generates the parameter update on the fly for whatever is retrieved. Knowledge enhancement then happens at test time without a bloated prompt. Costs fall because the document need not ride along in every context window. Generalization improves because the update is computed for the current need. The work tries to give retrieval systems the speed of parametric memory with the freshness of search.
Catalog record
| CREATOR | Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, Kang Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23895
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, Kang Liu Abstract Retrieval-augmented generation (RAG) enhances large language models (LLMs) by retrieving relevant documents from external sources and incorporating them into the context. While it improves reliability by providing factual texts, it significantly increases inference costs as context length grows and introduces challenging issue of RAG hallucination, primarily caused by the lack of corresponding parametric knowledge in LLMs. An efficient solution is to enhance the knowledge of LLMs at test-time. Parametric RAG (PRAG) addresses this by embedding document into LLMs parameters to perform test-time knowledge enhancement, effectively reducing inference costs through offline training. However, its high training and storage costs, along with limited generalization ability, significantly restrict its practical adoption. To address these challenges, we propose Dynamic Parametric RAG (DyPRAG) , a novel framework that leverages a lightweight parameter translator model to efficiently convert documents into parametric knowledge. DyPRAG not only reduces inference, training, and storage costs but also dynamically generates parametric knowledge, seamlessly enhancing the knowledge of LLMs and resolving knowledge conflicts in a plug-and-play manner at test-time. Extensive experiments on multiple datasets demonstrate the effectiveness and generalization capabilities of DyPRAG, offering a powerful and practical RAG paradigm which enables superior knowledge fusion and mitigates RAG hallucination in real-world applications. 1 Introduction Knowledge-intensive tasks, such as question answering (QA), require leveraging extensive world and domain knowledge [ 20 ; 22 ] . While Large Language Models (LLMs) have demonstrated impressive capabilities across various tasks and fields [ 28 ] , they struggle to handle knowledge-intensive tasks independently due to limitations in accessing external knowledge and a tendency to generate hallucinated information [ 10 ] . To mitigate incorrect internal knowledge and improve performance in QA, researchers have developed knowledge-augmented methods for LLMs. One of the most used methods called Retrieval-Augmented Generation (RAG) [ 13 ] , which retrieves relevant documents from external sources (e.g., Wikipedia) and integrates both the retrieved documents and the query into the model’s input [ 17 ] (as shown in Figure 1 (a)). Standard RAG methods [ 3 ; 18 ; 39 ] share a common approach: injecting external knowledge into LLMs by appending retrieved documents to the input context, referred to as in-context injection . This technique harnesses the reasoning capabilities of LLMs to address knowledge gaps and reduce misinformation [ 5 ] . However, it introduces several challenges. As the number and length of retrieved documents increase, the extended context leads to significantly higher inference costs. Additionally, when the internal knowledge of LLMs conflicts with the external context, a phenomenon known as RAG hallucination (or knowledge conflict) arises, causing LLMs to produce incorrect answers even when accurate documents are provided [ 53 ; 42 ] . Figure 1 : Compared to RAG and PRAG, the proposed DyPRAG offers multiple advantages, including lower inference, training and storage cost, strong generalization ability, and mitigation of RAG hallucination. Parametric RAG (PRAG) employs another way of injecting knowledge which integrates external knowledge directly into the parameters of LLMs, known as parameter injection (in Figure 1 (b)). PRAG first rewrites retrieved documents and transforms them into QA pairs, facilitating knowledge memorization and manipulation [ 1 ; 2 ] . The workflow of PRAG is divided into two stages. During the offline phase, these augmented documents are fine-tuned with LoRA [ 16 ] , encoding contextual knowledge directly into parameters. In the online phase, retrieved documents are replaced with loadable parameters. However, this approach has significant trade-offs. Augmenting, training, and storing parameters for each retrieved document incurs high computational and storage costs, coupled with non-generalization, severely limiting scalability in real-world applications (e.g., systems in domains requiring frequent knowledge updates or multi open-domain QA tasks). This presents a critical challenge for PRAG: how can we minimize the overhead of converting documents into parametric knowledge without compromising performance? In this work, we propose Dynamic Parametric RAG (DyPRAG) , a lightweight framework that converts documents into parameters directly, making DyPRAG seamlessly enhance the knowledge of LLMs in a plug-and-play manner at test-time (as shown in Figure 1 (c)). Rethinking of PRAG, its intrinsic goal is to obtain an underlying mapping function \mathcal{F} which transforms external documents into parameters through augmentation and training for each document. Building on this insight, we aim to establish a more generalized mapping function by directly modeling \mathcal{F} with a small hypernetwork \mathcal{F}^{\prime}_{\phi} , referred to as parameter translator. With well-trained \mathcal{F}^{\prime}_{\phi} , DyPRAG eliminates the need for the cumbersome training and storage process required by PRAG. After a detailed analysis of computation and storage overhead, our method significantly reduces the high inference costs of traditional RAG while eliminating the rigid training and storage costs of PRAG. Through extensive experiments, we derive two key findings: 1) DyPRAG enhances the test-time parametric knowledge of LLMs effectively. During evaluation, DyPRAG outperforms standard RAG across different scales of LLMs, demonstrating its ability to enhance the internal knowledge of LLMs. Furthermore, although PRAG learns \mathcal{F} by training separately for each document, DyPRAG achieves comparable or even better performance in various scenarios with significantly lower costs. 2) Combining test-time generated parametric knowledge with contextual knowledge leads to superior knowledge fusion. Following [ 41 ] , we further investigate the combination of in-context injection and parameter injection, referred to as DyPRAG-Combine. In both independent identically distributed and out-of-distribution settings, DyPRAG-Combine achieves the best results, showing strong generalization ability. Additionally, we find that DyPRAG-Combine effectively relieves RAG hallucination by first injecting context-related parameters to reduce the knowledge conflicts. Based on the experimental results on RAGTruth [ 35 ] benchmark, we observe that DyPRAG-Combine enables LLMs to internalize previous unseen knowledge. This suggests that integrating parametric and contextual knowledge using DyPRAG could be a powerful approach for building a trustworthy RAG system in real-world applications. We summarize our contributions as follows: • We propose Dynamic Parametric RAG (DyPRAG), a lightweight framework to efficiently convert documents into parameters in a plug-and-play manner with minimal cost. • We introduce a powerful and practical RAG paradigm that effectively combines contextual knowledge with test-time generated parametric knowledge, enabling superior knowledge fusion. • Experimental results show that DyPRAG excels in generalization, efficiently injects parameters, and seamlessly incorporates contextual knowledge, enhancing performance while reducing RAG hallucination. 2 Related Work 2.1 Retrieval Augmented Generation Large language models (LLMs) have demonstrated remarkable performance across diverse applications. However, their inherent knowledge often falls short in handling knowledge-intensive tasks, highlighting the need for external knowledge integration to ensure robust performance in such contexts. A prominent approach to bridging this gap is Retrieval-Augmented Generation (RAG), which enhances LLMs by incorporating relevant external knowledge sources [ 4 ; 46 ; 47 ; 14 ] . The retrieved documents are appended to the LLM’s input context, enabling it to leverage knowledge beyond its training data [ 24 ] . We refer to this as the in-context injection paradigm. However, this approach leads to high inference costs and can exceed context length limits as the number and length of retrieved documents increase [ 50 ] . To address this issue, recent study introduces Parametric RAG (PRAG) [ 41 ] , a paradigm that fine-tunes the model on augmented documents, encoding useful information