LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks
William Fleshman, Benjamin Van Durme · 2025
Read LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks by William Fleshman, Benjamin Van Durme (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05346.txt.
Introduction
William Fleshman and Benjamin Van Durme present LAG, a 2025 method for tapping libraries of small expert adapters, called LoRA modules, that fine-tune a language model for narrow jobs. As such experts multiply, picking and blending the right ones per task becomes the problem. LAG needs no extra training and no access to the experts' data. It filters and fetches adapters during generation, choosing them token by token and layer by layer.
On knowledge-heavy tasks, LAG beat rival data-free methods. Where extra data does exist, it also paired well with retrieval-augmented generation, the standard trick of fetching documents at answer time. The authors present LAG as a practical way to reuse a growing shelf of expert modules instead of training one giant model per job.
Catalog record
| CREATOR | William Fleshman, Benjamin Van Durme |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05346
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks William Fleshman, Benjamin Van Durme Abstract The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to data, and efficiently filters, retrieves, and applies experts on a per-token and layer basis. We evaluate LAG on various knowledge-intensive tasks, achieving superior performance over existing data-free methods. We explore scenarios where additional data is available, demonstrating LAG’s compatibility with alternative solutions such as retrieval-augmented generation (RAG). Introduction Modern language models (LMs) pretrained on large general-purpose text collections have led to rapid gains in task performance. Significant research is now focused on methods for for effectively deploying them in novel data domains and for specialized applications ( Lewis et al. 2020 ; Hu et al. 2022 ; Ilharco et al. 2023 ; Fleshman and Van Durme 2024 ; Yang et al. 2025 ) . One prominent approach involves leveraging techniques such as Retrieval-Augmented Generation (RAG), which dynamically injects relevant information from a collection of documents at inference time to guide the model output ( Lewis et al. 2020 ) . While highly effective, RAG necessitates the availability and efficient retrieval of these documents during the inference phase. Alternatively, LMs can be adapted to specific tasks or custom data domains through fine-tuning, a process that adjusts the model parameters using task-specific or domain-specific datasets ( Bapna and Firat 2019 ; Houlsby et al. 2019 ; Wei et al. 2022 ; Mangrulkar et al. 2022 ) . A particularly efficient and popular fine-tuning method is Low-Rank Adaptation (LoRA), which introduces a small number of trainable parameters, known as LoRA adapters, alongside the frozen pretrained model ( Hu et al. 2022 ) . This approach significantly reduces computational costs and storage requirements compared to full fine-tuning. The success and efficiency of LoRA has led to a rapid proliferation of these adapters, with numerous variations and specialized versions openly shared and readily accessible in public repositories such as Hugging Face ( Wolf et al. 2020 ) . The abundance of these readily available LoRA adapters, each potentially specialized for different tasks, domains, or styles, presents a unique opportunity. However, it also introduces a critical challenge: how to effectively select or combine these diverse adapters at inference time to achieve optimal performance. Many existing methods for leveraging LoRA adapters rely on access to the original training data corresponding to each adapter or require additional training to merge or select them ( Pfeiffer et al. 2021 ; Wang et al. 2022 ; Caccia et al. 2023 ; Ponti et al. 2023 ; Fleshman et al. 2024 ; Huang et al. 2024 ; Zadouri et al. 2024 ) . For example, Parametric-RAG (PRAG) uses a RAG-like retrieval mechanism over training documents, but loads an adapter corresponding to the selected document instead of including the content of the document in the prompt ( Su et al. 2025 ) . The dependency on data or additional training can be a significant bottleneck, especially in scenarios where the data is proprietary, unavailable, or too large to manage. These issues have led to the recent development of unsupervised routing methods. Arrow routing constructs rank-1 prototypes directly from the LoRA weights and uses them to efficiently select adapters on-the-fly ( Ostapenko et al. 2024 ) . Spectral routing (SpectR) improves the accuracy of Arrow at the cost of higher computational complexity ( Fleshman and Van Durme 2025 ) . We introduce LoRA-Augmented Generation (LAG), an approach developed to address the challenge of effectively utilizing existing LoRA adapters without requiring their corresponding data or any additional training. LAG delivers a flexible and efficient mechanism for leveraging the collective knowledge and capabilities embedded within a large set of LoRA adapters, enabling dynamic LoRA selection on a per-token and per-layer basis (Figure 1 ). Specifically we: • Develop an efficient LoRA selection and routing procedure to outperform other training-free approaches. • Leverage a library of over 1000 LoRAs to demonstrate our improved results on knowledge-intensive tasks; and • Compare and combine LAG with alternative methods for augmenting LMs with additional knowledge. Figure 1: Overview of LAG. LoRA adapters are converted offline via SVD to align representations and extract arrows . The token vector \mathbf{x} is processed in two stages: (1) Arrow routing is used to efficiently filter the large library of adapters to a smaller set of k potential LoRAs, and (2) Spectral routing is used to rank the filtered selection by measuring the length of the token representation in the basis of each adapter. The best adapter completes the new token representation \mathbf{x}^{*} . Background This section provides an overview of the key concepts and prior works motivating our problem setting and proposed LAG framework. We discuss existing paradigms for incorporating new knowledge into language models, focusing on retrieval-augmented generation and parameter-efficient fine-tuning methods, particularly LoRA. We then delve into recent advancements in knowledge acquisition, adapter retrieval, and unsupervised routing techniques, highlighting the strengths and limitations of each approach. Retrieval-Augmented Generation Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for grounding LMs with external knowledge, thereby mitigating issues like hallucination and enabling access to up-to-date information ( Lewis et al. 2020 ) . The core idea behind RAG is to augment the LM’s input with relevant passages retrieved from a knowledge base. A typical RAG pipeline involves several stages: a retriever component fetches relevant documents or passages based on a query, an optional reranker then reorders these retrieved passages to select the most pertinent ones, and finally, these selected passages are concatenated with the user query and fed into the generative model ( Glass et al. 2022 ) . The retriever can be based on sparse methods like BM25 or dense methods using embedding models ( Robertson and Zaragoza 2009 ; Karpukhin et al. 2020 ; Gao et al. 2024 ) . While highly effective, RAG systems face several challenges. Managing large and dynamic knowledge bases, ensuring retrieval relevance for diverse queries, and fitting retrieved context within the LM’s finite context window are all significant hurdles ( Liu et al. 2023 ; Gao et al. 2024 ; Barnett et al. 2024 ) . Moreover, the performance of a RAG solution heavily depends on the quality and freshness of the underlying knowledge base, making robust RAG difficult to implement, especially in rapidly evolving domains where the dataset itself is dynamic and constantly updated. Critically, RAG solutions require the external data to be available and retrieved at inference time, which might not always be feasible. In this work, we explore the case where knowledge is only available parametrically via adapters, but also compare to scenarios where relevant documents can be retrieved via RAG. Parameter-Efficient Fine-Tuning (PEFT) Beyond RAG, fine-tuning is another prominent approach to adapt LMs to new tasks or domains. However, full fine-tuning of large models is computationally expensive and memory-intensive, leading to the development of Parameter-Efficient Fine-Tuning (PEFT) methods ( Mangrulkar et al. 2022 ) . PEFT techniques aim to update only a small subset of model parameters while keeping the majority of the pretrained weights