KeyKnowledgeRAG ( K 2 R A G K^{2}RAG ): An Enhanced RAG method for improved LLM question-answering capabilities
Hruday Markondapatnaikuni, Basem Suleiman, Abdelkarim Erradi, Shijing Chen · 2025
Read KeyKnowledgeRAG ( K 2 R A G K^{2}RAG ): An Enhanced RAG method for improved LLM question-answering capabilities by Hruday Markondapatnaikuni, Basem Suleiman, Abdelkarim Erradi, Shijing Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07695.txt.
Introduction
Hruday Markondapatnaikuni and co-authors present a 2025 paper on better question answering with retrieval. Fine-tuning giant models to absorb new knowledge costs huge compute. Retrieval-augmented generation instead fetches facts from a store at answer time. Naive retrieval scales poorly and answers shakily. Their framework, called K2RAG, attacks both flaws. It is inspired by divide and conquer.
Dense and sparse search run together to catch facts worded in unlike ways. A knowledge graph adds structure over the fetched chunks. Key knowledge extraction focuses the model on what matters most. Enterprise teams get a retrieval design for large, shifting knowledge bases. Law, medicine, and support desks all need answers tied to sources. A stronger retrieval layer can lift answer trust without retraining the base model.
Catalog record
| CREATOR | Hruday Markondapatnaikuni, Basem Suleiman, Abdelkarim Erradi, Shijing Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-07695
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
KeyKnowledgeRAG ( K 2 R A G K^{2}RAG ): An Enhanced RAG method for improved LLM question-answering capabilities Thanks: Citation : Authors. Title. Pages…. DOI:000000/11111. Hruday Markondapatnaikuni, Basem Suleiman, Abdelkarim Erradi, Shijing Chen Abstract Fine-tuning is an immensely resource expensive process when trying to retrain Large Language Models (LLMs) to have access to a larger bank of knowledge. To alleviate this issue there have been many different fine-tuning techniques proposed which have shown good progress in trying to reduce time and computational resources to achieve fine-tuning but with LLMs becoming more intelligent and larger, this issue continues to arise. Hence a new method of enabling knowledge expansion on LLMs had to be devised. Retrieval-Augment-Generate (RAG) is a class of techniques where information is stored in a database and appropriate chunks of information are retrieved to help answer the question. However there are many limitations to naive RAG implementations. This paper proposes the KeyKnowledgeRAG ( K 2 R A G K^{2}RAG ) framework to address the scalability and answer accuracy limitations associated with naive RAG implementations. This framework takes inspiration from divide-and-conquer ideology, and combines dense and sparse vector search, knowledge graphs and text summarization to help address these limitations. The framework also involves a data pre-processing step to reduce training times. The MultiHopRAG dataset was used for evaluation where the our implemented K 2 R A G K^{2}RAG pipeline was trained with the document corpus and then evaluated with the test data. Results have shown that there is an improvement in answering questions compared to common Naive RAG implementations where K 2 R A G K^{2}RAG achieved the highest mean answer similarity at 0.57 and also was able to answer more questions with more ground truth similarity with a highest Q 3 Q_{3} quartile at 0.82 . Additionally our proposed framework is trainable much faster due to the inclusion of a train data corpus summarization step which reduces training times of the individual components by 93% on average. Furthermore K 2 R A G K^{2}RAG has shown to operate even faster than a traditional knowledge graph based naive RAG implementation with a mean execution times reduced by up to 40% . In addition to superior question-answering capabilities, K 2 R A G K^{2}RAG offers being a more scalable solution with VRAM requirements reduced by 3 times in comparison to implementations of several naive RAG pipelines evaluated in this paper. Hence K 2 R A G K^{2}RAG can help companies in sophisticated decision-making through implementing a more lightweight and robust question-answering systems built on internal documents. 1 Introduction Motivation: Customizing LLMs to be more knowledgeable on specific areas has been a popular task to accomplish in recent years. A technique known as fine-tuning has been proposed and popularized through various different methods of fine-tuning coming up over the past few years such as FLAN [ 1 ] , LoRA [ 2 ] and QLoRA [ 3 ] which all aim to address different setbacks associated with the LLM fine-tuning process such as improving the final efficacy as well as reducing computational resources. Fine-tuning LLMs is usually done on full LLMs and is generally not advised to fine-tune on or convert pre-finetuned models into quantized models due to potential loss of knowledge retention and accuracy which can be attributed to lower precision of the model’s weights. Even though existing techniques like QLoRA have shown results which indicate minimal performance degradation upon the resulting fine-tuned quantized models which are based on their Guanaco models, this is not applicable to all LLMs [ 3 ] . For example an empirical study [ 4 ] evaluated different LLAMA3 quantizations and fine-tune quantizations has shown a significant reduction in LLAMA3-8B scores on the MMLU benchmarks when compared to the standard full 16-bit model and a QLoRA fine-tune-quantization to 4bits. This is because although all LLMs follow the basic transformer architecture there are still differences in model designs which can impact the quality of fine-tunes and hence even previously considered efficient methods such as QLoRA face their limitations with newer models [ 4 ] . Hence a major setback from all fine-tuning methods is that they are still extremely time and computational resource intensive in order to yield satisfactory results. Hence a new approach needed to be devised whereby LLMs could gain access to more knowledge with more computational efficiency. Figure 1: Evolution of RAG since 2020 [ 5 ] . A class of techniques known as Retrieval Augment Generate (RAG) systems have been devised as an alternative to fine-tuning where they do not employ any step where a base LLM is modified as in the fine-tuning method but rather knowledge is extracted from documents and indexed in a knowledge holding system to then retrieve [ 6 ] . This helps eliminate the computationally expensive associated with fine-tuning methods and also comes with the added benefit of having a dynamic knowledge base which could be updated quicker than fine-tuning methods. Since 2020 the concept of RAG has since evolved to include many different implementations and approaches to performing information retrieval to enhance LLM answering capabilities on new datasets [Figure 1 ]. This highlights how the benefits gained from RAG are being researched in depth to further optimize this process. Figure 2: Graph showing how LLMs prefer extracting information from top or bottom of context to answer a question [ 7 ] . Answer Accuracy: Naive RAG implementations often struggle with the "Needle-in-a-Haystack" problem [ 8 ] , where the generator model fails to provide accurate answers due to an inability to extract relevant information from irrelevant or excessively long contexts. This issue typically arises from poorly optimized chunk sizes and retrieval processes [ 9 ] . Optimizing chunk size is critical and challenging, as smaller chunks risk losing essential context, while larger chunks may introduce irrelevant information and unnecessarily increase context length [Figure 2 ]. Optimizing the retrieval process to maximize the relevance of retrieved information blocks is crucial. This ensures that irrelevant data, which could act as noise and obscure the correct information, does not compromise the generator LLM’s ability to accurately answer the question. Another limitation of naive RAG systems is that embeddings which are similar but not actually relevant to answer the question might be retrieved [ 10 ] as the original sentences might be semantically similar or carry a similar meaning but might not be right in helping answer the question in that context. Scalability: Naive implementations of Retrieval-Augmented Generation (RAG) often rely on 16-bit floating-point large language models (LLMs) for the generation component. However, this approach introduces significant scalability challenges due to the increased memory demands required to host the LLM as well as longer inference times due to using a higher precision number type. To enable more efficient scaling, it is crucial to integrate methods or techniques that reduce the memory footprint and inference times of generator models. Quantized models offer more scalable solutions due to less computational requirements, hence when developing RAG systems we should aim to use quantized LLMs for more cost effective deployment as compared to a