CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correction
Harsh Maheshwari, Srikanth Tenneti, Alwarappan Nakkiran · 2025
Read CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correction by Harsh Maheshwari, Srikanth Tenneti, Alwarappan Nakkiran (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15629.txt.
Introduction
Harsh Maheshwari, Srikanth Tenneti, and Alwarappan Nakkiran present CiteFix in a 2025 paper on search systems that use large language models. Such systems, called RAG systems, pull in source texts and then write answers with citations. The authors note that the models often cite the wrong sources, which weakens trust in the answers.
CiteFix checks citations after an answer is written and fixes them before the user sees the result. The team tests three ways to match a claim to the right source, using word matching, a scored match model, and a small language model. Their tests show a clear gain in citation accuracy, with little added cost or delay. They also argue that better citations could let a product use a smaller, faster, and cheaper model.
Catalog record
| CREATOR | Harsh Maheshwari, Srikanth Tenneti, Alwarappan Nakkiran |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15629
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correction Harsh Maheshwari, Srikanth Tenneti, Alwarappan Nakkiran Abstract Retrieval Augmented Generation (RAG) has emerged as a powerful application of Large Language Models (LLMs), revolutionizing information search and consumption. RAG systems combine traditional search capabilities with LLMs to generate comprehensive answers to user queries, ideally with accurate citations. However, in our experience of developing a RAG product, LLMs often struggle with source attribution, aligning with other industry studies reporting citation accuracy rates of only about 74% for popular generative search engines. To address this, we present efficient post-processing algorithms to improve citation accuracy in LLM-generated responses, with minimal impact on latency and cost. Our approaches cross-check generated citations against retrieved articles using methods including keyword + semantic matching, fine tuned model with BERTScore, and a lightweight LLM-based technique. Our experimental results demonstrate a relative improvement of 15.46% in the overall accuracy metrics of our RAG system. This significant enhancement potentially enables a shift from our current larger language model to a relatively smaller model that is approximately 12x more cost-effective and 3x faster in inference time, while maintaining comparable performance. This research contributes to enhancing the reliability and trustworthiness of AI-generated content in information retrieval and summarization tasks which is critical to gain customer trust especially in commercial products. 1 Introduction Recent advancements in AI infrastructure and methodologies have enabled training Large Language Models (LLMs) over internet-scale data. These models demonstrate impressive competence in answering a wide range of general queries. However, when applied to specialized domains such as addressing questions based on internal company documents, off-the-shelf LLMs exhibit significant limitations. They often lack access to latest information, have difficulty interpreting domain specific language, struggle with source attribution, are prone to hallucinations Ji et al. (2023) , and are prone to overly broad responses. To overcome these challenges, two broad strategies have emerged. The first involves fine-tuning LLMs on domain-specific data. However, this approach is not only resource-intensive and requires frequent updates, but also risks unintended consequences such as catastrophic forgetting, where the model loses previously acquired general knowledge, thereby increasing the overall system complexity. The second, often more practical method is Retrieval-Augmented Generation (RAG). RAG is a process that combines information retrieval with text generation. It typically involves the following steps: (1) indexing a knowledge base of relevant information, (2) using a retrieval system to find content specifically relevant to a given user query, (3) providing the user query and the retrieved content to an LLM, instructing it to generate a response based on the retrieved content. RAG offers numerous benefits, including real-time access to up-to-date information, improved token generation Khandelwal et al. (2019) , reduced hallucinations, better source attribution Gao et al. (2023a) ; Hsu et al. (2024) and overall superior response generation Shuster et al. (2021) ; Béchard and Ayala (2024) . Additionally, RAG tends to be more cost-effective and transparent than full model fine-tuning. Examples of RAG-based products include Perplexity.ai Perplexity AI (2024) , bing search, GPT Search etc. Despite enabling a novel information retrieval experience for users, RAG systems today face key limitations. Table 1 illustrates results of a Subject Matter Expert based auditing of a RAG based system. Shown is a metric "Relative Mean Question Level Accuracy", which captures relevancy of cited chunks, correctness and completeness of the answer(Sec. 4.1 ), relative to Model C 1 1 1 Model names are anonymized following standard practice for proprietary/pre-release models. Publicly available models retain their original names. Model A, Model B and Model C are sufficiently large and powerful language model. With number of parameters in decreasing order for A, B and C. Model B however is the model trained on latest data with better methodologies accuracy. A prevalent form of error that contributes to lower performance is that of unverifiable facts in LLMs’ responses. Unverifiable facts are the facts in LLM response which cannot be validated by cited reference. In our analysis for Model C, notably around 80% of these unverifiable facts were not pure hallucinations, but rather errors in the model’s ability to cite the correct reference from which it generated the given factual point. These observations align with industry studies Liu et al. (2023) reporting citation accuracy rates of only about 74% for popular generative search engines. Incorrect citations not only reduce the actionability of the responses, but also dent customer trust, especially for commercial products. This paper focuses on this issue and proposes methods to address it. Figure 1: Improvements in RAG accuracy for various LLMs after employing our proposed citation correction methods. Results are shown as percentage improvements in Mean Question Level Accuracy(MQLA) over Model C baseline performance without citation correction. MQLA is a metric designed to capture relevancy, correctness and completeness (see Sec. 4.1 ). Model Cents per 1K Relative Mean Question % of factual % of factual points % of factual points O/P tokens Level Accuracy points unverifiable incorrectly cited purely hallucinated Model A +1100% +7.9% (+12%) Base (Base) 90.8% (65%) 9.1% (35%) Model B +220% +21.1% (+21.1%) Base (Base) 66.6% (66.6%) 34.4% (-33.4%) Model C Base Base (+15.4%) Base (Base) 80.6% (33.3%) 19.4%(-66.6%) Qwen 14-B open source +10.5% (+15.8%) Base (Base) 76.2% (70.8%) -13.8% (29.2%) Qwen 2-B open source -39% (NA) NA NA NA Table 1: Motivating the need for CiteFix: This table shows the prevalence of incorrect citations across LLMs and our method’s impact. Model C is the baseline for cost and accuracy columns. For the last three columns, the baseline is each model’s total percentage of unverifiable factual points. Numbers outside (inside) parentheses show performance before (after) CiteFix. Initially, incorrect citations significantly outnumber hallucinations. CiteFix balances this ratio and in absolute terms it drastically reduces incorrect citations. Qwen 2-B was excluded from detailed audit due to inconsistent citation generation. While previous studies have explored attributable text generation ( Nakano et al. (2022) ; Gao et al. (2023b) ) and simple prompting techniques for citation incorporation ( Malaviya et al. (2024) ; Sun et al. (2024) ; Li et al. (2024) ), systematic evaluations reveal significant performance gaps Gao et al. (2023b) . Recent work Huang et al. (2024) has only scratched the surface by demonstrating attribution quality degradation from ChatGPT to Llama 2 7B, leaving a critical need for deeper analysis and practical solutions. This paper offers two contributions: 1. Demonstrating the existence and extent of the incorrect citations issue across multiple LLMs, and highlighting the need to address the same. 2. Proposing six computationally light weight methods to address this issue, ranging from simple heuristic methods to more sophisticated learning-based solutions. Through extensive experimentation, we show that different citation correction approaches may be optimal for different LLMs - for instance, hybrid (lexical + semantic) matching works best with Model A, while fine-tuned BERTScore performs better with Model B. We provide detailed comparisons of their effectiveness and practical applicability. As seen in Fig 1 and Table 1 , our method resulted in an improvement of upto 15.46% relative improvement in accuracy when tested across four different LLMs. Through this work, we aim to not only advance the understanding of citation accuracy challenges in LLMs, but also provide practical low cost solutions for improving attribution in real-world applications. Sec. 2 presents