Dehallucinating Parallel Context Extension for Retrieval-Augmented Generation
Zexiong Ma, Shengnan An, Zeqi Lin, Yanzhen Zou, Jian-Guang Lou, Bing Xie · 2024
Read Dehallucinating Parallel Context Extension for Retrieval-Augmented Generation by Zexiong Ma, Shengnan An, Zeqi Lin, Yanzhen Zou, Jian-Guang Lou, Bing Xie (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-14905.txt.
Introduction
Zexiong Ma and co-authors present DePaC, a fix for made-up facts in retrieval-augmented chatbots, in a 2024 paper. Such systems fetch documents before answering, yet still invent claims. A technique that merges several fetched contexts in parallel made the problem worse. DePaC targets two failure modes: stating things the sources never said, and skipping things they did.
Against invention, the model is trained with negative examples that teach it to refuse when sources are off-topic. Against omission, the merger favors context windows that add the most new information. Across nine retrieval tasks, both kinds of hallucination dropped markedly. The work matters for any setting where a chatbot's answer must stay faithful to its sources.
Catalog record
| CREATOR | Zexiong Ma, Shengnan An, Zeqi Lin, Yanzhen Zou, Jian-Guang Lou, Bing Xie |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-14905
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dehallucinating Parallel Context Extension for Retrieval-Augmented Generation Zexiong Ma, Shengnan An, Zeqi Lin, Yanzhen Zou, Jian-Guang Lou, Bing Xie Abstract Large language models (LLMs) are susceptible to generating hallucinated information, despite the integration of retrieval-augmented generation (RAG). Parallel context extension (PCE) is a line of research attempting to effectively integrating parallel (unordered) contexts, while it still suffers from in-context hallucinations when adapted to RAG scenarios. In this paper, we propose DePaC ( De hallucinating Pa rallel C ontext Extension), which alleviates the in-context hallucination problem with context-aware negative training and information-calibrated aggregation . DePaC is designed to alleviate two types of in-context hallucination: fact fabrication (i.e., LLMs present claims that are not supported by the contexts) and fact omission (i.e., LLMs fail to present claims that can be supported by the contexts). Specifically, (1) for fact fabrication, we apply the context-aware negative training that fine-tunes the LLMs with negative supervisions, thus explicitly guiding the LLMs to refuse to answer when contexts are not related to questions; (2) for fact omission, we propose the information-calibrated aggregation which prioritizes context windows with higher information increment from their contexts. The experimental results on nine RAG tasks demonstrate that DePaC significantly alleviates the two types of in-context hallucination and consistently achieves better performances on these tasks. 1 Introduction Retrieval-augmented generation (RAG) ( Lewis et al. 2020 ; Gao et al. 2023 ) is nowadays a prevalent paradigm for incorporating large language models (LLMs) ( OpenAI 2023 ; Touvron et al. 2023 ; Jiang et al. 2023a ) with outside knowledge. RAG employs a retriever to fetch documents that are semantically closest to the question, and incorporates them into LLM’s prompt. Parallel Context Extension (PCE) ( Hao et al. 2022 ; Ratner et al. 2023 ; Su et al. 2024 ) is a line of research attempting to effectively integrating parallel contexts through an aggregation function. PCE is highly compatible with RAG scenarios, as the candidate retrieved documents of RAG are independ of each other. However, existing PCE approaches still face two types of in-context hallucination issues ( Ji et al. 2023 ; Rawte et al. 2023 ; Yang et al. 2023 ) : fact fabrication and fact omission. (1) fact fabrication occurs when the model presents fabricated claims that are inconsistent with the contextual facts. As shown in Figure 2(a) , LLM confidently produces a fabricated answer for the window with Doc_{2} , caused PCE to fabricate the wrong answer. (2) fact omission refers to windows lacking useful information may disproportionately affect the aggregation function, leading it to omit critical information present in other windows. This will make LLMs fail to present claims that can be supported by the contexts. As shown in Figure 2(b) , Doc_{3} does not contain required information, makes LLM confidently generate "Unknown" for the window with Doc_{3} , further leading to the wrong final answer. Figure 1: DePaC significantly reduces the occurrence of hallucinations in responses within RAG scenarios. (a) Fact fabrication example. Doc_{2} is useless to answer the question. The higher confidence in "Wendy" on Doc_{2} caused PCE to fabricate the answer "Alice’s grandfather is Wendy." (b) Fact omission example. Doc_{3} is useless to answer the question. The higher confidence in "unknown" on Doc_{3} caused PCE to omit the fact on Doc_{1} , resulting an incorrect final answer after aggregation. Figure 2: Existing PCE approaches face two types of in-context hallucination issues when applied to RAG: (1) Fact fabrication. LLM generates fabricated answers that are inconsistent with the contextual facts. (2) Fact omission. The absence of required information in certain windows disproportionately influence the aggregation function, leading to disregard critical information in other windows. In this paper, we propose DePaC to alleviate the hallucination issue of parallel context extension on RAG. DePaC contains two parts: NegTrain (Context-aware Neg ative Train ing) to address fact fabrication issue and ICA ( I nformation- C alibrated A ggregation) to address fact omission issue. (1) NegTrain guids the LLMs to refuse to answer when contexts are not related to the question. NegTrain consists of two parts of training data: one part comprises useful documents and questions as input, with corresponding answers as output. While the other part treats irrelevant documents and questions as input, with a rejection token as output. (2) ICA prioritizes context windows with higher information increment from their contexts. Specifically, we utilize Kullback-Leibler ( Kullback and Leibler 1951 ) divergence to measure the information increment of with-document compared to non-document. This approach enhances DePaC’s capability to identify useful information within parallel windows. Moreover, DePaC has lower computational complexity than vanilla inference approach. The inference time of DePaC increases linearly with the number of documents, while inference time of vanilla approach increases squarely. We conduct experiments on various RAG tasks, demonstrate that DePaC significantly alleviates the two types of hallucination and consistently achieves promising performances. Then we analyze the proportion of hallucination produced by different approaches, demonstrating that DePaC can effectively mitigate the two types of hallucination (Figure 1 ). We also conducte ablation study to identify that information-calibrated aggregation and context-aware negative training are both essential for DePaC performance. The main contents of this paper are organized as follows. Section 2 introduces the formalization of PCE and two existing aggregation methods for PCE. Section 3 introduces the methodology and implementation details of DePaC. Section 4 introduces the complexity analysis of DePaC. Section 5 introduces our experimental results on information seeking and DocQA. Section 6 discusses the related work. Finally, section 7 provides a conclusion regarding our work. 2 Background: Parallel Context Extension (PCE) The core idea of PCE involves aggregating information from multiple context windows into a unified representation space. Such a representation aggregation can be formalized on either the probability distributions of output tokens ( Su et al. 2024 ) , or the internal hidden states in attention layers ( Hao et al. 2022 ; Ratner et al. 2023 ) . Su et al. 2024 claimed the above two formalizations have similar practical performances. In this work, we adopt the formalization in ( Su et al. 2024 ) that takes the aggregation of output distributions. Given an question \mathcal{Q} , a set of retrieved documents \mathcal{D}=\{d_{1},d_{2},...,d_{n}\} , and a language model with parameters \theta , PCE first computes the output distribution of each context window, \mathbf{p_{i,j}}=p_{\theta}(\ \cdot\ |\ d_{j}\oplus\mathcal{Q}\oplus\mathcal{A}_{1:i-1}), (1) where \mathbf{p_{i,j}} is the probability distribution of the i -th token for output \mathcal{A} based on the d_{j} document, and \oplus represents the concatenation of sequences. Subsequently, these individual distributions are aggregated into a single distribution, \mathbf{p_{i}}=\mathrm{AGG}(\mathbf{p_{i,1}},\ \mathbf{p_{i,2}},\ ...,\ \mathbf{p_{i,n}}), (2) where \mathrm{AGG}(\cdot) represents the aggregation method. Finally, the output token \mathcal{A}_{i} will be sampled based on the aggregated distribution \mathbf{p_{i}} , \mathcal{A}_{i}\sim\hat{\mathbf{p_{i}}},\quad\hat{\mathbf{p_{i}}}=\mathbf{p_{i}}-\alpha\cdot\mathbf{p_{i,c}},\quad (3) \mathbf{p_{i,c}}=p_{\theta}(\ \cdot\ |\ \mathcal{Q}\oplus\mathcal{A}_{1:i-1}), (4) where the \hat{\mathbf{p_{i}}} is the calibrated distribution to facilitate generation. We set \alpha=0.2 following Su et al. 2024 . The effectiveness of the PCE paradigm is significantly influenced by the design of the aggregation method \mathrm{AGG}(\cdot) . Here, we discuss two aggregation methods used in existing studies. Average Aggregation ( Hao et al. 2022 ; Ratner et al. 2023 ) . The aggregated distribution is computed as the average of n individual distributions, \mathbf{p_{i}}=\frac{1}{n}\sum_{j=1}^{n}\mathbf{p_{i,j}}. (5) In practice, the size