Improving Factuality by Contrastive Decoding with Factual and Hallucination Prompts Authors: Bojie Lv, Ao Feng, Chenlong Xie Published: 2024 ABSTRACT Large language models have demonstrated impressive capabilities in many domains. But they sometimes generate irrelevant or nonsensical text, or produce outputs that deviate from the provided input, an occurrence commonly referred to as hallucination. To mitigate this issue, we introduce a novel decoding method that incorporates both factual and hallucination prompts (DFHP). It applies contrastive decoding to highlight the disparity in output probabilities between factual prompts and hallucination prompts. Experiments on both multiple-choice and text generation tasks show that our approach significantly improves factual accuracy of large language models without additional training. On the TruthfulQA dataset, the DFHP method significantly improves factual accuracy of the LLaMA model, with an average improvement of 6.4% for the 7B, 13B, 30B, and 65B versions. Its high accuracy in factuality makes it an ideal choice for high reliability tasks like medical diagnosis and legal cases. 1. Introduction Large language models (LLMs) have become pivotal in natural language processing (NLP), showcasing remarkable performance across various applications. Their ability to understand context and generate coherent text [1] allows them to capture subtle distinctions in language effectively. Additionally, LLMs exhibit impressive few-shot learning capabilities [2], enabling rapid adaptation to new tasks with minimal fine-tuning [3]. However, accelerated development of these models exposes a concerning issue. LLMs occasionally generate nonsensical or contextually irrelevant text, a phenomenon referred to as hallucination [4,5]. Hallucinations significantly compromise the reliability of LLMs in practical applications. In a medical setting, where large models are used to prescribe medications [6], hallucinations or inaccurate judgments could potentially worsen a patient’s condition or even pose life-threatening risks. The factors contributing to hallucinations include data quality, training processes, and inference challenges [7]. Flawed data sources and biases [8,9] can lead to the generation of false information, while the limitations of model architecture and training strategies, such as maximum likelihood estimation (MLE) [10,11], exacerbate the issue. Additionally, the stochastic nature of decoding strategies [12] may hinder the model’s ability to generate diverse outputs, increasing the likelihood of inaccurate predictions [13]. Addressing hallucinations typically requires substantial retraining, which is resource-intensive. To mitigate this, we propose a novel decoding strategy that optimizes model outputs to reduce hallucination frequency without compromising performance. DFHP enhances factual accuracy by filtering out non-factual information through a contrastive decoding approach. This involves two key objectives, improving factuality of the model’s outputs and eliminating non-factual content from them. We develop factual prompts to guide the model towards accuracy while steering it with hallucination-inducing prompts to generate hallucination-prone outputs. The contrastive decoding process allows the model to compare output distributions, effectively identifying and discarding hallucinated responses. Importantly, our DFHP method is compatible with existing pre-trained language models and requires no additional training. The key concept of DFHP is illustrated in Figure 1. Our method is built upon the LLaMA model [14], with the 7B parameter version as the main backbone. To assess its effectiveness, evaluations are conducted with both multiple-choice and open-ended generation tasks. In the multiple-choice setting, DFHP achieves the highest scores on both the TruthfulQA [15] and Factor [16] datasets, displaying significant improvement in the factual accuracy over the LLaMA-7B model. For open-ended generation tasks like TruthfulQA, it shows a slight decrease in informativeness but produces more truthful content, resulting in better %truth*info metrics. Results from the chain-of-thought reasoning datasets, including StrategyQA [17] and GSM8k [18], indicate that DFHP enhances the model’s factual accuracy as well as its reasoning capability. To gain a deeper understanding of DFHP, we conducted additional analyses, including a comparison of the effects of various prompts on DFHP and verifying its effectiveness across different sizes of the LLaMA model. 2. Related Work Huang et al. [7] categorize hallucinations in large language models into two main types: factuality hallucinations and faithfulness hallucinations. Factuality hallucinations occur when the model’s output contradicts or diverges from established knowledge. Faithfulness hallucinations, on the other hand, arise when the generated content fails to follow the user’s instructions, does not align with the provided context, or exhibits internal inconsistencies. This study primarily focuses on addressing factuality hallucination. Current approaches to mitigating hallucinations fall into two categories [19]: prompt engineering and developing models. Prompt engineering involves the systematic design and optimization of input prompts to guide the model’s responses, ensuring accuracy, relevance, and coherence in the generated output [20]. Prompt engineering has gained widespread attention due to its ability to effectively shape model outputs. By using carefully crafted prompts, researchers can significantly enhance the adaptability of large models across a wide range of tasks [21]. Research in this field has expanded rapidly, from basic methods involving comprehensive description [22] to more sophisticated approaches such as “chain of thought” prompting [23]. In this study, we mainly utilize two strategies: role prompting [24] and few-shot prompting [25]. Role prompting assigns the model a specific identity or role to guide its outputs, which proves highly effective. A well-known example is the “Grandmother loophole”, a case of prompt injection attacks using role-playing. Few-shot prompting, meanwhile, helps the model understand and perform specific tasks by providing a few output samples as examples, such as guiding the model to generate outputs in a particular style. Developing models aim to improve the model’s internal mechanisms to reduce hallucinations. Key measures include developing new decoding strategies [26], utilizing a loss function based on fidelity [27], and employing methods like supervised fine-tuning [11]. Compared to other interventions, addressing hallucinations at the inference level offers greater cost-effectiveness and better control. To develop a solution without extra training, this study focuses on prompt engineering and innovative decoding strategies. As highlighted in studies [28,29], well-structured prompts can help address issues such as hallucinations in machine-generated text. However, relying solely on prompts to elicit correct answers has limited effectiveness in improving the factual accuracy of the model. In this study, we combine prompt engineering with contrastive decoding techniques. As a similar approach, context-aware decoding (CAD) [30] is a proven strategy for reducing hallucination in text generation, which compares decoding outputs with and without contextual information. To fully leverage CAD, it is often crucial to supply rich context relevant to the problem. In specialized domains, the complexity of issues makes it particularly challenging to obtain such context, thereby limiting the utility of CAD. A closely related approach is contrastive decoding (CD), which aims to mitigate hallucinations in large language models by comparing outputs generated by a parameter-rich expert model and a smaller amateur model. Implementing CD requires an amateur model with fewer parameters that shares the same vocabulary, which can be impractical in some cases. Our method combines the strengths of CAD and CD with novel improvements. Instead of relying on external context or additional models, we leverage the power of the same model. By employing fact-based and hallucination-based prompts in contrastive decoding, we refine the next-word probability distribution with more accurate outputs. Experimental results demonstrate that DFHP outperforms the earlier methods in effectiveness. 3. Methods The core of our approach involves using two types of prompts to process the user’s input. First, we employ a factuality prompt to guide the model in generating a probability distribution that aligns with the target output. Then, we apply a hallucinations prompt to produce a probability distribution that reflects potential errors. The difference between these two distributions is then used to predict the next token’s distribution. This method allows us to demonstrate that the model can generate more realistic content. In the following, we will delve into a detailed analysis of the proposed method concerning the model’s predicted probability distribution in Section 3.1, while Section 3.2 will provide a comprehensive derivation of the specific mathematical formulas underpinning this method. 3.1. Model Predictive Probability Distributions When employing factual prompts, our objective is to steer the model toward producing entirely accurate answers. Although this approach enhances the model’s factual accuracy, the effect is limited, and the model may still yield incorrect responses. Conversely, when using hallucination prompts, we anticipate that the model will generate incorrect answers, which occurs in most instances. However, due to factors like the model’s robustness, these prompts do not always lead to failure. They may even yield correct responses in certain cases. In this context, our method can ensure effectiveness. For models using factual prompts, even if the final answer is incorrect, these prompts elevate the probability of the correct answer within the distribution. Similarly, while hallucination prompts may lead to incorrect outputs, they lower the likelihood of the correct answer. Consider this question “The word `Easter’ is connected with which goddess? Easter is connected with the goddess”. Inputting this into the large model using the factual prompt model, hallucination prompt model, and DFHP model allows us to predict the next word. We will extract the probabilities of the top three predictions from each method and display them in a bar chart (Figure 2), where the bar heights indicate the word probabilities. Closer bar lengths suggest similar probabilities, while greater disparities indicate more significant differences. In the factual prompt model, Ishtar appears with the highest probability, followed by Eostre, and then “of”, with small differences among them. In the hallucination prompt model, Ishtar retains the highest probability, but the likelihood of the correct answer, Eostre, decreases substantially, demonstrating the effectiveness of hallucination prompts in guiding the model to lower the correct answer’s probability. Conversely, in the DFHP model after contrastive decoding, Eostre shows a very high probability, surpassing Ishtar and “of” significantly. The process of contrastive decoding essentially resembles a process of amplifying differences. Specifically, factual prompts significantly enhance the likelihood of the correct answer, “Eostre”, while hallucination prompts correspondingly reduce this probability. Following contrastive decoding, the distinction between the correct answer and other options is amplified, thereby highlighting “Eostre” and giving it a dominant probability. DFHP’s core principle is to increase the predicted probability of the correct answer with factual prompts while decreasing it with hallucination prompts, ultimately emphasizing the correct answer’s probability through contrastive decoding. This example illustrates our method’s variations from the perspective of the model’s internal probability predictions, further validating its effectiveness. 3.2. Proposed Method For a given factuality prompt C+ and an input sequence {x1,x2,…,xt−1}, the logit function logit(xt|x