Uncertainty-Aware Fusion: An Ensemble Framework for Mitigating Hallucinations in Large Language Models Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May
Prasenjit Dey, Srujana Merugu, Sivaramakrishnan Kaveri · 2025
Read Uncertainty-Aware Fusion: An Ensemble Framework for Mitigating Hallucinations in Large Language Models Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May by Prasenjit Dey, Srujana Merugu, Sivaramakrishnan Kaveri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05757.txt.
Introduction
Prasenjit Dey, Srujana Merugu, and Sivaramakrishnan Kaveri fight language model hallucinations by fusing several models and listening to each one's self-doubt. Ensembles are an old trick, but they usually ignore how sure each member is. Recent work shows models can partly grade their own reliability. The catch is unevenness: different models shine in different situations. Writing in twenty twenty-five, the team's framework weights members by accuracy and self-assessment skill, then combines their answers on factoid questions.
Across several public benchmarks, the fusion beat leading hallucination fixes by a clear margin in factual accuracy. It also narrowed or reversed the gap to a top proprietary model. No representation surgery or retraining is needed. The recipe is deliberately simple: measure uncertainty, then vote accordingly.
Catalog record
| CREATOR | Prasenjit Dey, Srujana Merugu, Sivaramakrishnan Kaveri |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-05757
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncertainty-Aware Fusion: An Ensemble Framework for Mitigating Hallucinations in Large Language Models Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3701716.3715523 ISBN: 979-8-4007-1331-6/25/04 Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3701716.3715523 ISBN: 979-8-4007-1331-6/25/04 CCS: Computing methodologies Ensemble methods CCS: Computing methodologies Natural language generation Prasenjit Dey, Srujana Merugu, Sivaramakrishnan Kaveri Abstract . Large Language Models (LLMs) are known to hallucinate and generate non-factual outputs which can undermine user trust. Traditional methods to directly mitigate hallucinations, such as representation editing and contrastive decoding, often require additional training data and involve high implementation complexity. While ensemble-based approaches harness multiple LLMs to tap into the "wisdom of crowds", these methods overlook uncertainties in individual model responses. Recent studies reveal that uncertainty estimation can enable LLMs to self-assess the likelihood of generating hallucinations. In this work, we focus on factoid question answering (QA) and observe that LLMs accuracy and self-assessment capabilities vary widely with different models excelling in different scenarios. Leveraging this insight, we propose Uncertainty-Aware Fusion (UAF), an ensemble framework to reduces hallucinations by strategically combining multiple LLM based on their accuracy and self-assessment abilities. Empirical results on several public benchmark datasets show that UAF outperforms state-of-the-art hallucination mitigation methods by 8 % 8\% in factual accuracy, while either narrowing or surpassing the performance gap with GPT-4. † † cc-license: by Figure 1. Fraction of examples where each LLM generates the most confident correct answer. Optimal LLM ( high accuracy and confidence) varies across examples. 1. Introduction Large language models (LLMs) have yielded remarkable performance boosts across diverse natural language processing (NLP) tasks ( Achiam et al. 2023 ) . However, their tendency to ’hallucinate’, i.e., generating outputs not grounded in factual training data ( Ji et al. 2023 ) , remains a critical limitation, impeding deployment, particularly in high-stakes applications where factual accuracy is paramount. Related work: Existing work on hallucination mitigation can be broadly categorized into three groups. The first group of techniques tries to enhance truthfulness at inference time using contrastive decoding or representation editing. Representation editing methods like ITI ( Li et al. 2024 ) and TruthX ( Zhang et al. 2024 ) identify a truthful direction using human-labeled data and adjust model activations accordingly. Contrastive decoding approaches such as SH2 ( Kai et al. 2024 ) and DoLa ( Chuang et al. 2023 ) modify output probabilities by comparing distributions between different models or layers. However, these methods often face challenges with implementation complexity, reliance on extensive labeled data, and limited generalization across domains. The second group employs ensemble learning techniques to improve model performance. Methods such as LLM-Blender ( Jiang et al. 2023 ) learn to combine diverse candidate responses from multiple LLMs, while MLM ( Salazar et al. 2019 ) selects the response most similar to the input. Another approach exemplified by Consistent ( Wang et al. 2022 ) , generates multiple candidate responses and uses majority voting for the final output. However, these approaches often overlook the inherent uncertainties in candidate responses, potentially limiting their effectiveness. The third group of methods is based on the observation that LLMs often have an internal sense of truthfulness, even when producing false statements ( Kadavath et al. 2022 ; Chen et al. 2024 ) . This insight is exploited in various approaches such as logit probability-based methods ( Ren et al. 2022 ; Malinin and Gales 2020 ) , consistency-based techniques ( Kuhn et al. 2023 ; Manakul et al. 2023 ) , probing internal representations ( Chen et al. 2024 ; Du and Xiao 2024 ) , and prompting-based strategies ( Kadavath et al. 2022 ) to obtain uncertainty scores without additional training or model editing, offering promising avenues for hallucination detection and mitigation. Contributions: In this work, we focus on mitigating LLM hallucinations for factoid question answering (QA). First, we observe that the proliferation of open-source LLMs, each with distinct strengths and weaknesses, precludes any single model from consistently outperforming others in factual accuracy and self-assessment. Figure 1 illustrates this phenomenon for the TruthfulQA ( Lin et al. 2021 ) dataset, showing the distribution of examples where each LLM produces the most confident correct response. Despite Mistralv3-7B achieving the highest overall accuracy, it leads in confidence for only 18% of examples, indicating that the rest are better handled by other models. This diversity in model performance across examples presents an opportunity to leverage the complementary strengths of multiple LLMs to reduce hallucinations and enhance overall accuracy. Our key contributions are summarized below: 1. We demonstrate, through extensive experiments with six LLMs across multiple benchmarks, that the relative factual accuracy and self-assessment capabilities of LLMs vary significantly across examples, with no single dominant model. 2. We propose Uncertainty-Aware Fusion (UAF), an ensemble framework that strategically combines multiple LLMs to achieve superior performance. UAF consists of two key modules: (i) SELECTOR, which chooses the top K LLMs from the entire pool based on accuracy and hallucination detection abilities, and (ii) FUSER, which merges the outputs of these K LLMs to generate the final result. 3. Experiments on benchmarks such as TruthfulQA ( Lin et al. 2021 ) , TriviaQA ( Joshi et al. 2017 ) , and FACTOR ( Muhlgay et al. 2023 ) datasets reveal that our ensembling technique significantly enhances factual accuracy with UAF outperforming SOTA baselines by 8% in accuracy, while narrowing the performance gap with, or even surpassing, GPT-4 ( Achiam et al. 2023 ) . We also present ablation studies examining the impact of uncertainty measure, model selection criteria, and ensemble size on overall performance. Figure 2. Overview of our UAF architecture 2. Proposed Method Problem Statement: Let \mathcal{X} and \mathcal{Y} denote the input and output spaces, respectively, and D=\{(x_{i},y_{i})\}_{i=1}^{n} the dataset, where x_{i}\in\mathcal{X} and y_{i}\in\mathcal{Y} are the i^{th} question-answer pair. For each x_{i} , the goal is to generate a response \hat{y}_{i} that maximizes the overall accuracy. The goal is to achieve this using a pool of N pretrained foundational LLMs without additional training or fine-tuning. Algorithm 1 Components of UAF - SELECTOR and FUSER 0: D_{val} , Pool of LLMs \mathcal{M}=\{M^{j}\}_{j=1}^{N} , Uncertainty function U_{f}(.,.,.) , Ensemble size K , Test data point x_{test} 0: Test data response \hat{y}_{test} procedure SELECTOR ( D_{val},\mathcal{M},U_{f},K ) for M^{j}\in\mathcal{M} do for (x_{i},y_{i})\in D_{val} do \hat{y}_{i}^{j}=M^{j}(x_{i}) ; s_{i}^{j}=\mathbf{1}(\hat{y}_{i}^{j}==y_{i}) u_{i}^{j}=U_{f}(M^{j},x_{i},\hat{y}_{i}^{j}) end for Acc_{j}=\frac{1}{|D_{val}|}\sum_{i}s_{i}^{j} ; SAH_{j}=ROC\_AUC\_score(\{s_{i}^{j},u_{i}^{j}\}_{i}) Cscore_{j}=Acc_{j}\times SAH_{j} end for return: \mathcal{M}^{sel}=TopK(\{Cscore_{j}\}_{j=1}^{N}) , {//Top K LLMs} Acc^{sel}=\{Acc_{j}|j\in\mathcal{M}^{sel}\} {//Accuracy of selected K LLMs} procedure FUSER ( x_{test},\mathcal{M}^{sel},Acc^{sel},U_{f},K ) for M^{k}\in\mathcal{M}^{sel} do \hat{y}_{test}^{k}=M^{k}(x_{test}) u_{test}^{k}=U_{f}(M^{k},x_{test},\hat{y}_{test}^{k}) end for \hat{y}_{test}=\hat{y}_{test}^{k^{*}}\text{, where }k^{*}=argmax_{k\in\{1,\dots K\}}Acc_{k}\times(1-u_{test}^{k}) return: \hat{y}_{test} 2.1. Uncertainty Aware Fusion (UAF) Figure 2 provides an overview of our UAF framework. At a high level, UAF consists of two modules: SELECTOR and FUSER. Given a specific task, the SELECTOR selects the top K LLMs from a pool of N LLMs based on performance metric. FUSER then combines the outputs of these K LLMs to produce the final response. 2.1.1.