Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review

Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Leon Witt, Muhammad Asif Ali, Yukai Miao, Dan Li, Qingsong Wei · 2025

Read Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review by Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Leon Witt, Muhammad Asif Ali, Yukai Miao, Dan Li, Qingsong Wei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18346.txt.

Introduction

Toghrul Abbasli and co-authors systematically reviewed how researchers measure and reduce the uncertainty of large language models. These models often state wrong answers with full confidence, a failure called hallucination, so knowing when a model is unsure matters for safe use. Older statistics offered uncertainty tools for simpler models, and some have been adapted, but the field lacked a fair head-to-head test.

The review surveys the main measurement and calibration methods, then benchmarks seven of them on three widely used reliability datasets. The results back the review's main findings about which approaches hold up. The authors close with open problems and directions, aiming to give builders a clearer map of how to make model confidence mean something.

Catalog record

CREATORToghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Leon Witt, Muhammad Asif Ali, Yukai Miao, Dan Li, Qingsong Wei
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-18346

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Leon Witt, Muhammad Asif Ali, Yukai Miao, Dan Li, Qingsong Wei Abstract Large Language Models (LLMs) have been transformative across many domains. However, hallucination, i.e., confidently outputting incorrect information, remains one of the leading challenges for LLMs. This raises the question of how to accurately assess and quantify the uncertainty of LLMs. Extensive literature on traditional models has explored Uncertainty Quantification (UQ) to measure uncertainty and employed calibration techniques to address the misalignment between uncertainty and accuracy. While some of these methods have been adapted for LLMs, the literature lacks an in-depth analysis of their effectiveness and does not offer a comprehensive benchmark to enable insightful comparison among existing solutions. In this work, we fill this gap via a systematic survey of representative prior works on UQ and calibration for LLMs and introduce a rigorous benchmark. Using three widely used reliability datasets, we empirically evaluate seven related methods, which justify the significant findings of our review. Finally, we provide outlooks for key future directions and outline open challenges. To the best of our knowledge, this survey is one of the first dedicated studies to review the calibration methods and relevant metrics for LLMs. 1 1 1 Review list and source code : https://github.com/userTogrul/large-model-calibration-and-uncertainty I Introduction I-A Overview of Uncertainty and Calibration Large language models (LLMs) encompass a massive amount of parameters and are trained using extensively large-scale datasets to support diverse applications such as machine translation, content summarization, question answering, coding, and open conversation [ 1 , 2 , 3 , 4 ] . In past years, research and development practice has witnessed the power of scaling law, which shows that up-scaling the size of LLMs dramatically improves algorithmic, inference, and reasoning performance, a behavior widely known as the emergent phenomena [ 1 , 2 , 5 , 6 ] . However, these advancements have trade-offs, such as formidable costs and complexity. Meanwhile, the increase in the size of LLMs makes the interpretation and understanding of the internal scientific mechanisms and principles even harder [ 7 , 8 , 9 , 10 , 11 ] . For these reasons, research on the reliability of LLMs is crucial for enhancing stability and security while uncovering risks like misinformation, degraded cognitive capabilities, and hallucinations, especially in sectors such as finance, healthcare, and autonomous systems [ 12 , 13 , 5 ] . In other words, to achieve reliable performance, it is imperative to enable LLMs to “know what they do not know” in addition to tuning the prediction accuracy alone [ 14 , 15 ] . To this end, uncertainty serves as an important indicator of noisy, in-concrete categories or unknown content in decision-making [ 16 , 17 , 18 ] . Since uncertainty originates from both the internal structure of the model and the external environment, uncertainty quantification (UQ), i.e., detecting and measuring these sources, is fundamental to enhance the reliability of LLMs [ 19 , 20 , 13 ] . Meanwhile, LLMs must be capable of communicating the uncertainty of their predictions, a feature essential for users to know when to trust the model outcomes [ 21 , 22 , 23 , 11 ] . Although uncertainty can offer valuable insights into an LLM’s competence, these models often exhibit miscalibrated generations, displaying either over-confidence or under-confidence in their predictions, which provide incorrect measures of uncertainty [ 24 , 25 , 26 , 15 , 27 ] . This misalignment often results in a probability of prediction generation that does not reflect the model’s actual evaluation performance and introduces bias to the outputs [ 28 , 1 , 25 , 29 , 30 ] . Hence, calibrating models’ confidence is crucial for ensuring the reliable measurement and the accuracy of their probabilistic inferences. While recent surveys focus on works about uncertainty-aware techniques and hallucination [ 31 , 32 , 33 , 11 , 34 , 35 , 36 , 37 ] , we observe a lack of systematic review of calibration and evaluation of these methods for LLMs. This work seeks to bridge that gap by providing a comprehensive review of LLM calibration and uncertainty, by answering to the research questions (RQs) we set, and with the empirical analysis of them for the generation task. Our findings from the systematic review with PRISMA [ 38 ] method and generative question answering (QA) experiments revealed that recent LLMs are quite under-confident and over-confident in their predictions, and calibration methods are necessary to tackle this issue. While existing metrics are straightforward for measuring accuracy, calibration, and uncertainty, they fail to capture diverse nature predictions and the semantic and syntactic information of longer-form generations. We find that existing methods of calibration need to be adapted to internal layers and generation tasks to elicit reliable confidence. On the other hand, smaller LMs require a better calibration method. I-B Contributions The main contributions of the paper can be listed as follows: 1. To the best of our knowledge, this work is among the first to systematically review a broad set of calibration and uncertainty quantification (UQ) methods, considering both open-box and closed-box LLMs. 2. We conduct comparative experiments on LLMs using both calibration-specific and general performance metrics, addressing the current lack of effective evaluation regarding model truthfulness and uncertainty reasoning in generative QA tasks. While existing metrics are adequate for measuring accuracy and calibration, they do not fully capture the complexity of uncertainty or the semantic and syntactic richness of longer-form generations. We recommend adapting current methods to better account for internal model architectures and generation tasks in order to improve confidence estimation. 3. Our results show that smaller language models require improved calibration, whereas larger LLMs, with greater model and vocabulary sizes, can reduce output uncertainty through reasoning steps. For example, we observe that applying scaling to Mistral v0.3 7B improves its confidence estimates, and that models employing Chain-of-Thought (CoT) tend to be better calibrated. Experiments with Natural Questions (NQ) exhibit increased miscalibration, suggesting that factors such as question difficulty, distribution shift, and ambiguity affect model uncertainty. 4. We identify and summarize key research challenges and open problems to guide and accelerate future work. We hope our study increases awareness of the importance of aligning uncertainty estimation and accuracy in LLMs. Organization. The remainder of the paper is structured as follows: Section II presents the problem statement, reviews related surveys, and details our survey methodology. Section III provides the definition of uncertainty and UQ methods for DL and LLMs. Section IV examines calibration approaches for LLMs and the applicability of existing methods. Section V covers the evaluation of these methods and analyzes the results. Section VI discusses key findings, emerging topics, and future research challenges. Finally, Section VII concludes the paper. II Survey Methodologies II-A Related Survey Papers We examine existing survey papers to identify key research questions and determine gaps that need further investigation. Table I shows a detailed comparison of related review works. We mark topics that are covered as ✓ , topics that are not covered as ✗ , and topics that are briefly mentioned or slightly covered as \sim . Categories: UQ:

Read the complete text