A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions

Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, Anirudha Majumdar · 2024

Read A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions by Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, Anirudha Majumdar (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-05563.txt.

Introduction

Shorinwa and co-authors survey in 2024 how to measure doubt in the answers of large language models. These models can state false claims with great confidence, a flaw called hallucination. Spotting that doubt can flag answers that should not be trusted, from chat tools to robots.

The survey sorts known methods into a clear tree, with strengths and weak points for each branch. Some methods ask the model twice, some read its inner scores, and some train side tools. The authors map where each method is used and what problems stay open. They close with paths for making such checks cheaper and more sure.

Catalog record

CREATOROla Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, Anirudha Majumdar
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-05563

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, Anirudha Majumdar Abstract . The remarkable performance of large language models (LLMs) in content generation, coding, and common-sense reasoning has spurred widespread integration into many facets of society. However, integration of LLMs raises valid questions on their reliability and trustworthiness, given their propensity to generate hallucinations: plausible, factually-incorrect responses, which are expressed with striking confidence. Previous work has shown that hallucinations and other non-factual responses generated by LLMs can be detected by examining the uncertainty of the LLM in its response to the pertinent prompt, driving significant research efforts devoted to quantifying the uncertainty of LLMs. This survey seeks to provide an extensive review of existing uncertainty quantification methods for LLMs, identifying their salient features, along with their strengths and weaknesses. We present existing methods within a relevant taxonomy, unifying ostensibly disparate methods to aid understanding of the state of the art. Furthermore, we highlight applications of uncertainty quantification methods for LLMs, spanning chatbot and textual applications to embodied artificial intelligence applications in robotics. We conclude with open research challenges in uncertainty quantification of LLMs, seeking to motivate future research. DOI: 1111111.1111111 CCS: Computing methodologies CCS: Computing methodologies Artificial intelligence CCS: Computing methodologies Natural language processing CCS: Computing methodologies Natural language generation 1. Introduction Large language models have demonstrated remarkable language generation capabilities, surpassing average human performance on many benchmarks including math, reasoning, and coding ( Achiam et al., 2023 ; Anthropic, 2024 ; Brown, 2020 ; Touvron et al., 2023 ; Dubey et al., 2024 ; Chiang et al., 2023 ) . For example, recent (multi-modal) large language models were shown to achieve impressive scores, e.g., in the 90\% percentile, on simulated Law School Admission Test (LSAT) exams, the American Mathematics Competition (AMC) contests, the Multistate Bar Exam, and the Graduate Record Exam (GRE) General Test, outperforming a majority of test takers ( Katz et al., 2024 ; Achiam et al., 2023 ; Anthropic, 2024 ) . Likewise, LLMs have advanced the state of the art in machine translation, text summarization, and question-and-answer tasks. However, LLMs also tend to produce plausible, factually-incorrect responses to their input prompts, termed hallucinations ( Lee et al., 2018 ) . In some scenarios, the hallucinated response is overtly incorrect; however, in many cases, the factuality of the LLM response is harder to discern, posing significant risk as a user might falsely assume factuality of the response, which can result in devastating consequences, especially when safety is of paramount importance. As a result, hallucinations pose a notable danger to the safe, widespread adoption of LLMs. Figure 1 . A user asks an LLM the question: What is the lowest-ever temperature recorded in Antarctica? ; in response, the LLM answers definitively. Afterwards, the user asks the LLM how confident the LLM is. Although the LLM states that it is “100% confident," the LLM’s response fails to pass a fact-check test. Confidence scores provided by LLMs are generally miscalibrated. UQ methods seek to provide calibrated estimates of the confidence of LLMs in their interaction with users. A user asks an LLM the question: What is the lowest-ever temperature recorded in Antarctica? ; in response, the LLM answers definitively. Afterwards, the user asks the LLM how confident the LLM is. Although the LLM states that it is “100% confident," the LLM’s response fails to pass a fact-check test. Confidence scores provided by LLMs are generally miscalibrated. UQ methods seek to provide calibrated estimates of the confidence of LLMs in their interaction with users. A user asks an LLM the question: \emph{What is the lowest-ever temperature recorded in Antarctica?}; in response, the LLM answers definitively. Afterwards, the user asks the LLM how confident the LLM is. Although the LLM states that it is ``100\% confident," the LLM's response fails to pass a fact-check test. Confidence scores provided by LLMs are generally miscalibrated. UQ methods seek to provide calibrated estimates of the confidence of LLMs in their interaction with users. To ensure the trustworthiness of LLMs, substantial research has been devoted to examining the mechanisms behind hallucinations in LLMs ( Lee et al., 2018 ; Chen et al., 2023 ; Azamfirei et al., 2023 ; Xu et al., 2024a ; Ji et al., 2023 ) , detecting its occurrence, identifying potential causes, and proposing mitigating actions. However, even in the absence of hallucinations, LLMs are susceptible to doubt when given prompts at the boundary of their knowledge base. In these situations, prior work has shown that LLMs fail to accurately convey their uncertainty to a user, either implicitly or explicitly, unlike typical humans ( Liu et al., 2023c ; Alkaissi and McFarlane, 2023 ) . In fact, LLMs tend to be overconfident even when they should be uncertain about the factuality of their response ( Xiong et al., 2023 ; Groot and Valdenegro-Toro, 2024 ) . We provide an example in Figure 1 , where an LLM is asked: “What is the lowest-ever temperature recorded in Antarctica?", to which the LLM responds definitively. Even when prompted for its confidence in its answer, the LLM claims that it is “100% confident." However, the LLM’s answer fails to pass a fact-check test. Knowing how much to trust an LLM-generated response is critical for users ( Kim et al., 2024a ) , helping inform the development of contingency strategies commensurate with the degree of uncertainty of the LLM in its response. For example, in applications such as robotics, an LLM-equipped robot could seek human guidance ( Ren et al., 2023a ) or necessitate further review in the judicial practice ( Delacroix, 2024 ) . Uncertainty quantification (UQ) methods for LLMs seek to address this challenge by providing users with an estimate of an LLM’s confidence in its response to a given prompt. Indeed, uncertainty quantification can be important in factuality analysis ( Huang et al., 2023a ) . The rapid adoption of LLMs in many applications has contributed to the fast-pace development of UQ methods for LLMs to promote their safe integration into a wide range of applications. However, the huge volume of UQ methods for LLMs has made it particularly challenging to ascertain the research scope and guarantees provided by existing UQ methods, complicating the identification of useful UQ methods for practitioners seeking to leverage them in their application areas, as well as the identification of impactful future directions for research. We claim that this challenge arises from the lack of a taxonomy that unifies related existing methods and presents an organized view of existing work in this research area. Through this survey, we seek not only to enumerate existing work in UQ for LLMs, but also to provide a useful taxonomy of UQ methods for LLMs to aid understanding the state of the art in this research area. We reiterate that the introduction of an effective taxonomy for these methods can facilitate their adoption in wide-ranging applications, such as in factuality analysis, hallucination detection, and robotics. We categorize existing uncertainty quantification methods for LLMs into four main classes: (1) token-level uncertainty quantification methods; (2) self-verbalized uncertainty quantification methods; (3) semantic-similarity uncertainty quantification methods; and (4)

Read the complete text