Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles
Yuxi Xia, Pedro Henrique Luz de Araujo, Klim Zaporojets, Benjamin Roth · 2025
Read Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles by Yuxi Xia, Pedro Henrique Luz de Araujo, Klim Zaporojets, Benjamin Roth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03991.txt.
Introduction
Yuxi Xia and co-authors study what shapes calibration in large language models, in this 2025 paper. Calibration means a model's stated confidence matches how often it is right, which matters for safe use. The team sets a controlled test bed with twelve models and four prompt styles. They ask whether pooling answers from several models, and choosing the right training loss, can sharpen confidence estimates.
Their framework trains a helper model that reads the agreement among several models' answers. Agreement plus a focal loss both improved calibration over baselines. Few-shot prompts suited the helper approach best. Helpers stayed reliable as accuracy shifted, beating the models' own stated confidence. The authors map these factors to guide safer deployment of language models.
Catalog record
| CREATOR | Yuxi Xia, Pedro Henrique Luz de Araujo, Klim Zaporojets, Benjamin Roth |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 105 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-03991
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles Yuxi Xia, Pedro Henrique Luz de Araujo, Klim Zaporojets, Benjamin Roth Abstract Calibration, the alignment between model confidence and prediction accuracy, is critical for the reliable deployment of large language models (LLMs). Existing works neglect to measure the generalization of their methods to other prompt styles and different sizes of LLMs. To address this, we define a controlled experimental setting covering 12 LLMs and four prompt styles. We additionally investigate if incorporating the response agreement of multiple LLMs and an appropriate loss function can improve calibration performance. Concretely, we build Calib-n, a novel framework that trains an auxiliary model for confidence estimation that aggregates responses from multiple LLMs to capture inter-model agreement. To optimize calibration, we integrate focal and AUC surrogate losses alongside binary cross-entropy. Experiments across four datasets demonstrate that both response agreement and focal loss improve calibration from baselines. We find that few-shot prompts are the most effective for auxiliary model-based methods, and auxiliary models demonstrate robust calibration performance across accuracy variations, outperforming LLMs’ internal probabilities and verbalized confidences. These insights deepen the understanding of influence factors in LLM calibration, supporting their reliable deployment in diverse applications. 1 1 1 Code and data will be released upon acceptance Figure 1: Overview of calibration training of Calib-n ( n indicates the number of LLMs that provide responses). We consider the effect of different prompt styles on calibration and design four diverse prompts to query n target LLMs to provide answers to a question. n joint strings of the question with each answer presenting the response agreement of LLMs are passed to the auxiliary model to generate probabilities for each answer. The auxiliary model is optimized with three loss functions respectively on the correctness of the LLM answers. Brier score (one of four metrics) is used to evaluate the calibration performance of the auxiliary model. 1 Introduction Improving the calibration of Large Language Models (LLMs), i.e., aligning the model’s confidence with the accuracy of its predictions, can maintain their reliability, usability, and ethical deployment in domains like medicine, law, and education Guo et al. 2017a ; Jiang et al. 2021 ; Geng et al. 2024 . As LLMs are increasingly integrated into decision-making processes, poor calibration can amplify risks of misinformation, propagate biases, and foster over-reliance among users Raji et al. 2020 . Recent studies Ni et al. 2024 indicate that LLMs struggle to accurately express their internal confidence in natural language, particularly in the form of verbalized confidence Tian et al. 2023 . Liu et al. 2024 addresses this issue by training a linear layer to adjust the hidden states of the LLM’s final layer for confidence prediction, but this approach only applies to LLMs with accessible weights. In contrast, Ulmer et al. 2024 introduces an auxiliary model for confidence estimation using only the generations of the target LLM but is constrained by its evaluation on just two LLMs and two prompt styles. These narrow experimental setups limit the generalization of their findings. Our study tackles this limitation by comprehensively examining the generalization of different methods over 12 LLMs and four prompt styles. Additionally, we investigate the influence of diverse factors on the calibration of LLMs, which includes response agreements among LLMs, three loss functions, and four prompt styles. Concretely, we introduce Calib-n (illustrated in Fig. 1 ), a novel framework aggregating responses from multiple LLMs ( n indicates the number of LLMs) to train a single auxiliary model for confidence estimation. Calib-n captures inter-model agreement, mitigating overfitting and reducing overconfidence associated with individual LLMs for calibration Kim et al. 2023 . Except for standard binary cross-entropy (BCE) loss, we incorporate focal (FL) Lin et al. 2020 and AUC surrogate (AUC) Yuan et al. 2021 loss functions that are effective for improving the calibration of non-transformer type of neural networks Mukhoti et al. 2020 ; Moon et al. 2020 . Finally, we systematically study the effects of prompt styles, testing four diverse prompt types: Verbalized Tian et al. 2023 , Chain-of-Thought (CoT) Wei et al. 2022 , Zero-shot, and Few-shot prompts. Our experiments span four open-ended question answering datasets and 12 LLMs, including seven small models (2–9B parameters) and five large models (27–72B parameters) from five distinct model families. The results indicate that no single method consistently outperforms all others across the various models, datasets, and prompt configurations. Yet, after aggregating the results and counting the number of overall wins for each method, we uncover new insights regarding the calibration factors of the analyzed settings: • Response Agreement: By leveraging inter-model response agreement, Calib-n outperforms the state-of-the-art baselines. • Loss Functions: FL loss improves calibration compared to BCE and AUC losses, demonstrating its effectiveness for both Calib-1 (i.e., using responses from one LLM) and Calib-n. Calib-1 trained with FL yields the best results. • Prompt Style: We find that the effectiveness of methods is highly influenced by the prompt styles, with few-shot prompts proving to be the most beneficial for training auxiliary models. • Accuracy-Calibration Correlation: With accuracy variances caused by different dataset complexities, prompt styles and models, we find that auxiliary models maintain robust calibration performance across accuracy changes, in contrast to the fluctuating calibration of LLMs that rely on internal probabilities and verbalized confidences. Our findings underscore the importance of reexamining calibration strategies for LLMs, suggesting the use of response agreement, focal loss and few-shot prompts to improve the calibration of LLMs. 2 Related Work Calibration The concept of calibration of neural networks was introduced by Guo et al. 2017b . Lin et al. 2022 show that GPT-3 can learn to express uncertainty about its own answers without the use of logits. Later, Tian et al. 2023 demonstrate that verbalized confidence is generally better calibrated than the conditional probabilities w.r.t the consistency of the LLMs. However, Zhang et al. 2024 show that LLM probabilities and verbalized confidence tend to overly concentrate within a fixed range. Similarly, Ni et al. 2024 analyze and compare probabilistic and verbalized perceptions of the knowledge boundaries of LLM, highlighting their challenges in confidence estimation. To address these issues, Liu et al. 2024 train a linear layer to adjust the last layer’s hidden states of LLMs for confidence generation. Ulmer et al. 2024 propose a method to estimate LLM confidence based only on textual input and output. However, none of these works perform a comprehensive analysis of different influence factors in LLM calibration . These factors can be loss functions, response agreement and prompt styles. Mukhoti et al. 2020 prove that focal loss Lin et al. 2020 can improve the calibration of neural networks. AUC surrogate loss Yuan et al. 2021 and correctness ranking loss Moon et al. 2020 calibrate models by adjusting the ranking of logits to ensure positive samples are ranked higher than negative samples. Kim et al. 2023 suggest combining the predictions from multiple models (response agreement) can mitigate overfitting and reduce overconfidence for calibration. Min et al. 2022 ; Wang et al. 2023 ; Chen et al. 2024 showcase that LLMs output is highly dependent on the prompt and different prompt styles can significantly influence the performance of LLMs. Our work comprehensively studies the influence