Does Calibration Affect Human Actions?

Meir Nizri, Amos Azaria, Chirag Gupta, Noam Hazon · 2025

Read Does Calibration Affect Human Actions? by Meir Nizri, Amos Azaria, Chirag Gupta, Noam Hazon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18317.txt.

Introduction

Meir Nizri and co-authors test whether showing people better-calibrated probabilities actually changes what they decide. Their 2025 study runs a human-computer interaction experiment where non-expert users act on a classifier's predictions. Calibration, making stated confidence match real accuracy, is often assumed to build trust and better choices.

Calibration alone did not do the job. The decisive addition was a correction drawn from prospect theory, the behavioral economics of how people weigh gains and losses. With that adjustment, users' decisions aligned much more closely with the model's predictions. Yet when asked directly, users did not report trusting the model more. The gap between stated trust and revealed trust is the paper's sharpest finding. Designers of decision aids should measure behavior, not questionnaires. And a technically sound probability, the authors show, can still be psychologically misformatted for the human who must act on it.

Catalog record

CREATORMeir Nizri, Amos Azaria, Chirag Gupta, Noam Hazon
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-18317

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Does Calibration Affect Human Actions? Meir Nizri, Amos Azaria, Chirag Gupta, Noam Hazon Abstract Calibration has been proposed as a way to enhance the reliability and adoption of machine learning classifiers. We study a particular aspect of this proposal: how does calibrating a classification model affect the decisions made by non-expert humans consuming the model’s predictions? We perform a Human-Computer-Interaction (HCI) experiment to ascertain the effect of calibration on (i) trust in the model, and (ii) the correlation between decisions and predictions. We also propose further corrections to the reported calibrated scores based on Kahneman and Tversky’s prospect theory from behavioral economics, and study the effect of these corrections on trust and decision-making. We find that calibration is not sufficient on its own; the prospect theory correction is crucial for increasing the correlation between human decisions and the model’s predictions. While this increased correlation suggests higher trust in the model, responses to “Do you trust the model more?" are unaffected by the method used. 1 Introduction Machine learning (ML) models often assume the role of assistants rather than sole decision-makers, aiding humans in making the final decision that leads to a tangible action. However, in some areas where decisions are made, such as weather forecasts, medical diagnoses, risk assessments, fraud detection, and financial forecasting, it is essential that the model not only provides a predicted class but also associates its classification with a probability. In a classification setting, confidence takes the form of reporting a probability associated with the predicted class or probability. As a simple motivating example, consider a couple planning an outdoor wedding. It is insufficient for a model to predict that there will be no rain on their wedding day; the model must also provide a probability. It is quite likely that even if this probability is less than 50%, but not negligible (e.g., 30%), the couple will organize their wedding indoors. That is, the couple is not concerned only with whether the model predicts rain or not; they are interested in the probability of rain. Indeed, classification models are typically inherently probabilistic; that is, their raw outcome is a probability (or a set of probabilities) that is translated into a class prediction. It is common practice to present a probability to the user alongside each prediction; this probability is often referred to as a confidence score . When confidence scores are reported, the human decision-maker consuming them must use them in a way that is consistent with what the machine learning expert had in mind when designing the model. The first consideration we can make is whether the model is calibrated. That is, does the probability of the predicted class align with the confidence score provided? Many out-of-the-box approaches lead to classifiers that produce over-confident predictions. For instance, a model might predict a rainy day with a 90\% probability. However, if we analyze all the model predictions calling for rain with a 90\% probability, only 70\% might correspond to actual rainy days. Specifically, it is well known that modern neural networks suffer from over-confident predictions Guo et al. 2017 . A well-calibrated model should produce a probability that accurately reflects the true likelihood that the event will occur. For example, if a well-calibrated model predicts a rainy day with an 80% probability, it should rain on approximately 80% of the days with such a prediction. We note that if, in practice, either a higher or lower percentage of days are rainy than predicted, the model is not considered calibrated. Several papers have proposed methods to produce classifiers that are calibrated; see Gupta 2023 for a recent review of the literature. For completeness, we also review some calibration methods and their quantitative evaluation metrics in Appendix B and C respectively. However, despite the existence of many different calibration methods, which compose an entire field in machine learning, little is known about how humans react to a calibrated model in comparison to the original uncalibrated model. A human-focused evaluation of calibration is natural to ask for. Such an evaluation offers multiple benefits as noted next. First, human evaluation provides a means to assess the real-world impact and effectiveness of the model. While traditional evaluation metrics offer quantitative insights, human subjects can provide qualitative feedback that captures nuances and contextual considerations that may be missed by automated assessments alone ( Bostrom and Yudkowsky 2018 ) . Second, human evaluation allows for assessing the usability and user experience of the model. Feedback from humans can provide valuable insights into how well the model meets the needs and expectations of users ( Kaplan and Haenlein 2019 ) . Furthermore, human evaluation helps to ensure that the model aligns with ethical and legal standards. It allows for the examination of the fairness, transparency, and accountability of the model, as well as its compliance with regulations and guidelines ( Mittelstadt et al. 2016 ) . 1.1 An additional layer of Kahneman and Tversky’s prospect theory In this paper, we introduce an additional layer on top of the existing calibration methods, based on the prospect theory from behavioral economics ( Kahneman and Tversky 1979 ; Kahneman and Tversky 1992 ) . Prospect theory provides a behavioral or psychological framework that seeks to explain how individuals perceive and evaluate probabilities of potential gains and losses. We posit that considering prospect theory is crucial since it is the perceived probability that influences individuals’ decision-making and trust in the system, and not the actual reported probability. Prospect theory uses a weighting function to describe how people subjectively weigh probabilities. In a nutshell, events with very low probabilities (i.e., close to 0) are often perceived as more likely than they truly are, while events with very high probabilities (i.e., near 1) are perceived as less likely than they truly are. In our approach, we propose transforming each calibrated probability based on the inverse of the prospect theory weighting function to better match user perceptions. Thus, by using the inverse function, we can derive a probability that users perceive as matching the original prediction. For instance, if individuals perceive an event with a reported probability of 90\% as actually having an 80\% chance of occurring, then for an actual probability of 80\% , we would report it as 90\% to align with their perception. 1.2 Experimental method and summary of findings We performed a human study to evaluate our prospect theory-based calibration method on two distinct settings. The first domain is rain forecasting, and the second is loan approval. Both domains are commonly associated with probabilistic predictions and are easily interpretable by a general audience. We used a neural network as the uncalibrated model. We tested several calibration methods and selected the isotonic regression method, which performed better than other methods. We compare our method, which adds the prospect theory correction on top of the calibrated model, to four baselines. Namely, the uncalibrated model, the model calibrated with isotonic regression, the prospect theory correction directly over the uncalibrated model, and a random method using our method’s probabilities. In the rain-forecasting domain, we presented each participant with a rain prediction system. Following each prediction, participants were asked to indicate, assuming they are planning an outdoor activity for that day, how

Read the complete text