On Calibration in Multi-Distribution Learning

Rajeev Verma [email protected], Volker Fischer [email protected], Eric Nalisnick [email protected] · 2024

Read On Calibration in Multi-Distribution Learning by Rajeev Verma [email protected], Volker Fischer [email protected], Eric Nalisnick [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-14142.txt.

Introduction

Learning one predictor that serves several data distributions at once is now common in work on robustness and fairness. Rajeev Verma, Volker Fischer, and Eric Nalisnick ask in 2024 whether such a predictor can stay calibrated, its stated confidence matching reality, on every distribution at once. Using classic decompositions of scoring losses, they derive the best possible rule for this setting.

That ideal rule keeps worst-case loss minimal, yet its calibration errors can still differ across distributions. A built-in trade-off between calibration and refinement persists even at the optimum. The authors warn designers to watch for uneven trustworthiness when one model serves many populations.

Catalog record

CREATORRajeev Verma [email protected], Volker Fischer [email protected], Eric Nalisnick [email protected]
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-14142

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On Calibration in Multi-Distribution Learning Rajeev Verma [email protected], Volker Fischer [email protected], Eric Nalisnick [email protected] Abstract Modern challenges of robustness, fairness, and decision-making in machine learning have led to the formulation of multi-distribution learning (MDL) frameworks in which a predictor is optimized across multiple distributions. We study the calibration properties of MDL to better understand how the predictor performs uniformly across the multiple distributions. Through classical results on decomposing proper scoring losses, we first derive the Bayes optimal rule for MDL, demonstrating that it maximizes the generalized entropy of the associated loss function. Our analysis reveals that while this approach ensures minimal worst-case loss, it can lead to non-uniform calibration errors across the multiple distributions and there is an inherent calibration-refinement trade-off, even at Bayes optimality. Our results highlight a critical limitation: despite the promise of MDL, one must use caution when designing predictors tailored to multiple distributions so as to minimize disparity. 1 Introduction The ever increasing applicability of machine learning systems in safety-critical applications requires robustness, fairness, and reliable decision-making. Contrary to traditional machine learning, which is operationalized by assuming samples originate from a single fixed distribution, multi-distribution learning (MDL) aims to learn a predictor across a set of distributions. Moreover, the predictor should perform well for any distribution in the set. MDL has the potential to provide a unified framework for tackling the aforementioned challenges. For example, to improve robustness, MDL can be applied by specifying the set as distributions within a small perturbation of the training distribution. This setup is also known as distributionally robust optimization , and thus MDL can be seen as a generalization. To turn to fairness, equitable decisions must be made across a collection of heterogeneous populations defined by protected group-membership. This setting can be considered a form of MDL by taking the heterogeneous populations to be the set of interest. A related problem is that of decision-making where the learner’s loss is aggregated by averages (or expectations). However, taking averages can cause disparate performance across populations ( Blum et al., 2021 ) and also cause a fair model to become unfair ( Hashimoto et al., 2018 ) . A unified study of MDL in learning theory ( Haghtalab et al., 2022 ; Blum et al., 2017 ; Mohri et al., 2019 ; Zhang et al., 2024 ) extends the celebrated agnostic learning framework ( Valiant, 1984 ; Kearns et al., 1992 ) and has primarily focused on the sample complexity of learning across the multiple distributions. For instance, Haghtalab et al. (2022) propose adaptive sampling to achieve data-efficient MDL algorithms. However, there are no studies on the calibration ( Dawid, 1982 ) of predictive models under the MDL framework. Calibration properties of predictors under the traditional single-distribution framework are well-documented ( Degroot & Fienberg, 1983 ; Bröcker, 2009 ) , and have long guided the practice of machine learning. Besides the general discriminativeness (refinement) of predictors, calibration has important consequences in safety-critical applications, especially to drive equitable decision-making ( Grünwald, 2018 ; Dwork et al., 2021 ; Rothblum & Yona, 2021 ; Zhao et al., 2021 ) . For example, perfect calibration guarantees that a healthcare facility can reliably use the same threshold for everyone on the predicted risk of a disease to recommend further expensive tests. Since the motivations of MDL of robustness and fairness aligns with the usual motivations of calibration, it is imperative to study how well these two properties align theoretically. In this work, we take a foundational approach and study the calibration of a predictor under the MDL framework, independent of specific learning algorithms. One of the challenges in studying calibration for MDL is that the MDL learning algorithms are usually randomized complicating their tractable study. Thus, we also give an attainable lower bound for MDL and study the calibration at this lower bound. Our work reveals the existence of non-uniform calibration disparities across different distributions in MDL—a problem that resembles calibration disparity across different (sub)groups in the single distribution literature ( Hebert-Johnson et al., 2018 ) . However, while it can be eliminated theoretically and algorithmically for the single-distribution case, calibration disparity in MDL is fundamental and there exists a calibration-refinement trade-off . Thus, our analysis shows that the promise of MDL to aid robustness and equitable decision-making is tenuous. Summary of Results 1. Proposition 3.2 states the attainable lower bound on the error of a predictor in MDL. 2. Proposition 4.1 and the next results establish calibration error bound for any distribution in MDL and characterize the calibration disparity in terms of the performance disparity for a predictor across multiple distributions—arguing that if a predictor leads to low error for some distribution, it compensates for it by increasing the calibration error. Proposition 4.4 states the decision-theoretic consequence of calibration in MDL, and argue that decision-makers need to be careful to fully realize the potential of MDL. 4. Finally, Section 6 states the consequences of our findings to the use case of distributional robust optimization and fairness, two popular applications of MDL. 2 Background In this section, we motivate MDL framework and give background for calibration and refinement and its connection with risk minimization. Notation and Single Distribution Learning Traditionally, practical machine learning starts with an access to some finite dataset D=\{\left({\bm{x}}_{i},y_{i}\right)\}_{i=1}^{N} , where \left({\bm{x}},y\right)\in\mathcal{X}\times\mathcal{Y} . The goal, then, is to design a predictor h_{\theta}:\mathcal{X}\rightarrow\Delta^{|\mathcal{Y}|} , \Delta^{|\mathcal{Y}|} denotes a simplex over the label space \mathcal{Y} where we assume |\mathcal{Y}| is finite. Here, this predictor is parameterized by \theta , and is a member of some hypothesis class \mathcal{H}=\{h_{\theta}:\theta\in\Lambda\} . However, moving forward we will suppress the dependence of the hypothesis h on \theta . Given some pre-specified loss \ell:\mathcal{Y}\times\Delta^{|\mathcal{Y}|}\rightarrow[0,M] , one aims to find h^{*} as a solution to the following optimization problem h^{*}\in\arg\min_{\mathcal{H}}\frac{1}{N}\sum_{i=1}^{N}\left[\ell\left(y_{i},h\left({\bm{x}}_{i}\right)\right)\right], To elaborate, we assume the loss function \ell is bounded. If the loss function is not bounded from above, e.g. the log-loss -\log h\left({\bm{x}}\right) , we assume it can be bounded by re-defining it as \min\left\{M,-\log h\left({\bm{x}}\right)\right\} , for example. The above exercise makes sense if one is to assume some base measure P on \mathcal{X}\times\mathcal{Y} , as then the law of large numbers suggests that the goodness of the predictor h^{*} as measured by the loss \ell is controlled in terms of the expectation over the population P , i.e. the expected population loss, also referred to as the risk denoted as \mathcal{R}_{P}=\mathbb{E}_{P}\left[\ell\left({\textnormal{y}},h\left({\mathbf{x}}\right)\right)\right] , is controlled. We use {\mathbf{x}} and y to denote random variables. 2.1 Calibration and Refinement While the learning procedure stated above controls the risk, decision makers also care about the confidence of a predictor. The output of the hypothesis h\left({\bm{x}}\right)\in\Delta^{|\mathcal{Y}|} is considered as the forecast over \mathcal{Y} , and to reliably use it as a notion of confidence, the decision makers aim to certify the calibration property of h as defined below: Definition 2.1 . (Canonical Calibration) . Given d some divergence measure, e.g. squared error, a confidence predictor h:\mathcal{X}\rightarrow\Delta^{|\mathcal{Y}|} is said to be (perfectly) canonically calibrated if the following holds true: \mathbb{E}_{\left({\mathbf{x}},{\textnormal{y}}\right)\sim P}\left[d\left(P\left[\ {\textnormal{y}}\ |\ h\left({\mathbf{x}}\right)\ \right],h\left({\mathbf{x}}\right)\right)\right]=0. (1) Canonical calibration asserts that on average the confidence predictor means what it says, i.e. P\left[{\textnormal{y}}\ |\ h\left({\mathbf{x}}\right)\right]=h\left({\mathbf{x}}\right). When \mathcal{Y}=\{0,1\} , this translates that if the predictor outputs that

Read the complete text