Sequential Harmful Shift Detection Without Labels

Salim I. Amoukou, Tom Bewley, Saumitra Mishra · 2024

Read Sequential Harmful Shift Detection Without Labels by Salim I. Amoukou, Tom Bewley, Saumitra Mishra (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-12910.txt.

Introduction

Salim Amoukou and five co-authors set out a way to spot, step by step, when a shift in data starts to harm a working learning model, with no true labels at hand. Their paper is from two thousand twenty-four.

In live use, a model's world drifts, and harm can creep in unseen. Past work could track harm over time when labels were there to count errors. The new way stands in a trained guesser of the model's error rate for the true count. Tests show the method has strong power to catch shifts, while it keeps false alarms in check. The shifts tried span field shifts, label shifts, and real drifts across place and time.

Catalog record

CREATORSalim I. Amoukou, Tom Bewley, Saumitra Mishra
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-12910

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sequential Harmful Shift Detection Without Labels Salim I. Amoukou, Tom Bewley, Saumitra Mishra Abstract We introduce a novel approach for detecting distribution shifts that negatively impact the performance of machine learning models in continuous production environments, which requires no access to ground truth data labels. It builds upon the work of Podkopaev and Ramdas (2022) , who address scenarios where labels are available for tracking model errors over time. Our solution extends this framework to work in the absence of labels, by employing a proxy for the true error. This proxy is derived using the predictions of a trained error estimator. Experiments show that our method has high power and false alarm control under various distribution shifts, including covariate and label shifts and natural shifts over geography and time. 1 Introduction When deploying a machine learning model in production, it is common to encounter changes in the data distribution, such as shifts in covariates ( Shimodaira, 2000 ) , labels ( Saerens et al., 2002 ; Lipton et al., 2018 ) or concepts ( Gonçalves Jr et al., 2014 ) . Many methods exist for detecting such distribution shifts. However, a distinct but equally important challenge is assessing whether a shift has a harmful impact on the prediction error of a given model, which may necessitate interventions such as ceasing production or retraining the model. Not all distribution shifts are harmful, but traditional methods for shift detection are unable to distinguish harmful and benign shifts. While some approaches address the specific issue of performance shift, most require access to ground truth data labels in the production environment ( Gama et al., 2013 ; Gama et al., 2014 ; Bayram et al., 2022 ) . In scenarios where predictions concern future outcomes, such as medical diagnosis or credit scoring, immediate access to labels in production is not feasible. This work focuses on the challenge of detecting harmful distribution shifts — those that increase model error in production — without requiring access to labels. As Trivedi et al. (2023) note, current methods for harmful shift detection without labels rely on disparate heuristics, often lacking a solid theoretical foundation. Such methods include proxies based on aggregate dataset-level statistics ( Deng and Zheng, 2021 ) , optimal transport mappings between training and production distributions ( Koebler et al., 2023 ) , and model-specific metrics such as input margins ( Mouton et al., 2023 ) , perturbation-sensitivity ( Ng et al., 2023 ) , disagreement-metrics ( Chen et al., 2023 ; Ginsberg et al., 2022 ) , and prediction confidence ( Guillory et al., 2021 ; Garg et al., 2022 ) . While such methods may be practically effective in certain contexts, they rely on assumptions and correlations that do not hold universally, so can provide no guarantees. Furthermore, conventional methods rely on two-sample or batch testing, which involves comparing the statistical properties of a production dataset with those of a control sample. These methods have inherent limitations, as the sample size is prespecified. This is a problem because the necessary amount of data to detect any given shift is unknown beforehand. Furthermore, in real-world scenarios, data typically arrive sequentially over time and shifts may occur either suddenly or gradually. In such scenarios, it may be desirable to detect harmful shifts as early as possible. Batch testing is ill-suited to the sequential context ( Maharaj et al., 2023 ) , as it does not accommodate the collection of additional data for retesting without adjusting for multiple testing, leading to diminished power. The most principled and relevant work to our problem is that of Podkopaev and Ramdas (2022) , which tackles the problem of sequential harmful shift detection with false alarm control but assumes the availability of ground truth labels in production. Our work builds on the foundation established by Podkopaev and Ramdas (2022) , extending the methodology to detect harmful shifts in unlabeled production data while effectively managing false alarms. Our approach leverages a secondary model to estimate the errors of the primary model. While learning such a model might seem challenging at first, consider a situation where the primary model performs well overall but struggles with specific data subgroups. Sagawa et al. (2019) demonstrate that this phenomenon can occur in natural distributions. In such cases, learning to predict “error given X” might be easier than the primary task of predicting “Y given X”, because the error estimator only needs to identify those subgroups where the primary model struggles. This approach has shown promise in recent studies ( Zrnic and Candès, 2024 ; Amoukou and Brunel, 2023 ) . More generally, Zrnic and Candès (2024) note that predicting the magnitude of the error, rather than its direction, is often easier. Furthermore, our approach is based on estimating the proportion of high-error observations over time. For this task, the error estimator does not need to be very accurate; it only needs to assign higher values to observations with higher errors. That is, the estimator only needs to correctly order most observations from low to high error, which is easier than precisely predicting the error itself. We demonstrate in Section 4.1 that even a relatively inaccurate error estimator can be effective at identifying high-error observations, and thus provides the functionality required by our framework. Although this paper uses a learned error estimator’s predictions as a proxy for error, we note that any scalar function correlated with error could suffice to isolate high-error observations. For example, for a well-calibrated binary classification model, we could instead use that model’s predicted probability, tracking observations with predictions near 0.5 to identify uncertain predictions. Figure 1: Overview of the proposed approach. Left: calibrating an estimated error threshold to separate low/high true errors. Right: sequentially tracking production data exceeding the threshold and raising an alarm upon a significant increase. Figure 1 gives an overview of our approach. We first fit the secondary error estimator model to predict the error of the primary model, then use labeled data to calibrate an estimated error threshold ( — ) that separates observations with low ( x ) and high ( x ) true error as fully as possible. We run the error estimator on all observations encountered in production and continually monitor the proportion of observations whose estimated error falls above the threshold. We raise an alarm when this exceeds the rate of high-error observations ( x ) in the calibration set plus a tolerance threshold \epsilon_{\text{tol}} and correction terms to deal with the sequential setting and account for uncertainty in the estimates. In the example shown, this occurs at time t=10 . The rest of this paper is organized as follows. Section 2 outlines the problem definition and Section 3 provides an overview of the foundational work of Podkopaev and Ramdas (2022) as background. Section 4 is dedicated to the presentation and theoretical analysis of our sequential statistical test. Section 5 demonstrates the empirical efficacy of our method, showcasing its strong detection capabilities and controlled false alarm rates across various types of harmful shift. 2 Problem Definition Let \mathcal{X} and \mathcal{Y} be input and label spaces, f:\mathcal{X}\to\mathcal{Y} be a predictive model, and \ell:\mathcal{Y}^{2}\to\mathcal{E} be a measurable and bounded error function that is selected for monitoring

Read the complete text