Improving self-training under distribution shifts via anchored confidence with theoretical guarantees

Taejong Joo & Diego Klabjan · 2024

Read Improving self-training under distribution shifts via anchored confidence with theoretical guarantees by Taejong Joo & Diego Klabjan (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00586.txt.

Description

Abstract Self-training often falls short under distribution shifts due to an increased discrepancy between prediction confidence and actual accuracy. This typically necessitates computationally demanding methods such as neighborhood or ensemble-based label corrections. Drawing inspiration from insights on early learning regularization, we develop a principled method to improve self-training under distribution shifts based on temporal consistency.

Catalog record

CREATORTaejong Joo & Diego Klabjan
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00586

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving self-training under distribution shifts via anchored confidence with theoretical guarantees Taejong Joo & Diego Klabjan Abstract Self-training often falls short under distribution shifts due to an increased discrepancy between prediction confidence and actual accuracy. This typically necessitates computationally demanding methods such as neighborhood or ensemble-based label corrections. Drawing inspiration from insights on early learning regularization, we develop a principled method to improve self-training under distribution shifts based on temporal consistency. Specifically, we build an uncertainty-aware temporal ensemble with a simple relative thresholding. Then, this ensemble smooths noisy pseudo labels to promote selective temporal consistency. We show that our temporal ensemble is asymptotically correct and our label smoothing technique can reduce the optimality gap of self-training. Our extensive experiments validate that our approach consistently improves self-training performances by 8% to 16% across diverse distribution shift scenarios without a computational overhead. Besides, our method exhibits attractive properties, such as improved calibration performance and robustness to different hyperparameter choices. 1 Introduction In this work, we address the challenge of adapting pre-trained neural networks at test time under distribution shifts, a problem known as test-time adaptation (TTA) or source-free domain adaptation (SFDA). Distribution shifts—where a model trained on one distribution is then tested on a different one—are ubiquitous in many practical scenarios due to demographic subpopulation shift [ 1 ] and changes in data collection environments [ 2 , 3 , 4 ] . Despite the robust performance of neural networks under independent and identically distributed (i.i.d.) settings, they often suffer from substantial performance degradation under such shifts [ 5 , 6 ] . Recently, TTA and SFDA have proven their effectiveness in resolving these critical issues by effectively leveraging information about the distribution shifts contained in unlabeled samples given at the test time. Self-training is the basis of many state-of-the-art methods in TTA and SFDA [ 7 , 8 , 9 ] , utilizing pseudo labels generated from the model’s own predictions to train a model on unlabeled samples [ 10 ] . Since pseudo labels are regarded as true labels in self-training, the success of self-training methods highly depends on how to filter incorrect pseudo labels to prevent self-confirmation bias [ 11 ] . This issue has been effectively handled via simple confidence-based thresholding in i.i.d. settings [ 12 , 13 , 14 ] . However, the distribution shifts make it hard to filter incorrect pseudo labels due to high noise rates even under high threshold [ 15 ] . Thus, sophisticated methods filter incorrect pseudo labels based on a neighborhood structure of the data [ 16 , 17 , 18 ] and consistency of multiple predictions under different models [ 19 ] or augmentations [ 20 ] (cf. Section 5 ), which are computationally intensive by nature. Recent insights in [ 8 ] suggest an alternative strategy can be also effective: promoting temporal consistency can enhance self-training performance in SFDA without the computational burden of previous methods. The temporal consistency regularizer, so called early learning regularization (ELR) [ 21 ] , was originally developed to address neural networks’ tendency to learn clean information first and then gradually memorize noisy labels [ 22 , 23 ] ; this setting is naturally connected to self-training scenarios when we regard the pseudo labels as random noisy labels. However, the impacts of ELR on self-training have not been fully understood. Also, since ELR does not consider unique characteristics of distribution shifts, we aim to answer the following question: Is there any principled way to improve the way of memorizing all past predictions tailored self-training under distribution shifts? In this work, we show that the answer is affirmative by proposing Anchored Confidence ( AnCon ) that uses confident predictions to support a generalized notion of temporal consistency . Specifically, we construct a generalized temporal ensemble, which weighs predictions based on predictive uncertainty, and then use the ensemble as a smoothing vector in label smoothing [ 24 ] . Then, through rigorous theoretical analyses, we show that our simple heuristic for the generalized temporal ensemble is asymptotically correct and that the label smoothing formulation can reduce the optimality gap. As a result, AnCon can correct wrong pseudo labels without expensive computations and can be easily applied to self-training methods by replacing one-hot pseudo labels with smooth pseudo labels, unlike neighborhood-based or centroid methods. Through extensive experiments, we show that AnCon improves self-training under diverse distribution shift scenarios and posses many attractive properties. Our contribution can be summarized as follows: 1) We develop AnCon , which is the first algorithm that attempts to improve self-training under distribution shifts by generalizing a notion of temporal consistency with theoretical guarantees; 2) Without any additional forward passes or neighborhood search, AnCon improves self-training performances 8% and 16% under domain shifts and image corruptions, respectively; 3) Remarkably, we also show that AnCon significantly improves calibration performance and is robust with respect to model selection methods and hyperparameter choices. 2 Background Notation and setup For an input space {\mathcal{X}}\subseteq\mathbb{R}^{d} and a label space {\mathcal{Y}}=[K]:=\{1,2,\cdots,K\} , we define X and Y to be random variables of input and output with probability densities p_{X} and p_{Y} , respectively. We also define a neural network f(\cdot;\theta):{\mathcal{X}}\rightarrow\triangle^{K-1} parameterized by a parameter \theta\in\mathbb{R}^{p} where \triangle^{K-1} is the probability simplex with K elements. Our goal is to minimize the cross-entropy loss \min_{\theta\in\mathbb{R}^{p}}l(\theta):=\mathbb{E}_{XY}[H(f(X;\theta),p_{Y|X})] where H(f(x;\theta),p_{Y|X=x}):=-\sum_{k\in[K]}p(Y=k|x)\log f_{k}(x;\theta) . In the SFDA setting, we are given an initial parameter \theta_{0} that is trained on a different data generating distribution (X^{\prime},Y^{\prime}) , e.g., \theta_{0}\in\argmin_{\theta\in\mathbb{R}^{p}}\mathbb{E}_{X^{\prime}Y^{\prime}}[H(f(X^{\prime};\theta),p_{Y^{\prime}|X^{\prime}})] . We can think about this setting as either (1) (X^{\prime},Y^{\prime}) being pre-training data and \theta_{0} the foundation model with the task of fine-tuning the model on unlabeled X or (2) a transfer learning problem with (X^{\prime},Y^{\prime}) being the source domain data and X the target domain. Here, we assume only covariate shift without concept shift; that is, p_{X}\neq^{D}p_{X^{\prime}} but p_{Y|X}=^{D}p_{Y^{\prime}|X^{\prime}} . Even in this case, we note that suboptimality under the distribution shift, i.e., \min_{\theta\in\mathbb{R}^{p}}l(\theta)-l(\theta_{0}) , can be large. Self-training In this work, we tackle the distribution shifts by using the self-training method that replaces the true label Y(x) by the pseudo label, i.e., \min_{\theta_{new}}\hat{l}(\theta_{new};\theta)=\mathbb{E}_{X}[-\log f_{\hat{Y}(x;\theta)}(x;\theta_{new})] where \hat{Y}(x;\theta):=\argmax_{k\in[K]}f_{k}(x;\theta) is a pseudo label under \theta . Specifically, the algorithmic framework of self-training is as follows: given \theta_{0} , we iteratively find model parameter \theta_{m+1} for m=0,1,\cdots with \theta_{m+1}\in\argmin_{\theta_{new}}\hat{l}(\theta_{new};\theta_{m}) . For later use, we also define a prediction confidence c(x;\theta):=\max_{k\in[K]}f_{k}(x;\theta) and \theta_{0:m}:=(\theta_{0},\cdots,\theta_{m}) . Early learning regularization In the learning from noisy labels (LFN) scenario, [ 21 ] identified an "early-learning phenomenon" where neural networks initially learn information contained in clean labels before gradually memorizing noisy labels, leading to a performance deterioration as training progresses. To mitigate this issue, ELR penalizes predictions that deviate from earlier predictions by defining a target network from the past predictions: \bar{f}_{ELR}(x;\theta_{0:m}):=\sum_{j=0}^{m}(1-\gamma)\cdot\gamma^{m-j}f(x;\theta_{j}) . Then, adding an auxiliary loss of L_{ELR}(\theta;\theta_{0:m})=\mathbb{E}_{X}[\log(1-f(X;\theta)^{T}\bar{f}_{ELR}(X;\theta_{0:m}))] to \hat{l}(\theta;\theta_{m}) can prevent memorization of noisy labels while preserving correct patterns. Notably, this insight has recently been confirmed to be applicable in the SFDA setting by [ 8 ] . This observation is appealing because ELR can be efficiently implemented by reusing past predictions without additional forward passes or neighborhood searching, unlike dominant methods in SFDA [ 16 , 17 , 18 , 19

Read the complete text