Characterizing and Correcting Effective Target Shift in Online Learning

Ziyan Li, Naoki Hiratani · 2026

Read Characterizing and Correcting Effective Target Shift in Online Learning by Ziyan Li, Naoki Hiratani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07886.txt.

Introduction

Ziyan Li and Naoki Hiratani expose a hidden distortion in online learning, where a model updates as data streams in rather than training on a fixed set. Studying kernel regression, they derive a closed form for the function that online learning actually produces and compare it with the offline answer.

Online kernel regression, they show, is exactly equivalent to offline regression fed shifted, inaccurate target outputs: an implicit target shift baked into the teaching signal. Correcting for that shift, in closed form or iteratively, makes online learning provably recover the same predictor as offline training. On continual image classification benchmarks, online gradient descent with corrected targets even beat learning with the true targets.

Catalog record

CREATORZiyan Li, Naoki Hiratani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07886

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Characterizing and Correcting Effective Target Shift in Online Learning Ziyan Li, Naoki Hiratani Abstract Online learning from a stream of data is a defining feature of intelligence, yet modern machine learning systems often struggle in this setting, especially under distributional shift. To understand its basic properties, we study the relationship between online and offline learning in the context of kernel regression. We derive a closed-form expression for the function learned by online kernel regression, revealing that online kernel regression is equivalent to offline regression with shifted, inaccurate target outputs. Conversely, we show that by compensating for this effective shift in the teaching signal through target correction, online kernel-based learning can provably learn the same predictor as its offline counterpart. We derive both a closed-form expression for this target correction and an iterative form that can be applied sequentially. Applying this framework to image classification tasks on CIFAR-10 and CORe50, we show that online stochastic gradient descent with iteratively corrected targets outperforms learning with the true targets in continual learning settings. This work therefore provides a basic framework for analyzing and improving online learning in non-stationary environments. 1 Introduction Humans and animals can adaptively learn new skills and knowledge from streams of sensory inputs and environmental feedback. This online form of learning in non-stationary environments is becoming increasingly important in machine learning applications, including large language model fine-tuning [ 39 , 7 ] , robotics [ 33 , 58 ] , and medical diagnosis [ 30 ] . However, neural network-based models typically require many epochs of updates over the same static dataset, using gradients averaged over many samples. Consequently, in the presence of large distributional shifts or continual changes in the task objective, neural networks often struggle to learn in an online manner and suffer from catastrophic forgetting [ 41 , 56 , 20 , 39 ] . There has been substantial empirical progress in developing methods to mitigate forgetting in continual learning [ 17 , 26 , 60 , 49 , 6 , 42 ] , and there has also been important theoretical progress [ 4 , 32 , 12 , 15 , 34 ] . Nevertheless, the theoretical basis of forgetting remains incompletely understood, especially in the stricter setting of online continual learning, where data must be processed in a single pass [ 38 , 1 , 40 ] . Traditional online learning theory [ 65 , 48 , 11 ] provides rigorous regret bounds for streaming data, but these analyses often assume convex objectives or stationary distributions and therefore do not directly capture the dynamics of modern deep learning under distribution shift. Thus, we still lack a mechanistic understanding of how the sequential nature of online updates alters the learned predictor relative to a joint, multi-task offline ideal. Here, we address why online continual learning is structurally difficult, and how to mitigate this difficulty, by asking: what kind of offline learning is an online learner actually performing? We address this question primarily in the context of kernel regression [ 51 , 23 ] and its online formulation [ 18 , 8 , 27 , 14 , 53 ] . This provides a theoretically grounded framework that is also intrinsically linked to the training dynamics of wide deep neural networks via the Neural Tangent Kernel (NTK) [ 24 , 31 , 2 ] . First, we rewrite online kernel regression as a closed-form offline expression. This highlights the directional, causal sample-to-sample interactions inherent in online learning, contrasting with the bi-directional inter-sample interactions of offline batch learning. Importantly, this formulation allows us to compare online and offline learning directly. We reveal that online kernel regression is mathematically equivalent to offline kernel regression trained on shifted targets, and we provide a closed-form expression for these effective shifts. Second, we reveal that it is possible to reverse this effective label shift by actively correcting the teaching signals used during online learning. Specifically, we prove that there exists a label correction that allows online kernel regression to exactly achieve the performance of its offline counterpart. While this exact correction requires a priori knowledge of all future samples, we derive an iterative approximation that respects causality. Finally, we apply this target correction framework heuristically to non-linear models outside the strict kernel regime. By using corrected targets based on the evolving empirical NTK as training signals for mini-batch stochastic gradient descent (SGD), we evaluate deep neural networks on the CIFAR-10 [ 29 ] and CORe50 [ 37 ] datasets. We show that in continual learning settings, target correction with periodic NTK updates significantly outperforms vanilla SGD. This result provides empirical support for a counterintuitive conclusion from our theory: online continual learning can be improved by training on corrected target labels rather than the ground-truth targets. 2 Related Work The online implementation of kernel regression has been studied extensively [ 27 , 18 , 8 , 14 , 53 ] . Various data sub-sampling and dictionary-building methods to keep these algorithms memory-efficient have also been proposed [ 10 , 50 , 36 , 9 ] . Theoretical analyses of convergence rates have demonstrated that, in the absence of temporal distribution shifts, online kernel regression is competitive with its offline counterpart under appropriate step size choices [ 59 , 55 , 11 , 57 , 61 ] . However, these works primarily focus on independent and identically distributed (i.i.d.) settings. In contrast, our work explicitly addresses non-stationary environments, including class-incremental learning. While online learning in non-stationary environments has been explored, particularly in online convex optimization settings [ 65 , 5 ] , our approach provides a fine-grained sample level analysis of the sub-optimality. The concept of an effective label shift is conceptually adjacent to influence functions, a diagnostic tool used to estimate the impact of individual training points on model predictions [ 21 , 28 ] . The key distinction lies in the direction of the analysis: while influence functions measure how the model’s error changes when a data point is perturbed, our effective target shift measures how the data itself is effectively perturbed by the online learning process. Furthermore, our target correction method finds parallels in label smoothing [ 54 , 43 ] and knowledge distillation [ 22 ] , where training targets are softened to improve generalization. Unlike global target modifications employed in these methods, here we provide sample-level vectorized target correction to exactly reverse the bias stemming from online learning. This method is also conceptually similar to reward shaping [ 13 , 44 , 62 ] in the reinforcement learning (RL) literature, which accelerates learning by augmenting the environmental reward. In the context of continual learning, our target correction is related to methods for mitigating catastrophic forgetting by regularizing the weights to protect prior knowledge [ 26 , 60 ] or by sparsifying/orthogonalizing gradients to prevent interference [ 17 , 52 , 4 ] . The proposed target correction framework offers a conceptually different perspective: rather than constraining the weights or gradients directly, we modulate the target outputs to naturally align the online trajectory with the multi-task (offline) ideal. This is also related to Dark Experience Replay [ 6 ] , in which past predictions, rather than ground-truth targets, are replayed. Our framework

Read the complete text