Towards regularized learning from functional data with covariate shift
Markus Holzleitner [email protected], Sergiy Pereverzyev Jr [email protected], Sergei V. Pereverzyev [email protected], Vaibhav Silmana [email protected], S. Sivananthan [email protected] · 2026
Read Towards regularized learning from functional data with covariate shift by Markus Holzleitner [email protected], Sergiy Pereverzyev Jr [email protected], Sergei V. Pereverzyev [email protected], Vaibhav Silmana [email protected], S. Sivananthan [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21019.txt.
Introduction
Markus Holzleitner and four co-authors study learning when training and test data come from different worlds. Their 2026 paper builds a regularization framework for unsupervised domain adaptation. The setting is vector-valued regression under covariate shift. Covariate shift means the input distributions differ between training and test. The outputs may be vectors, not single numbers. Such shift poses significant challenges for reliable learning.
The framework uses vector-valued reproducing kernel Hilbert spaces. These spaces give a rich, structured way to fit functions. Regularization keeps the fit from chasing noise in the source data. No labels from the target domain are assumed. The paper develops the theory for this general case. It serves fields like medicine and sensing, where labeled data sit in one setting and use happens in another.
Catalog record
| CREATOR | Markus Holzleitner [email protected], Sergiy Pereverzyev Jr [email protected], Sergei V. Pereverzyev [email protected], Vaibhav Silmana [email protected], S. Sivananthan [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21019
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards regularized learning from functional data with covariate shift Markus Holzleitner [email protected], Sergiy Pereverzyev Jr [email protected], Sergei V. Pereverzyev [email protected], Vaibhav Silmana [email protected], S. Sivananthan [email protected] Abstract This paper investigates a general regularization framework for unsupervised domain adaptation in vector-valued regression under the covariate shift assumption, utilizing vector-valued reproducing kernel Hilbert spaces (vRKHS). Covariate shift occurs when the input distributions of the training and test data differ, introducing significant challenges for reliable learning. By restricting our class of hypothesis space, we develop a practical operator-learning algorithm capable of handling functional outputs. We establish optimal convergence rates for the proposed framework under a general source condition, providing a theoretical foundation for regularized learning in this setting. We also propose an aggregation-based approach that forms a linear combination of estimators corresponding to different regularization parameters and different kernels. The proposed approach addresses the challenge of selecting appropriate tuning parameters that is crucial for constructing a good estimator, and we provide a theoretical justification for its effectiveness. Further, we illustrate our proposed method on real-world face image dataset, demonstrating the method’s robustness and effectiveness in mitigating distributional discrepancies under covariate shift. † † editor: … keywords Aggregation, Covariate shift, Operator learning, vRKHS, Regularization 1 Introduction In statistical learning theory, the fundamental goal is to learn a relationship between input and output variables based on observed data. Formally, given an input space X and an output space Y , we assume that the dataset \mathbf{z}= \{(x_{i},y_{i})\}_{i=1}^{n}\subset X\times Y , is drawn independently according to an unknown probability distribution \rho on X\times Y . The aim of the learning algorithm is to construct a function f:X\rightarrow Y that approximates the underlying dependency between the input x and the output y , achieving a small prediction error with respect to \rho . However, in many real-world situations, the training and testing data are not sampled from the same distribution; this leads to the problem of domain adaptation . Let us denote the source distribution as p(x,y) and the target distribution as q(x,y) on X\times Y . If the source and target distributions are entirely unrelated, then it is impossible to minimize the target risk based solely on source samples 5 . Hence, to make domain adaptation feasible, one typically assumes a structural relationship between the two distributions. A particularly important and tractable case is the covariate shift setting. Generally, covariate shift refers to a change in the input distribution compared to the training data used to build learning algorithms. In case of finite dimensional inputs and outputs, this issue has been extensively discussed in the literature, starting with the seminal work by 29 . To the best of our knowledge, the covariate shift issue is much less studied in the case of functional data, when both the output and the input variables are functions. At the same time, functional data with covariate shift frequently appear in various practical applications where changes in input distributions are caused by environmental and operational factors, as it is the case, for example, in Structural health monitoring discussed in the study by 30 . In that study, the authors propose an adjustment to covariate-induced variations by training a functional mixed additive model for function-on-function regression, in which the regression function is assumed to be expanded in a finite sum of chosen basis functions such as B-splines, for example. Note that such family of models is covered by class of learning algorithms in vector-valued reproducing kernel Hilbert spaces (vRKHS) that has been recently investigated in the study by 24 . At the same time, we are not aware of any results on learning in vRKHS from functional data with covariate shift, and this is the goal of the present study to shed light on the possibility of such learning. For this aim, we combine the technique developed in 24 with the ideas presented in 14 . Mathematically, the covariate shift assumption posits the following relationship between the source and target distributions: p(x,y)=p(y|x)\,p_{X}(x),\quad q(x,y)=p(y|x)\,q_{X}(x), where p_{X}(x) and q_{X}(x) are the marginal distributions and p(y|x) is the same conditional distribution that is related by the expression dp(y|x)=\mathcal{K}(dy,x) to the so-called Markov kernel \mathcal{K}(dy,x) , which has been employed in the analysis 24 . Under this setting, one typically has access to labeled samples \{(x_{i},y_{i})\}_{i=1}^{n} from the source distribution p(x,y) and unlabeled inputs \{x^{\prime}_{j}\}_{j=1}^{m} from the target marginal distribution q_{X} . The goal is to minimize the expected risk over the target domain (X\times Y,q) : \mathcal{R}_{q}(f)=\int_{X\times Y}\big\|f(x)-y\big\|_{Y}^{2}\,dq(x,y). (1) Assuming \int_{X\times Y}\|y\|_{Y}^{2}\,dq(x,y)<\infty, it can be shown that the minimum of the expected risk \mathcal{R}_{q}(f) over Bochner space L_{2}(q_{X},Y) of square-integrable functions f:X\to Y with respect to the marginal distribution q_{X} on X is attained at the regression function f_{q}(x)=\int_{Y}y\,dp(y|x). Since the conditional distribution is identical, the regression functions corresponding to the source and target domains coincide, i.e., f_{p}=f_{q} . We denote this common regression function by f^{*} . Hence, given the available data, the goal is to minimize the target risk over L_{2}(q_{X},Y) by constructing an estimator f_{\mathbf{z}} that closely approximates the regression function f^{*} , with the key distinction that, unlike standard supervised learning where the minimization is performed in L_{2}(p_{X},Y) under the source marginal p_{X} , domain adaptation seeks to minimize the error in L_{2}(q_{X},Y) determined by the target marginal q_{X} . The quality of this approximation is typically quantified in terms of the excess risk , \mathcal{R}_{q}(f_{\mathbf{z}})-\mathcal{R}_{q}(f^{*}) . From 7 we know that it can be expressed as \mathcal{R}_{q}(f_{\mathbf{z}})-\mathcal{R}_{q}(f^{*})=\big\|f_{\mathbf{z}}-f^{*}\big\|_{L_{2}(q_{X},Y)}^{2}\;. Covariate shift has been studied in finite-dimensional and scalar-output settings. Recent advances have extended this analysis to more general setups with improved convergence rates (see, 12 , 14 , 29 ). In 14 , convergence rates were established under general source condition characterized by splitting form of the index function. Moreover, a number of recent studies have investigated methods for addressing the case of unbounded density ratios; see 15 ; 23 ; 13 ; 22 . Furthermore, convergence analyses have been established for computationally efficient methods such as Nyström subsampling 11 , which substantially reduce the computational complexity of kernel-based algorithms while preserving optimal statistical guarantees. The vRKHS framework offers an approach to regression problems involving functional or infinite-dimensional outputs (see, 20 , 28 , 24 ). Notably, 28 derived optimal convergence rates for Tikhonov and general regularization schemes under general source conditions for trace-class kernels, thereby extending the classical scalar-valued results to the vector-valued framework. More recently, 24 provided a comprehensive theoretical framework for vector-valued regularization methods, encompassing both well-specified and misspecified cases. Within this framework, optimal convergence rates were derived for the general multiplicative kernel under interpolation space conditions in the \gamma -norm. Together, these results highlight that vRKHS provides a principled framework for modern learning problems such as multi-task regression, functional data analysis, where outputs are inherently in high or infinite-dimensional space. Motivated by this, we first propose an operator learning algorithm and then establish convergence rates under a general source condition for a broad class of regularization schemes in vector-valued regression under covariate shift. It is also important to note that the performance of regularized learning with reproducing kernels crucially depends on how regularization parameters and kernels are chosen. On the other hand, to the best of our knowledge,