Wasserstein Transfer Learning
Kaicheng Zhang, Sinian Zhang, Doudou Zhou, Yidong Zhou · 2025
Read Wasserstein Transfer Learning by Kaicheng Zhang, Sinian Zhang, Doudou Zhou, Yidong Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17404.txt.
Introduction
Transfer learning borrows strength from related datasets, but most versions assume the answer is a number or a short vector. Kaicheng Zhang and co-authors extend the idea in a 2025 paper to answers that are whole probability distributions, curves living in what mathematicians call Wasserstein space. Their framework covers regression problems where each output is a distribution.
When the helpful source datasets are known, their estimator comes with proven convergence rates that quantify how similarity between domains pays off. When the helpful set is unknown, a data-driven procedure guards against negative transfer, borrowing that hurts. Theory, simulations, and real applications support the methods. The authors release code, aiming at fields like distributional forecasting where the borrowed knowledge itself is a shape, not a point.
Catalog record
| CREATOR | Kaicheng Zhang, Sinian Zhang, Doudou Zhou, Yidong Zhou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17404
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Wasserstein Transfer Learning Kaicheng Zhang, Sinian Zhang, Doudou Zhou, Yidong Zhou Abstract Transfer learning is a powerful paradigm for leveraging knowledge from source domains to enhance learning in a target domain. However, traditional transfer learning approaches often focus on scalar or multivariate data within Euclidean spaces, limiting their applicability to complex data structures such as probability distributions. To address this limitation, we introduce a novel transfer learning framework for regression models whose outputs are probability distributions residing in the Wasserstein space. When the informative subset of transferable source domains is known, we propose an estimator with provable asymptotic convergence rates, quantifying the impact of domain similarity on transfer efficiency. For cases where the informative subset is unknown, we develop a data-driven transfer learning procedure designed to mitigate negative transfer. The proposed methods are supported by rigorous theoretical analysis and are validated through extensive simulations and real-world applications. The code is available at https://github.com/h7nian/WaTL . Keywords: Data Aggregation, Domain Adaptation, Multitask Learning, non-Euclidean Output, Optimal Transport 1 Introduction In recent years, transfer learning ( Torrey & Shavlik 2010 ) has emerged as a powerful paradigm in machine learning, enabling models to leverage knowledge acquired from one domain and apply it to related tasks in another. This approach has proven especially valuable in scenarios where data collection and labeling can be costly, or where tasks exhibit inherent similarities in structure or representation. While early successes focused on conventional data types such as images ( Shin et al. 2016 ) , text ( Raffel et al. 2020 ) , and tabular data ( Weiss et al. 2016 ) , there is growing interest in extending these methods to more complex data structures. Such data often reside in non-Euclidean spaces and lack basic algebraic operations like addition, subtraction, or scalar multiplication, posing challenges for traditional learning algorithms. A key example is probability distributions ( Petersen et al. 2022 ) , where for example the sum of two density functions does not yield a valid density. Samples of univariate probability distributions are increasingly encountered across various research domains, such as mortality analysis ( Ghodrati & Panaretos 2022 ) , temperature studies ( Zhu & Müller 2023 ) , and physical activity monitoring ( Lin et al. 2023 ) , among others ( Petersen et al. 2022 ) . Recently, there has been a growing focus on directly modeling distributions as elements of the Wasserstein space, a geodesic metric space related to optimal transport ( Villani 2009 , Panaretos & Zemel 2020 ) . The absence of a linear structure in this space motivates the development of specialized transfer learning techniques that respect its intrinsic geometry. To address this gap, we introduce Wasserstein Transfer Learning (WaTL), a novel transfer learning framework for regression models where outputs are univariate probability distributions. WaTL effectively leverages knowledge from source domains to improve learning in a target domain by intrinsically incorporating the Wasserstein metric, which provides a natural way to measure discrepancies between probability distributions. 1.1 Contributions The primary contributions of this work are summarized as follows: Methodology. We propose a novel transfer learning framework for regression models with distributional outputs, addressing the challenges inherent in the Wasserstein space, which lacks a conventional linear structure. Our framework includes an efficient algorithm for cases where the informative subset of source domains is known, and a data-driven algorithm for scenarios where the subset is unknown. To the best of our knowledge, this is the first comprehensive transfer learning approach specifically designed for regression models with outputs residing in the Wasserstein space. Theoretical analysis. We establish the asymptotic convergence rates for the WaTL algorithm in both the case where the informative set is known and the more challenging scenario where it must be estimated. In the latter case, we also prove that the informative set can be consistently identified. In both settings, we demonstrate that WaTL effectively improves model performance on the target data by leveraging information from the source domain. The proofs rely heavily on empirical process theory and a careful analysis of the covariate structure. Our key theoretical results extend beyond responses lying in the Wasserstein space, offering potential applications to other complex outputs. Simulation studies and real-world applications. We evaluate WaTL through simulations and real data applications, demonstrating its effectiveness in improving target model performance by leveraging source domain information. The benefits become more pronounced with larger source sample sizes, underscoring its ability to harness transferable knowledge. 1.2 Related Work Transfer learning. Transfer learning aims to improve performance in a target population by leveraging information from a related source population and has seen wide application across domains ( Huang et al. 2013 , Gong et al. 2012 , Choi et al. 2017 , Zhou, Liu, Li & Cai 2024 , Zhou, Li, Cai & Liu 2024 , e.g.,) . Recent theoretical developments have focused on regression in Euclidean settings, including high-dimensional linear ( Li et al. 2022 ) and generalized linear models ( Tian & Feng 2023 ) , nonparametric regression ( Cai & Pu 2024 , Lin & Reimherr 2024 ) , and function mean estimation from discretely sampled data ( Cai et al. 2024 ) . In parallel, optimal transport has been used to measure distributional shifts for domain adaptation ( Courty et al. 2017 , Redko et al. 2017 ) . However, to the best of our knowledge, no existing work has investigated transfer learning in regression models where outputs are probability distributions residing in the Wasserstein space. This represents a significant gap in the literature, highlighting the need for novel methodologies that address this challenging yet important setting. Distributional data analysis. The increasing prevalence of data where distributions serve as fundamental units of observation has spurred the development of distributional data analysis ( Petersen et al. 2022 ) . Recent advancements in this field include geodesic principal component analysis in the Wasserstein space ( Bigot et al. 2017 ) , autoregressive models for time series of distributions ( Zhang et al. 2022 , Zhu & Müller 2023 ) , and distribution-on-distribution regression ( Ghodrati & Panaretos 2022 , Chen et al. 2023 ) . Leveraging the Wasserstein metric, regression models with distributional outputs and Euclidean inputs can be viewed as a special case of Fréchet regression ( Petersen & Müller 2019 ) , which extends linear and local linear regression to outputs residing in general metric spaces. In practical scenarios where only finite samples from the unknown distributional output are available, empirical measures have been utilized as substitutes for the unobservable distributions in regression models ( Zhou & Müller 2024 ) . 2 Preliminaries 2.1 Notations Let L^{2}(0,1) be the space of square-integrable functions over the interval (0,1) , with the associated L^{2} norm and metric denoted by \|\cdot\|_{2} and d_{L^{2}} , respectively. To be specific, \|g\|_{2}=(\int_{0}^{1}g^{2}(z)dz)^{1/2}\text{ and }d_{L^{2}}(g_{1},g_{2})=\|g_{1}-g_{2}\|_{2} . For a vector Z , \|Z\| denotes the Euclidean norm. Given a matrix \Sigma , we define its spectrum as the set of its singular values. For a sub-Gaussian random vector X , we define the sub-Gaussian norm as \|X\|_{\Psi_{2}}:=\sup_{\|v\|=1}\inf\left\{t>0:E(e^{\langle X,v\rangle^{2}/t^{2}})\leq 2\right\}. We write a_{n}\lesssim b_{n} if there exists a positive constant C such that a_{n}\leq Cb_{n} when n is large enough and a_{n}\asymp b_{n} if