Efficient Inference under Label Shift in Unsupervised Domain Adaptation
Seong-ho Lee, Yanyuan Ma, Jiwei Zhao · 2025
Read Efficient Inference under Label Shift in Unsupervised Domain Adaptation by Seong-ho Lee, Yanyuan Ma, Jiwei Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17780.txt.
Introduction
Lee, Ma, and Zhao build a sound way to draw facts about a new group from a model trained on an old one in twenty twenty-five. Labels are costly to gather for the new group, so only unlabeled data exist there. When the mix of labels shifts between groups, naive reuse misleads. Their method models how outcome density changes between the groups.
The estimate grows in three stages: a rough guess, then a steady estimate, then a sharp one. Each stage feeds the next, a self improving path that is new in this field. They prove the math traits of the final estimate. Tests on fake and real data beat past methods, they report.
Catalog record
| CREATOR | Seong-ho Lee, Yanyuan Ma, Jiwei Zhao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 133 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-17780
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient Inference under Label Shift in Unsupervised Domain Adaptation Seong-ho Lee, Yanyuan Ma, Jiwei Zhao Abstract In many real-world applications, researchers aim to deploy models trained in a source domain to a target domain, where obtaining labeled data is often expensive, time-consuming, or even infeasible. While most existing literature assumes that the labeled source data and the unlabeled target data follow the same distribution, distribution shifts are common in practice. This paper focuses on label shift and develops efficient inference procedures for general parameters characterizing the unlabeled target population. A central idea is to model the outcome density ratio between the labeled and unlabeled data. To this end, we propose a progressive estimation strategy that unfolds in three stages: an initial heuristic guess, a consistent estimation, and ultimately, an efficient estimation. This self-evolving process is novel in the statistical literature and of independent interest. We also highlight the connection between our approach and prediction-powered inference (PPI), which uses machine learning models to improve statistical inference in related settings. We rigorously establish the asymptotic properties of the proposed estimators and demonstrate their superior performance compared to existing methods. Through simulation studies and multiple real-world applications, we illustrate both the theoretical contributions and practical benefits of our approach. , and ?? Department of Statistics, University of Seoul, Seoul, South Korea , ?? ?? Department of Statistics, Pennsylvania State University, PA, USA , ?? ?? Departments of Statistics and of Biostatistics & Medical Informatics, University of Wisconsin-Madison, WI, USA , ?? 1 Introduction In many real-world applications, researchers are interested in deploying models and algorithms trained on a source domain to a target domain where obtaining labeled data is often expensive, time-consuming, or even infeasible. For example, in healthcare, disease prediction models trained on data from one hospital may be applied to another population of patients at risk but not yet diagnosed. In social science, models are often evaluated for their generalizability across survey populations that might differ demographically or culturally, for which the outcome of interest is not fully collected and time-consuming to collect. In autonomous driving, models trained on simulated data or limited environment must adapt to new, unseen locations. In the literature, this type of research problems was termed unsupervised domain adaptation (also known as transductive transfer learning); see an excellent review paper by Kouw and Loog 2019 . In such scenarios where acquiring gold-standard labels in the target domain is costly and labor-intensive, machine learning (ML)-based algorithms and models offer an efficient solution, substantially reducing the cost of outcome collection and accelerating scientific research across disciplines ( Wang et al. 2023 ) . For instance, in electronic health records (EHR), true disease statuses are accessible through expert-led manual chart review, and this manual review requires substantial costs due to the large scale nature of EHR. As a cost-effective solution, ML models can provide approximation of the disease statuses with accuracy comparable to manual review, thereby streamlining downstream clinical analyses ( Yang et al. 2022 ; Zhou et al. 2022 ) . Similarly, although population biobanks provide extensive data for high-throughput genetic research, many phenotypes and disease outcomes remain unmeasured due to prohibitive costs. Resolving this practical issue, ML approaches have been proven effective in facilitating novel genetic discoveries and advancing scientific inquiry ( An et al. 2023 ; McCaw et al. 2024 ; Miao et al. 2024 ) . However, the inherent complexity and the use of ML predictions pose significant challenges in statistical inferences. Using such predictions without recognizing their differences from observed gold-standard data can result in biased outcomes and misleading scientific conclusions. To address this, researchers have introduced methods that fuse extensive ML predictions with limited gold-standard data to ensure the validity of ML-assisted statistical inference. Over the past five years, this topic has gained significant importance and popularity in the literature. For easier exposition, consider the availability of both labeled data \displaystyle(Y_{i},{\mathbf{X}}_{i}),i=1,\dots,n,\mbox{ drawn as a random sample from source population }\mathcal{P}, (1) and unlabeled data \displaystyle{\mathbf{X}}_{i},i=n+1,\dots,N,\mbox{ drawn as a random sample from target population }\mathcal{Q}, (2) where the assumption \mathcal{P}=\mathcal{Q} is tentatively adopted for simplicity. Wang, McCormick and Leek 2020 proposed a post-prediction inference procedure, and Motwani and Witten 2023 conducted a more rigorous investigation on this inference after prediction procedure, and showed that, the method proposed in Wang, McCormick and Leek 2020 could only provide a valid inference in some restrictive scenarios in the context of least squares estimation. More recently, Angelopoulos et al. 2023 proposed the approach termed prediction-powered inference (PPI), which yields valid inference even when the ML predictions were of a low quality. To illustrate, PPI estimates the outcome mean \theta=\mathrm{E}(Y) as \displaystyle\widehat{\theta} \displaystyle= \displaystyle\underbrace{\frac{1}{N-n}\sum_{i=n+1}^{N}\widehat{y}_{i}}_{\begin{subarray}{c}\text{Direct estimation using}\\ \text{predictions on unlabeled data}\end{subarray}}\quad+\underbrace{\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\widehat{y}_{i})}_{\begin{subarray}{c}\text{Rectifier: measure the bias of using}\\ \text{predictions on labeled data}\end{subarray}}, (3) where \widehat{y}_{i} ’s, representing the conditional mean \mathrm{E}(Y\mid{\mathbf{x}}_{i}) , are the predicted outcomes produced by the ML algorithm/model. A few extensions of PPI were also proposed in the past two years, including but not limited to, Zrnic and Candès 2024 which considered the availability of the ML predictions, and Angelopoulos, Duchi and Zrnic 2023 ; Miao et al. 2023 ; Gronsbell et al. 2024 which investigated the statistical efficiency gains from different perspectives. It is worthwhile to note, in the statistical literature, terms as semi-supervised inference , researchers have also proposed different methods of utilizing the unlabeled data for different purposes ( Chakrabortty and Cai 2018 ; Yuval and Rosset 2022 ; Azriel et al. 2022 ; Livne, Azriel and Goldberg 2022 ; Hou, Guo and Cai 2023 ; Song, Lin and Zhou 2024 ) , including settings under high dimensionality ( Zhang, Brown and Cai 2019 ; Cai and Guo 2020 ; Zhang and Bradic 2022 ; Deng et al. 2024 ) . Despite significant progress in conducting inference under such a scenario, several key aspects remain unclear in the literature. This paper is motivated by the challenge of distribution shift ( Quiñonero-Candela et al. 2009 ) between source domain labeled data ( 1 ) and target domain unlabeled data ( 2 ), in the sense that the unlabeled data were collected under conditions different from those of the labeled data. The scenario \mathcal{P}\neq\mathcal{Q} is more realistic and more applicable in many practical settings. Indeed, various forms of distribution shift have been studied extensively in the literature, including covariate shift (the conditional distribution of Y given {\mathbf{X}} remains the same while the marginal distribution of {\mathbf{X}} shifts), and label shift (the conditional distribution of {\mathbf{X}} given Y remains the same while the marginal distribution of Y shifts). The goal of this paper is to develop an efficient procedure for estimation and inference under label shift , where efficiency is defined in the context of statistical estimation ( Bickel et al. 1993 ; Tsiatis 2006 ) . It is worth noting that in the original PPI paper, Angelopoulos et al. 2023 addressed scenarios involving distribution shift. However, under label shift, their method is limited to cases with discrete Y , similar to the approaches in Lipton, Wang and Smola 2018 and Tian, Zhang and Zhao 2023 . Moreover, estimation efficiency was not considered in their work. To the best of our knowledge, efficient estimation with label shift under this setting