Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 3–7, 2025; Toronto, ON, Canada DOI: 10.1145/3711896.3737161 ISBN: 979-8-4007-1454-2/2025/08 Proceedings of th

Tianrun Yu, Jiaqi Wang, Haoyu Wang, Mingquan Lin, Han Liu, Nelson S. Yee, Fenglong Ma · 2025

Read Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 3–7, 2025; Toronto, ON, Canada DOI: 10.1145/3711896.3737161 ISBN: 979-8-4007-1454-2/2025/08 Proceedings of th by Tianrun Yu, Jiaqi Wang, Haoyu Wang, Mingquan Lin, Han Liu, Nelson S. Yee, Fenglong Ma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08617.txt.

Introduction

Tianrun Yu, Fenglong Ma, and five co-authors make federated learning fair when data shifts unevenly. Their paper, from twenty twenty-five, tackles imbalanced covariate shift. In federated learning, many clients train one shared model without pooling data. Fairness suffers when some clients' data drifts from the rest.

Their analysis finds drift hides mostly in the samples clients get wrong. Well-judged samples look alike across clients. Their method, FedAKD, swaps knowledge in two stages. Clients first learn with the global model held fixed. Then confident, correct samples update the global model. The server still merges all client models as usual. Prediction quality and fairness rise together.

Catalog record

CREATORTianrun Yu, Jiaqi Wang, Haoyu Wang, Mingquan Lin, Han Liu, Nelson S. Yee, Fenglong Ma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08617

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 3–7, 2025; Toronto, ON, Canada DOI: 10.1145/3711896.3737161 ISBN: 979-8-4007-1454-2/2025/08 Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’25), August 3–7, 2025, Toronto, ON, Canada CCS: Information systems Information systems applications Tianrun Yu, Jiaqi Wang, Haoyu Wang, Mingquan Lin, Han Liu, Nelson S. Yee, Fenglong Ma Abstract . Collaborative fairness is a crucial challenge in federated learning. However, existing approaches often overlook a practical yet complex form of heterogeneity: imbalanced covariate shift . We provide a theoretical analysis of this setting, which motivates the design of FedAKD (Federated Asynchronous Knowledge Distillation)—a simple yet effective approach that balances accurate prediction with collaborative fairness. FedAKD consists of client and server updates. In the client update, we introduce a novel asynchronous knowledge distillation strategy based on our preliminary analysis, which reveals that while correctly predicted samples exhibit similar feature distributions across clients, incorrectly predicted samples show significant variability. This suggests that imbalanced covariate shift primarily arises from misclassified samples. Leveraging this insight, our approach first applies traditional knowledge distillation to update client models while keeping the global model fixed. Next, we select correctly predicted high-confidence samples and update the global model using these samples while keeping client models fixed. The server update simply aggregates all client models. We further provide a theoretical proof of FedAKD ’s convergence. Experimental results on public datasets (FashionMNIST and CIFAR10) and a real-world Electronic Health Records (EHR) dataset demonstrate that FedAKD significantly improves collaborative fairness, enhances predictive accuracy, and fosters client participation even under highly heterogeneous data distributions. 1 1 1 Source code is available at https://github.com/Tianrun-Yu/FedAKD . 1. Introduction Federated Learning (FL) has emerged as a promising distributed paradigm that enables multiple participants (or clients) to collaboratively train a global model without sharing their raw local data ( Konecný 2016 ; Li et al. 2020a ; Zhao et al. 2018 ; McMahan et al. 2017 ; Yan et al. 2023 ) . However, disparities in data quality, quantity, and distribution among clients make uniform treatment in global model aggregation unfair, particularly for those with higher-quality or larger datasets. To address this, collaborative fairness (CF) has been introduced to ensure that each client’s final reward or benefit is commensurate with its contribution to the global model ( Lyu et al. 2020 ) . In other words, clients with a greater impact on model performance should receive proportionally higher gains. Figure 1 . (a) KL divergence vs. sample size for each client’s local data, revealing both data imbalance and feature covariate shifts. (b)-(e) compare different fairness methods’ contribution definitions to each client’s standalone training accuracy. Their poor correlation highlights the limitations of explicit contribution metrics under extra covariate shifts. Several fairness-aware FL approaches, such as CGSV ( Xu et al. 2021 ) , CFFL ( Lyu et al. 2020 ) , FedAve ( Wang et al. 2024a ) , and FedSAC ( Wang et al. 2024b ) , have been developed to assign different contribution-based rewards (i.e., weights) during model aggregation. While these methods help mitigate fairness disparities, they still face the following challenges: \bullet Unrealistic assumptions about data heterogeneity. Existing CF approaches primarily assume that data heterogeneity arises from imbalanced data sizes ( Xu et al. 2021 ; Lyu et al. 2020 ) , imbalanced class distributions ( Xu et al. 2021 ; Lyu et al. 2020 ) , or both ( Gao et al. 2022 ; Yurochkin et al. 2019 ) . However, real-world datasets exhibit greater complexity. Beyond these imbalances, client data often differ significantly in feature distributions, leading to the covariate shift problem ( Karimireddy et al. 2020 ; Gao et al. 2022 ; Li et al. 2020a ; Goksu and Pugeault 2024 ) . Figure 1 (a) presents a preliminary analysis on a real-world electronic health record (EHR) dataset for pancreatic cancer prediction 2 2 2 Details on the EHR dataset and the preliminary analysis are provided in Sections 5.1 and 2 , respectively. . The x -axis represents client dataset sizes, while the y -axis shows the Kullback–Leibler (KL) divergence between each client’s fitted latent feature distribution and that of the entire dataset. Each circle represents a client, i.e., a U.S. state in the EHR dataset. The preliminary data analysis reveals that not only do clients have varying dataset sizes, but their latent feature distributions also significantly diverge from the global reference distribution. Thus, a more realistic FL heterogeneity setting should account for imbalanced covariate shift rather than just data quantity or class imbalance. \bullet Weak correlation between client accuracy and assigned contributions. Beyond unrealistic data distribution assumptions, existing approaches to collaborative fairness typically follow a two-step pipeline: (1) explicitly defining a contribution metric and (2) allocating rewards based on this metric . For example, CGSV ( Xu et al. 2021 ) estimates client contributions via gradient similarity, CFFL ( Lyu et al. 2020 ) and FedAVE ( Wang et al. 2024a ) rely on performance improvements measured by a global validation set, and FedSAC ( Wang et al. 2024b ) bases contributions on standalone local training results. However, in real-world datasets characterized by imbalanced covariate shift, these approaches fail to establish a strong correlation between client accuracy and assigned contributions, contradicting their underlying design assumptions. Figures 1 (b)–(e) illustrate the relationship between client standalone accuracy ( x -axis) on the testing set and the learned contribution value ( y -axis) either on the training set or validation set under different methods – CFFL, CGSV, FedAVE, and the baseline FedAvg ( McMahan et al. 2017 ) – using the real-world EHR dataset, which is the same as we analyzed in Figure 1 (a). Each dot represents an individual client, and in the case of FedAvg, the contribution score is simply the proportion of data owned by the client. The results show that existing CF-based methods exhibit significantly lower Pearson correlation scores compared to the simple FedAvg baseline. These findings highlight the limitations of existing approaches in handling realistic imbalanced covariate shift settings. Theoretical analysis on imbalanced covariate shift. This paper aims to develop a novel model to ensure collaborative fairness in federated learning under the realistic yet challenging imbalanced covariate shift setting. To achieve this, in Section 2 , we first provide a theoretical analysis in Theorem 2.1 demonstrating that imbalanced covariate shift – quantified as the KL divergence between each client’s empirical data distribution and the ideal global distribution – is primarily influenced by the perturbation \delta . Specifically, if the underlying data distributions of individual clients and the entire dataset follow a multivariate normal distribution \mathcal{N}(\mu,\Sigma) , where \mu is the mean vector, we show that \delta and the covariance \Sigma play key roles in determining the extent of distributional divergence in Theorem 2.2 . The two theorems motivate us to mitigate the imbalanced covariate shift to achieve collaborative fairness by correctly qualifying \delta and \Sigma . However, directly calculating these values is infeasible, as the true data distributions are inherently unknown . Motivations of model design. To address this challenge, we conducted a preliminary analysis on the results of FedAvg

Read the complete text