Federated Learning with Domain Shift Eraser
Zheng Wang, Zihui Wang, Zheng Wang, Xiaoliang Fan, Cheng Wang · 2025
Read Federated Learning with Domain Shift Eraser by Zheng Wang, Zihui Wang, Zheng Wang, Xiaoliang Fan, Cheng Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13063.txt.
Introduction
The authors propose a federated learning framework that erases domain shift in a 2025 paper. Federated learning trains a shared model while data stay on client devices. When client data come from different domains, the shift between them degrades the shared representation and hurts performance.
Their framework splits each network layer into a domain agnostic feature extractor and a domain specific skew eraser. A forward pass alternately extracts and deskews features. A regularizer pulls local statistics toward global ones. Extractors are averaged fairly across clients, while erasers are personalized by similarity, so each client shift is handled differently.
Catalog record
| CREATOR | Zheng Wang, Zihui Wang, Zheng Wang, Xiaoliang Fan, Cheng Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13063
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Federated Learning with Domain Shift Eraser Zheng Wang, Zihui Wang, Zheng Wang, Xiaoliang Fan, Cheng Wang Abstract Federated learning (FL) is emerging as a promising technique for collaborative learning without local data leaving their devices. However, clients’ data originating from diverse domains may degrade model performance due to domain shifts, preventing the model from learning consistent representation space. In this paper, we propose a novel FL framework, Federated Domain Shift Eraser (FDSE), to improve model performance by differently erasing each client’s domain skew and enhancing their consensus. First, we formulate the model forward passing as an iterative deskewing process that extracts and then deskews features alternatively. This is efficiently achieved by decomposing each original layer in the neural network into a Domain-agnostic Feature Extractor (DFE) and a Domain-specific Skew Eraser (DSE). Then, a regularization term is applied to promise the effectiveness of feature deskewing by pulling local statistics of DSE’s outputs close to the globally consistent ones. Finally, DFE modules are fairly aggregated and broadcast to all the clients to maximize their consensus, and DSE modules are personalized for each client via similarity-aware aggregation to erase their domain skew differently. Comprehensive experiments were conducted on three datasets to confirm the advantages of our method in terms of accuracy, efficiency, and generalizability. 1 Introduction Federated learning has emerged as a promising paradigm for training machine learning models on distributed data while preserving privacy McMahan et al. 2017 . However, a notable challenge arises when clients originate from diverse domains, where domain shifts across clients may result in performance degradation of the aggregated model Huang et al. 2023 ; Chen et al. 2020 . For example, hospitals located in different regions may collect data from diverse patient populations using various machines and protocols, leading to distinct domains in their feature distributions Li et al. 2021c . The misalignment in feature spaces can hinder client consensus on the representation space (e.g., Figure 1 left), finally reducing model performance in two ways. First, the model is compelled to additionally learn generalizable representations for vastly different samples across domains, competitively preventing it from focusing on the task objective. Second, the training process may be dominated by a single domain when clients’ model updates exhibit conflicts and substantial variations in magnitude Chen et al. 2024 ; Wang et al. 2021 , which causes the model to overfit on the prevailing domain while sacrificing its utility on others Hu et al. 2022 . Figure 1 : Inllustration of the characteristic of the proposed FDSE v.s. Vallina FL McMahan et al. 2017 . FDSE decomposes the model to respectively erase domain skew for each client by f_{DSE}^{(i)} and extract generalizable features for all the clients by f_{DFE} . This promotes consistency in representation space across domains since knowledge is fine-grainedly decoupled for learning and aggregation. Recent works addressing the FL domain shift problem can be categorized into two main groups. The first group (i.e., consensus-based methods) Zhou et al. 2023 ; Tan et al. 2022 ; Chen et al. 2024 enhances the consensus among clients at various levels to improve the model generalizability across domains. In contrast, another group (i.e., personalization-based methods) Li et al. 2021c ; Liang et al. 2020 ; Sun et al. 2021 mitigates the need for clients’ consensus by personalizing their models to fit their local data distributions. Since the two groups exhibit complementary strengths in modeling different types of knowledge (i.e., general and personal knowledge), it’s natural to consider a hybrid solution that synergizes their advantages for further improvement. However, model personalization may severely hinder clients’ consensus due to over-focusing on clients’ local benefits, thereby resulting in invalid integration. This motivates us to rethink the FL domain shift from the hybrid perspective How to personalize models to enhance clients’ consensus? In this work, we address this issue by using personalized parameters to erase domain-specific skew for each client while enhancing clients’ consensus on other parameters based on two key observations. First, visual learning tasks often mitigate samples’ domain-specific skew differently. For example, in point cloud segmentation, high-density and low-density point clouds from channel-varying lidars are respectively downsampled and upsampled to achieve their alignment in feature space Yuan et al. 2024 . These inherently opposing processes of feature alignment necessitate personalized modeling. Second, a unified treatment is usually applied to different samples after their domain-specific skew is mitigated Jiang et al. 2022 ; Yuan et al. 2024 ; Huang et al. 2023 , revealing the importance of achieving post-deskewing consensus in a domain-agnostic way. Following the above paradigm, we develop a novel FL framework, F ederated D omain S hift E raser ( FDSE ), aiming to differently erase clients’ domain-specific skew and then enhance their consensus in a decoupled manner. First, we formulate the model forward passing as an iterative deskewing process that extracts and then deskews features alternatively. This is efficiently achieved by decomposing each original layer in the neural network into a Domain-agnostic Feature Extractor (DFE) and a Domain-specific Skew Eraser (DSE). Then, a regularization term is applied to enhance the effectiveness of feature deskewing by pulling local statistics of each DSE’s output close to the globally consistent ones. Finally, DFE modules are fairly aggregated and broadcast to all the clients to maximize their consensus, while DSE modules are respectively aggregated for each client based on their similarity to erase heterogeneous domain skew collaboratively. Compared to the Vallina FL in Figure 1 (left), our FDSE benefits from the design that emphasizes fine-grained personalization and consensus maximization on decoupled objectives, promoting both the consistency in the representation space and the efficiency of aggregating different types of knowledge. Our contributions are summarized as follows: • We rethink the domain shift problem in FL from a novel hybrid perspective that integrates the advantages of two distinct sets of existing methods, i.e., consensus-based and personalization-based methods. • We develop a novel framework, FDSE, to improve cross-domain FL by differently erasing clients’ domain skew while enhancing their consensus in a decoupled manner. • We conduct comprehensive experiments on three datasets (i.e., Office-Caltech10, PACS, and DomainNet) to confirm FDSE’s superiority in terms of accuracy, efficiency, and generalizability against the state-of-the-art methods. 2 Related Works 2.1 Data Heterogeneity in FL FL often faces challenges due to data heterogeneity, complicating the training process and degrading overall model accuracy McMahan et al. 2017 ; Zhao et al. 2018 . Previous efforts have been devoted to handling heterogeneous data by maintaining consistency between local objective and global objective Li et al. 2020 ; Li et al. 2021a ; Acar et al. 2021 ; Karimireddy et al. 2020 or smoothening the clients’ loss landscape during model training Sun et al. 2023 ; Mendieta et al. 2022 ; Qu et al. 2022 ; Dai et al. 2023 . Another series of works instead focuses on enhancing models’ local performance under data heterogeneity via model personalization techniques, such as partial parameter sharing Liang et al. 2020 ; Sun et al. 2021 ; Husnoo et al. 2022 , meta-learning Li et al. 2021b ; Fallah et al. 2020 , latent representation space alignment Tan et al. 2022 ; Oh et al. 2021 , hyper-network Shamsian et al. 2021 , and knowledge decoupling Chen and Chao