FedAPM: Federated Learning via ADMM with Partial Model Personalization Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
Shengkun Zhu, Feiteng Nie, Jinshan Zeng ∗, Sheng Wang ∗, Yuan Sun, Yuan Yao, Shangfeng Chen, Quanqing Xu, Chuanhui Yang · 2025
Read FedAPM: Federated Learning via ADMM with Partial Model Personalization Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 by Shengkun Zhu, Feiteng Nie, Jinshan Zeng ∗, Sheng Wang ∗, Yuan Sun, Yuan Yao, Shangfeng Chen, Quanqing Xu, Chuanhui Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04672.txt.
Introduction
Zhu and eight co-authors built a shared learning frame in twenty twenty-five that lets each site keep part of the model its own. Sites hold unlike data, so one shared model fits poor-ly. Split models drift from the shared core, though, which slows the joint work.
Their frame adds two fix terms that pull local and shared parts back in line. Proofs show runs settle from any start at three speed kinds. On four mixed data sets it beat top past frames on truth by wide shares. It also took fewer talk rounds to get there.
Catalog record
| CREATOR | Shengkun Zhu, Feiteng Nie, Jinshan Zeng ∗, Sheng Wang ∗, Yuan Sun, Yuan Yao, Shangfeng Chen, Quanqing Xu, Chuanhui Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 143 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-04672
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FedAPM: Federated Learning via ADMM with Partial Model Personalization Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD ’25), August 3–7, 2025, Toronto, ON, Canada DOI: 10.1145/3711896.3736954 ISBN: 979-8-4007-1454-2/2025/08 Thanks: *Corresponding authors: Sheng Wang and Jinshan Zeng CCS: Computing methodologies Distributed artificial intelligence CCS: Computing methodologies Distributed algorithms CCS: Computing methodologies Regularization Shengkun Zhu, Feiteng Nie, Jinshan Zeng ∗, Sheng Wang ∗, Yuan Sun, Yuan Yao, Shangfeng Chen, Quanqing Xu, Chuanhui Yang Abstract . In federated learning (FL), the assumption that datasets from different devices are independent and identically distributed (i.i.d.) often does not hold due to user differences, and the presence of various data modalities across clients makes using a single model impractical. Personalizing certain parts of the model can effectively address these issues by allowing those parts to differ across clients, while the remaining parts serve as a shared model. However, we found that partial model personalization may exacerbate client drift (each client’s local model diverges from the shared model), thereby reducing the effectiveness and efficiency of FL algorithms. We propose an FL framework based on the alternating direction method of multipliers (ADMM), referred to as FedAPM , to mitigate client drift. We construct the augmented Lagrangian function by incorporating first-order and second-order proximal terms into the objective, with the second-order term providing fixed correction and the first-order term offering compensatory correction between the local and shared models. Our analysis demonstrates that FedAPM , by using explicit estimates of the Lagrange multiplier, is more stable and efficient in terms of convergence compared to other FL frameworks. We establish the global convergence of FedAPM training from arbitrary initial points to a stationary point, achieving three types of rates: constant, linear, and sublinear, under mild assumptions. We conduct experiments using four heterogeneous and multimodal datasets with different metrics to validate the performance of FedAPM . Specifically, FedAPM achieves faster and more accurate convergence, outperforming the SOTA methods with average improvements of 12.3% in test accuracy, 16.4% in F1 score, and 18.0% in AUC while requiring fewer communication rounds. 1. Introduction With the widespread use of mobile devices, vast amounts of data are generated that fuel various machine learning applications ( Xiao et al. 2018 ; Wang et al. 2023 ; Yadav et al. 2021 ) . However, traditional cloud-based training methods face issues of privacy leakage and data transmission costs ( Kairouz et al. 2021a ; Smith et al. 2017b ) . Additionally, with the implementation of privacy regulations like the California Consumer Privacy Act (CCPA) ( Wik 2018a ) and General Data Protection Regulation (GDPR) ( Wik 2018b ) , protecting user data has become even more critical. Federated learning (FL) is a distributed machine learning framework that protects privacy by enabling local model training without the need for data centralization ( McMahan et al. 2017 ; Li et al. 2020a ) , addressing challenges like data silos and insufficient samples ( Li et al. 2022 ; Kairouz et al. 2021b ) , and fostering advancements in various intelligent applications ( Ye et al. 2024b ; Kuang et al. 2024 ) . In FL, it is typically assumed that the datasets from various devices are sampled from the same or highly similar distributions ( Li et al. 2020b ; Li et al. 2022 ; Luo et al. 2021 ) . However, due to the distinct characteristics of users and the growing prominence of personalized on-device services, the assumption of independent and identically distributed (i.i.d.) data often does not hold in practical scenarios ( Ye et al. 2024a ; Zhao et al. 2018 ; Sattler et al. 2020 ) . This leads to statistical heterogeneity, which can significantly impact the effectiveness of FL algorithms ( Ye et al. 2024a ) . Moreover, different clients may have data in various modalities ( Feng et al. 2023 ; Chen and Zhang 2024 ; Chen and Zhang 2022 ) , such as images, videos, text, and audio, each requiring distinct models like convolutional neural networks (CNNs) for images ( LeCun et al. 2015 ) and recurrent neural networks (RNNs) for audio ( Goodfellow 2016 ) . Therefore, relying on a single model for all cases is ineffective and impractical ( Pillutla et al. 2022 ; Sun et al. 2023 ) . Partial model personalization ( Pillutla et al. 2022 ) is a method designed to address the challenge of heterogeneous and multimodal data by dividing the model parameters into two groups: shared and personalized model parameters. The dimensions of personalized models can vary among clients, enabling personalized components to differ in the number of parameters or even in their architecture, while the shared model is common among clients and maintains the same structure. However, while personalized models can effectively address the impact of heterogeneous and multimodal data, the shared model cannot, leading to the common saying: it started with a bang and ended with a whimper . This limitation arises because each client’s local objectives differ. When solving for the shared model, the result is often a stationary point with respect to the local objective, rather than a stationary point with respect to the global objective. This issue is commonly referred to as client drift ( Karimireddy et al. 2020 ) . Several studies consider using full model personalization to address client drift ( Dinh et al. 2020 ; Li et al. 2021 ; Lin et al. 2022 ) , which involves personalizing all model parameters. However, each model requires twice the memory footprint of the full model, which limits the size of trainable models. Moreover, Pillutla et al. 2022 proposed that full model personalization may not be essential for modern deep learning architectures, which consist of multiple simple functional units often structured in layers or other complex interconnected designs. Focusing on personalizing the appropriate components, guided by domain expertise, can yield significant advantages with only a minimal increase in memory requirements. Current partial model personalization methods can be categorized into two types based on their iterative approach: one using Gauss-Seidel iteration ( Collins et al. 2021 ; Pillutla et al. 2022 ; Singhal et al. 2021 ) , and the other employing Jacobi iteration ( Liang et al. 2020 ; Arivazhagan et al. 2019 ; Hanzely et al. 2021 ) , referred to as FedAlt and FedSim , respectively. Pillutla et al. 2022 theoretically demonstrated the convergence of both methods and experimentally validated that FedAlt achieves better test performance than FedSim . However, these methods do not fundamentally address client drifts, as multiple local updates can still lead to deviations of each client’s local model from the optimal representation. Moreover, we find that partial model personalization sometimes even exacerbates client drift, as shown in Figure 1 . The local updates in FedAlt and FedSim , compared to FedAvg (a de facto FL framework without personalization) ( McMahan et al. 2017 ) , bring the local model closer to the optimal local model while moving it further from the optimal shared model. Figure 1. Client drift in FedAvg ( McMahan et al. 2017 ) and FedAlt