Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation Conference: Proceedings of the ACM Web Conference 2026; April 13–17, 2026; Dubai, United Arab Emirates Proceedings of the ACM Web Conference 2026 (WWW ’26), April 13–17, 2026, Dubai, United Arab Emirates DOI: 10.1145/3774904.3792579 ISBN: 979-8-4007-2307-0/2026/04 CCS: Information systems Collaborative search CCS: Security and privacy Privacy protections
Fengyuan Yu, Xiaohua Feng, Yuyuan Li, Changwang Zhang, Jun Wang, Chaochao Chen · 2026
Read Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation Conference: Proceedings of the ACM Web Conference 2026; April 13–17, 2026; Dubai, United Arab Emirates Proceedings of the ACM Web Conference 2026 (WWW ’26), April 13–17, 2026, Dubai, United Arab Emirates DOI: 10.1145/3774904.3792579 ISBN: 979-8-4007-2307-0/2026/04 CCS: Information systems Collaborative search CCS: Security and privacy Privacy protections by Fengyuan Yu, Xiaohua Feng, Yuyuan Li, Changwang Zhang, Jun Wang, Chaochao Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11503.txt.
Introduction
Yu and co-authors present FedRecGEL, a federated recommendation framework that centers on item embeddings. Federated systems train together while user data stays on each device, sharing only model parameters. Item embeddings carry the shared knowledge across clients, so their quality matters system-wide. Heterogeneous, sparse local data make stable, general embeddings hard to learn. The team recasts the problem as multi-task learning from an item-centered view. Sharpness-aware minimization then steadies training, and tests on four data sets show clear gains.
Catalog record
| CREATOR | Fengyuan Yu, Xiaohua Feng, Yuyuan Li, Changwang Zhang, Jun Wang, Chaochao Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66673 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11503
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation Conference: Proceedings of the ACM Web Conference 2026; April 13–17, 2026; Dubai, United Arab Emirates Proceedings of the ACM Web Conference 2026 (WWW ’26), April 13–17, 2026, Dubai, United Arab Emirates DOI: 10.1145/3774904.3792579 ISBN: 979-8-4007-2307-0/2026/04 CCS: Information systems Collaborative search CCS: Security and privacy Privacy protections Fengyuan Yu, Xiaohua Feng, Yuyuan Li, Changwang Zhang, Jun Wang, Chaochao Chen Abstract . Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue , i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Fed erated Rec ommendation with G eneralized E mbedding L earning ( FedRecGEL ). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL . † † cc-license: by 1. Introduction Recommender systems ( 15 ; 54 ; 18 ; 43 ; 35 ; 48 ; 57 ) are designed to leverage user-item interaction information to provide personalized content to users. They play a pivotal role across diverse domains, helping users discover items that match their preferences, thus improving user experience and driving economic value. Conventional recommender systems typically rely on centralized collection of user behavior data, which raises significant privacy concerns ( 42 ) . To mitigate these issues, federated learning ( 38 ; 37 ) is introduced, and federated recommender systems ( 30 ; 40 ) have emerged as a promising solution. In such a system, each client holds user interaction data locally and shares only essential model parameters for collaborative training. Federated recommender systems operating under the cross-device setting ( 4 ; 59 ) involve a vast number of clients, where each client corresponds to an individual user. This cross-device setting further exacerbates data heterogeneity and sparsity, thereby substantially increasing the difficulty of training. Existing work on federated recommendation generally falls into two categories aimed at improving the training process and thus enhancing recommendation performance. First , a number of studies focus on clustering similar users based on user embeddings or item embeddings updated by clients, and then conducting collaborative training within each cluster ( 14 ; 26 ; 28 ; 31 ; 27 ) . These approaches seek to stabilize the training process by grouping clients with relatively small data distributional differences, thereby alleviating the effects of data heterogeneity and sparsity within clusters. Second , other studies aim to train personalized models for each client by adapting the global model through local fine-tuning ( 51 ; 52 ; 22 ) or by fusing the global and local models ( 21 ; 20 ; 49 ; 58 ) . These methods allocate more model parameters to capture client-specific data distribution, mitigating the negative impact of heterogeneity across clients. However, both of the aforementioned categories of methods still overlook a critical challenge , i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. In federated recommender systems, the same item can be interacted with by multiple users, and the corresponding updates are aggregated on the server to capture global information, which is then distributed back to the clients for local training, thereby facilitating knowledge sharing across clients. Consequently, it is essential to learn a generalized item embedding that not only characterizes the global distribution but also adapts to diverse local distributions. Additionally, in the cross-device setting, the local data distribution exhibits significant heterogeneity and sparsity, with only a small subset of items participating in local training, further exacerbating the difficulty of learning generalized embeddings ( 16 ; 41 ; 39 ) . Taken together, these factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. Clustering-based approaches only learn item embeddings specific to a group of similar users, while personalized methods do not focus on improving the generalization of global embeddings. To fill this gap, we propose a new federated recommendation framework, named Fed erated Rec ommendation with G eneralized E mbedding L earning ( FedRecGEL ). Different from the aforementioned approaches, FedRecGEL directly addresses the challenge of learning generalized embeddings. FedRecGEL adopts an item-centered perspective to formalize the federated recommendation problem as a multi-task learning problem, with the objective of improving the generalization capability of the learned embeddings. Through theoretical analysis, we demonstrate that this generalization challenge can be effectively addressed using sharpness-aware minimization. Building on this insight, FedRecGEL modifies both the local training and global aggregation processes, employing sharpness-aware minimization to learn robust and generalized embeddings. This design stabilizes the training procedure and ultimately enhances recommendation performance. We summarize the main contributions of this paper as follows: • We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, highlighting the importance of learning generalized item embeddings. • Through theoretical analysis, we reveal that the generalization problem of item embedding learning in the multi-task learning framework can be effectively addressed using sharpness-aware minimization. • Based on this insight, we propose a novel framework, FedRecGEL , which incorporates sharpness-aware minimization into both local training and global aggregation to stabilize the training process and improve the generalization of embeddings. • We conduct extensive experiments on four real-world datasets and validate the effectiveness of FedRecGEL . Our method, FedRecGEL , consistently outperforms all baselines across diverse scenarios. Its advantage grows as the user-item ratio increases. 2. Related Work 2.1. Federated Recommendation Recommendation systems ( 15 ; 54 ; 18 ; 8 ; 7 ) are designed to leverage user-item interaction information to provide personalized content to users ( 13 ) . In federated recommendation, each user is treated as a client, holding their own interaction data locally ( 16 ; 5 ; 17 ) . Existing research on federated recommender systems generally falls into two categories. First, a number of studies focus on clustering similar users based on user embeddings or item embeddings updated by clients, and then conducting collaborative training within each cluster ( 14 ; 26 ; 28 ; 31 ; 27 ) . These approaches aim to stabilize the training process by grouping clients with relatively small data distributional differences, thereby alleviating the effects of data heterogeneity and sparsity within clusters. However, obtaining reliable clustering results is challenging due to the limited and sparse representations of individual client data. Second, other studies aim to train personalized models for each client by