Sharpness-aware Federated Graph Learning Conference: Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining; February 22–26, 2026; Boise, ID, USA Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining (WSDM ’26), February 22–26, 202
Ruiyu Li, Peige Zhao, Guangxia Li, Pengcheng Wu, Xingyu Gao, Zhiqiang Xu · 2025
Read Sharpness-aware Federated Graph Learning Conference: Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining; February 22–26, 2026; Boise, ID, USA Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining (WSDM ’26), February 22–26, 202 by Ruiyu Li, Peige Zhao, Guangxia Li, Pengcheng Wu, Xingyu Gao, Zhiqiang Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-16247.txt.
Introduction
Ruiyu Li and five co-authors improve privacy-preserving graph learning in their 2025 paper on arXiv, accepted to WSDM 2026. Organizations hold graph data they cannot pool, so federated graph learning trains a shared neural model without moving private data. Two problems plague it: each client's data differs, and local training settles into sharp valleys that generalize poorly.
Their algorithm, SEAL, optimizes both the loss and the sharpness of the loss surface, steering models toward flat regions that cope better with varied data. A regularizer based on the correlation matrix of learned representations stops those representations from collapsing into a few dimensions. On several graph classification benchmarks, SEAL consistently beat leading federated baselines. Gains spread across participants, which matters because federated systems live or die by keeping members willing to join.
Catalog record
| CREATOR | Ruiyu Li, Peige Zhao, Guangxia Li, Pengcheng Wu, Xingyu Gao, Zhiqiang Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-16247
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sharpness-aware Federated Graph Learning Conference: Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining; February 22–26, 2026; Boise, ID, USA Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining (WSDM ’26), February 22–26, 2026, Boise, ID, USA DOI: 10.1145/3773966.3777989 ISBN: 979-8-4007-2292-9/2026/02 CCS: Information systems Data mining CCS: Computing methodologies Distributed algorithms CCS: Computing methodologies Distributed artificial intelligence Ruiyu Li, Peige Zhao, Guangxia Li, Pengcheng Wu, Xingyu Gao, Zhiqiang Xu Abstract . One of many impediments to applying graph neural networks (GNNs) in processing large-volume real-world graph-structured data is that it disapproves of a centralized training scheme which involves gathering data belonging to different organizations due to privacy concerns. As a distributed data processing scheme, federated graph learning (FGL) enables learning GNN models collaboratively without sharing participants’ private data. Though theoretically feasible, a core challenge in FGL systems is the variation of local training data distributions among clients, also known as the data heterogeneity problem. Most existing solutions suffer from two problems: (1) The typical optimizer based on empirical risk minimization tends to cause local models to fall into sharp valleys and weakens their generalization to out-of-distribution graph data. (2) The prevalent dimensional collapse in the learned representations of local graph data has an adverse impact on the classification capacity of the GNN model. To this end, we formulate a novel optimization objective that is aware of the sharpness (i.e., the curvature of the loss surface) of local GNN models. By minimizing the loss function and its sharpness simultaneously, we seek out model parameters in a flat region with uniformly low loss values, thus improving the generalization over heterogeneous data. By introducing a regularizer based on the correlation matrix of local representations, we relax the correlations of representations generated by individual local graph samples, so as to alleviate the dimensional collapse of the learned model. The proposed S harpness-aware f E derated gr A ph L earning (SEAL) algorithm can enhance the classification accuracy and generalization ability of local GNN models in federated graph learning. Experimental studies on several graph classification benchmarks show that SEAL consistently outperforms SOTA FGL baselines and provides gains for more participants. † † cc-license: by 1. Introduction Graph learning is a powerful tool to process graph-structured data. Most graph learning methods, especially graph neural networks (GNNs), are trained on large-scale centralized graph data. These methods have been extensively applied in various domains, including drug discovery ( 39 ; 51 ) , social networks ( 50 ; 57 ) , knowledge graphs ( 10 ; 4 ) , and recommender systems ( 49 ; 36 ) . However, in many real-world scenarios, graph data are geographically distributed and cannot be directly shared due to privacy concerns. For example, in bioinformatics, graph classification techniques are employed to learn protein representations and classify them as enzymes or non-enzymes. These private protein molecules typically reside with multiple individual owners (i.e., clients). Due to intellectual property concerns and the intrinsic commercial value of molecules, academic labs, national labs, and private organizations are often unable to share their molecular data, leading to the data isolation problem. (a) Intra-domain heterogeneity (b) Inter-domain heterogeneity Figure 1. Graph data heterogeneity. 1(a) Intra-domain heterogeneity involves two cases: first, clients possess data drawn from the same dataset but with inconsistent statistical properties; second, clients’ data are sourced from different datasets within the same domain. 1(b) Inter-domain heterogeneity implies that clients’ data are derived from entirely different domains. Federated graph learning (FGL), a distributed learning paradigm for collaboratively training GNN models, has emerged as a promising direction for exploiting GNNs on decentralized graph data ( 43 ; 21 ) . A major challenge in FGL is the underlying discrepancy in graph data distribution across clients, commonly referred to as graph data heterogeneity. As illustrated in Fig. 1 , the heterogeneity manifests both intra and inter domains. Intra-domain heterogeneity arises when clients hold graph data from the same dataset or domain (e.g., small molecules) but with distinct statistical properties. In contrast, inter-domain heterogeneity occurs when each client’s graph data is drawn from different domains, leading to discrepancies in both statistical characteristics and graph structural information. Such heterogeneity can severely destabilize the training process and degrade performance. While traditional federated learning approaches can effectively alleviate statistical inconsistency in computer vision tasks ( 35 ; 23 ; 33 ; 45 ) , they fall short when it comes to handling heterogeneous graph data. Prior studies have sought to address the graph data heterogeneity problem from multiple perspectives, including optimization-based methods ( 33 ; 23 ) , personalized FGL ( 55 ; 43 ) , and clustered FGL ( 6 ; 48 ) . For example, FedEgo ( 55 ) and FedStar ( 43 ) share graph structural information across clients and introduce personalized feature knowledge learning modules on each client. FedCG ( 6 ) and GCFL+ ( 48 ) group clients into non-overlapping clusters, within which clients share model parameters. However, personalized approaches introduce additional trainable parameters, while clustered approaches are highly sensitive to the quality of client clustering. Since GNNs are typically over-parameterized ( 60 ) , all of these methods incur substantial computational cost and communication overhead ( 29 ) , and they still suffer from the following issues: (1) GNN models trained under the empirical risk minimization (ERM) based optimizers tend to overfit local graph data and fail to generalize to out-of-distribution data. (2) Heterogeneous graph data can cause local GNN models to undergo dimensional collapse in the representation space (as illustrated in Fig. 2(a) and Fig. 2(c) ). To address the first issue, we note that models trained with ERM-based optimizers may fall into the sharp valley of the loss surface ( 14 ) , which leads to poor generalization, even when the training and test data distributions are similar. Previous efforts ( 12 ; 8 ; 25 ) have explored the relationship between the flatness of local minima and the generalization ability of deep models to improve generalization and have applied it within federated learning ( 5 ) , but their application to FGL has not been thoroughly investigated. In FGL, each local GNN model only has access to a subset of the global graph data and thus tends to overfit its local data to minimize the training loss, which harms its ability to generalize to the test data. This study investigates the higher-order information underlying GNN generalization, namely, the curvature of the loss surface, to enhance the generalization ability of as many local GNN models as possible. To this end, we equip each local GNN model with a sharpness-aware minimization optimizer that jointly optimizes for low training loss and a flatter loss landscape. The objective is to identify model parameters that reside in a neighborhood with uniformly low loss values, rather than at a single point that simply minimizes the loss, which naturally leads to a min–max optimization problem. We solve it via a two-step procedure: (1) applying gradient ascent to find a perturbation point that increases the loss within a fixed-radius neighborhood of the current parameters, and (2) updating the current parameters by gradient descent using the gradient evaluated at this perturbed point. To address the