FedGrAINS: Personalized SubGraph Federated Learning with AdaptIve Neighbor Sampling
Emir Ceyani, Han Xie, Baturalp Buyukates, Carl Yang, Salman Avestimehr · 2025
Read FedGrAINS: Personalized SubGraph Federated Learning with AdaptIve Neighbor Sampling by Emir Ceyani, Han Xie, Baturalp Buyukates, Carl Yang, Salman Avestimehr (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12592.txt.
Introduction
Emir Ceyani and co-authors present FedGrAINS, a method for training graph models across private data silos. Graphs model linked data in biology, finance, and social webs, yet privacy rules often bar pooling the raw links. Federated learning trains on site and shares only model updates, but each client's subgraph differs wildly.
Uneven node degrees and missing cross links make joint training shaky. FedGrAINS rates each node's worth for a client's task using generative flow networks. It then reshapes message passing so prized nodes speak more, tuned by a balance goal. Added as a regularizer, the method lifted scores over baselines in test after test. The gain held across varied client graphs, which suggests a sturdy route for private graph learning at scale.
Catalog record
| CREATOR | Emir Ceyani, Han Xie, Baturalp Buyukates, Carl Yang, Salman Avestimehr |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-12592
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FedGrAINS: Personalized SubGraph Federated Learning with AdaptIve Neighbor Sampling Emir Ceyani, Han Xie, Baturalp Buyukates, Carl Yang, Salman Avestimehr Abstract Graphs are crucial for modeling relational and biological data. As datasets grow larger in real-world scenarios, the risk of exposing sensitive information increases, making privacy-preserving training methods like federated learning (FL) essential to ensure data security and compliance with privacy regulations. Recently proposed personalized subgraph FL methods have become the de-facto standard for training personalized Graph Neural Networks (GNNs) in a federated manner while dealing with the missing links across clients’ subgraphs due to privacy restrictions. However, personalized subgraph FL faces significant challenges due to the heterogeneity in client subgraphs, such as degree distributions among the nodes, which complicate federated training of graph models. To address these challenges, we propose FedGrAINS , a novel data-adaptive and sampling-based regularization method for subgraph FL. FedGrAINS leverages generative flow networks (GFlowNets) to evaluate node importance concerning clients’ tasks, dynamically adjusting the message-passing step in clients’ GNNs. This adaptation reflects task-optimized sampling aligned with a trajectory balance objective. Experimental results demonstrate that the inclusion of FedGrAINS as a regularizer consistently improves the FL performance compared to baselines that do not leverage such regularization. Keywords: federated learning, graph neural networks, reinforcement learning, node classification, generative models, generative flow networks 1 Introduction Graph Neural Networks (GNNs) operate over a single graph, with nodes and edges stored in a central server. For example, in social networks, each user, along with their connections form a large graph of all users and their links [ 2 ] . However, parties may not send their private graph datasets to a central server due to privacy concerns, raising the need to train GNN models over multiple distributed graph datasets. For instance, hospitals may maintain their patient interaction networks to track physical contacts or co-diagnoses of diseases. However, they cannot share these graphs with other entities due to privacy restrictions [ 52 ] . Here, the question is how to collaboratively train GNNs across clients’ distributed subgraphs without sharing the actual graph data. Federated Learning (FL) of GNNs aims to solve this problem so that each client individually trains a local GNN on their local data, while a central server aggregates locally updated weights from multiple clients into a global model using federated optimization [ 19 ] . A key challenge in naïvely applying FL methods to GNNs is the potential loss of crucial information due to missing links between subgraphs distributed across parties. Recent approaches to subgraph FL address this issue by extending local subgraphs with information from other subgraphs [ 52 , 47 ] . Specifically, they extend the local subgraph by either precisely adding the relevant nodes from other subgraphs from different clients [ 47 ] or by predicting the nodes using node information from the other subgraphs via graph mending [ 52 ] . However, sharing node information can compromise data privacy and lead to high communication costs. While [ 4 ] mitigates the issues of missing links and subgraph heterogeneity through personalized aggregation and local sparse masks, all existing approaches are mainly based on image data modality, failing to fully exploit the graph-structured data to address heterogeneity, thus resulting in sub-optimal solutions to personalize GNN models. On the other hand, dropout-based data pruning in FL [ 44 , 21 ] and stochastic regularization in GNNs [ 30 , 17 ] have shown to be successful in regularizing deeper models without any significant overhead by randomly dropping the network weights and/or data components. Existing dropout-based data pruning approaches in FL, however, are not suitable for graph ML models due to GNNs’ invariance requirements, which forces us to utilize dropout methods for graph-structured data [ 40 , 11 ] . Further, applying dropout techniques naïvely to deeper GNNs without accounting for the data structure can degrade node and structural information quality due to Laplacian smoothing, leading to over-smoothened representations that are uninformative in higher layers [ 31 ] . Moreover, using the same dropout parameters across different clients is also not an effective strategy due to heterogeneity over data, system, and the model, raising the need for coordination based on the users’ requirements. To effectively personalize GNN models by respecting graph structures, we propose a novel data-adaptive, sampling-based regularization method for subgraph FL, named FedGrAINS by estimating and selecting essential nodes at each layer of GNNs every round for every client in a federated manner. To estimate the adaptive neighborhood distribution of each node in the datasets, we leverage Generative Flow Networks (GFlowNets), the state-of-the-art generative models tailored for structured discrete objects like graphs [ 7 ] . Here, for each node in the clients’ graph, we leverage GFlowNets to estimate the importance of the neighbors of a node so that we can prune the graph for each node adaptively in parallel to optimize the clients’ personalized task performances. We refer to this framework as Personalized Sub Gr aph Federated Learning with A dapt I ve N eighbor S ampling ( FedGrAINS ). We extensively validate the performance of FedGrAINS across six popular graph datasets, evaluating overlapping and disjoint subgraph FL scenarios with varying numbers of clients. Our comprehensive experimental results show that FedGrAINS significantly outperforms relevant baselines. Our contributions are summarized as follows: • We introduce a new type of heterogeneity, node-degree heterogeneity , showing that it cannot be handled with previous personalized FL approaches. • We propose FedGrAINS , a novel framework for personalized subgraph FL. FedGrAINS performs layer-wise node-importance estimation using GFlowNets and dynamically updates the GNN message-passing steps to combat neighborhood heterogeneity caused by missing links and subgraph heterogeneity between multiple clients. • We validate our personalized FL framework FedGrAINS across six real-world datasets under disjoint and overlapping node scenarios, demonstrating its effectiveness against relevant baselines. 2 Preliminaries In this section, we describe GNNs and personalized subgraph FL in more detail. Then, we present the core component of our proposed algorithm, GFlowNets. GNNs: Let \mathcal{G}=(\mathcal{V,E}) be an undirected graph with a set of N nodes \mathcal{V}=\left\{v_{1},\dots v_{N}\right\} and edges \mathcal{E} . \bm{X}=\left\{\bm{x}_{1},\dots,\bm{x}_{N}\right\}\in\mathbb{R}^{N\times d} , where \mathbf{x}_{i}\in\mathbb{R}^{d} is the node feature vector for node v_{i} , indicating the features of each node i\in\{1,\ldots,N\} . According to the Message-Passing Neural Network (MPNN) paradigm [ 12 ] , the flow of hidden representations in GNNs, based on the nodes’ neighborhoods and features, is described as follows: (2.1) \bm{H}^{l+1}_{v}=\text{UPD}^{l}\left(\bm{H}^{l}_{v},\text{AGG}^{l}\left(\left\{\bm{H}^{l}_{u}:\forall u\in\mathcal{N}(v)\right\}\right)\right), where \bm{H}^{l}_{v} denotes the features of node v at the l -th layer, and \mathcal{N}(v) denotes the set of neighbor nodes of node v : \mathcal{N}(v)=\left\{u\in\mathcal{V}\;|\;(u,v)\in\mathcal{E}\right\} . AGG aggregates the features of the neighbors of node v . We note that AGG can be mean or even a black-box function. UPD updates node v ’s representation given its previous representation and the aggregated representations from its neighbors. \bm{H}^{1} is initialized as \bm{X} . As the number of layers increases, the embedding computation for a node v incorporates information from neighbors several hops away. Increasing the number of layers expands the depth of neighbors for each node included in the training set, resulting in the exponential increase in the neighborhood size for each node, thus causing high computation time on the user side. Reducing high computation at the edge is