Personalized Subgraph Federated Learning with Differentiable Auxiliary Projections
Wei Zhuo, Zhaohuan Zhan, Han Yu · 2025
Read Personalized Subgraph Federated Learning with Differentiable Auxiliary Projections by Wei Zhuo, Zhaohuan Zhan, Han Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23864.txt.
Introduction
Wei Zhuo, Zhaohuan Zhan, and Han Yu personalize federated learning for graph data in a 2025 paper. Each client in their setting holds a different piece of one big graph, such as a social or citation network, and the pieces differ in character. Sharing raw data or node embeddings is off the table.
Their method, FedAux, gives each client a learnable projection vector that squeezes node embeddings into a ranked one-dimensional signature. The server compares signatures to judge which clients resemble each other, then mixes their models in proportion. Each client ends with a personal model that still borrows strength from similar peers. Tests on graph benchmarks beat the baselines on both accuracy and personal fit, with theory supplied for convergence.
Catalog record
| CREATOR | Wei Zhuo, Zhaohuan Zhan, Han Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-23864
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Personalized Subgraph Federated Learning with Differentiable Auxiliary Projections Wei Zhuo, Zhaohuan Zhan, Han Yu Abstract Federated Learning (FL) on graph-structured data typically faces non-IID challenges, particularly in scenarios where each client holds a distinct subgraph sampled from a global graph. In this paper, we introduce Fed erated learning with Aux iliary projections ( FedAux ), a personalized subgraph FL framework that learns to align, compare, and aggregate heterogeneously distributed local models without sharing raw data or node embeddings. In FedAux , each client jointly trains (i) a local GNN and (ii) a learnable auxiliary projection vector ( APV ) that differentiably projects node embeddings onto a 1D space. A soft-sorting operation followed by a lightweight 1D convolution refines these embeddings in the ordered space, enabling the APV to effectively capture client-specific information. After local training, these APV s serve as compact signatures that the server uses to compute inter - client similarities and perform similarity - weighted parameter mixing, yielding personalized models while preserving cross - client knowledge transfer. Moreover, we provide rigorous theoretical analysis to establish the convergence and rationality of our design. Empirical evaluations across diverse graph benchmarks demonstrate that FedAux substantially outperforms existing baselines in both accuracy and personalization performance. The code is available at https://github.com/JhuoW/FedAux . 1 Introduction Real - world data often manifests as relational structures, ranging from social interactions [ 30 , 46 ] and financial networks [ 28 , 45 ] to molecular graphs [ 35 , 44 ] , whose scale and privacy constraints increasingly require training to be carried out in a federated manner [ 9 ] , whereby multiple clients collaboratively learn a Graph Neural Network (GNN) model without exchanging their raw data. However, applying federated learning to graph-structured data, such as social networks, faces severe challenges due to non-identically and independently distributed (non-IID) data across clients. For example, consider a federated learning scenario involving multiple regional social networking platforms, each representing a distinct subgraph of a global social network. Users within each region exhibit unique interaction patterns and distinct interests, resulting in significant heterogeneity in local graph structures and node attributes. This inherent diversity among subgraphs leads to substantial difficulties when attempting to aggregate local GNN models into a unified global model, as traditional FL algorithms [ 20 , 14 ] typically assume homogeneous data distributions across clients. To tackle the non-IID challenges inherent in subgraph federated learning, personalized FL [ 29 ] has recently emerged as a promising paradigm, which aims to provide client-specific GNN models rather than enforcing a universal global solution. Existing personalized subgraph FL approaches commonly achieve personalization by clustering clients on the server side, necessitating a reliable measure of client similarity without direct access to client-side data. In this work, we impose even stricter privacy constraints: neither raw data nor embeddings are shared, and only model learnable parameters can be exchanged. Although the server could compare clients by directly measuring similarity between their parameter matrices uploaded, the high dimensionality of these matrices makes such metrics unreliable under the curse of dimensionality [ 4 ] . Recent improvements have proposed measuring similarity by comparing communication - level parameter gradients [ 35 ] or generating a common anchor graph on the server as a neutral testbed [ 3 ] . Although these strategies mitigate some limitations, they remain largely heuristic and do not explicitly model the heterogeneity inherent in subgraph clients (See extended discussion in Appendix A ). Figure 1 : The overall framework of FedAux . Left: The server maintains a global GNN model together with learnable auxiliary projection vectors ( APV s) that are broadcast to all clients at the start of each communication round. Middle: Clients jointly optimize the GNN and APV during local training, where the APV projects node embeddings onto a 1D \bm{a}_{k} -space that positions related nodes closer together. Right: After local training, clients transmit their optimized GNN parameters and personalized APV s to the server. The server computes a client similarity matrix by comparing the learned APV s, which captures the heterogeneity across subgraphs without accessing raw data. These similarities determine personalized aggregation weights. At the end of each round, the updated global parameters are broadcast for the next communication round. Motivation Our key insight is that a compact, low-dimensional proxy, derived directly from the client’s own model parameters, can faithfully summarize local subgraph characteristics without leaking sensitive node features or embeddings. Such a proxy remains compact enough to avoid the pitfalls of high-dimensional similarity measures, yet expressive enough to reflect meaningful differences between clients. By learning this proxy jointly with the GNN parameters in each client, and using it to guide both local adaptation and server-side aggregation, we obtain a principled, privacy-preserving mechanism for personalization that directly leverages model parameters as a stand-in for subgraph information. Contribution In this work, we propose FedAux , which employs differentiable auxiliary projections to effectively capture and exploit client-specific heterogeneity for subgraph FL. As illustrated on the left of Fig. 1 , the server stores not only a global GNN but also a learnable auxiliary projection vector ( APV ) that accompanies the model parameters. At the start of the first communication round, the server broadcasts the global GNN and the current APV to all clients. Each client projects its node embeddings onto the APV , which is treated as a one-dimensional latent space. A differentiable soft-sorting operator then orders the projected embeddings by similarity, after which a simulated 1D convolution aggregates the sorted embedding sequence. The aggregated representations drive a supervised loss that simultaneously refines the local GNN and the APV , so that the optimized APV preserves the relational structure of the client subgraph. Upon completing local training, clients send their updated GNN weights and personalized APV s back to the server. Since the APV reveals only the latent space that best preserves local node relationships while concealing the exact position of every node in this space, it acts as a compact privacy-preserving summary of the client subgraph. Then the server computes similarity among the returned APV s to quantify inter-client affinity and yields client-specific aggregation weights. The server uses these weights to combine the incoming parameters, producing a personalized model for each client that respects both shared knowledge and local subgraph idiosyncrasies. Furthermore, we establish comprehensive and rigorous theoretical analyses that justify the soundness and interoperability of every technique used in FedAux . Extensive federated node classification experiments on six datasets, spanning diverse graph domains and client scales, demonstrate that FedAux achieves better accuracy and stronger personalization than state-of-the-art personalized subgraph FL baselines. 2 Problem Statement: Subgraph Federated Learning In Federated Learning (FL), multiple clients collaboratively train a global model without exchanging their raw data. In the subgraph federated learning setting, each client holds a subgraph of a larger graph. Formally, let a graph \mathcal{G} be partitioned (or subsampled) into K subgraphs \{G_{1},G_{2},\cdots,G_{K}\} as K clients, where G_{k}=(V_{k},E_{k},X_{k},Y_{k}) . Here, V_{k}=\{v_{k,1},\cdots,v_{k,N_{k}}\} is the set of nodes in the k -th subgraph with N_{k}=|V_{k}| nodes, E_{k} the set of edges among those nodes, X_{k}\in\mathbb{R}^{N_{k}\times d} the node features, and Y_{k} the labels relevant to the learning task. In our FL scenario, each client G_{i} has access only to its local