Toward a universal foundation model for graph-structured data
Sakib Mostafa, Lei Xing, Md Tauhidul Islam · 2026
Read Toward a universal foundation model for graph-structured data by Sakib Mostafa, Lei Xing, Md Tauhidul Islam (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06391.txt.
Introduction
Mostafa, Xing, and Islam build a graph foundation model for biomedical graphs: molecular, gene, cell, and knowledge networks. Current graph networks train on one dataset. Their model learns transferable structure from feature-agnostic properties: degree, centrality, community, and diffusion signatures, encoded as structural prompts. These prompts join a message-passing backbone for one shared space. Pretrained once, it reuses on unseen datasets. It matches or beats strong baselines, with superior zero-shot and few-shot generalization. For SagePPI, fine-tuning reaches 95.5 percent ROC-AUC, beating the best baseline by 22 percent.
Catalog record
| CREATOR | Sakib Mostafa, Lei Xing, Md Tauhidul Islam |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80864 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.06391
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Toward a universal foundation model for graph-structured data Sakib Mostafa, Lei Xing, Md Tauhidul Islam Abstract Graphs are a central representation in biomedical research, capturing molecular interaction networks, gene regulatory circuits, cell–cell communication maps, and knowledge graphs. Despite their importance, currently there is not a broadly reusable foundation model available for graph analysis comparable to those that have transformed language and vision. Existing graph neural networks are typically trained on a single dataset and learn representations specific only to that graph’s node features, topology, and label space, limiting their ability to transfer across domains. This lack of generalization is particularly problematic in biology and medicine, where networks vary substantially across cohorts, assays, and institutions. Here we introduce a graph foundation model designed to learn transferable structural representations that are not specific to specific node identities or feature schemes. Our approach leverages feature-agnostic graph properties, including degree statistics, centrality measures, community structure indicators, and diffusion-based signatures, and encodes them as structural prompts. These prompts are integrated with a message-passing backbone to embed diverse graphs into a shared representation space. The model is pretrained once on heterogeneous graphs and subsequently reused on unseen datasets with minimal adaptation. Across multiple benchmarks, our pretrained model matches or exceeds strong supervised baselines while demonstrating superior zero-shot and few-shot generalization on held-out graphs. On the SagePPI benchmark, supervised fine-tuning of the pretrained backbone achieves a mean ROC-AUC of 95.5%, a gain of 21.8% over the best supervised message-passing baseline. The proposed technique thus provides a unique approach toward reusable, foundation-scale models for graph-structured data in biomedical and network science applications. 1 Introduction Figure 1: Architecture of the Graph Foundation Model. a , Structural prompt construction. For each node, local context descriptors—degree, clustering coefficient, k -core number, ego-network statistics, and PageRank—are combined with global context indicators derived from two community detection algorithms, Label Propagation and SCoDA, each contributing community identity, community size, and intra-community density. These descriptors are assembled into a natural-language node profile and tokenized by a miniature language model (MiniLM) to produce a context embedding. Node features are projected into a matching space and concatenated with this embedding to form the Graph Feature Embedding (GFE), a representation that encodes both attribute content and structural role without relying on dataset-specific feature semantics. b , Multi-graph pretraining. A GraphSAGE backbone ingests multiple heterogeneous graphs simultaneously, producing a Graph Structure Embedding (GSE) for every node via neighborhood aggregation. Contrastive alignment is applied between the GFE and the GSE: the text embedding of node i is matched against the graph embeddings of structurally related nodes and repelled from structurally dissimilar nodes, training the backbone to map diverse graphs into a shared topological representation space. c , Downstream deployment. The frozen or fine-tuned GFM generates GFE and GSE representations for nodes in an unseen graph. These are concatenated and passed to a lightweight task-specific head for node-level classification or regression, requiring minimal labeled data on the target graph. Graphs are widely used to represent relational systems in biomedical science, such as molecular interaction networks [ 3 ] , gene regulatory circuits [ 11 ] , and cell–cell communication maps [ 1 ] . They are also the inherent representation for biomedical knowledge graphs that link genes, diseases, drugs, and phenotypes [ 30 ] . This wide applicability of graph representations makes graph learning critically important for both biological discovery and clinical translation. In recent years, deep learning has transformed representation learning in domains with fixed input formats, most notably language and vision, where large pretrained models can be reused across tasks with minimal adaptation [ 7 , 13 , 5 ] . However, despite major advances in graph neural networks, there is still no broadly reusable foundation model that generalizes reliably across graphs from different domains. This challenge is particularly pronounced in biology and medicine, where graph structure can vary across cohorts, assays, and institutions, causing models trained on one graph to transfer poorly to another. Most existing graph neural networks are trained on a single dataset and learn representations that are specific to that graph’s node features, topology, and label space [ 44 ] . Common message-passing architectures, including Graph Convolutional Networks (GCN) [ 23 ] , Graph Isomorphism Networks (GIN) [ 45 ] , Graph Attention Networks (GAT) [ 42 ] , and GraphSAGE [ 17 ] , can be used on semi-supervised learning and node classification tasks when training and testing occur within the same graph. However, the learned parameters typically reflect the statistics and semantics of one dataset at a time. Recent self-supervised approaches, including Deep Graph Infomax (DGI) [ 43 ] , GRACE [ 48 ] , and BGRL [ 40 ] , have the same limitations. They define objectives on a single graph and learn representations that are most reliable within that same structural and feature distribution. As a result, graph representation learning remains dataset-specific in practice. Models are often retrained from scratch for each new graph, and knowledge learned from one dataset is not consistently carried forward to another [ 49 ] , even when the graphs describe related biological systems. This generalizability limitation stems from properties that are inherent to graphs. Unlike images or text, graphs do not have a canonical input space [ 6 ] . Node features can take many forms, such as bag-of-words vectors, molecular fingerprints, continuous attributes, or learned embeddings, and their meaning varies widely across applications. The meaning of edges also changes across domains. An edge may represent a physical interaction, a regulatory relationship, a citation, or a transactional link. This variability makes it difficult to define a shared representation that works across datasets. There are also architectural constraints. Most graph models rely on local message passing, where each layer aggregates information from a node’s neighbors. This works well for local structure, but long-range dependencies require deeper networks. As depth increases, node representations can become overly similar [ 24 ] , and information from distant regions of the graph can be compressed into small vectors. These effects weaken the model’s ability to preserve global structural signals consistently. Pretraining objectives do not automatically fix this. Many objectives are tuned to the distribution of a single graph and may not remain stable under large shifts in structure, feature space, or semantics. Together, these issues make cross-graph generalization difficult, even when models perform well within a single dataset. For a reusable foundation model, the ideal representation must rely on components of graph organization that are intrinsic and broadly available, rather than being tied to dataset-specific features or labels. Graph topology provides such a basis. Many structural properties are defined without reference to feature semantics. These include degree statistics, centrality measures, community structure, diffusion statistics, and higher-order connectivity patterns [ 29 ] . They can be computed for any graph and can be compared across graphs, even when the underlying node attributes differ. If these structural signals can be expressed through a common interface, then heterogeneous graphs may be mapped into a shared embedding space without requiring common node identities or a shared feature schema. This provides a practical route to cross-domain learning in settings where raw features are incompatible, but topology remains meaningful. Here, to address the limitations of existing