Billion-Scale Graph Foundation Models
Maya Bechler-Speicher, Yoel Gottlieb, Andrey Isakov, David Abensur, Ami Tavory, Daniel Haimovich, Ido Guy, Udi Weinsberg · 2026
Read Billion-Scale Graph Foundation Models by Maya Bechler-Speicher, Yoel Gottlieb, Andrey Isakov, David Abensur, Ami Tavory, Daniel Haimovich, Ido Guy, Udi Weinsberg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04768.txt.
Description
Maya Bechler-Speicher and 7 co-authors present a recipe for billion-scale graph foundation models in a 2026 paper. Graphs underpin many applications, but foundation models that pretrain once and adapt cheaply, as in language and vision, have lagged for real-world graph data.
Their framework, GraphBFF, centers on a flexible transformer architecture built for heterogeneous graphs, where nodes and edges come in many types. The paper gives methods for batching, pretraining, and fine-tuning at that scale, plus neural scaling laws for such graphs. A billion-parameter model trained by the recipe was tested on ten real downstream tasks involving unseen graphs. It beat baselines consistently, sometimes by large margins, including in few-shot settings with very little task data.
Catalog record
| CREATOR | Maya Bechler-Speicher, Yoel Gottlieb, Andrey Isakov, David Abensur, Ami Tavory, Daniel Haimovich, Ido Guy, Udi Weinsberg |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-04768