Billion-Scale Graph Foundation Models

Maya Bechler-Speicher, Yoel Gottlieb, Andrey Isakov, David Abensur, Ami Tavory, Daniel Haimovich, Ido Guy, Udi Weinsberg · 2026

Read Billion-Scale Graph Foundation Models by Maya Bechler-Speicher, Yoel Gottlieb, Andrey Isakov, David Abensur, Ami Tavory, Daniel Haimovich, Ido Guy, Udi Weinsberg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04768.txt.

Description

Maya Bechler-Speicher and 7 co-authors present a recipe for billion-scale graph foundation models in a 2026 paper. Graphs underpin many applications, but foundation models that pretrain once and adapt cheaply, as in language and vision, have lagged for real-world graph data.

Their framework, GraphBFF, centers on a flexible transformer architecture built for heterogeneous graphs, where nodes and edges come in many types. The paper gives methods for batching, pretraining, and fine-tuning at that scale, plus neural scaling laws for such graphs. A billion-parameter model trained by the recipe was tested on ten real downstream tasks involving unseen graphs. It beat baselines consistently, sometimes by large margins, including in few-shot settings with very little task data.

Catalog record

CREATORMaya Bechler-Speicher, Yoel Gottlieb, Andrey Isakov, David Abensur, Ami Tavory, Daniel Haimovich, Ido Guy, Udi Weinsberg
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-04768

Browse the full library · About the Tower of Babel