Generalizable Insights for Graph Transformers in Theory and Practice

Timo Stoll, Luis MüllerChristopher MorrisDepartment of Computer ScienceRWTH Aachen UniversityAachen, Germany [email protected] · 2025

Read Generalizable Insights for Graph Transformers in Theory and Practice by Timo Stoll, Luis MüllerChristopher MorrisDepartment of Computer ScienceRWTH Aachen UniversityAachen, Germany [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08028.txt.

Description

Timo Stoll, Luis Muller, and Christopher Morris search for graph transformer design rules that hold across many domains. These models vary widely in attention style, positional information, and power, and most theory ties to one narrow design. That leaves builders without guidance that survives a change of task or scale.

The authors define a Generalized Distance Transformer using standard attention plus recent advances. They analyze how attention and positional encodings shape its representational power. Experiments span more than eight million graphs from images, molecules, code, and reasoning tasks. A few design and training choices perform well almost everywhere, including few shot transfer.

Catalog record

CREATORTimo Stoll, Luis MüllerChristopher MorrisDepartment of Computer ScienceRWTH Aachen UniversityAachen, Germany [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE167 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08028

Browse the full library · About the Tower of Babel