Heterogeneous Graph Alignment for Joint Reasoning and Interpretability

Zahra Moslemi, Ziyi Liang, Norbert Fortin, Babak Shahbaba · 2026

Read Heterogeneous Graph Alignment for Joint Reasoning and Interpretability by Zahra Moslemi, Ziyi Liang, Norbert Fortin, Babak Shahbaba (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22593.txt.

Introduction

Zahra Moslemi, Ziyi Liang, Norbert Fortin, and Babak Shahbaba built a transformer that learns across many kinds of graphs at once. Real graph sets differ in shape, scale, and meaning, with no shared node names. Their 2026 preprint presents the Multi-Graph Meta-Transformer for that messy case.

The frame aims to pull joint signals while keeping each graph's sense clear, so results stay interpretable. Brains, molecules, and social nets could all feed one such learner. The work is a bid for a general graph engine. It values clear reasons as much as raw scores. That breadth is the point of the design.

Catalog record

CREATORZahra Moslemi, Ziyi Liang, Norbert Fortin, Babak Shahbaba
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE120 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22593

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Heterogeneous Graph Alignment for Joint Reasoning and Interpretability Zahra Moslemi, Ziyi Liang, Norbert Fortin, Babak Shahbaba Abstract Multi-graph learning is crucial for extracting meaningful signals from collections of heterogeneous graphs. However, effectively integrating information across graphs with differing topologies, scales, and semantics, often in the absence of shared node identities, remains a significant challenge. We present the Multi-Graph Meta-Transformer (MGMT), a unified, scalable, and interpretable framework for cross-graph learning. MGMT first applies Graph Transformer encoders to each graph, mapping structure and attributes into a shared latent space. It then selects task-relevant supernodes via attention and builds a meta-graph that connects functionally aligned supernodes across graphs using similarity in the latent space. Additional Graph Transformer layers on this meta-graph enable joint reasoning over intra- and inter-graph structure. The meta-graph provides built-in interpretability: supernodes and superedges highlight influential substructures and cross-graph alignments. Evaluating MGMT on both synthetic datasets and real-world neuroscience applications, we show that MGMT consistently outperforms existing state-of-the-art models in graph-level prediction tasks while offering interpretable representations that facilitate scientific discoveries. Our work establishes MGMT as a unified framework for structured multi-graph learning, advancing representation techniques in domains where graph-based data plays a central role. 1 Introduction Graphs are fundamental data structures in many domains including neuroscience ( Shahbaba et al. 2022 ; Zhou et al. 2024 ) , social networks ( Fan et al. 2019 ; Zhang et al. 2022 ) and molecular biology ( Wieder et al. 2020 ; Xu et al. 2022 ; Li et al. 2022 ) . While powerful models like Graph Neural Networks (GNNs) ( Scarselli et al. 2008 ; Kipf & Welling 2016 ; Yu et al. 2023 ) and the more recent Graph Transformers (GTs) ( Wu et al. 2021 ; Kreuzer et al. 2021 ; Rampášek et al. 2022 ; Kim et al. 2022 ) excel at learning from single graphs, many real-world problems require integrating information across multiple heterogeneous graphs, where the heterogeneity may stem from differences in modalities, views, or population characteristics. For instance, neuroscience experiments studying brain dynamics often generate graphs from multiple subjects, each with distinct connectivities and node sets ( Shahbaba et al. 2022 ) . Enhancing prediction performance or extracting common neural patterns in such settings requires a framework capable of effectively integrating information across structurally heterogeneous graphs. Nonetheless, the question of how to optimally adapt powerful architectures such as the GT to the multi-graph integration problem remains an active area of research, particularly with heterogeneous structures, unaligned node sets, and a need for fine-grained cross-graph reasoning, conditions that arise in many scientific domains. Recent work on multi-graph learning partially addresses this challenge by either operating on a single unified heterogeneous graph with aligned nodes ( He et al. 2025 ; Zhang et al. 2019 ; Zheng et al. 2022 ) or by learning graph-level embeddings from multiple modality-specific graphs and combining them via shared contexts, adaptive weights, or correlation-based objectives ( Hayat et al. 2022 ; Xu et al. 2024 ; Xing et al. 2024 ; D’Souza et al. 2023 ; Nakhli et al. 2023 ; Fu et al. ) . However, these methods model cross-graph interactions only through pooled graph embeddings or shared tokens, without explicit fine-grained message passing between structurally similar subgraphs across unaligned graphs, which limits interpretability and makes it challenging to identify which substructures across graphs interact and contribute to the model’s predictions. To address these limitations, we propose the Multi-Graph Meta-Transformer (MGMT), a flexible framework for integrating information across collections of heterogeneous graphs. Under the umbrella term “multi-graph,” our unified approach accommodates common scenarios, including: multimodal (graphs from different measurement channels), multi-view (different structural views of the same data), and multi-subject (graphs from heterogeneous subjects in the same experiment). MGMT first processes each input graph using a graph-transformer encoder and aggregates intermediate layer outputs through a depth-aware mixing scheme. This yields node embeddings that adaptively integrate information across multiple receptive-field sizes. The framework is backbone-agnostic and can be implemented with either localized or global GT layers. It then selects a small set of informative supernodes from each graph using attention scores and constructs an explicit meta-graph over these task-relevant substructures, preserving within-graph connectivity while selectively introducing cross-graph (inter-graph) edges between functionally aligned regions. Additional Transformer layers on the resulting meta-graph perform fine-grained cross-graph message passing and generate the final prediction. This design directly addresses the key limitations of existing multi-graph approaches. Instead of representing each graph solely through global graph-level or context-level embeddings at fusion time, MGMT preserves structure at the node and subgraph levels and performs fusion by attending over an explicit meta-graph of supernodes, encoding both intra-graph and inter-graph structure. More specifically, supernodes summarize local patterns within each graph, while superedges align substructures across graphs and capture their interactions. The resulting meta-graph also provides insights into task-relevant subgraphs and their relationships, which can help with interpreting the results. Taken together, MGMT utilizes depth-aware, structure-preserving GT encoders within each graph, identifies supernodes, and uses them to build an explicit meta-graph that supports cross-graph message passing and provides a principled and scalable approach to aggregating information across a collection of heterogeneous graphs with unaligned node sets. Our main contributions are as follows: (1) For heterogeneous graphs (multimodal, multi-view, or multi-subject), we formalize a data-fusion framework, MGMT, which provides a backbone-agnostic, depth-aware, and structure-preserving architecture with interpretable outputs through an explicitly constructed meta-graph over attention-selected supernodes; (2) we provide theoretical results demonstrating that MGMT’s depth-aware aggregation can recover general L -hop neighborhood mixing and characterize conditions under which the induced meta-graph function class offers strictly improved approximation capacity relative to late-fusion strategies that operate only on pooled graph embeddings; (3) using synthetic benchmarks and real-world applications, we show that MGMT outperforms state-of-the-art multimodal and graph-based methods, including recent multi-graph and transformer architectures; and (4) we demonstrate that MGMT can detect meaningful neurobiological patterns, thereby offering insights for scientific investigations, particularly for understanding neural mechanisms underlying memory and factors contributing to Alzheimer’s disease. 2 Related Work Graph Representation Learning. GNN is the cornerstone of modern graph machine learning, which learns node embeddings via local message passing ( Scarselli et al. 2008 ; Kipf & Welling 2016 ; Velickovic et al. 2017 ) . Attention-based models such as GAT ( Velickovic et al. 2017 ) learn neighbor-specific attention weights instead of using fixed aggregation rules, allowing them to prioritize more informative neighbors. Nevertheless, because they still aggregate information only from local neighborhoods, their ability to distinguish graph structures remains fundamentally limited by the expressive power of the 1-Weisfeiler–Lehman (1-WL) test. More recent GT architectures with structured self-attention have outperformed message-passing GNNs on a variety of benchmarks ( Dwivedi & Bresson 2020 ; Vaswani 2017 ) . Global-attention models such as EGT ( Hussain et al. 2022 ) replace or complement convolution with fully-connected self-attention augmented with structural encodings, thereby extending expressivity. Sparse and structure-aware GTs, including Exphormer ( Shirzad et al. 2023 ) and GRIT ( Ma et al. 2023 ) , introduce scalable attention patterns and graph inductive biases that retain or approximate the expressivity of dense Transformers while reducing computational cost. Hierarchical and distance-structured GT variants, such as HDSE ( Luo et al. 2024 ) , further refine how multi-scale

Read the complete text