UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs
Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, Bryan Hooi · 2025
Read UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs by Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, Bryan Hooi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-00806.txt.
Introduction
Yufei He and co-authors build a foundation model for graphs whose nodes carry more than text. Their 2025 paper introduces UniGraph2, aimed at multimodal graphs: networks where each node bundles features from several modalities, like images and text, and edges carry the relationships. Existing graph foundation models mostly assume text-only nodes and stumble on this richer structure.
Modality-specific encoders feed a graph neural network that learns one shared embedding space. A new pretraining scheme spans many graph domains at once, and a mixture-of-experts component aligns features across domains and modalities. Across representation, transfer, and generative tasks the model beat the state of the art. The ambition mirrors what unified embedding models did for web search and recommendation, extended to structured data. If graphs are where an organization's knowledge actually lives, a model that reads all their modalities at once is a platform, not a feature.
Catalog record
| CREATOR | Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, Bryan Hooi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-00806
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, Bryan Hooi Abstract . Existing foundation models, such as CLIP, aim to learn a unified embedding space for multimodal data, enabling a wide range of downstream web-based applications like search, recommendation, and content classification. However, these models often overlook the inherent graph structures in multimodal datasets, where entities and their relationships are crucial. Multimodal graphs (MMGs) represent such graphs where each node is associated with features from different modalities, while the edges capture the relationships between these entities. On the other hand, existing graph foundation models primarily focus on text-attributed graphs (TAGs) and are not designed to handle the complexities of MMGs. To address these limitations, we propose UniGraph2 1 1 1 The code is available at https://github.com/yf-he/UniGraph2 , a novel cross-domain graph foundation model that enables general representation learning on MMGs, providing a unified embedding space. UniGraph2 employs modality-specific encoders alongside a graph neural network (GNN) to learn a unified low-dimensional embedding space that captures both the multimodal information and the underlying graph structure. We propose a new cross-domain multi-graph pre-training algorithm at scale to ensure effective transfer learning across diverse graph domains and modalities. Additionally, we adopt a Mixture of Experts (MoE) component to align features from different domains and modalities, ensuring coherent and robust embeddings that unify the information across modalities. Extensive experiments on a variety of multimodal graph tasks demonstrate that UniGraph2 significantly outperforms state-of-the-art models in tasks such as representation learning, transfer learning, and multimodal generative tasks, offering a scalable and flexible solution for learning on MMGs. Conference: Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Proceedings of the ACM Web Conference 2025 (WWW ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3696410.3714818 ISBN: 979-8-4007-1274-6/25/04 CCS: Information systems Data mining CCS: Computing methodologies Neural networks CCS: Information systems Social networks † † cc-license: by 1. Introduction Real-world web applications increasingly rely on multimodal data, where information is derived from a variety of sources such as text, images, and audio ( Ngiam et al., 2011 ; Baltrušaitis et al., 2018 ; Sui et al., 2025 ; He et al., 2025 ; Chen et al., 2025 ; Liu et al., 2025 ) . Recent foundation models have focused on learning a unified embedding space across different modalities that allows for the seamless integration of multimodal data, thereby enabling effective cross-modal interactions and supporting downstream applications ( Radford et al., 2021 ; Girdhar et al., 2023 ; Gao et al., 2025 ) . Models such as CLIP ( Radford et al., 2021 ) have demonstrated the power of learning from multimodal data by mapping text and images into a shared embedding space. However, CLIP and similar models are fundamentally limited by their reliance on a 1-to-1 mapping between paired modalities, such as text-to-image alignment, ignoring more complex structures where nodes can be connected through many-to-many relationships and involve multiple modalities. These models fail to account for the graph structure present in numerous web domains, from social networks to e-commerce networks ( Yoon et al., 2023 ; Zhu et al., 2024 ; Ektefaie et al., 2023 ; Zhang et al., 2021 ; He and Ma, 2022 ) , where entities and their interactions are crucial to understanding the underlying relationships. For example, in e-commerce platforms, recommendation systems rely on complex networks of products, users, and their interactions ( Schafer et al., 2001 ) . Each node represents a a user or a product, and edges represent interactions like purchases, views, or reviews. Additionally, both users and products are associated with rich multimodal data: product descriptions (text), images (visual), and user reviews (text), and demonstration videos (audio and visual). Integrating these diverse data types within the graph structure is essential for accurate recommendations and personalized user experiences ( Gao et al., 2022 ) . To address these challenges, Multimodal Graphs (MMGs) have been introduced as a framework that combines graph structures with multimodal data ( Zhu et al., 2024 ; Ektefaie et al., 2023 ) . On MMGs, nodes are enriched with information from multiple modalities, allowing for a more comprehensive representation of entities and their relationships. However, existing MMGs learning methods can only train models individually for a specific graph and task ( Yoon et al., 2023 ; Chen et al., 2022 ; Zeng et al., 2023 ) , and cannot achieve cross-graph and cross-task transfer like foundation models do without retraining or fine-tuning. Recently, there has been considerable progress in learning foundation models for text-attributed graphs (TAGs) ( He et al., 2024d ; He et al., 2024e ; He et al., 2024a ; Chien et al., 2022 ) , which can be viewed as a special case of MMGs where the node features are are exclusively in the text modality. One prominent effort in this direction is UniGraph ( He et al., 2024d ) , which introduces a unified embedding space that combines graph structure and node-level textual information for all TAGs. UniGraph employs a masked prediction framework ( He et al., 2022 ; Kenton and Toutanova, 2019 ; Radford, 2018 ) , inspired by the success of masked language models (MLMs) ( Kenton and Toutanova, 2019 ) . In this framework, UniGraph performs self-supervised pre-training by masking node-level text attributes and learning to predict the missing information based on the graph context. Despite its effectiveness on TAGs, UniGraph faces two significant limitations when extended to more complex settings. First, it is limited in its ability to generalize to MMGs, where nodes may contain features from diverse modalities such as images, in addition to text. Second, UniGraph focuses on pre-training on a single graph from one domain, which restricts its capacity to leverage knowledge across multiple domains. In training a foundation model, it is essential to employ more diverse pre-training data from different domains to enhance the model’s generalization ( Radford et al., 2021 ; Achiam et al., 2023 ; Girdhar et al., 2023 ) . Presented Work. In this work, we propose UniGraph2, a graph foundation model for MMGs that provides a unified embedding space across graph domains and modalities, as shown in Figure 2 . In UniGraph2, nodes are not restricted to textual attributes; instead, they can incorporate features from any combination of modalities. Similar to UniGraph, UniGraph2 adopts a masked prediction framework, but generalizes the masked prediction task to accommodate multimodal data. In this setup, the model is tasked with predicting missing node attributes, which could be text, image features, or any other modality, based on the graph structure and the available multimodal information. This allows the model to learn rich, unified representations that capture both the multimodal features of each node and the relationships encoded in the graph. Furthermore, while UniGraph focuses on pre-training within a single graph domain, UniGraph2 introduces a more robust multi-graph pre-training strategy. In real-world applications, data often comes from multiple sources, each with different graph structures and node modalities. To handle this, UniGraph2 proposes a cross-domain multi-graph pre-training framework, which enables the model to learn compact and transferable knowledge across a diverse set of graph datasets with varying modality and domain distributions.