MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning
Xunkai Li, Yuming Ai, Yinlin Zhu, Haodong Lu, Yi Zhang, Guohao Fu, Bowen Fan, Qiangqiang Dai, Rong-Hua Li, Guoren Wang · 2026
Read MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning by Xunkai Li, Yuming Ai, Yinlin Zhu, Haodong Lu, Yi Zhang, Guohao Fu, Bowen Fan, Qiangqiang Dai, Rong-Hua Li, Guoren Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22416.txt.
Introduction
Xunkai Li and a large team built MM-OpenFGL, a broad test for learning on graphs that mix many data types. Such graphs pair links with images, text, and more. Their 2026 preprint notes that real graphs sit split across firms and labs, barred from pooling by privacy and trade rules.
Federated learning trains across those splits without moving the raw data. The bench tests such methods in a shared, fair way. Good benches keep claims honest and progress clear. This one covers a fast-growing, messy corner of the field. It gives builders a common yardstick.
Catalog record
| CREATOR | Xunkai Li, Yuming Ai, Yinlin Zhu, Haodong Lu, Yi Zhang, Guohao Fu, Bowen Fan, Qiangqiang Dai, Rong-Hua Li, Guoren Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22416
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning Xunkai Li, Yuming Ai, Yinlin Zhu, Haodong Lu, Yi Zhang, Guohao Fu, Bowen Fan, Qiangqiang Dai, Rong-Hua Li, Guoren Wang Abstract Multimodal-attributed graphs (MMAGs) provide a unified framework for modeling complex relational data by integrating heterogeneous modalities with graph structures. While centralized learning has shown promising performance, MMAGs in real-world applications are often distributed across isolated platforms and cannot be shared due to privacy concerns or commercial constraints. Federated graph learning (FGL) offers a natural solution for collaborative training under such settings; however, existing studies largely focus on single-modality graphs and do not adequately address the challenges unique to multimodal federated graph learning (MMFGL). To bridge this gap, we present MM-OpenFGL, the first comprehensive benchmark that systematically formalizes the MMFGL paradigm and enables rigorous evaluation. MM-OpenFGL comprises 19 multimodal datasets spanning 7 application domains, 8 simulation strategies capturing modality and topology variations, 6 downstream tasks, and 57 state-of-the-art methods implemented through a modular API. Extensive experiments investigate MMFGL from the perspectives of necessity, effectiveness, robustness, and efficiency, offering valuable insights for future research on MMFGL. 1 Introduction Multimodal-attributed graph (MMAG) serves as a powerful modeling technique for relational data in which nodes are associated with heterogeneous modalities (e.g., text and images) and edges encode interactions among entities. By jointly modeling complementary multimodal signals, recent MMAG methods have achieved strong performance across a wide range of applications. However, these advances predominantly rely on a centralized learning paradigm, implicitly assuming that all modalities and graph structures are accessible to a single graph learning system. However, such an assumption is often violated in practice, where MMAGs are naturally distributed across multiple data owners due to privacy constraints or commercial competition. For example, different platforms independently maintain social graphs over disjoint user populations, where posts combine textual content with visual attachments, yet cross-platform data sharing is prohibited. As a result, existing centralized MMAG frameworks are fundamentally limited in their ability to exploit decentralized multimodal knowledge, hindering global cross-modal alignment and the discovery of collective multimodal intelligence. To address these challenges, a natural recourse is the adoption of federated graph learning (FGL), a paradigm that has demonstrated significant efficacy across diverse decentralized graph learning scenarios. By facilitating the collaborative training of graph models through the exchange of model updates instead of raw data, FGL preserves data privacy while simultaneously capitalizing on decentralized knowledge. Nevertheless, existing FGL frameworks are predominantly tailored for single-modality attributed graphs characterized by homogeneous node features. In such settings, the primary learning obstacles arise from structural heterogeneity or shifts in label distributions. Extending FGL to multimodal environments gives rise to multimodal federated graph learning (MMFGL), a transition that is fundamentally non-trivial. As evidenced by our empirical study in Sec. 4.1 ( Q2 ), naive adaptations of conventional FL or FGL methodologies fail to reconcile cross-modal semantic conflicts and cross-client structural mismatches, occasionally yielding results inferior to those of isolated training. Consequently, there is an urgent necessity to establish a systematic foundation for MMFGL. Table 1: Comparison between existing FGL benchmarks and our proposed MM-OpenFGL for MMFGL. Benchmark Multimodal Simulation Algorithms Downstream Tasks Evaluations Bkd-FedGNN ( 35 ) ✗ 2 (Label) 4 2 Graph-Level 6 OpenFGL ( 31 ) ✗ 4 (Topo. & Label) 18 5 Graph-Level 11 FedGraph ( 66 ) ✗ 2 (Label) 14 3 Graph-Level 7 MM-OpenFGL ✓ 8 (2 Modal. & Topo. & Label) 57 4 Graph-Level 5 Modality-Level 14 Despite its importance, MMFGL still remains largely underexplored, primarily due to three fundamental gaps: ❶ Lack of Problem Formalization: There is no unified formulation that systematically characterizes different MMFGL settings and tasks in the current literature. This lack of clarity prevents researchers from pinpointing the unique challenges arising from the intersection of MMAGs and decentralized learning. ❷ Lack of Benchmarking System: Current FGL libraries (Table 1 ) lack support for multimodal graph datasets, modality-specific simulation processes, and related learning tasks. A standardized, accessible benchmark featuring open datasets and reference implementations is essential to foster the development of MMFGL methodologies. ❸ Lack of Directional Insights: Systematic empirical studies on MMFGL are largely absent. Without experimental evidence regarding how factors like modality heterogeneity or cross-modal alignment influence federated training dynamics, it remains unclear which challenges are most critical. This empirical void hinders the principled design and rigorous evaluation of MMFGL algorithms. To this end, we propose MM-OpenFGL, the first comprehensive benchmark specifically designed to formalize the MMFGL paradigm. MM-OpenFGL can be viewed as a systematic extension of OpenFGL: while OpenFGL focuses on single-modal graphs and provides a foundation for FGL evaluation, MM-OpenFGL inherits its modular API but substantially extends its scope to MMAGs and federated graph foundation model (Appendix A.3 for details). Specifically, MM-OpenFGL integrates 19 multimodal graph datasets across 7 application domains, 8 simulation strategies, 57 state-of-the-art algorithms, and 9 graph- and modality-based downstream tasks providing a unified and scalable framework for benchmarking MMFGL methods. Using this benchmark, we conduct an extensive empirical study of existing baselines and derive 10 key insights, covering Necessity, Effectiveness, Robustness, and Efficiency. Our Contributions. ❶ Problem Formalization. To the best of our knowledge, we are the first to introduce the multimodal federated graph learning (MMFGL) problem. ❷ Comprehensive Benchmark. We present MM-OpenFGL, a pioneering MMFGL benchmark spanning 19 multimodal datasets across 7 domains. It defines 8 distributed simulation settings based on modality, label and topology, incorporates 57 algorithms, and evaluates them from four perspectives: necessity, effectiveness, robustness, and efficiency. ❸ Valuable Insights. Through extensive empirical studies, we derive 10 conclusions, outlining promising research directions for further MM-FGL community. ❹ Community Resources. We release an open-source library and a curated repository including a systematic literature review to lower the entry barrier for MMFGL. All code, datasets, and tutorials are available at https://anonymous.4open.science/r/TEST-SA7D7A . Table 2: An overview of our proposed MM-OpenFGL benchmark. Scenarios Data Datasets Movies, Cloth, Ele-fasion, Bili cartoon, MultiMET facebook, PixelRec50K… Encoder Qwen2.5-3B-Instruct, Qwen2-7B-Instruct, Clip-Vit-Large, Llama-3.2-11B-Vision-Instruct,… Simulation Modality-IID / Modality-NonIID, Topology-Available / Topology-Unavailable, Label-IID / Label-NonIID Tasks Graph-Level(Node Classification, Link Prediction,…), Modality-Level(Modality Match, Modality Generation,…) Method Algorithms GNN GCN, GAT, GraphSAGE, SGC, GIN, ChebNet, MMA, RevGAT, BUDDY, … MM-GNN MMGCN, MGAT, GSMN, MGNet, GraphMAE2, MMGCL, LGMRec, COGMEN, DMGC, … Standard-FL FedGM, FedSPA, FedIIH, FedSSP, S2FGL, FedLap, FCGL, FGC, FedGLS, FedDEP, FedAGHN, … Heterogeneous-FL FedTGP, FML, FIARSE, ReeFL, FedTSA, MH-pFLID, PEPSY, FedMVP, FedMAC, FedMM, … GFM GFT, OFA, UniGraph, GQT, GraphCLIP, AnyGraph, GFSE, SwapGT, RAGraph Experiment Evaluations Data Analysis Feature KL Divergence, Label Distribution, Topology Statistics Effectiveness Accuracy, Precision, F1, AUC-ROC, Recall@K, MRR, CLIP Score, BLEU Robustness Noise, Sparsity, Homophily, Federated Scenario Generalization, Privacy Preserve Efficiency Convergence, Scalability, Communication, FLOPS, Time&Space Complexity 2 Problem Statement To provide a comprehensive evaluation of MMAGs in federated scenarios, we classify training pipelines into two paradigms: the End-to-End (standard FGL) pipeline for task-specific collaboration and the Two-Stage (graph foundation model) pipeline for task-agnostic representation learning, which are detailed as follows: 2.1 End-to-End Pipeline This pipeline follows the classical federated learning approach, training a task-specific model from scratch through iterative communication. Following the OpenFGL protocol, the training process at communication round t involves four steps: ❶ Receive Message : The server distributes the current global model to selected clients. Each client initializes its local model with the received parameters: \mathbf{w}^{T}_{k}\leftarrow\mathbf{\hat{w}}^{T}