Graffe: Graph Representation Learning via Diffusion Probabilistic Models
Dingshuo Chen, Shuchen Xue, Liuji Chen, Yingheng Wang, Qiang Liu, Shu Wu, Zhi-Ming Ma, Liang Wang · 2025
Read Graffe: Graph Representation Learning via Diffusion Probabilistic Models by Dingshuo Chen, Shuchen Xue, Liuji Chen, Yingheng Wang, Qiang Liu, Shu Wu, Zhi-Ming Ma, Liang Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04956.txt.
Introduction
Dingshuo Chen and colleagues turn diffusion models, famous for generating images, into graph representation learners. Their system, Graffe, encodes a source graph into a compact representation. That representation then conditions a diffusion decoder's denoising process. Learning to denoise becomes learning to understand the graph.
Theory backs the design: the denoising loss lower-bounds the conditional mutual information between data and representation, so training provably squeezes information into the code. Case studies validate the theory's predictions. Under linear probing, Graffe proved competitive on node and graph classification. It reached state-of-the-art results on nine of eleven real-world data sets. The findings argue that strong generative models double as strong representation learners. Diffusion's talent for structure, the paper shows, transfers from pixels to graphs.
Catalog record
| CREATOR | Dingshuo Chen, Shuchen Xue, Liuji Chen, Yingheng Wang, Qiang Liu, Shu Wu, Zhi-Ming Ma, Liang Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04956
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Graffe: Graph Representation Learning via Diffusion Probabilistic Models Dingshuo Chen, Shuchen Xue, Liuji Chen, Yingheng Wang, Qiang Liu, Shu Wu, Zhi-Ming Ma, Liang Wang Abstract Diffusion probabilistic models (DPMs), widely recognized for their potential to generate high-quality samples, tend to go unnoticed in representation learning. While recent progress has highlighted their potential for capturing visual semantics, adapting DPMs to graph representation learning remains in its infancy. In this paper, we introduce Graffe , a self-supervised diffusion model proposed for graph representation learning. It features a graph encoder that distills a source graph into a compact representation, which, in turn, serves as the condition to guide the denoising process of the diffusion decoder. To evaluate the effectiveness of our model, we first explore the theoretical foundations of applying diffusion models to representation learning, proving that the denoising objective implicitly maximizes the conditional mutual information between data and its representation. Specifically, we prove that the negative logarithm of the denoising score matching loss is a tractable lower bound for the conditional mutual information. Empirically, we conduct a series of case studies to validate our theoretical insights. In addition, Graffe delivers competitive results under the linear probing setting on node and graph classification tasks, achieving state-of-the-art performance on 9 of the 11 real-world datasets. These findings indicate that powerful generative models, especially diffusion models, serve as an effective tool for graph representation learning. I Introduction Self-supervised learning (SSL), which enables effective data understanding without laborious human annotations, is emerging as a key paradigm for addressing both generative and discriminative tasks. When we revisit the evolution of SSL across these two tasks, interestingly, a mutually reinforcing manner becomes evident: Progress in one aspect often stimulates progress in the other. For instance, autoencoder [ 1 ] , which initially made a mark in feature extraction, laid the foundation for the success of VAEs [ 2 ] for sample generation. Conversely, breakthroughs in generative tasks like autoregression [ 3 ] and adversarial training [ 4 ] , have deepened our understanding of representation learning, driving the development of iGPT [ 5 ] and BigBiGAN [ 6 ] . Recently, diffusion models [ 7 , 8 ] have demonstrated astonishing generation quality in different domains, particularly in terms of realism, detail depiction, and distribution coverage. A natural question arises: can we draw on the successful experiences of diffusion models to enhance representation learning? This issue is particularly pressing in the context of graph learning, since generation—the ability to create—plays a less critical role compared to discrimination on graphs, e.g., social networks, citation networks, and recommendation networks. The question seems not difficult to address, as generation is considered one of the highest manifestations of learning thus having powerful capability to learn high-quality representation [ 9 , 10 , 11 , 12 ] ; however, the reality is much more complex. To generalize the representation learning power of diffusion models on graph data, two main impediments must be addressed: ① the non-Euclidean nature of graph data , which complicates the direct application of diffusion models and necessitates consideration of both structural and feature information [ 13 , 14 ] ; ② the absence of an encoder component in diffusion model prevents us from obtaining explicit data representation and finetuning encoder in downstream tasks. Motivated to overcome these challenges, we investigate how to adapt diffusion models to graph representation learning and enhance their discrimination performance. Fig. 1: The overall framework of Graffe . (Left) The input graph has certain nodes corrupted and is subsequently fed into a GNN encoder to obtain node representations as the condition. The decoder then receives both the noisy graph features \mathbf{x}_{t} and the condition \mathbf{z} as inputs to perform denoising, aiming to restore the original node features \mathbf{x}_{0} . (Right) The diffusion process of graph features and the architecture of GraphU-Net decoder. This work is particularly relevant to approaches that use diffusion models to capture high-level semantics for classification tasks while enhancing representational capacity. Those approaches can be broadly categorized into two main groups: (i) one treats part of the diffusion model itself as a feature extractor ( implicit-encoder pattern ) [ 15 , 16 , 17 ] . They obtain the latent representation from a certain intermediate layer, which inevitably exposes them to challenge ②. (ii) Another line of work jointly trains the diffusion model and an additional feature extractor ( explicit-encoder pattern ) [ 18 , 11 , 12 ] . However, the latter pattern have struggled to surpass their contrastive and auto-encoding counterparts. In this paper, we propose Graffe , which shares a philosophy similar to the explicit-encoder pattern. Starting with the optimization objective for diffusion-based SSL, we analyze diffusion representation learning (DRL) and show that it maximizes the mutual information lower bound between the learned representation and the original input, with more informative representations leading to lower denoising score matching loss, and vice versa. This suggests that DRL implicitly follows a principle akin to the InfoMax principle [ 19 , 20 ] , which we call the Diff-InfoMax principle. Furthermore, we observe from the frequency domain of graph features that DRL excels in capturing high-frequency information. Inspired by our theoretical insights, we instantiate our model with a graph neural network (GNN) encoder for explicit representation extraction and a tailored diffusion decoder, both trained from scratch in tandem. The encoder transforms the graph structure and feature information into a compact representation, which acts as a condition for the decoder together with noisy features to guide the denoising process. The main contributions of this work are three-fold: ❶We theoretically prove that the negative logarithm of the denoising score matching loss is a tractable lower bound for conditional mutual information. Building on this, we introduce the Diff-InfoMax principle, an extension of the standard InfoMax principle, showing that DRL implicitly follows it. ❷We propose an effective diffusion-based representation learning method catering to graph tasks, termed as Graffe . Equipped with random node masking and customized diffusion architecture for different task types, it can achieve sufficient graph understanding and obtain representations with rich semantic information. ❸We conduct extensive experiments on 11 classification tasks under the linear protocol, spanning node- and graph-level tasks of diverse domains. Our method can achieve state-of-the-art or near-optimal performance across all datasets. On Computer , Photo , and COLLAB datasets, our model set a new accuracy record of 91.3%, 94.2% and 81.3%, respectively. II Related Work II-A Self-supervised Learning on Graphs Contrastive methods Being popular in SSL, contrastive methods aim to learn discriminative representations by contrasting positive and negative samples. The key to obtain distinguishable representations lies in the way of constructing contrastive pairs. DGI [ 21 ] and InfoGraph [ 22 ] , based on MI maximization, corrupt graph feature and topology to construct negative samples. To avoid the underlying risk of semantic damage, GRACE [ 23 ] , GCA [ 24 ] , and GraphCL [ 25 ] use other graphs within the same batch as negatives. This approach helps to mitigate issues related to graph-specific distortions while still maintaining the contrastive nature of the objective. Other works, i.e., BGRL [ 26 ] and CCA-SSA [ 27 ] , propose to achieve contrastive learning free of negatives yet demanding strong