Simple Graph Contrastive Learning via Fractional-order Neural Diffusion Networks
Yanan Zhao, Ji Feng, Kai Zhao, Xuhao Li, Qiyu Kang, Wenfei Liang, Yahya Alkhatib, Xingchao Jian, Wee Peng Tay · 2025
Read Simple Graph Contrastive Learning via Fractional-order Neural Diffusion Networks by Yanan Zhao, Ji Feng, Kai Zhao, Xuhao Li, Qiyu Kang, Wenfei Liang, Yahya Alkhatib, Xingchao Jian, Wee Peng Tay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16748.txt.
Introduction
Yanan Zhao and co-authors build a graph learning method that needs no data augmentation and no negative samples. Contrastive learning teaches a model by pulling matching views together and pushing others apart. On graphs, making those views has meant either heavy editing tricks or special encoders. Their 2025 paper draws the views from fractional diffusion instead.
Learnable encoders follow fractional differential equations, whose order setting steers each view toward local or global structure. Two views of one graph then differ in a principled way, ready for contrast. The method works on graphs where neighbors mostly match and on those where they mostly differ. Graph learning readers get a simpler contrastive recipe with the augmentation machinery removed.
Catalog record
| CREATOR | Yanan Zhao, Ji Feng, Kai Zhao, Xuhao Li, Qiyu Kang, Wenfei Liang, Yahya Alkhatib, Xingchao Jian, Wee Peng Tay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-16748
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Simple Graph Contrastive Learning via Fractional-order Neural Diffusion Networks Yanan Zhao, Ji Feng, Kai Zhao, Xuhao Li, Qiyu Kang, Wenfei Liang, Yahya Alkhatib, Xingchao Jian, Wee Peng Tay Abstract Graph Contrastive Learning (GCL) has recently made progress as an unsupervised graph representation learning paradigm. GCL approaches can be categorized into augmentation-based and augmentation-free methods. The former relies on complex data augmentations, while the latter depends on encoders that can generate distinct views of the same input. Both approaches may require negative samples for training. In this paper, we introduce a novel augmentation-free GCL framework based on graph neural diffusion models . Specifically, we utilize learnable encoders governed by Fractional Differential Equations (FDE). Each FDE is characterized by an order parameter of the differential operator. We demonstrate that varying these parameters allows us to produce learnable encoders that generate diverse views, capturing either local or global information, for contrastive learning. Our model does not require negative samples for training and is applicable to both homophilic and heterophilic datasets. We demonstrate its effectiveness across various datasets, achieving state-of-the-art performance. mathx"17 1 Nanyang Technological University, Singapore 2 Anhui University, China 1 Introduction Contrastive learning is a powerful unsupervised learning technique that has gained significant attention in representation learning. It focuses on learning meaningful representations by distinguishing between similar and dissimilar feature embeddings generated from different encoders . The learning process pulls similar instances closer together while pushing dissimilar ones apart in the feature space. This approach enables models to capture important patterns without requiring extensive labeled data. Contrastive learning has been widely applied in areas such as computer vision, natural language processing, and recommender systems. When this technique is applied to unsupervised learning involving graph-structured data, it is known as Graph Contrastive Learning (GCL) . GCL methods can be categorized into augmentation-based and augmentation-free approaches (see Section 2 ). Augmentation-based methods rely on complex data augmentations, while augmentation-free methods depend on encoders that can generate distinct views of the same input. Both approaches may require negative samples for training. We focus on the augmentation-free approach due to its simplicity and independence from the quality of augmentations. However, the success of an augmentation-free approach hinges on two factors: (a) the ability of the encoders to generate high-quality feature embeddings, and (b) the capability of contrasting encoders to produce distinct views of the same input. To address these requirements, we propose a novel GCL framework that utilizes neural diffusion-based encoders to generate contrasting views of node features. To explain our main insight regarding applying neural diffusion in GCL, recall that recent works Chamberlain et al. (2021b) ; Thorpe et al. (2022) ; Chamberlain et al. (2021a) ; Song et al. (2022) ; Rusch et al. (2022) ; Kang et al. (2023) have introduced graph diffusion based on ordinary differential equations (ODE). This approach is analogous to heat diffusion over a graph and can be viewed as a continuous substitution for the message passing of node features in models such as GCN. Diffusions based on fractional-order differential equations (FDE) with the differential operator \frac{\,\mathrm{d}^{\alpha}}{\,\mathrm{d}t^{\alpha}} , which generalize ODE-based diffusions, have been proposed in Kang et al. (2024) ; Zhao et al. (2024) . The key parameter is the order \alpha\in(0,1] of the derivative of features with respect to (w.r.t.) time t . For instance, \alpha=1 corresponds to the usual derivative \frac{\mathrm{d}{}}{\mathrm{d}{t}} . FDE allows \alpha to be a real number, e.g., 0<\alpha<1 , with the interpretation that it governs how much “non-local” information (from the past feature evolution history) is incorporated in the diffusion. Therefore, we can adjust \alpha to control whether the features contain more global or local information. By choosing different \alpha values for different FDE-based encoders, we can generate features with distinct views, which is essential for augmentation-free GCL. The term diffusion has been associated with different meanings depending on the context. In our paper, it specifically refers to the dynamic of the features following a prescribed differential equation. Our main contributions are as follows: • We introduce a novel augmentation-free GCL model utilizing neural diffusion-based encoders on graphs. This model is both simple and flexible, with the feature view properties primarily governed by a single parameter within the continuous domain (0,1] . • We provide a theoretical analysis of the model using the framework of FDE-based graph neural diffusion models, offering insights into the features generated by contrasting encoders. Based on the observations, we propose a new way to regularize contrastive loss, avoiding the use of negative samples. • We conduct extensive numerical experiments on datasets with varying characteristics (e.g., homophilic vs. heterophilic) and demonstrate the model’s superior performance compared to existing benchmarks. 2 Related Works In this section, we summarize recent advancements in GCL that are related to our work. Additional details on these methods can be found in Appendix B . Augmentation-based GCL models typically use various data augmentations, such as edge removal and node feature masking, to create diverse graph views. These models optimize the loss by maximizing mutual information between the augmented views ( Velickovic et al., 2019 ; Hassani & Khasahmadi, 2020a ; Zhang et al., 2023 ; Zhu et al., 2020 ; Zhu et al., 2021 ; Chen & Kou, 2023 ; You et al., 2020 ; Xiao et al., 2022 ; Chen et al., 2022 ) . This approach enhances model robustness and generalization. In contrast, augmentation-free methods do not rely on complex augmentations. They directly input the same graph and features into different encoders to generate contrasting views ( Peng et al., 2020 ; Zhang et al., 2022 ) . Notably, methods such as GraphACL ( Xiao et al., 2023 ) and SP-GCL ( Wang et al., 2023 ) strictly avoid augmentations but still use negative samples. Similarly, PolyGCL ( Chen et al., 2024 ) applies feature shuffling for negative samples. These approaches move away from the traditional homophily assumption, enabling more effective learning in heterophilic and structurally diverse graphs by incorporating advanced encoding and spectral filtering techniques. A notable trend in GCL models is the elimination of negative sample pairs. For instance, BGRL ( Thakoor et al., 2022 ) and CCA-SSG ( Zhang et al., 2021 ) minimize the need for augmentations and completely remove negative sampling, focusing on maximizing correlations between graph views. AFGRL ( Lee et al., 2022 ) takes this further by eliminating both augmentations and negative samples, directly using the original graph to create positive samples. However, its applicability is primarily limited to homophilic datasets, posing challenges in handling heterophilic or structurally diverse graphs. Our approach neither requires any data augmentation nor negative sampling , while effectively handling both homophilic and heterophilic datasets. Therefore, we regard our model with these characterizations as a “simple” form of GCL in the title. As highlighted in Section 1 , unlike previous works, we use encoders based on fractional-order neural diffusion to generate pairs of feature representations for different views. As far as we are aware of, this is the first work along this line. 3 Preliminaries Setup and problem formulation Consider an undirected graph \mathcal{G}=(\mathcal{V},\mathcal{E}) , where \mathcal{V}=\{v_{1},\ldots,v_{N}\} is a finite set of N nodes and \mathcal{E}\subset\mathcal{V}\times\mathcal{V} is the set of edges. The raw features of the nodes are represented by the