Differentiated Information Mining: A Semi-supervised Learning Framework for GNNs
Long Wang ID, Kai Liu ID · 2025
Read Differentiated Information Mining: A Semi-supervised Learning Framework for GNNs by Long Wang ID, Kai Liu ID (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08769.txt.
Introduction
Long Wang and Kai Liu build a training scheme that helps graph networks learn from mostly unlabeled data. Their 2025 paper notes that a common trick, checking one model's labels against an independent second view, breaks down because true second sources are rare and rarely independent. Their framework, DiFac, instead draws several distinct factors from the same source and demands that they agree.
During training, samples whose factors clash are dropped, and candidate labels are ranked by a shortest-stave rule that resists overconfidence. The team also feeds in text knowledge from a large multimodal model as an extra factor, guarded by an accountability score. Across benchmark datasets, DiFac beat rival methods most clearly when labels were scarce.
Catalog record
| CREATOR | Long Wang ID, Kai Liu ID |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08769
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Differentiated Information Mining: A Semi-supervised Learning Framework for GNNs Long Wang ID, Kai Liu ID Abstract In semi-supervised learning (SSL) for enhancing the performance of graph neural networks (GNNs) with unlabeled data, introducing mutually independent decision factors for cross-validation is regarded as an effective strategy to alleviate pseudo-label confirmation bias and training collapse. However, obtaining such factors is challenging in practice: additional and valid information sources are inherently scarce, and even when such sources are available, their independence from the original source cannot be guaranteed. To address this challenge, In this paper we propose a Differentiated Factor Consistency Semi-supervised Framework (DiFac), which derives differentiated factors from a single information source and enforces their consistency. During pre-training, the model learns to extract these factors; in training, it iteratively removes samples with conflicting factors and ranks pseudo-labels based on the “shortest stave” principle, selecting the top candidate samples to reduce overconfidence commonly observed in confidence-based or ensemble-based methods. Our framework can also incorporate additional information sources. In this work, we leverage the large multimodal language model to introduce latent textual knowledge as auxiliary decision factors, and we design a accountability scoring mechanism to mitigate additional erroneous judgments introduced by these auxiliary factors. Experiments on multiple benchmark datasets demonstrate that DiFac consistently improves robustness and generalization in low-label regimes, outperforming other baseline methods. I Introduction Graph Neural Networks (GNNs) have demonstrated remarkable performance across various graph-structured data analysis tasks, such as node classification, link prediction, and graph classification, by effectively integrating node attributes with structural information [ 1 ] . However, their success largely depends on the availability of large-scale, high-quality labeled datasets, which are often expensive or even infeasible to obtain in real-world scenarios. Semi-Supervised Learning (SSL) offers an effective remedy by leveraging abundant unlabeled data to improve model generalization, especially in low-label settings [ 2 , 3 ] . Among existing SSL techniques, pseudo-labeling has been widely adopted for its simplicity and scalability [ 4 ] . This approach assigns “pseudo-labels” to unlabeled samples based on high-confidence predictions and incorporates them into the training set to strengthen supervision. However, pseudo-labeling is prone to confirmation bias and training collapse [ 5 , 6 ] : incorrect early predictions, once accepted, are recursively reinforced in subsequent training. In GNNs, the message-passing mechanism amplifies this issue by propagating local errors across the network. Moreover, the confidence estimates of deep models are not always reliable—high confidence does not guarantee correctness—making overconfidence particularly harmful in graph models [ 7 ] . Existing solutions can be broadly categorized into ensemble-based methods and consistency regularization methods. Ensemble-based methods train multiple models or apply multiple perturbations to the same model, and select pseudo-labels based on consensus or voting [ 8 ] . While such approaches can alleviate overconfidence, they incur substantial computational costs. Consistency regularization methods [ 9 , 4 ] enhance prediction stability under perturbations to improve robustness, but still rely on potentially flawed confidence scores. Both paradigms implicitly assume access to multiple independent “views” or factors, which is inherently difficult to achieve in many real-world scenarios—especially in single-source environments. To address this challenge, we propose a heuristic sample selection framework based on differentiated factor consistency from a single information source (Differentiating-Factor Consistency Semi- supervised Framework, DiFac). During pretraining, the model learns multiple independent or weakly correlated factors from the same source, each serving as an “internal view” of the data. In pseudo-label generation, we first perform consistency filtering: samples with conflicting predictions across factors are discarded, retaining only those with high cross-factor agreement. This selective acceptance mitigates noisy supervision, alleviates confirmation bias, and preserves the integrity of learned representations—without requiring additional models. Building upon this, we introduce a Pseudo-Label Ranking mechanism to further refine sample selection after consistency filtering. Unlike traditional approaches that rank by the highest confidence score, we use the minimum confidence across all factors as the ranking criterion. This means a sample’s final rank is determined by its weakest supporting factor, ensuring that all “views” provide sufficiently strong evidence for the label. This strategy not only theoretically suppresses the negative effects of overconfidence from individual factors but also yields more stable performance improvements in practice. Furthermore, our framework naturally extends to multi-source integration. Additional information such as graph topology, node textual descriptions, visual features, or external knowledge bases—can serve as extra factors to enrich decision-making. Recent advances in multimodal large models make it possible to generate augmented descriptions of nodes or edges, providing additional factors to enhance sample selection robustness. However, the predictive quality of additional sources can vary widely, and directly involving them in pseudo-labeling may degrade overall performance due to noisy or misleading information. To address this, we propose an Accountability Scoring mechanism: external factors do not participate in the initial filtering stage but contribute their confidence scores only during the ranking process. This “limited involvement” preserves the potential benefits of auxiliary information while avoiding its direct interference with initial selection, striking a balance between leveraging multi-source information and maintaining decision stability. Extensive experiments on benchmark citation networks such as Cora, Citeseer, and Pubmed, as well as other graph datasets, demonstrate that our method consistently improves GNN robustness and generalization in low-label regimes, outperforming conventional pseudo-labeling and confidence-based SSL approaches while avoiding the computational overhead of ensemble training. The contributions of our work are as follows: • We propose a novel pseudo-label sample selection framework based on the consistency of differentiated factors derived from a single information source, effectively mitigating confirmation bias and learning collapse inherent in traditional pseudo-labeling methods. • We design an iterative selective acceptance mechanism that ranks candidate samples by the minimum confidence across multiple factors, prioritizing reliable samples for subsequent training iterations. This strategy effectively suppresses the adverse effects of model overconfidence on pseudo-label reliability. • We demonstrate the efficacy of augmented data from large models as independent decision factors, paving the way for future extensions incorporating multimodal information sources. Additionally, we introduce an accountability scoring mechanism that preserves the potential benefits of auxiliary information while preventing its direct interference during initial pseudo-label selection. • To comprehensively evaluate our semi-supervised learning framework, we conduct extensive experiments on multiple benchmark graph datasets. Results show that under low-label and high-noise conditions, our method consistently outperforms existing pseudo-labeling and semi-supervised approaches, achieving superior accuracy, enhanced robustness, and improved training stability, thereby demonstrating strong practical value and application potential. This work not only offers a novel perspective for semi-supervised graph learning research but also contributes both theoretical insights and practical value toward developing more reliable machine learning systems. The remainder of the paper is organized as follows: Section II reviews related work. Section III presents the Differentiated Factor Consistency Sample Selection Framework (DiFac) and its application on auxiliary information sources from LMM. Efficiency comparisons between DiFac and manifold-based baseline models are provided in Section IV . Finally, conclusions and future directions are discussed in Section V . II Related Works In this section, we review recent advances in graph semi-supervised learning and knowledge enhancement via large multimodal models (LMMs), broadly divided into the following subsections: Graph-oriented Gemi-supervised Learning Techniques, and Data Enrichment via LMMs. II-A graph-oriented semi-supervised learning techniques The effectiveness of Graph Neural Networks (GNNs) has been demonstrated across a wide