GraphPPD: Posterior Predictive Modelling for Graph-Level Inference

Soumyasundar Pal, Liheng Ma, Amine Natik, Yingxue Zhang, Mark Coates · 2025

Read GraphPPD: Posterior Predictive Modelling for Graph-Level Inference by Soumyasundar Pal, Liheng Ma, Amine Natik, Yingxue Zhang, Mark Coates (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16995.txt.

Introduction

Soumyasundar Pal and co-authors present a framework for uncertainty-aware prediction on whole graphs, in work from 2025. Graph neural networks classify networks such as molecules or social graphs. Knowing how unsure a prediction is helps users make safer decisions, yet most uncertainty methods target single nodes or links, not whole graphs.

Their framework learns the distribution of likely labels for a graph, built on an embedding from an existing graph network. The approach adapts to the data rather than fixing a shape in advance. Tests on several benchmark datasets show the method works well in practice. The authors position it as a general add-on for graph-level tasks like molecule screening, where flagging doubtful predictions matters as much as the predictions themselves.

Catalog record

CREATORSoumyasundar Pal, Liheng Ma, Amine Natik, Yingxue Zhang, Mark Coates
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-16995

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GraphPPD: Posterior Predictive Modelling for Graph-Level Inference Soumyasundar Pal, Liheng Ma, Amine Natik, Yingxue Zhang, Mark Coates Abstract Accurate modelling and quantification of predictive uncertainty is crucial in deep learning since it allows a model to make safer decisions when the data is ambiguous and facilitates the users’ understanding of the model’s confidence in its predictions. Along with the tremendously increasing research focus on graph neural networks (GNNs) in recent years, there have been numerous techniques which strive to capture the uncertainty in their predictions. However, most of these approaches are specifically designed for node or link-level tasks and cannot be directly applied to graph-level learning problems. In this paper, we propose a novel variational modelling framework for the posterior predictive distribution (PPD) to obtain uncertainty-aware prediction in graph-level learning tasks. Based on a graph-level embedding derived from one of the existing GNNs, our framework can learn the PPD in a data-adaptive fashion. Experimental results on several benchmark datasets exhibit the effectiveness of our approach. 1 Introduction Recently, graph neural networks (GNNs) have become the de-facto tool to analyze and learn from network structured data, which is pervasive in neuroscience ( Griffa et al. 2017 ) , chemistry ( Duvenaud et al. 2015 ; Gilmer et al. 2017 ) , recommendation ( Monti et al. 2017 ; Ying et al. 2018a ) , and social sciences ( Kipf & Welling 2017 ; Monti et al. 2019 ) . Several avenues have been explored for designing more expressive GNN architectures ( Xu et al. 2019 ; Morris et al. 2019 ) . Despite their success, most of these approaches suffer from the major weakness that they cannot gauge the uncertainty associated with their predictions. Assessment of predictive uncertainty is crucial in many practical applications if a decision is to be made using the predictions. In order to address this drawback, there have been research efforts to equip Graph Neural Networks with uncertainty characterization. However, most of these techniques are tailored to node classification ( Zhang et al. 2019 ; Ma et al. 2019 ; Elinas et al. 2020 ) ), link prediction ( Pal et al. 2020 ; Opolka & Lió 2022 ) , and recommendation ( Sun et al. 2020 ) and cannot handle graph-level learning tasks such as those appearing in ( Xu et al. 2019 ; Hu et al. 2021 ; Dwivedi et al. 2020 ; Morris et al. 2020 ; Yang et al. 2022 ) . Graph-level inference has diverse applications in molecular property understanding ( Hu et al. 2021 ) , circuit performance prediction ( Hakhamaneshi et al. 2022 ) , protein structure inference ( Xu et al. 2019 ) , and many other fields. Existing techniques from the conventional Bayesian deep learning ( Gal & Ghahramani 2016 ; Li et al. 2016 ) literature are either ineffective for quantifying uncertainty or computationally costly when applied to model the uncertainty associated with the parameters of GNNs. In this paper, we consider amortized approximation of the posterior predictive density (PPD). We employ a variational inference formulation for tackling graph-level learning tasks in a supervised setting. This framework permits a principled utilization of the training set of graphs and their labels during the inference stage when forming predictions on the test set graphs. Our design of the PPD module is based on a cross-attention mechanism between the embeddings of the query graphs and those of selected training set graphs with known labels. This allows for a direct comparison between different labeled graphs during training, which leads to improved learning of a generalizable PPD module. Comparison with labelled training graphs during the inference stage ameliorates the prediction performance. We evaluate our designed GraphPPD module on 18 commonly used graph classification and regression tasks, as well as on a selective prediction ( Kivlichan et al. 2021 ) task to validate the effectiveness of its predictive uncertainty. The novel contributions in this paper are as follows: 1) We propose a novel variational inference based PPD learning framework for application in graph-level learning tasks. 2) Our framework can incorporate existing GNNs as graph encoders and can be extended to other domains using suitable feature extractor modules. 3) In addition to end-to-end training, our approach allows use of pre-trained graph encoders for learning the PPD module with considerably lower training time. 4) Experimental results on diverse benchmark graph-level learning tasks show that the incorporation of several GNNs and a graph transformer for graph-level representation learning inside our framework proves beneficial in most cases. Besides, we show our predictive uncertainty can effectively serve as the review criterion in the selective prediction ( Kivlichan et al. 2021 ) task. 2 Preliminary In a supervised setting, the learning goal is to provide a label \mathbf{y}^{\prime} for a data instance with features \mathbf{x}^{\prime} . In presence of a training dataset \mathcal{D}_{\mathcal{L}}=\{(\mathbf{x}_{i},\mathbf{y}_{i})\}_{i\in\mathcal{L}} , obtained by i.i.d. sampling from the same underlying data distribution p_{true}(\mathbf{x},\mathbf{y}) as that of (\mathbf{x}^{\prime},\mathbf{y}^{\prime}) , we aim to learn a conditional distribution model p(\mathbf{y}|\mathbf{x};\gamma) , parameterized by \gamma , which is subsequently used for labelling \mathbf{x}^{\prime} . In the Bayesian setting, the model parameter \gamma is assumed to be random variable and a suitable prior p(\gamma) is adopted. In order to account for parameter uncertainty while forming the prediction for a new input, the posterior predictive distribution (PPD) is evaluated as the expected predictive density w.r.t. to the posterior distribution of \gamma , as shown below: \displaystyle p(\mathbf{y}^{\prime}|\mathbf{x}^{\prime},\mathcal{D}_{\mathcal{L}})=\int p(\mathbf{y}^{\prime}|\mathbf{x}^{\prime};\gamma)p(\gamma|\mathcal{D}_{\mathcal{L}})\,d\gamma\,. (1) The training objective is to compute the posterior distribution of \gamma , which follows Bayes’ rule: \displaystyle p(\gamma|\mathcal{D}_{\mathcal{L}}) \displaystyle\propto p(\gamma)\prod_{i\in\mathcal{L}}p(\mathbf{y}_{i}|\mathbf{x}_{i};\gamma)\,. (2) If the chosen family of models contains the ‘true’ conditional distribution, i.e., if there exists a \gamma^{*} such that p(\mathbf{y}|\mathbf{x};\gamma^{*})=p_{true}(\mathbf{y}|\mathbf{x}) holds for all (\mathbf{x},\mathbf{y}) , the PPD results in optimal prediction ( Nogales 2022 ) . In most practical cases, neither computing the posterior in eq. ( 2 ) nor evaluating the PPD in eq. ( 1 ) can be done analytically. Several techniques for tractable approximation of the posterior p(\gamma|\mathcal{D}_{\mathcal{L}}) have been proposed based on expectation propagation ( Hernández-Lobato & Adams 2015 ) and variational inference (VI) ( Gal & Ghahramani 2016 ; Sun et al. 2017 ; Louizos et al. 2017 ) . Typically, such approximations are easy to sample from (e.g., multivariate normal distribution with diagonal covariance structure), which allows us to form Monte Carlo estimates of the PPD subsequently. Another class of algorithms resorts to sampling from the posterior via various Markov Chain Monte Carlo (MCMC) methods ( Neal 1993 ; Korattikara et al. 2015 ; Li et al. 2016 ; Izmailov et al. 2021 ) . Although these approaches are effective, VI generally introduces a difficult-to-characterize bias in estimating the PPD and underestimates the predictive uncertainty. On the other hand, MCMC approaches often require prohibitively expensive computation. In contrast, recently there has been another line of work ( Gordon et al. 2019 ; Garnelo et al. 2018b ; Müller et al. 2021 ) , which attempts to learn an approximation of the PPD from the training data directly using neural networks. 3 Problem Statement We address variational modelling of the PPD for graph-level learning tasks, where our goal is to approximate the distribution of the label \mathbf{y} of

Read the complete text