Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?

Pedro C. Vieira, Pedro Ribeiro, Viacheslav Borovitskiy · 2026

Read Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks? by Pedro C. Vieira, Pedro Ribeiro, Viacheslav Borovitskiy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22593.txt.

Introduction

Vieira, Ribeiro and Borovitskiy test whether deep ensembles, the default route to uncertainty in deep learning, work for graph neural networks. Across eight varied data sets, ensembles beat a single model only slightly, mostly by averaging away training noise in point forecasts rather than by sharpening uncertainty. The authors trace this to epistemic collapse: separately trained nets land on near identical forecasts, so their disagreement, the very signal ensembles need, vanishes. A bound tied to graph shape and smoothing energy links the collapse to message passing itself.

Catalog record

CREATORPedro C. Vieira, Pedro Ribeiro, Viacheslav Borovitskiy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46519
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22593

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks? Pedro C. Vieira, Pedro Ribeiro, Viacheslav Borovitskiy Abstract While deep ensembles are widely considered to be the default method for uncertainty quantification in deep learning, their effectiveness for graph-structured data is often simply assumed based on successes in domains like computer vision. We investigate standard deep ensembles specifically for message-passing graph neural networks. Benchmarking across seven datasets representing varied tasks and complexities, we reveal that ensembles provide surprisingly little improvement over a single model. Instead, the observed marginal gains stem primarily from stabilizing optimization noise in point predictions rather than yielding meaningfully better uncertainty estimates. Through an aleatoric-epistemic decomposition, we identify epistemic collapse : independently trained networks consistently converge to overly similar predictions. Because disagreement is the fundamental mechanism through which ensembles capture epistemic uncertainty, this lack of diversity neutralizes their key advantage. Analyzing this phenomenon further, we suggest this collapse is driven by functional rather than weight-space convexity, where distinct parameter solutions induce almost identical behavior. Our results suggest that deep ensemble success does not seamlessly transfer to graph machine learning. 1 Introduction 0 0 footnotetext: Code available at https://github.com/PedrV/gnn-uq-inspector . Uncertainty quantification is essential in high-stakes or automated decision-making applications ( 37 ; 56 ; 18 ) . Among practical approaches to uncertainty quantification, deep ensembles (DEs) 28 have emerged as a widely used method. Their appeal lies in their simplicity: building a standard deep ensemble requires only the repeated training of a single architecture. Crucially, the disagreement among these independently trained models captures uncertainty that a single model cannot. This mechanism has proven highly effective, establishing deep ensembles as the default for uncertainty quantification in deep learning, provided the added computational cost is not prohibitive ( 38 ; 44 ) . When processing graph-structured data—such as molecules or road networks—uncertainty quantification is much less studied. Here, message-passing neural networks, like graph convolutional (GCN) 25 or attention (GAT) 46 networks, have emerged as one of the dominant architecture classes. While other classes like graph transformers ( 13 ; 40 ) exist, we focus exclusively on message-passing architectures, which we refer to simply as graph neural networks (GNNs) throughout this work. Unlike standard deep learning architectures, GNN computation flow is defined by the structure of the input graph, a property that fundamentally alters the hypothesis space and how optimization trajectories explore it 54 ; 42 . Naturally, this structural difference can significantly affect how ensembles of such models behave, for better or worse. At first glance, the literature suggests that DEs are a successful, out-of-the-box solution for uncertainty quantification in graph learning tasks ( 48 ; 27 ) . However, this reputation relies largely on the assumption that the strong uncertainty quantification capabilities of DEs transfer seamlessly from classical deep learning settings like computer vision to graphs. In reality, studies that rigorously scrutinize the uncertainty quality of standard DEs for GNNs are extremely rare. 1 1 1 Modified ensemble methods exist for specific graph tasks (discussed in Section 2 ), but here we focus strictly on the standard DE mechanism of repeatedly training the same architecture from different initializations. Because this simple approach is the out-of-the-box standard in classical deep learning, it must serve as the fundamental baseline. However, this fundamental baseline is particularly poorly studied. One notable exception is 43 : focusing exclusively on molecular property prediction with GCNs, they report mixed results. More alarmingly, anecdotal evidence from the PEMS road network benchmark ( 8 ; 35 ) (a graph task explicitly designed to benchmark uncertainty quantification) shows DEs of GNNs failing dramatically and underperforming even a trivial baseline—a failure we revisit in Section 3 . Motivated by these mixed signals, in this paper we aim to provide a sober empirical evaluation of DEs of GNNs. We systematically benchmark these models across multiple tasks, focusing explicitly on the quality of their uncertainty estimates. Our results demonstrate that their behavior in graph settings falls significantly short of expectations inherited from classic deep learning settings, most notably computer vision, where DEs are most widely tested. A central finding of our study is an empirical phenomenon we term epistemic collapse : individual GNNs within a DE exhibit surprisingly limited diversity. Although they are trained independently, their predictions converge to highly similar functions. Because prediction disagreement is the primary mechanism through which deep ensembles capture epistemic uncertainty—the key advantage they offer over a single model—this lack of diversity severely limits their utility. Thus, the ensemble provides little uncertainty improvement over a much cheaper single-model baseline, even though ensemble averaging yields moderate gains in prediction accuracy. Our contributions are: 1. We systematically benchmark standard deep ensembles of GNNs across multiple classic and modern graph machine learning datasets. 2. We empirically demonstrate the phenomenon of epistemic collapse, showing that GNNs tend to converge to highly similar functions regardless of initialization. 3. We demonstrate that the minor improvements in the negative log likelihood (NLL) score provided by DEs are primarily attributable to moderately improved prediction accuracy rather than better uncertainty calibration. 4. We analyze this behavior through the lens of convexity. We hypothesize that the structural properties of GNNs restrict the optimization landscape, driving models toward the same functional solutions and explaining why the success of DEs does not transfer here. 2 Background Graph neural networks. As mentioned in Section 1 , we focus on message passing graph neural networks ( 19 ) . Let \mathcal{G} be a graph where each node i is associated with a feature vector \boldsymbol{x}_{i} . Message passing GNNs build node representations by iteratively exchanging and processing information within local neighborhoods. A typical update at layer \ell takes the form: \displaystyle\boldsymbol{h}^{(\ell+1)}_{i}=\gamma_{\mathbf{\Theta}_{1}}\bigg(\boldsymbol{h}^{(\ell)}_{i},\bigoplus\nolimits_{j\in\mathcal{N}(i)}\psi_{\mathbf{\Theta}_{2}}\left(\boldsymbol{h}^{(\ell)}_{i},\boldsymbol{h}^{(\ell)}_{j}\right)\bigg), \displaystyle\boldsymbol{h}^{(0)}_{i}=\boldsymbol{x}_{i}. (1) Here, \mathcal{N}(i) represents the neighbors of node i , a neural network \psi_{\mathbf{\Theta}_{2}} with weights \mathbf{\Theta}_{2} computes messages associated with every neighbor, \bigoplus is a permutation-invariant aggregation function, and \gamma_{\mathbf{\Theta}_{1}} post-processes the aggregated messages. While specific architectures vary—for instance, by incorporating convolution logic as in GCN ( 25 ) , or attention mechanisms as in GAT ( 46 ) —the fundamental inductive bias and computation flow remain tied strictly to the graph topology via ( 1 ). Uncertainty quantification and deep ensembles. Uncertainty quantification aims to measure not just what a model predicts, but how reliable that prediction is. Predictive uncertainty is conceptually divided into aleatoric uncertainty, which reflects irreducible noise in the data, and epistemic uncertainty, which reflects the model’s lack of knowledge ( 24 ) . Originally proposed by 28 , deep ensembles (DEs) provide a straightforward, non-Bayesian approach to uncertainty quantification. The method aggregates predictions from M copies of the same architecture trained from different random weight initializations. In a deep ensemble, aleatoric and epistemic uncertainty can be explicitly separated. For regression, the m -th member of a DE predicts the mean \mu_{m}(x) and variance \sigma^{2}_{m}(x) of a Gaussian distribution. The ensemble prediction is modeled as a Gaussian mixture, yielding an aggregated point prediction \mu(x)=\frac{1}{M}\sum_{m=1}^{M}\mu_{m}(x) . The total predictive uncertainty is the variance of this mixture, which elegantly decomposes into the average variance and the variance of the means: \underbrace{\sigma^{2}(x)}_{\text{Total}}=\underbrace{\frac{1}{M}\sum\nolimits_{m=1}^{M}\sigma_{m}^{2}(x)}_{\text{Aleatoric }\sigma^{2}_{\mathrm{ale}}(x)}+\underbrace{\frac{1}{M}\sum\nolimits_{m=1}^{M}\left(\mu_{m}(x)-\mu(x)\right)^{2}}_{\text{Epistemic }\sigma^{2}_{\mathrm{epi}}(x)}. (2) For classification, the m -th member of a DE outputs a probability distribution p_{m}(y\mid x) over classes. The ensemble outputs a

Read the complete text