The Oversmoothing Fallacy: A Misguided Narrative in GNN Research

MoonJeong Park, Sunghyun Choi, Jaeseung Heo, Eunhyeok Park, Dongwoo Kim · 2025

Read The Oversmoothing Fallacy: A Misguided Narrative in GNN Research by MoonJeong Park, Sunghyun Choi, Jaeseung Heo, Eunhyeok Park, Dongwoo Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04653.txt.

Introduction

MoonJeong Park and co-authors argue in 2025 that a famous problem in graph neural networks has been misdiagnosed. Deep networks over graphs were said to fail because repeated averaging, called oversmoothing, makes all nodes look alike. That story has steered the field away from deep designs.

Looking at the network's three operations, the authors show the real culprit is the vanishing gradient, caused by the transform and activation steps, not the averaging step. The same failure appears in ordinary networks, so it is not special to graphs. Classic fixes, skip connections and normalization, let deep graph networks stack layers without collapse. The paper urges the field to revisit deep architectures.

Catalog record

CREATORMoonJeong Park, Sunghyun Choi, Jaeseung Heo, Eunhyeok Park, Dongwoo Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04653

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Oversmoothing Fallacy: A Misguided Narrative in GNN Research MoonJeong Park, Sunghyun Choi, Jaeseung Heo, Eunhyeok Park, Dongwoo Kim Abstract Oversmoothing has been recognized as a main obstacle to building deep Graph Neural Networks (GNNs), limiting the performance. This position paper argues that the influence of oversmoothing has been overstated and advocates for a further exploration of deep GNN architectures. Given the three core operations of GNNs, aggregation, linear transformation, and non-linear activation, we show that prior studies have mistakenly confused oversmoothing with the vanishing gradient, caused by transformation and activation rather than aggregation. Our finding challenges prior beliefs about oversmoothing being unique to GNNs. Furthermore, we demonstrate that classical solutions such as skip connections and normalization enable the successful stacking of deep GNN layers without performance degradation. Our results clarify misconceptions about oversmoothing and shed new light on the potential of deep GNNs. 1 Introduction Motivated by the success of deeper neural networks in various domains, many studies have sought to increase the depth of Graph Neural Networks (GNNs) to achieve better performance. However, these attempts often result in performance degradation rather than improvement, primarily attributed to a phenomenon known as oversmoothing, a situation in which each node embedding converges to a non-informative point as the number of layers increases [ 20 ] . The concept of oversmoothing was first introduced by Li et al. 2018 , quickly gaining acceptance within the community due to the intuitive idea that aggregation steps inherently cause embeddings of neighboring nodes to become similar. Early theoretical studies confirmed the asymptotic emergence of oversmoothing. Subsequent works, including those by Oono and Suzuki 2020 and Wu et al. 2023a , extended this perspective by demonstrating that oversmoothing measures decrease exponentially even for relatively shallow GNNs. Following this, numerous methodological innovations, including novel model architectures, regularization strategies, and normalization methods, were proposed to better analyze and mitigate oversmoothing [ 6 , 9 , 21 , 25 , 29 , 36 , 46 , 47 ] . This paper argues that there exists a common misunderstanding of oversmoothing in the community, and the actual influence of oversmoothing induced from a unique property of GNN is marginal. First, we observe that despite the initial identification of two distinct definitions of oversmoothing, subsequent studies typically adopted only one, creating confusion and hindering a clear theoretical understanding. Moreover, many previous studies overly emphasize the role of the aggregation step in oversmoothing, among the various components of GNNs. To address this gap, we systematically analyze the contributions of each of the three core components of GNN layers: i) aggregation, ii) linear transformation, and iii) nonlinear activation, to the oversmoothing. Our analysis quantitatively demonstrates that, contrary to popular belief, aggregation alone has a minimal impact on oversmoothing. Instead, our findings reveal that the nonlinear activation and linear transformation steps are responsible for the zero-collapsing phenomenon, whereby node embeddings converge toward the zero vector. Therefore, we conjecture that many previous studies have mistakenly confused zero-collapsing with oversmoothing. Through additional experiments, we demonstrate that when oversmoothing occurs (distinct from zero-collapsing), GNNs can still effectively learn and mitigate its effects. Fig. 1 illustrates how input node features, initialized identically across all nodes, evolve across layers when a model is properly trained. Zero-collapsing has long been observed in traditional multilayer perceptrons (MLPs) and recurrent neural networks, and it is widely recognized as a leading cause of vanishing gradients. Leveraging this insight, we apply well-established solutions for mitigating zero-collapsing, the residual connections and the batch normalization, to GNN architectures. Remarkably, we show that by properly integrating these two simple yet effective strategies, it is feasible to train extremely deep GNNs, successfully scaling up to 1,024 layers. Finally, we reflect on why the oversmoothing research community has treated the vanishing gradient, a classic deep learning problem, as a unique property of GNNs and discuss how this perception has limited research on deep GNNs. Figure 1: Changes in node embeddings across GCN layers, where all node features are initialized to the same value (thus initially oversmoothed), visualized via t-SNE [ 40 ] . The result implies that a GCN can recover from oversmoothing even after it has occurred . We summarize the key messages of our work as follows: • We clarify the common misconception that oversmoothing, typically understood as embeddings becoming similar, only reflects one of two fundamentally distinct phenomena. • We demonstrate that many prior studies have incorrectly labeled zero-collapsing as oversmoothing and clarify that the aggregation step in GNNs has a marginal effect on oversmoothing. • We show that the celebrated graph convolutional network can effectively overcome oversmoothing, provided that zero-collapsing is addressed separately. • We empirically verify that standard techniques used to mitigate zero-collapsing in neural networks, such as residual connections and batch normalization, remain efficient solutions. • Finally, we discuss why the current research community is missing this seemingly evident conclusion and provide historical context and potential reasons. 2 Background The term “oversmoothing” has permeated the graph neural network (GNN) community, often invoked to explain why deeper GNNs fail to yield improved performance. In this section, we introduce a general framework of GNN with two representative examples ( Section 2.1 ), a formal description of oversmoothing ( Section 2.2 ), and related works proposed to solve the oversmoothing ( Section 2.3 ). 2.1 Gentle introduction to graph neural networks We consider an undirected graph \mathcal{G}=(\mathcal{V},\mathcal{E}) , where \mathcal{V} is a set of N\in\mathbb{N} nodes and \mathcal{E}\subseteq\mathcal{V}\times\mathcal{V} is a set of edges, with additionally observed d -dimensional node feature \mathbf{X}\in\mathbb{R}^{N\times d} . We denote the d -dimensional feature of node i as \mathbf{x}_{i}\in\mathbb{R}^{d} . The connectivity between nodes in an undirected and unweighted graph can also be represented through a binary symmetric adjacency matrix \mathbf{A}\in\{0,1\}^{N\times N} , where an edge between node i and j is represented as an entry \mathbf{A}_{ij}=\mathbf{A}_{ji}=1 . In this work, we consider two variants of GNNs, graph convolutional network (GCN) [ 17 ] and graph attention network (GAT) [ 41 ] , since these two models are considered as “the target model” that needs to be fixed in many previous research on oversmoothing. Both GCN and GAT transform the input feature \mathbf{X} through L layers. At each layer \ell\in[0,L-1] , input node feature \mathbf{X}^{(\ell)} is updated as {\mathbf{X}}^{({\ell+1})}=\sigma(\hat{\mathbf{A}}\mathbf{X}^{(\ell)}{\mathbf{W}}^{({\ell})}), (1) where {\mathbf{X}}^{({0})}\coloneqq\mathbf{X} , \sigma(\cdot) is a non-linear activation function, and {\mathbf{W}}^{({\ell})} is a learnable weight matrix at layer \ell . The matrix \hat{\mathbf{A}} serves as the aggregation operator, and its structure differs significantly between GCN and GAT. In GCN, \hat{\mathbf{A}} is a fixed, symmetrically normalized adjacency matrix defined as \hat{\mathbf{A}}_{\text{{GCN}}}=\tilde{\mathbf{D}}^{-\frac{1}{2}}\tilde{\mathbf{A}}\tilde{\mathbf{D}}^{-\frac{1}{2}} where \tilde{\mathbf{A}}=\mathbf{A}+\mathbf{I} is an adjacency matrix with self-loops, \tilde{\mathbf{D}} is a degree matrix with diagonal entries (\tilde{\mathbf{D}})_{ii}=d_{i}+1 , and d_{i}=\sum_{j=1}^{N}\mathbf{A}_{ij} is the node degree. In contrast, GAT constructs \hat{\mathbf{A}} dynamically using a learnable attention mechanism. The resulting matrix \hat{\mathbf{A}}_{\text{{GAT}}} is right-stochastic, satisfying \sum_{j}(\hat{\mathbf{A}}_{\text{{GAT}}})_{ij}=1 for all i . A single layer of GCN and GAT can be factorized further into three sequential steps: aggregation, transformation, and activation. Formally, these steps are expressed as \displaystyle{\hat{\mathbf{X}}}^{({\ell+1})} \displaystyle=\hat{\mathbf{A}}\mathbf{X}^{(\ell)}\;(\text{aggregation}), (2) \displaystyle{\tilde{\mathbf{X}}}^{({\ell+1})} \displaystyle={\hat{\mathbf{X}}}^{({\ell+1})}{\mathbf{W}}^{({\ell})}\;(\text{transformation}), (3) \displaystyle{\mathbf{X}}^{({\ell+1})} \displaystyle=\sigma({\tilde{\mathbf{X}}}^{({\ell+1})})\;(\text{activation}). (4) We consider Rectified Linear Unit (ReLU) as the activation function, which has been widely used in most GNN architectures [ 17 , 41 , 43 ] . 2.2 A brief

Read the complete text