Resolving Oversmoothing with Opinion Dissensus

Keqin Wang, Yulong Yang, Ishan Saha, Christine Allen-Blanchette · 2025

Read Resolving Oversmoothing with Opinion Dissensus by Keqin Wang, Yulong Yang, Ishan Saha, Christine Allen-Blanchette (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19089.txt.

Introduction

Keqin Wang, Yulong Yang, Ishan Saha, and Christine Allen-Blanchette attack a known failure of deep graph neural networks: oversmoothing. Their 2025 paper observes that as these networks deepen, every node's representation converges toward the same value, erasing the differences predictions need. The authors trace the flaw to an analogy with opinion dynamics, where message passing mirrors people averaging their neighbors' views until all agree.

Their model, BIMP, swaps the linear averaging rule for a nonlinear one that can settle into lasting disagreement, called dissensus. In experiments the network resisted oversmoothing past one hundred steps. It also beat existing architectures, even ones patched with standard anti-smoothing fixes. The opinion-dynamics lens, they argue, is a design tool, not just a diagnosis.

Catalog record

CREATORKeqin Wang, Yulong Yang, Ishan Saha, Christine Allen-Blanchette
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-19089

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Resolving Oversmoothing with Opinion Dissensus Keqin Wang, Yulong Yang, Ishan Saha, Christine Allen-Blanchette Abstract While graph neural networks (GNNs) have allowed researchers to successfully apply neural networks to non-Euclidean domains, deep GNNs often exhibit lower predictive performance than their shallow counterparts. This phenomena has been attributed in part to oversmoothing, the tendency of node representations to become increasingly similar with network depth. In this paper we introduce an analogy between oversmoothing in GNNs and consensus (i.e., perfect agreement) in the opinion dynamics literature. We show that the message passing algorithms of several GNN models are equivalent to linear opinion dynamics models which have been shown to converge to consensus for all inputs regardless of the graph structure. This new perspective on oversmoothing motivates the use of nonlinear opinion dynamics as an inductive bias in GNN models. In our Behavior-Inspired Message Passing (BIMP) GNN, we leverage the nonlinear opinion dynamics model which is more general than the linear opinion dynamics model, and can be designed to converge to dissensus for general inputs. Through extensive experiments we show that BIMP resists oversmoothing beyond 100 time steps and consistently outperforms existing architectures even when those architectures are amended with oversmoothing mitigation techniques. We also show that BIMP has several desirable properties including well behaved gradients and adaptability to homophilic and heterophilic datasets. 1 Introduction The underlying structure of a broad class of real-world systems is naturally captured by graphs. For example, in molecules, atoms can be represented by graph nodes and atomic bonds can be represented by edges [ 1 ] ; in animal groups, individuals can be represented as nodes and their relative proximity can be represented by edges [ 2 ] ; and in transportation networks, bus stops can be represented as nodes and public transit routes can be represented by edges [ 3 ] . Because of the broad applicability of graphs, their classification, regression, and generation are of strong interest across scientific communities. While MLPs can be adapted to operate on graph data, graph neural networks (GNNs) are specifically designed to respect the graph permutation symmetry (i.e., invariance to node relabeling) and can therefore learn generalizable graph representations, whereas MLPs are not. GNNs have been successfully applied to tasks as diverse as recommendation [ 4 ] , molecular prediction [ 5 ] , protein design [ 6 , 7 ] , and computer chip design [ 8 ] . However, GNN representations often become increasingly similar with network depth; a phenomena known as oversmoothing [ 9 , 10 , 11 ] . A number of approaches have been proposed to address oversmoothing in GNNs, including the use of residual connections [ 9 , 12 , 13 , 14 ] , feature normalization [ 15 , 16 , 17 ] , and alternative architectures [ 18 , 19 , 20 ] . While many alternative architectures work to incorporate higher-order features [ 21 , 22 , 23 , 24 ] , continuous-depth GNNs instead interpret conventional GNN architectures as discretizations of a continuous process [ 25 , 26 , 27 ] . This interpretation allows the application of techniques developed for modeling and analyzing dynamical systems [ 28 , 29 , 30 , 31 ] . For example, GraphCON [ 32 ] avoids oversmoothing by enforcing stability conditions on hidden states of a coupled and damped oscillator system; and ACMP [ 33 ] avoids oversmoothing by introducing repulsive forces traditionally used to control clustering in particle systems. These are just a few among many that have shown incorporating inductive biases from physical processes can mitigate oversmoothing while maintaining expressivity [ 34 ] . In this paper, we propose a continuous-depth GNN inspired by behavioral interaction, instead of physical processes. First, we introduce an analogy likening node features in a GNN to opinions in an opinion dynamics model, feature aggregation to opinion exchange, and graph outputs to opinion outcomes (Section 3 ). Using this analogy, we show that oversmoothing will always occur in GNN models with layer-wise aggregation schemes that are equivalent to linear opinion dynamics (Section 4 ). With this new understanding of oversmoothing, we leverage the nonlinear opinion dynamics model [ 35 , 36 ] to design a novel continuous-depth GNN that is provably robust to oversmoothing with desirable characteristics such as well behaved gradients and adaptability to heterophilic datasets (Section 5 ). Finally, we empirically validate our Behavior-Inspired Message Passing (BIMP) GNN on several datasets and against competitive baselines (Section 6 ). 2 Related Work Oversmoothing in GNNs. Contrary to conventional feed forward networks [ 37 , 38 ] , deep discrete GNNs suffers degradation of performance from oversmoothing of node features [ 9 , 10 , 11 ] . Further analysis across wide classes of GNN models have resulted in ways to understand this phenomenon. Focusing on specific architectures, works have revealed that simple linear GNNs exhibits oversmoothing as the added layers exacerbate the denoising and mixing effect of the network [ 39 ] ; GCNs [ 40 ] learns representations that attempts to counteract an inherently oversmoothing prone network structure [ 41 ] ; and attention based networks such as GAT [ 42 ] oversmooths at an exponential rate due to the ergodicity of infinite matrix products [ 43 ] . More generally, oversmoothing have been characterized by energy minimization of gradient flows [ 44 ] , representational rank collapse [ 45 ] , and exceeding a theoretical limit of smoothing in mean aggregation [ 46 ] . Continuous-depth GNNs. Continuous-depth networks such as NeuralODE [ 47 ] define the network depth implicitly through the simulation of differential equations. GDE [ 25 ] leverages this notion to construct continuous-depth GNNs by propagating inputs through continuum of GNN layers governed by an underlying ODE. In order to better control and understand node dynamics, further work focuses on leveraging physical inductive biases such as heat diffusion [ 26 ] , Beltrami flows [ 22 ] , wave equations [ 27 ] , coupled damped oscillators [ 32 ] , energy source terms [ 48 ] , Allen-Cahn reaction diffusion processes [ 33 ] , blurring-sharpening forces [ 49 ] , oscillator synchronization [ 50 ] , and Ricci flow [ 51 ] . These embedded dynamics provide a systemic approach to counteract known limitations of discrete GNNs such as oversmoothing of deep networks and degradation of performance on heterophilic graphs [ 34 ] . Opinion dynamics. Since opinions can be interpreted as latent preferences, modeling opinion dynamics provide insights into understanding and predicting agents’ behavior. Specifically, consensus dynamics [ 52 ] is commonly used in multi-agent settings such as coordinating multi-vehicle movements [ 53 , 54 ] , understanding network systems [ 55 , 56 ] , and learning on graphs [ 57 ] . However, the naive implementation of linear models for opinion formulation results in dynamics that only converge to consensus [ 58 , 59 ] . This noted short comings of consensus dynamics models are resolved in nonlinear opinion dynamics [ 35 ] . The nonlinearity results in bifurcations [ 60 ] and hence opinions can evolve to dissensus rapidly even under weak input signals [ 36 ] . Nonlinear opinion dynamics have been shown

Read the complete text