Gate the Filter, Not the Message: Node-Channel Mixtures for Pre-Propagation GNNs
Zichao Yue, Zhiru Zhang, [0.5em] School of Electrical, Computer Engineering, Cornell UniversityIthaca, New York, USA · 2026
Read Gate the Filter, Not the Message: Node-Channel Mixtures for Pre-Propagation GNNs by Zichao Yue, Zhiru Zhang, [0.5em] School of Electrical, Computer Engineering, Cornell UniversityIthaca, New York, USA (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01660.txt.
Introduction
Yue and Zhang rethink filter design in pre-propagation graph networks. These networks push graph computation into a preprocessing step. Training then runs on dense hop features alone. That makes them highly scalable in practice. A puzzle: complex hop aggregators do not reliably beat simple ones. A plain MLP aggregator often matches hop-attention variants. The team revisit this from a graph-filter perspective. Over a diffusion basis, designs differ in how coefficients are shared. Sharing varies across nodes and feature channels.
FilterMoE fills a missing regime in that design space. A small bank of Chebyshev filter experts serves the network. A 3D gating tensor routes them jointly over nodes and channels. Eleven benchmarks, homophilic and heterophilic, test the design. FilterMoE wins on nine datasets and all large-scale ones.
Catalog record
| CREATOR | Zichao Yue, Zhiru Zhang, [0.5em] School of Electrical, Computer Engineering, Cornell UniversityIthaca, New York, USA |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01660
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Gate the Filter, Not the Message: Node-Channel Mixtures for Pre-Propagation GNNs Zichao Yue, Zhiru Zhang, [0.5em] School of Electrical, Computer Engineering, Cornell UniversityIthaca, New York, USA Abstract Pre-propagation graph neural networks (PP-GNNs) push all graph-dependent computation into a preprocessing step and train only on the resulting dense hop features, which makes them highly scalable. A puzzle in this regime is that more complex hop aggregators do not reliably outperform simpler ones: on many benchmarks, a plain MLP-based aggregator matches or beats hop-attention variants. We revisit this behavior from a graph-filter perspective. Over a precomputed diffusion basis, existing PP-GNNs differ mainly in how filter coefficients are shared across nodes and feature channels, rather than simply in raw aggregator capacity. MLP-based architectures learn channel-dependent filters that are largely shared across nodes, while hop-attention-based architectures learn node-dependent mixtures that are largely shared across channels. This reveals a missing regime in standard PP-GNN designs: joint node- and channel-adaptive filtering under the pre-propagation computational contract. We propose FilterMoE, a mixture-of-experts PP-GNN in which a small bank of learnable Chebyshev filter experts is routed jointly over nodes and channels by a 3D gating tensor. Across eleven homophilic and heterophilic benchmarks, FilterMoE outperforms strong PP-GNN baselines on nine datasets and ranks first on all three large-scale benchmarks, improving the average test score by 1.53 points. These results establish joint node–channel filter routing as a robust alternative to dataset-specific hop-aggregator selection. 1 Introduction Graph neural networks (GNNs) have become a standard tool for learning on relational data, but their scalability is often limited by repeated neighborhood aggregation during training ( 13 ; 15 ) . In message-passing-based GNNs (MP-GNNs), aggregation is performed at every layer and every epoch, leading to irregular sparse computation, high memory traffic, and the well-known neighbor-explosion problem ( 15 ) . Pre-propagation GNNs (PP-GNNs) address this bottleneck by moving message passing to a preprocessing stage and training only on the resulting dense node features ( 10 ; 38 ; 6 ) . This decoupled design makes PP-GNNs attractive for large-scale node classification, combining one-time graph preprocessing with up to two orders of magnitude faster training on large graph benchmarks ( 10 ; 38 ; 36 ) . Given the preprocessed node features, PP-GNNs differ mainly in the mechanisms employed for feature aggregation. However, a more complex feature aggregator does not necessarily lead to better accuracy. One might expect an attention-based feature aggregator, such as the one used in HOGA, to dominate a simpler MLP-based model such as SIGN. Our experiments do not support such a uniform conclusion: on several datasets, SIGN is as strong as or stronger than HOGA, as shown in Figure 1 . This observation suggests that the main limitation of current PP-GNNs is not feature-aggregator capacity alone. Figure 1: \Delta test score (%) of HOGA vs. SIGN on eight graphs. HOGA loses to SIGN on heterophilic graphs while outperforming SIGN on homophilic ones. Setting HOGA \mathrm{heads}=\mathrm{hidden\_channels} (triangles) does not close the gap consistently. A useful way to explain this behavior is through the lens of graph filtering. Let \mathbf{Z}=\left[\mathbf{X},\mathbf{S}\mathbf{X},\mathbf{S}^{2}\mathbf{X},\ldots,\mathbf{S}^{K}\mathbf{X}\right] denote the stack of precomputed node features generated by a diffusion operator \mathbf{S} . A PP-GNN then learns a predictor over this diffusion basis. Assuming a simple linear aggregator, this view becomes f_{\theta}(\mathbf{Z})=\sum_{k=0}^{K}\mathbf{S}^{k}\mathbf{X}\mathbf{W}_{k}, which can be interpreted as learning a graph filter over the precomputed diffusion basis. When the aggregator is more complex, as in most PP-GNNs, the learned object can be viewed as an implicit filter over the same precomputed basis. From this perspective, the key distinction between existing PP-GNNs is not raw architectural complexity, but how coefficients over the diffusion basis are shared . SIGN realizes a channel-dependent but largely node-shared filter, whereas hop-wise attention variants such as HOGA induce node-dependent but largely channel-shared mixtures. Thus, a more expressive hop aggregator is not automatically better: it may provide a different form of adaptivity, rather than a universally stronger one. This view immediately raises a second question: why not simply combine both forms of adaptivity by increasing the number of attention heads or by directly predicting nearly independent filters for all node–channel pairs? Empirically, we find that this strategy is ineffective. In particular, setting the number of HOGA heads equal to the channel dimension does not yield consistent gains, as shown in Figure 1 . Conceptually, such a design imposes a weak inductive bias: instead of assuming that many node–channel pairs share a compact repertoire of useful spectral responses, it effectively asks the model to generate almost unconstrained filters for a very large number of pairs. Prior work in spectral GNNs has also largely explored channel-adaptive filtering and node-adaptive filtering in separate lines ( 39 ; 7 ; 14 ) , leaving their joint realization largely unexplored. We therefore propose FilterMoE, a mixture-of-experts (MoE) PP-GNN that introduces joint node- and channel-adaptive filtering while preserving the pre-propagation paradigm. Our key idea is to replace direct prediction of node–channel-specific polynomial coefficients with a small bank of learnable Chebyshev filter experts and an input-dependent routing mechanism. Concretely, each expert is a Chebyshev polynomial filter, which provides an explicit spectral response and efficient preprocessing. A 3D gating tensor \mathbf{G}\in\mathbb{R}^{N\times M\times F} then mixes the M experts jointly across nodes and feature channels, so that each node–channel pair receives its own effective filter while all filters remain compositions of a small shared bank. We instantiate the router in two practical forms: a simple MLP router directly mapping the expert outputs to the 3D gating tensor, and an attention-style router that matches node representations to learned expert embeddings and then applies channel-wise adjustment. To prevent the expert bank from collapsing to near-duplicate filters, we further introduce a diversity regularizer defined in spectral-response space rather than coefficient space. Across 11 benchmark graphs, FilterMoE provides a robust alternative to manually selecting a hop aggregator for each dataset. Compared with the strongest PP-GNN baseline on each benchmark, FilterMoE improves the average test score by 1.53 points, and wins on 9 of 11 datasets. The advantage is especially clear in the large-graph regime where PP-GNNs are most useful: FilterMoE ranks first on all three large-scale benchmarks, outperforming the best PP-GNN baseline by +1.76 on pokec, +0.93 on ogbn-products, and +0.71 on ogbn-papers100M. Compared with MP-GNNs, FilterMoE achieves competitive accuracy while preserving the pre-propagation training contract. On the three large graphs, it outperforms all the evaluated MP-GNN baselines. Overall, these results suggest that the main limitation of existing PP-GNNs is not the lack of a more complex hop-feature aggregator, but the lack of a structured mechanism for learning graph filters that are jointly node- and channel-adaptive while retaining the scalability advantage of pre-propagation. Contributions. Our main contributions are as follows: • We use a graph-filter perspective to analyze existing PP-GNN hop aggregators and show that their main difference lies in how coefficients over a precomputed diffusion basis are shared across nodes and channels, which explains why more complex hop aggregators do not necessarily outperform simpler ones. • We identify the missing regime of joint node- and channel-adaptive filtering in PP-GNNs, and propose FilterMoE, a mixture-of-experts PP-GNN that routes a small bank of shared Chebyshev filter experts to each node–channel pair through a 3D gating mechanism. • We show that FilterMoE acts as