Transferring Social Network Knowledge from Multiple GNN Teachers to Kolmogorov–Arnold Networks
Yuan-Hung Chao, Chia-Hsun Lu, and Chih-Ya Shen · 2025
Read Transferring Social Network Knowledge from Multiple GNN Teachers to Kolmogorov–Arnold Networks by Yuan-Hung Chao, Chia-Hsun Lu, and Chih-Ya Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06663.txt.
Introduction
Yuan-Hung Chao, Chia-Hsun Lu, and Chih-Ya Shen work on faster learning for social network data in this 2025 paper. Graph neural networks read links between users well, but leaning on those links can make them slow and hard to scale. The authors try a newer network type that learns simple curve shapes instead.
They blend the new network into three known graph models, making three hybrid versions. Then they train one small student model to copy what all three hybrids know, without needing the graph at run time. Tests on standard data sets showed better scores for sorting users into classes. The student model gained a lot from the shared teaching. The authors say this path keeps accuracy while cutting run-time cost.
Catalog record
| CREATOR | Yuan-Hung Chao, Chia-Hsun Lu, and Chih-Ya Shen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06663
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Transferring Social Network Knowledge from Multiple GNN Teachers to Kolmogorov–Arnold Networks Yuan-Hung Chao, Chia-Hsun Lu, and Chih-Ya Shen Abstract Graph Neural Networks (GNNs) have shown strong performance on graph-structured data, but their reliance on graph connectivity often limits scalability and efficiency. Kolmogorov-Arnold Networks (KANs), a recent architecture with learnable univariate functions, offer strong nonlinear expressiveness and efficient inference. In this work, we integrate KANs into three popular GNN architectures—GAT, SGC, and APPNP—resulting in three new models: KGAT, KSGC, and KAPPNP. We further adopt a multi-teacher knowledge amalgamation framework, where knowledge from multiple KAN-based GNNs is distilled into a graph-independent KAN student model. Experiments on benchmark datasets show that the proposed models improve node classification accuracy, and the knowledge amalgamation approach significantly boosts student model performance. Our findings highlight the potential of KANs for enhancing GNN expressiveness and for enabling efficient, graph-free inference. I Introduction With the rapid development of deep learning technologies, Graph Neural Networks (GNNs) have demonstrated exceptional performance in processing graph-structured data [ 1 ] . GNNs have been widely applied in social network analysis, recommendation systems, knowledge graphs, and bioinformatics. However, the reliance of GNNs on graph structure data for inference poses challenges to their scalability and efficiency when dealing with large-scale graph data. To address these limitations, researchers have proposed various improvements, including Graph Convolutional Networks (GCNs) [ 2 ] , Graph Attention Networks (GATs) [ 3 ] , Simple Graph Convolution (SGC) [ 4 ] , and APPNP [ 5 ] . These methods enhance model expressiveness and computational efficiency through different mechanisms but still fall short of completely solving the bottlenecks of GNNs in large-scale applications. On the other hand, Kolmogorov-Arnold Networks (KANs) [ 6 ] , as a novel neural network architecture, replace traditional fixed activation functions with learnable univariate functions, demonstrating strong non-linear representation capabilities and interpretability. KANs have proven their superior performance in tasks such as mathematical and physical function fitting, achieving high accuracy with fewer training parameters. Moreover, KANs feature lower resource consumption, avoidance of catastrophic forgetting, and greater interpretability. Recent studies have integrated KANs into GNNs [ 7 , 8 , 9 , 10 ] , showing some performance improvements in experiments. However, current research mainly focuses on integrating KAN with GCN, with limited exploration of other common GNN architectures. Therefore, our goal is to combine KAN with other GNN architectures (e.g., GAT [ 3 ] , SGC [ 4 ] , APPNP [ 5 ] ) to further validate the advantages of KAN Layers in GNNs. Knowledge Distillation (KD) [ 11 ] is a model compression and acceleration technique that trains a smaller student model to mimic a larger, more efficient teacher model. Hinton et al. first proposed KD [ 12 ] , demonstrating its effectiveness in image classification tasks. KD has also been applied to GNNs to reduce computational load and enhance inference speed [ 13 , 14 , 15 , 16 , 17 ] . Multi-Teacher Knowledge Amalgamation (MTACL) [ 17 ] is an extension of KD, which merges knowledge from multiple teacher models into a student model. This method leverages the diversity of heterogeneous teacher models, allowing the student model to learn a broader range of knowledge from different perspectives, thus improving its generalization ability and performance. Current research mainly focuses on integrating KAN with GCN [ 7 , 8 , 9 , 10 ] , with limited exploration of other common GNN architectures. Our goal is to combine KAN with other GNN architectures (e.g., GAT [ 3 ] , SGC [ 4 ] , APPNP [ 5 ] ) to further validate the advantages of KAN Layers in GNNs. The study by MTACL [ 17 ] has limitations in the choice of student models, primarily comparing different sizes of MLP student models without attempting other structures. We aim to use KAN as the student model to explore its effectiveness in knowledge distillation tasks. Additionally, one of MTACL’s conclusions is that heterogeneous teacher models (those with different structures) can distill better student models. Based on this conclusion, we introduce KAN-based GNN teacher models and combine them with common GNNs as teacher pairs to verify the effect of heterogeneous teacher models in knowledge distillation. We hope to distill more performant student models through more structurally diverse teacher combinations. We choose KAN [ 6 ] as the student model for the following reasons: KAN has demonstrated superior performance compared to MLP in multiple studies, indicating the potential for better performance in knowledge distillation tasks. Given the inclusion of KAN-based GNNs as teacher models, we hypothesize that using KAN as the student model will result in better learning outcomes due to structural similarities. Distilling knowledge into graph-structure-independent KAN helps optimize inference time, particularly for more complex network structures like KAN-based GNNs. The contributions of this paper are summarized as follows. • We propose three new network architectures—KGAT, KSGC, and KAPPNP—that combine the powerful non-linear representation capabilities of KAN [ 6 ] with the advantages of GAT [ 3 ] , SGC [ 4 ] , and APPNP [ 5 ] , respectively, enhancing the performance of GNNs. • We design a training framework based on multi-teacher knowledge amalgamation, merging knowledge from multiple teacher models and transferring it to graph-structure-independent KAN student models, thereby improving inference efficiency. • Through experiments, we validate the superior performance of the proposed methods on multiple benchmark datasets and test various KAN variants in knowledge distillation tasks, conducting ablation studies to analyze the effects of different components. The innovation of this study lies in combining KAN with GNNs and leveraging multi-teacher knowledge amalgamation techniques to overcome the bottlenecks of GNNs in large-scale applications, providing new ideas and directions for future research. In addition to its technical contributions, our research offers significant practical advantages. Our model provides more precise performance compared to competitors, achieving higher accuracy across various tasks. By using Knowledge Amalgamation, we enhance efficiency while maintaining performance, which is crucial in resource-constrained environments. Moreover, our model excels in scenarios with limited training data, maintaining strong performance. This makes our research both innovative in theory and competitive in practice. II Related Works II-A Kolmogorov-Arnold Networks (KANs) Kolmogorov-Arnold Networks (KANs) [ 6 ] are a novel neural network architecture designed based on the Kolmogorov-Arnold representation theorem [ 18 , 19 , 20 ] . This theorem states that any multivariable function can be represented as a combination of several univariate functions, providing a mathematical foundation for KANs. The unique feature of KANs is the replacement of traditional linear weights with learnable spline-based univariate functions, which serve as activation functions. KANs overcome the limitations of traditional multilayer perceptrons (MLPs) in terms of efficiency and interpretability. They reduce information loss and demonstrate high performance with low computational resource consumption and good model interpretability. Additionally, KANs perform well in continuous learning tasks, effectively avoiding catastrophic forgetting. II-B Graph Neural Networks (GNNs) Graph Neural Networks (GNNs) [ 1 ] are a specialized neural network model designed for processing graph data. Unlike traditional Neural Networks, GNNs aggregate and propagate node features through the graph structure, effectively capturing the dependencies between nodes. This type of network model has demonstrated its advantages in various fields such as social network analysis [ 21