From Model to Data (M2D): Shifting Complexity from GNNs to Graphs for Transparent Graph Learning

Debolina Halder Lina, Arlei Silva · 2026

Read From Model to Data (M2D): Shifting Complexity from GNNs to Graphs for Transparent Graph Learning by Debolina Halder Lina, Arlei Silva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06814.txt.

Introduction

Debolina Halder Lina and Arlei Silva introduce model-to-data distillation, which moves what a complex graph neural network learned into the graph itself. Desirable traits like accuracy, fairness, and robustness usually stay locked inside the sophisticated model that learned them, limiting transfer to simpler models. Their method instead jointly learns augmented node features and graph structure that encode the teacher's behavior.

The enriched graph becomes a lasting transfer medium: simple networks trained on it normally recover much of the teacher's behavior. Experiments cover fairness-aware networks, attention networks, and graph transformers as teachers, with student models keeping comparable predictive performance while inheriting teacher traits. The paradigm trades model complexity for data complexity.

Catalog record

CREATORDebolina Halder Lina, Arlei Silva
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06814

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Model to Data (M2D): Shifting Complexity from GNNs to Graphs for Transparent Graph Learning Debolina Halder Lina, Arlei Silva Abstract Graph Neural Networks (GNNs) achieve high performance but can be opaque to humans, making it difficult to understand and compare the many proposed architectures. While existing explainability methods attribute individual predictions to nodes, edges, or features, they do not provide architectural transparency or explain the fundamental performance gap between simple and more complex models. To address this limitation, we introduce Model-to-Data (M2D) distillation, a new framework that increases transparency by transferring model complexity into the data space. M2D distills the teacher model into an augmented graph with enriched features and structure, enabling a simple student to match the teacher’s performance. By materializing model behavior in the data, our approach allows humans to inspect architectural advantages directly. We show that M2D reveals underlying mechanisms such as fairness objectives and attention-based aggregation in an interpretable way, enhancing GNN transparency while preserving performance. 1 Introduction Graph Neural Networks (GNNs) have achieved state-of-the-art performance on graph tasks such as node classification, link prediction, and graph classification ( 15 ; 30 ; 48 ; 19 ) . However, as GNNs become more complex to address limitations such as biased predictions, lack of robustness to data perturbations, over-smoothing, and limited expressiveness, they also become increasingly opaque to humans. This epistemic opacity has practical implications ranging from limiting the adoption of GNNs in high-stakes settings to hindering our ability to understand why certain architectures outperform others on specific benchmarks. Our work aims to make GNNs more transparent by enabling humans to (1) understand their logic and (2) audit them for correctness and bias 32 . While traditional explainers identify which input features influenced a prediction, they fail to explain the architectural suitability of one model over another for a specific dataset. For instance, it is unclear how the self-attention mechanism translates to the performance gains of a Graph Attention Network (GAT) over a Graph Convolutional Network (GCN) on the Cora dataset 48 . If we can materialize this architectural advantage into static data augmentations, we enable humans to audit the model’s logic directly within the data space, revealing why a complex architecture outperforms a simpler one. To this end, we introduce Model-to-Data (M2D) distillation, which transfers complexity from a teacher model \mathcal{M} into an augmented graph \mathbb{D} characterized by enriched node features and modified adjacency weights. This allows a lightweight student m to achieve teacher-level performance while maintaining a human-auditable data representation. We view M2D in a unified space of knowledge and data distillation, illustrated in Fig. 2 . While Knowledge (K) distillation 16 reduces model complexity (vertical shift) and Data (D) distillation reduces data complexity 31 (horizontal shift), M2D distillation trades model complexity for data complexity so that humans can compare models \mathcal{M} and m based on the differences between the corresponding datasets D and \mathbb{D} . Figure 1 : Increasing transparency of a fair Graph Convolutional Network (GCN) via M2D using a standard GCN as a comparison. The mechanism that enables the Fair GCN to generate fair predictions is not transparent to the user (middle). M2D makes the Fair GCN more transparent by distilling it into an augmented graph so that the GCN predictions are fair (right). Intuitively, the graph augmentation should capture how the Fair GCN addresses the bias in the features and topology of the input graph. Figure 1 illustrates our approach in the context of fair machine learning, where the goal is to better understand how a fair Graph Convolutional Network (GCN) mitigates biases in node classification compared to a simple GCN. More specifically, although a fair GCN produces bias-mitigated predictions, the internal mechanisms that enforce fairness are often opaque (middle). Figure 2 : Distillation design space with data (x-axis) and model (y-axis) complexity. Knowledge (K) distillation reduces model complexity (vertical), and data (D) distillation reduces data complexity (horizontal). Model-to-Data (M2D) distillation transfers capacity from the model to the data (diagonal). By shifting complexity to the data (right), fairness is no longer hidden within opaque adversarial training or complex loss functions but is instead materialized as modifications to the graph. This is particularly critical for high-stakes auditing: an auditor can inspect the specific edges added or features modified to mitigate bias, providing a more intuitive explanation than the internal gradients of a Fair GCN. Furthermore, we show that M2D can effectively ‘capture’ the global context of Graph Transformers, distilling their long-range dependencies into local feature augmentations that a simple GCN can process. M2D has other applications beyond transparency. The graph augmentations can be shared as benchmarks for simpler models that can be trained under resource constraints (e.g., on edge devices). Moreover, M2D can be applied to reverse-engineer a black-box model using only access to its logits. We summarize the contributions of our paper as follows: (1) we introduce model-to-data (M2D) distillation as a general framework to exchange model and data complexity in GNNs; (2) we instantiate M2D through an iterative framework using feature and structure learning and multi-objective optimization; and (3) we empirically show how M2D increases the transparency of fair GNNs, Graph Attention Networks, and Graph Transformers. 1.1 Related Work Graph explainers: GNN explainers focus on instance-level explanations, aiming to interpret individual predictions. Gradient-based methods (e.g., SA, Guided BP, CAM, Grad-CAM) attribute predictions using gradients or activations; perturbation-based methods (e.g., GNNExplainer, PGExplainer, GraphMask, DnX, SubgraphX) learn masks over nodes, edges, or features to preserve predictions; surrogate methods (e.g., GraphLime, RelEx, PGM-Explainer) approximate local model behavior with interpretable models; and decomposition methods (e.g., LRP, Excitation BP, GNN-LRP) distribute prediction scores back to the input space 3 ; 40 ; 60 ; 37 ; 41 ; 39 ; 62 ; 24 ; 68 ; 49 . Model-level explainers (e.g., XGNN, PAGE, GNNInterpreter, GLGExplainer, GCFExplainer, MOSE) seek to provide global explanations by identifying patterns, subgraphs, or concepts that consistently influence predictions across a target class 44 ; 46 ; 54 ; 29 ; 33 . While these methods explain what drives predictions, they do not explain why a particular architecture outperforms another on a dataset. In contrast, M2D targets architectural transparency by translating model behavior into explicit modifications of the data, enabling direct comparison of models through their induced data transformations. Knowledge distillation on graphs: Knowledge distillation on graphs varies along two axes: the type of knowledge transferred and the distillation framework. Existing methods distill either logits by aligning output distributions (e.g., TinyGNN, GFKD, DFAD-GNN, KDGA, CPF, GLNN) 55 ; 7 ; 71 ; 52 ; 56 ; 64 , structural knowledge by preserving local and global topology (e.g., LSP, FreeKD, Alignahead, GNN-SD, CKD, G-CRD, ROD, MSKD, Cold Brew, GKD, NOSMOG) 58 ; 9 ; 18 ; 5 ; 50 ; 27 ; 66 ; 63 ; 69 ; 57 ; 45 , or intermediate embeddings by matching representations across layers (e.g., T2-GNN, SAIL, GraphAKD, RDD) 25 ; 61 ; 21 ; 67 . From a framework perspective, prior work includes teacher-free self-distillation (e.g., GNN-SD, CKD, RDD) and teacher–student paradigm, where pretrained GNNs transfer knowledge to either compact GNNs (e.g., TinyGNN, GraphAKD, LSP) or simpler models such as MLPs (e.g., CPF, GLNN, Cold Brew, NOSMOG). Data distillation on graphs: Graph data distillation is

Read the complete text