Draw a Portrait of Your Graph Data: An Instance-Level Profiling Framework for Graph-Structured Data

Tianqi Zhao, Russa Biswas, Megha Khosla · 2025

Read Draw a Portrait of Your Graph Data: An Instance-Level Profiling Framework for Graph-Structured Data by Tianqi Zhao, Russa Biswas, Megha Khosla (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-12094.txt.

Introduction

Two graph models can post the same headline score. Yet they may fail on completely different nodes. Tianqi Zhao and two co-authors build NodePro in 2025 to expose that gap. NodePro is a profiling tool. It grades single nodes, one by one.

Each node gets plain profile scores from two families of signals. Data-side signals cover how unlike its features are, how unsure its label is, and how muddy its local structure is. Model-side signals track confidence and steadiness during training. Lining conduct up against profiles shows gaps that headline scores erase. Profiles also carry to unseen nodes. That supports trust judgments with no labels at hand. The tool also proves itself in one more way. It sniffs out bad or corrupted entries inside a structured knowledge graph.

Catalog record

CREATORTianqi Zhao, Russa Biswas, Megha Khosla
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-12094

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Draw a Portrait of Your Graph Data: An Instance-Level Profiling Framework for Graph-Structured Data Tianqi Zhao, Russa Biswas, Megha Khosla Abstract Graph machine learning models often achieve similar overall performance yet behave differently at the node level—failing on different subsets of nodes with varying reliability. Standard evaluation metrics such as accuracy obscure these fine-grained differences, making it difficult to diagnose when and where models fail. We introduce NodePro , a node profiling framework that enables fine-grained diagnosis of model behavior by assigning interpretable profile scores to individual nodes. These scores combine data-centric signals—such as feature dissimilarity, label uncertainty, and structural ambiguity—with model-centric measures of prediction confidence and consistency during training. By aligning model behavior with these profiles, NodePro reveals systematic differences between models, even when aggregate metrics are indistinguishable. We show that node profiles generalize to unseen nodes, supporting prediction reliability without ground-truth labels. Finally, we demonstrate the utility of NodePro in identifying semantically inconsistent or corrupted nodes in a structured knowledge graph, illustrating its effectiveness in real-world settings. Introduction Graph machine learning models have achieved remarkable success on node classification tasks. However, their performance is far from uniform across all nodes. Existing evaluation practices remain largely coarse-grained, relying on aggregate metrics such as accuracy or F1 score. While these metrics are valuable for benchmarking, they obscure important variation in how models behave at the node level. In particular, they fail to reveal why certain nodes are consistently harder to classify whether due to feature sparsity, structural ambiguity, or label noise. Crucially, models trained on the same dataset and achieving similar overall accuracy may still generalize in fundamentally different ways—failing on different subsets of nodes or exhibiting varying levels of predictive uncertainty. Without a principled approach to capturing node-level learning difficulty and failure modes, we lack the tools to assess the trustworthiness of individual predictions. Bridging this gap between high-level evaluation and fine-grained model behavior is essential for developing robust, interpretable, and trustworthy graph learning systems. This limitation motivates our central research question: How can we characterize fine-grained differences in model behavior across individual nodes, beyond aggregate performance metrics? To address this question, we propose NodePro , a node profiling framework designed to characterize individual nodes and explain model behavior at a fine-grained level. Rather than focusing on improving model performance through data augmentation or architectural changes, NodePro seeks to uncover where models fail by identifying nodes that are consistently misclassified and analyzing the underlying factors that contribute to their difficulty. In doing so, our approach introduces a novel contribution to the growing field of data-centric graph machine learning ( Zheng et al. 2023 ; Jin 2024 ) , which has traditionally emphasized improving datasets to enhance model performance. NodePro instead shifts the emphasis toward understanding model behavior in relation to intrinsic node characteristics, offering a complementary path toward building more robust and interpretable graph learning systems. NodePro profiles nodes along two complementary dimensions: data-centric characteristics, which describe a node’s context in the input graph, and model-centric behavior, which captures prediction dynamics during training. The data-centric component constructs interpretable node profiles based on three key properties: (i) feature dissimilarity (how distinct a node’s features are relative to others in the same class), (ii) local label uncertainty (the diversity of class labels in its immediate neighborhood), and (iii) higher-order structural ambiguity (how consistently distant neighbors share the node’s label). Together, these properties summarize the intrinsic difficulty of a node from a data perspective. To understand how models behave in relation to these profiles, NodePro analyzes two additional signals: prediction consistency indicating how stable predictions are across training checkpoints, and prediction confidence which quantifies how far the predicted probability for the true label deviates from a random guess. By aligning these model-level signals with data-centric node profiles, NodePro provides a powerful lens into model generalization and reliability. We demonstrate that NodePro not only explains node-level performance but also enables new capabilities such as detecting potentially incorrect predictions and flagging atypical or anomalous nodes, particularly in noisy or structurally complex graphs where traditional evaluation metrics fall short. Related Work Data-Centric Graph Machine Learning(DC-GML) Methods in DC-GML aim to improve model performance by focusing on the quality and manipulation of graph data itself, rather than solely on model architecture ( Zheng et al. 2023 ; Jin 2024 ) . Recent research categorizes DC-GML into stages such data collection, improvement, and maintenance, emphasizing interventions on graph topology, features, and labels to combat noise, missing information, and mislabeling ( Zheng et al. 2023 ; Sun et al. 2024 ; Jin 2024 ) . Key strategies include augmenting node features to handle incompleteness, refining the labels, and using graph condensation or synthesis to improve the efficiency of the models ( Zheng et al. 2023 ; Jin 2024 ; Guo et al. 2023a ; Huang et al. 2024 ) . These techniques have proven to be efficient to improve the performance of the existing SOTA models on tasks like GNN calibration, anomaly detection, and discovering hidden graph hierarchies. Within the DC-GML context, graph anomaly detection introduces taxonomies to classify nodes or entire graphs as hard/easy, typical/atypical, in distribution (ID) or OOD ( Guo et al. 2024 ; Ma et al. 2024 ; Gavrilev and Burnaev 2023 ) . These methods are often applied to domain-specific datasets in bio-informatics, social networks, and finance. The goal is to make OOD representations more distinguishable from ID ones. Detection techniques include density-based heuristics ( Breunig et al. 2000 ) , distance-based methods such as K -nearest neighbors ( Sehwag, Chiang, and Mittal 2021 ) , and learnable scoring strategies that increase the separation between ID and OOD during training ( Guo et al. 2023b ) . While most approaches focus on graph-level detection, ( Ma et al. 2024 ) propose a diffusion-based graph generator to synthesize node-level training data, which can be directly integrated into existing GNN models to enhance performance. Despite its promise, DC-GML still lacks frameworks for understanding how individual nodes affect model learning process. Most existing methods focus on global data properties or modifications, overlooking instance-level behavior. Data Hardness in Data-Centric AI Data hardness in data-centric AI refers to identifying and managing difficult samples within datasets that impede model learning, particularly in image, tabular and visual tasks ( Seedat, Imrie, and van der Schaar 2024 ; Seedat et al. 2022 ) . These hard samples include mislabeled, ambiguous, or rare examples that challenge standard models. Addressing data hardness involves techniques like data augmentation to diversify training data and improve model robustness. Hardness Characterization Methods are algorithmic tools used to detect such challenging samples. Recent research emphasizes evaluation of the data hardness quantitatively across benchmarks such as CIFAR and MNIST, aiming for reliable hardness detection and improved generalization of the models ( Seedat, Imrie, and van der Schaar 2024 ) . Managing data hardness is critical for improving model accuracy, robustness, and trustworthiness in data-centric AI ( Seedat, Imrie, and van der Schaar 2024 ; Jarrahi, Memariani, and Guha 2023 ) . In node-level tasks on graphs, hardness is influenced not just by the features of a node, as in Euclidean data, but also by its surrounding topology, from local structure to higher-order neighborhoods. This added complexity calls for a

Read the complete text