HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning
Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song · 2025
Read HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning by Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17458.txt.
Introduction
Guiquan Sun, Xikun Zhang, Jingchao Ni, and Dongjin Song teach graph models that keep learning without forgetting. Real graph data, full of mixed entity types and relations, change constantly, but a model retrained on new tasks tends to erase what old tasks taught. In such graphs, old knowledge hides not just in the target nodes but in their typed neighborhoods and relation-specific responses.
Their framework, HERO, replays a few chosen nodes together with compact, rebuilt neighborhoods, since replaying nodes alone provably skews the old lessons. A distillation step also aligns the model's predictions and its semantic responses with its past self, and a look-ahead step keeps it flexible for the new task. Across four data sets and three model backbones, HERO matched or beat the best rivals while forgetting competitively little.
Catalog record
| CREATOR | Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17458
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song Abstract Machine learning on heterogeneous graphs has advanced rapidly, driven by the diverse entities, relations, and semantics inherent in real-world data. However, most existing studies assume static graphs, whereas real-world graph data are often continuously updated. Continual learning in this setting is particularly challenging because historical knowledge is encoded not only in target-node representations, but also in their heterogeneous structural contexts and relation-dependent semantic responses. To this end, we introduce HERO, a heterogeneous continual graph learning framework that preserves historical knowledge at both structural and semantic levels while adapting to incoming tasks. We analyze experience replay as an approximation to the unavailable historical gradient and decompose its error into target-selection and context-reconstruction terms. This analysis shows that replaying target nodes alone is generally insufficient for relation-sensitive HGNNs, as removing typed neighborhood context can alter both representations and historical gradients. Motivated by this, HERO employs DiSCo to select representative target nodes and reconstruct compact heterogeneous contexts through relation-aware multi-type neighbor expansion. To further preserve knowledge that cannot be captured by replayed subgraphs alone, HERO aligns task-conditioned predictions and heterogeneous semantic responses through multi-level knowledge distillation. A lightweight look-ahead adaptation step is additionally used to improve plasticity on incoming tasks. Experiments on four datasets with three HGNN backbones show that HERO achieves the best or tied-best average performance among state-of-the-art baselines, while maintaining competitive forgetting. Our code is available at https://github.com/gqBond/HCGL . † † footnotetext: ∗ Corresponding authors 1 Introduction Heterogeneous graphs model complex real-world systems through multiple node and relation types ( Shi et al., 2018 ) , while heterogeneous graph neural networks (HGNNs) learn expressive representations by capturing their type-specific structural and semantic interactions ( Sun and Han, 2012 ; Wang et al., 2019 ; Hu et al., 2020 ) . Most existing HGNNs, however, are developed under static settings, whereas many real-world applications require models to adapt to continuously evolving heterogeneous graphs without retaining full historical observations. This gives rise to heterogeneous continual graph learning (HCGL), where an HGNN must adapt to incoming tasks while preserving knowledge acquired from previous ones. Directly applying existing continual learning methods to HCGL overlooks how historical knowledge is distributed across heterogeneous graph computation. Existing replay methods mainly preserve selected instances, while distillation methods typically constrain model outputs. However, an HGNN prediction also depends on typed neighborhoods and relation-specific semantic interactions, which can change even when the target nodes or output predictions are preserved. Consequently, preserving a target node does not fully reconstruct the historical computation that produced its prediction, while preserving its output does not guarantee that the underlying relation-dependent semantics remain unchanged. We argue that effective HCGL therefore requires preserving historical knowledge at both the structural and semantic levels. We analyze the structural challenge from an optimization perspective. When past data are unavailable, experience replay estimates the historical gradient using a limited-budget memory buffer. We characterize the resulting increase in historical risk through the gradient approximation error, which can be decomposed into two sources: target-selection error, caused by unrepresentative replay samples of past data, and context-reconstruction error, caused by incomplete heterogeneous neighborhoods. For relation-sensitive HGNNs, we further show that target-node-only replay is generally insufficient to recover the historical prediction function, since the same target nodes can induce different representations under different typed neighborhoods. Effective heterogeneous replay must therefore preserve both representative targets and the structural contexts needed to recover their historical predictions. Based on this observation, we propose HERO , a framework for preserving heterogeneous structural contexts and semantic responses in continual graph learning. HERO first introduces DiSCo ( Di versity S ampling with semantic Co nsistency), which reduces replay approximation error through representative target-node selection and relation-aware multi-type neighbor expansion, producing compact replay subgraphs that better reconstruct historical heterogeneous contexts. Yet context reconstruction does not constrain how the model uses these contexts: the roles of individual relations and semantic channels may still drift as new tasks are learned. HERO addresses this problem with HKD (Heterogeneity-aware Knowledge Distillation), which aligns the current model with historical teachers at two levels: task-conditioned predictions and heterogeneous semantic responses. A lightweight look-ahead step adapts the model to a representative subset of the incoming task, improving plasticity to new knowledge. Our main contributions are summarized as follows: 1. We provide an optimization view of heterogeneous replay, showing that replay-based retention depends on approximating the unavailable historical gradient. We decompose this approximation error into target-selection and heterogeneous-context reconstruction terms, and establish that target-node-only replay is generally insufficient for relation-sensitive HGNNs. 2. Guided by this analysis, we develop HERO to preserve historical knowledge at both structural and semantic levels. DiSCo reconstructs compact heterogeneous replay contexts, while HKD aligns task-conditioned predictions and relation-dependent semantic responses across learning stages. 3. We evaluate HERO on four heterogeneous graph benchmarks with three representative HGNN backbones and different task granularities. HERO achieves the best or tied-best average performance among strong continual graph learning baselines while maintaining competitive forgetting. Extensive analyses validate the roles of heterogeneous context reconstruction and semantic preservation. 2 Preliminaries We study continual node classification over a growing heterogeneous graph. The graph is revealed as a sequence of snapshots, where each snapshot is associated with the arrival of a new task. At each step, the learner receives the data of the current task and may retain only a bounded memory buffer from previous tasks. The objective is to adapt a shared HGNN to the incoming task while preserving its performance on all previously learned tasks. Definition 1 (Growing Heterogeneous Graph). A heterogeneous graph snapshot at step t is denoted by \mathcal{G}_{t}=(V_{t},E_{t},\mathcal{X}_{t},\phi,\psi) , where V_{t} and E_{t} are the node and edge sets, respectively, \phi:V_{t}\rightarrow\mathcal{A} maps each node to a node type, and \psi:E_{t}\rightarrow\mathcal{R} maps each edge to a relation type. The graph is heterogeneous if |\mathcal{A}|+|\mathcal{R}|>2 . For each node type \tau\in\mathcal{A} , let V_{\tau,t}=\{v\in V_{t}\mid\phi(v)=\tau\} and let X_{\tau,t}\in\mathbb{R}^{|V_{\tau,t}|\times d_{\tau}} denote its type-specific feature matrix. We consider a fixed heterogeneous schema consisting of \mathcal{A} and \mathcal{R} , while the graph grows through newly introduced node and edge instances. In particular, consecutive snapshots satisfy V_{t-1}\subseteq V_{t} and E_{t-1}\subseteq E_{t} for t=2,\ldots,T . Definition 2 (Heterogeneous Continual Graph Learning). Each snapshot \mathcal{G}_{t} introduces a node classification task \mathcal{T}_{t} with task-specific classes. The tasks are class-disjoint, and each task contains K classes. Its target nodes are divided into training and test sets V_{t}^{\mathrm{tr}} and V_{t}^{\mathrm{te}} , respectively. All tasks sequentially update the same HGNN encoder h_{\varphi} and the K -way classifier g_{\omega} , i.e., f_{\theta}(v,\mathcal{G}_{t})=g_{\omega}(h_{\varphi}(v,\mathcal{G}_{t})) with \theta=(\varphi,\omega) . At evaluation, the task context t is known to the evaluator and is used only to interpret the local prediction; it is not used to select task-specific model parameters. At step t , the model may retain a replay buffer \mathcal{B} satisfying |\mathcal{B}|\leq B ( B is the buffer size). 3 Methodology In this section, we present the HERO framework (see Figure 1 ). HERO addresses continual learning from two complementary aspects: retaining historical heterogeneous knowledge and maintaining plasticity to incoming tasks. We first introduce DiSCo to preserve representative heterogeneous contexts, and then present HKD to retain historical prediction and semantic responses beyond explicit replay. Finally, we describe a