Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning

Zhen-Hao Xie, Yan Wang, Hao Sun, Han-Jia Ye, De-Chuan Zhan, Da-Wei Zhou · 2026

Read Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning by Zhen-Hao Xie, Yan Wang, Hao Sun, Han-Jia Ye, De-Chuan Zhan, Da-Wei Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08839.txt.

Introduction

Zhen-Hao Xie and five coauthors unify two hard learning problems: adapting to new domains and learning new classes without forgetting old ones. A driving system trained on sunny city roads may later face rural highways in rain, needing both shifts handled at once. Their CORF framework refines training samples by spatial contribution maps that highlight informative regions, and weighs samples by prediction confidence to favor domain agnostic features.

For memory, a cascaded distillation scheme transfers relational knowledge between samples across several feature levels from earlier tasks. The framework plugs into existing incremental learning methods and lifts their results across benchmarks. Systems that must keep growing in changing worlds gain one tool for both pressures.

Catalog record

CREATORZhen-Hao Xie, Yan Wang, Hao Sun, Han-Jia Ye, De-Chuan Zhan, Da-Wei Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08839

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cross-Sample Relational Fusion: Unifying Domain Generalization and Class-Incremental Learning Zhen-Hao Xie, Yan Wang, Hao Sun, Han-Jia Ye, De-Chuan Zhan, Da-Wei Zhou Abstract Class-Incremental Learning (CIL) requires a learning system to learn new classes while retaining previously learned knowledge. However, in real-world scenarios such as autonomous driving, a system trained on urban roads in sunny weather may later need to operate in rural or highway environments with different traffic patterns and weather conditions. This requires the model not only to overcome catastrophic forgetting, but also to effectively handle domain shifts. In this paper, we propose CrOss-sample Relational Fusion (C ORF ), a unified framework to address domain shift and catastrophic forgetting simultaneously. To enhance generalizability, we perform selective refinement of training samples by leveraging spatial contribution maps to highlight semantically informative regions. Furthermore, we incorporate predictive confidence to adaptively weigh samples, thereby facilitating the learning of domain-agnostic representations. To alleviate forgetting, we propose a cascaded distillation framework that captures cross-sample relational dependencies across multiple feature hierarchies, enabling multi-grained knowledge transfer from previous tasks. C ORF can be seamlessly integrated into existing CIL algorithms to enhance their generalizability, achieving competitive performance across various benchmark datasets. Code is available: https://github.com/LAMDA-CL/TMM26-CORF . † † footnotetext: This work is partially supported by NSFC (62506160), Basic Research Program of Jiangsu (BK 20251251), JSTJ-2025-147, Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (No. JYB2025XDXM118), the 111 Center (No. B26023). (Corresponding author: Da-Wei Zhou.) Zhen-Hao Xie, Yan Wang, Hao Sun, Han-Jia Ye, De-Chuan Zhan and Da-Wei Zhou are with the School of Artificial Intelligence and the State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China (e-mail: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]). I Introduction The advent of deep learning has significantly advanced neural networks, enabling strong performance across real-world applications [ 20 ] , [ 12 ] . In open-world scenarios, data often arrives continuously, requiring models to incrementally incorporate new class knowledge. This process is known as Class-Incremental Learning (CIL) [ 72 , 78 ] , in which models must continuously acquire new class knowledge while retaining previously learned knowledge. A central challenge in CIL is catastrophic forgetting [ 19 ] , as the model tends to overwrite old knowledge when learning new tasks. However, even when forgetting is alleviated, in practical scenarios the model may still encounter data from domains that differ from the training distribution, resulting in domain shifts [ 30 , 56 ] that further intensify the challenge. For example, in autonomous driving or robotics, new object categories such as unfamiliar road signs, emerging transport modes, and diverse environments frequently appear, demanding continual adaptation without erasing previously acquired visual knowledge. Existing CIL methods [ 32 , 63 ] , [ 83 , 47 ] primarily aim to mitigate catastrophic forgetting [ 51 , 25 ] , [ 59 ] . However, they often struggle under domain shifts due to the lack of a generalization mechanism. In contrast, Domain Generalization (DG) techniques [ 23 ] , [ 15 ] , [ 66 ] , [ 22 ] enhance robustness across domains by learning domain-invariant representations [ 17 ] , [ 55 , 35 ] . Nevertheless, they lack a mechanism to resist forgetting and therefore fail to preserve prior knowledge when learning from continuous data streams. Fig. 1 : Illustration of CDCIL. During training, novel classes are introduced incrementally, with samples drawn from multiple seen domains. During inference, the model is evaluated on all previously seen classes across both seen and unseen domains. For example, in task 2, although only horse and elephant are observed during training (from seen domains shown in blue, yellow, and green), inference covers all encountered classes ( e.g. , snake, eagle, horse, and elephant) across both seen and unseen domains (shown in grey). Both CIL and DG fail to address the challenge of learning new classes sequentially under domain shift, leading to poor generalizability or severe forgetting. To address both forgetting and domain shift in a unified manner, we formulate the problem as Cross-Domain Class-Incremental Learning (CDCIL). As illustrated in Fig. 1 , the model must learn new classes incrementally while generalizing across domains. Tackling CDCIL presents two key challenges: 1) Constructing a domain-agnostic representation space. Since the model is evaluated on an unseen domain, it must effectively handle domain shifts by learning representations that are compact within the same class across domains while remaining well-separated between classes. 2) Developing a domain-agnostic strategy to mitigate forgetting. Since samples come from diverse domains, mitigating forgetting requires a fine-grained approach that captures both inter-class and cross-domain dependencies. Instead of supervising each sample independently, the model should retain knowledge across structurally related samples to preserve shared patterns and interactions. In this paper, we propose CrOss-sample Relational Fusion (C ORF ) to address the challenges of CDCIL. To enhance cross-domain generalizability, we introduce a contribution- and confidence-sensitive refinement mechanism. High-confidence samples are blended with same-class counterparts from different domains in low-contribution regions, while low-confidence samples are fused with different-class instances from the same domain in high-contribution regions. Jointly optimizing original and synthetic samples not only enhances representation discrimination but also facilitates the construction of a domain-agnostic representation space. We further propose a hierarchical kernel-based distillation strategy that aligns globalized information and embeds cross-domain and inter-class relational cues into the transfer process. Together, the refinement and distillation components complement each other by enhancing both the discriminative power and structural consistency of learned representations, thereby promoting robust and generalizable feature learning in CDCIL. II Related Works Class-Incremental Learning To address the challenge of catastrophic forgetting [ 10 , 19 , 42 ] in CIL [ 78 , 61 ] , [ 24 , 9 ] , incremental learning methods must maintain the performance of previously learned tasks while adapting to new ones. CIL algorithms can be categorized into several categories. Specifically, replay-based methods [ 25 , 39 ] focus on revisiting old knowledge to avoid forgetting, using an exemplar set to store representative instances from previous tasks and reintroduce them during the learning of new tasks. Apart from directly saving instances in the exemplar set, generative models are also widely applied to model the distribution of previous tasks for replay, e.g. , GAN [ 57 ] , diffusion model [ 11 ] and VAE [ 16 ] . Data regularization-based methods [ 44 ] , [ 1 , 68 ] mitigate catastrophic forgetting by imposing additional constraints on the loss function. These regularization methods can be applied to the weights by estimating the importance of the model parameters, ensuring that relevant weights do not drift significantly. Dynamic network-based methods [ 69 ] , [ 49 , 79 ] enhance the model’s representation capacity by dynamically adjusting and expanding the network in incremental learning, thereby better adapting to new categories. Such methods are typically categorized into three types of expansions: neuron expansion [ 69 , 63 , 73 ] , backbone expansion [ 79 , 29 ] , and prompt expansion [ 49 , 60 ] . Model rectification-based methods [ 62 , 2 ] mitigate catastrophic forgetting by correcting weights, logits, or embeddings, which enables a more balanced classification between old and new classes. More recently, CMoA [ 5

Read the complete text