Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning
Gyutae Oh, Jungwoo Bae, Jitae Shin · 2026
Read Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning by Gyutae Oh, Jungwoo Bae, Jitae Shin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12816.txt.
Introduction
Gyutae Oh and co-authors address catastrophic forgetting in domain-incremental continual learning, where task labels are absent. Prompt methods adapt a frozen backbone, yet prompt choice and classifier drift can still fail. Their Residual SODAP pairs sparse prompt selection with residual aggregation. It adds data-free distillation with pseudo-feature replay and drift detection from prompt use. An uncertainty-aware balance weighs the losses. Across three benchmarks with no stored data, it set the best average accuracy and forgetting scores.
Catalog record
| CREATOR | Gyutae Oh, Jungwoo Bae, Jitae Shin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61841 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.12816
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning Gyutae Oh, Jungwoo Bae, Jitae Shin Abstract Continual learning (CL) suffers from catastrophic forgetting, which is exacerbated in domain-incremental learning (DIL) where task identifiers are unavailable and storing past data is infeasible. While prompt-based CL (PCL) adapts representations with a frozen backbone, we observe that prompt-only improvements are often insufficient due to suboptimal prompt selection and classifier-level instability under domain shifts. We propose Residual SODAP, which jointly performs prompt-based representation adaptation and classifier-level knowledge preservation. Our framework combines α \alpha -entmax sparse prompt selection with residual aggregation, data-free distillation with pseudo-feature replay, prompt-usage–based drift detection, and uncertainty-aware multi-loss balancing. Across three DIL benchmarks without task IDs or extra data storage, Residual SODAP achieves state-of-the-art AvgACC/AvgF of 0.850/0.047 (DR), 0.760/0.031 (Skin Cancer), and 0.995/0.003 (CORe50). 1 Introduction Artificial intelligence has progressively evolved toward emulating the cognitive capabilities of living organisms [ 7 , 46 ] . While biological systems flexibly accommodate new information without compromising previously acquired knowledge, artificial neural networks frequently fail to do so [ 26 , 24 , 34 ] —a phenomenon known as catastrophic forgetting. In dynamic environments, the ability to continuously assimilate new knowledge is therefore essential not only for biological agents but also for artificial intelligence systems [ 20 , 37 , 6 ] . In the visual domain, variations in acquisition conditions, devices, and environments give rise to heterogeneous data distributions, causing models trained on one domain to suffer sharp performance degradation in another. Continual learning (CL) has been proposed to address this challenge [ 2 , 14 , 30 , 47 ] . Moreover, real-world deployments often impose strict constraints on data export and long-term storage due to privacy regulations [ 32 ] , driving growing demand for CL methods that remain effective under such restrictions [ 45 ] . Against this backdrop, we address Domain Incremental Learning (DIL), where a model must incorporate new domain knowledge without retaining any previous data. Prompt-based continual learning (PCL) has recently emerged as a promising paradigm for this setting [ 14 , 42 , 43 ] ; however, existing PCL methods exhibit two key limitations. Limited prompt selection schemes. Hard selection (Top-k) methods [ 43 ] restrict the model to a fixed subset of prompts, limiting expressiveness and potentially causing training instability due to the non-differentiability of the selection process [ 31 ] . Soft selection (Softmax) methods [ 14 ] alleviate this issue by enabling gradient-based learning over all prompts; however, the weighted-sum nature of softmax allows even irrelevant prompts to exert influence, leading to noise accumulation [ 41 ] . Neglect of the classifier structure. Existing PCL methods have predominantly focused on prompt or prompt-pool design to improve CL performance and mitigate forgetting [ 14 , 43 , 42 ] . While this enables effective representation adaptation, [ 21 ] suggests that classifier design can be equally important. Through simple empirical observations, we show that jointly considering the classifier structure alongside prompt design is indeed necessary. We note that, although multi-head classifiers have been proposed to mitigate forgetting by maintaining separate heads per task [ 39 ] , such methods generally assume task identity (Task-ID) at inference time. Since our DIL setting does not provide explicit Task-ID, our approach differs from these existing multi-head methods in the applicable problem scope. Building on these observations, we propose Residual Self-Organizing Domain-Adaptive Prompting (Residual SODAP) , which jointly improves both the prompt pool and the classifier. Our contributions are fourfold : • \alpha -Entmax-based Residual Prompt Selection. We enhance the input query with a memory bank and perform sparse prompt selection via \alpha -entmax [ 27 ] combined with residual connections, structurally preserving prior knowledge through frozen prompts while adapting to new domains. • Statistical Knowledge Preservation. Without accessing past data, we store class-wise feature distribution statistics and leverage them for feature distillation and pseudo feature replay during classifier training, thereby mitigating catastrophic forgetting. • Prompt Usage-based Drift Detection. We detect domain drift by jointly monitoring entropy changes in prompt selection patterns and structural shifts in selection behavior, enabling automatic prompt expansion and switching. • Uncertainty Weighting. By learning the uncertainty of each loss term, we enable stable joint optimization without manual weight tuning. 2 Related Work Domain adaptation (DA) [ 40 ] and domain generalization (DG) [ 38 ] also address distribution shifts across domains [ 17 , 33 ] . However, DA typically assumes prior or concurrent access to target-domain data, whereas DG is usually trained offline on multiple source domains without observing the target domain. As a result, these paradigms rely on different assumptions than our sequential setting where domains arrive over time. In contrast, this study focuses on a DIL setting where domains are presented sequentially, past data is inaccessible, and no Task-ID is provided [ 39 ] . Due to these fundamental differences in problem definition and evaluation protocols, we restrict our comparisons to continual learning baselines. Continual Learning. Traditional CL methods can be broadly categorized into three groups [ 45 ] . First, Regularization-based CL (Reg-CL) mitigate catastrophic forgetting by constraining changes to parameters deemed important during training. However, under complex distribution shifts or challenging settings, performance may degrade due to limited expressivity and adaptability [ 30 , 48 , 19 ] . Second, Rehearsal-based CL (Reh-CL) achieve strong performance by storing or replaying a subset of samples from previous tasks and training them jointly with current data, but their applicability is limited in practice due to constraints on data storage and reuse [ 2 , 46 , 23 ] . Third, Architectural-based CL (Arch-CL) alleviate forgetting by progressively expanding the model architecture, yet they often require a substantial number of additional parameters and increased computational cost [ 8 , 47 , 29 ] . Prompt-based Continual Learning. Recently, PCL has attracted attention because it can achieve high performance by training only a small number of prompts or a prompt pool, without requiring past-data storage or additional Task-ID [ 25 , 39 ] . Owing to these properties, a range of studies has explored PCL from both performance and efficiency perspectives. Nevertheless, existing methods often exhibit structural limitations in prompt selection. Hard selection (Top-k) methods [ 43 ] use only a subset of selected prompts, limiting expressiveness, and may suffer from training instability because gradients are not directly propagated through the selection process [ 31 ] . In contrast, soft selection (Softmax) methods [ 14 ] can utilize all prompts and learn them via gradients; however, due to the weighted-sum nature of softmax, even irrelevant prompts can exert influence, leading to accumulated noise [ 41 ] . Figure 1 : PCL exhibits classifier-level forgetting. We apply the cross-composition (backbone \times classifier) diagnostic of [ 21 ] to a [ 14 ] -based baseline under the skin-cancer DIL setting. Residual SODAP mitigates performance degradation caused by classifier instability and improves overall accuracy. The experimental setup and interpretation are provided in Sup. 0.A – 0.B . Distinctiveness and Proposed Approach. While our method also involves limited parameter expansion and prompt selection, it does not rely on naive architectural expansion. Instead, we focus on encouraging harmony between prior knowledge and new