Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning

Congren Dai, Huichi Zhou, Jiahao Huang, Zhenxuan Zhang, Fanwen Wang, Yijian Gao, Guang Yang, Fei Ye · 2025

Read Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning by Congren Dai, Huichi Zhou, Jiahao Huang, Zhenxuan Zhang, Fanwen Wang, Yijian Gao, Guang Yang, Fei Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18101.txt.

Introduction

Congren Dai and co-authors present a 2025 paper on teaching neural networks new data streams without erasing old skills. Online continual learning feeds a model one batch after another, and catastrophic forgetting wipes earlier knowledge. Their framework, ODEDM, keeps two memories: a quick short-term store and a long-term store split into sub-buffers around cluster centers.

A clustering step picks the prototype samples worth keeping, and an optimal transport rule guards the most representative ones. A divide-and-conquer update scheme breaks memory maintenance into smaller problems to cut computing cost. The module plugs into existing rehearsal methods rather than replacing them. Tests under both balanced and skewed data streams put it at the front of the pack.

Catalog record

CREATORCongren Dai, Huichi Zhou, Jiahao Huang, Zhenxuan Zhang, Fanwen Wang, Yijian Gao, Guang Yang, Fei Ye
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18101

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning Congren Dai, Huichi Zhou, Jiahao Huang, Zhenxuan Zhang, Fanwen Wang, Yijian Gao, Guang Yang, Fei Ye Abstract Online Continual Learning (OCL) involves sequentially arriving data and is particularly challenged by catastrophic forgetting, which significantly impairs model performance. To address this issue, we introduce a novel framework, Online Dynamic Expandable Dual Memory (ODEDM), that integrates a short-term memory for fast memory and a long-term memory structured into sub-buffers anchored by cluster prototypes, enabling the storage of diverse and category-specific samples to mitigate forgetting. We propose a novel K K -means-based strategy for prototype identification and an optimal transport-based mechanism to retain critical samples, prioritising those exhibiting high similarity to their corresponding prototypes. This design preserves semantically rich information. Additionally, we propose a Divide-and-Conquer (DAC) optimisation strategy that decomposes memory updates into subproblems, thereby reducing computational overhead. ODEDM functions as a plug-and-play module that can be seamlessly integrated with existing rehearsal-based approaches. Experimental results under both standard and imbalanced OCL settings show that ODEDM consistently achieves state-of-the-art performance across multiple datasets, delivering substantial improvements over the DER family as well as recent methods such as VR-MCL and POCL. † † footnotetext: Corresponding author. 1 Introduction Humans exhibit a remarkable ability to acquire new knowledge continuously from their environments while retaining previously learned information. This capability, known as Continual Learning (CL) [ 41 ] , has emerged as a prominent research area within the field of deep learning. In contrast to traditional deep learning paradigms [ 27 ] , which involve retraining on entire datasets multiple times, CL focuses on developing models that can sustain high performance across all prior tasks while assimilating new information. Still, contemporary learning models often experience considerable performance degradation in CL scenarios, primarily due to a phenomenon known as catastrophic forgetting [ 41 ] . The challenge of catastrophic forgetting arises when a model, while learning new tasks, gradually overwrites previously acquired knowledge [ 41 ] . To mitigate this, existing work generally falls into three categories: rehearsal-based methods that store a small set of past examples in a memory buffer and replay them during training; regularisation methods that add penalty terms to the loss function to prevent drastic changes in important parameters; and architecture-based methods that expand the network by adding or removing modules for each incoming task. However, most rehearsal-based approaches rely on a single memory buffer, which struggles to balance the retention of both recent and distant experiences under limited capacity, leaving the model vulnerable to forgetting. Additionally, another major challenge in CL is that most existing methodologies typically assume a static learning setting in which all past data can be revisited repeatedly and each class or task is perfectly balanced. These assumptions are rarely met in real-world scenarios. This focus overlooks a more complex and realistic CL context, characterised by the condition that each data batch is encountered only once, which we refer to as Online Continual Batch-to-Batch Learning (OCBBL). In addition, the data samples in the real-world scenario would be imbalanced, and such a scenario is less explored in OCCBBL. In this paper, we endeavour to tackle catastrophic forgetting under both the OCBBL and the dataset imbalance settings by introducing an innovative memory-based strategy. The biological findings indicate that knowledge is continuously processed in the brain through both fast and slow memory mechanisms [ 40 ] . Drawing inspiration from these findings, we introduce an innovative memory framework termed Online Dynamic Expandable Dual Memory (ODEDM). This framework comprises a short-term memory system designed to accommodate dynamic and transient information, alongside a long-term memory system intended to retain enduring knowledge. Specifically, we implement the short-term memory system utilising a buffer, which facilitates the dynamic storage of new data samples while minimising computational overhead. For the long-term memory system, we propose segmenting it into a series of cluster prototypes, each capable of forming an expandable sub-memory buffer to retain data samples that exhibit similar semantic characteristics as seeing more samples from different categories. To tackle the issue of imbalanced data samples, we present a novel sample selection strategy that ensures an equal number of cluster prototypes per category through a K -means clustering algorithm. Each cluster prototype serves as the focal point of its corresponding sub-memory buffer, designed to maintain an equivalent number of data samples for each distinct category, thereby ensuring balance across the memory system. To accumulate critical samples to an appropriate sub-memory buffer, we propose a novel memory optimisation approach that evaluates the knowledge similarity between each cluster prototype and incoming samples via an Optimal Transport (OT) distance. This approach selectively stores the data samples that contain significantly different information from the corresponding cluster prototype. Such a mechanism can promote each sub-memory buffer to store semantically informative data samples, leading to good generalisation performance. Furthermore, we propose a new dynamic memory allocation approach, which automatically assigns more capacity for the short-term memory buffer to provide sufficient training samples in the initial learning phase, which benefits the model convergence. As the number of training steps increases, the proposed approach incrementally assigns more capacities for the long-term memory buffer to preserve more critical past samples, which effectively relieves network forgetting. We also propose a new Divide-and-Conquer (DAC) strategy to reduce the computational cost of the memory optimisation process. Specifically, the DAC strategy recursively groups the incoming samples into a series of cluster prototypes and then combines the samples that are close to each cluster prototype. We conduct a series of experiments on both standard and imbalanced settings to evaluate the performance of various models, and the empirical results demonstrate that the proposed approach achieves state-of-the-art performance in various CL scenarios. The contributions of this paper are summarised into three parts: (1) inspired by the biological findings, we propose a novel training framework that incorporates short- and long-term memory buffers with dynamically changing sizes to address catastrophic forgetting in the OCBBL scenario; (2) ODEDM, a plug-and-play framework, is designed to be compatible with rehearsal-based models, and it is shown experimentally to enhance their performance; (3) we also propose a novel DAC approach that can significantly reduce the computational cost of ODEDM. 2 Related work Recent advances in OCL have explored memory management and sample selection to mitigate catastrophic forgetting. OnPro [ 53 ] maintains class-wise prototypes for contrastive learning but assumes clean streams and may struggle with data imbalance. OCM [ 26 ] uses mutual information (MI) maximisation to learn holistic feature representations and preserve past knowledge. It enhances representation robustness via MI objectives between inputs, outputs, and past features, using data augmentations to tighten MI bounds. PuriDivER [ 3 ] handles noisy and ambiguous streams via a hybrid memory strategy and semi-supervised learning on memory partitions. VR-MCL [ 56 ] improves OCL via momentum-based variance reduction for stable Hessian approximation, while POCL [ 57 ] employs Pareto-optimised gradient aggregation to model inter-task relations and balance past–current performance. Dual memory systems have been widely adopted to mitigate catastrophic forgetting in continual learning. DGM [ 32 ] proposes a generative dual-memory network that consolidates short-term learning via generative replay. DualMEM [ 39 ] emulates the PFC-HC circuitry with a stratified short-term memory and long-term memory design,

Read the complete text