Generate to Discriminate: Expert Routing for Continual Learning
Yewon Byun 1, Sanket Vaibhav Mehta 1, Saurabh Garg 1,2, Emma Strubell 1, Michael Oberst 3, Bryan Wilder 1, Zachary C. Lipton 1 · 2024
Read Generate to Discriminate: Expert Routing for Continual Learning by Yewon Byun 1, Sanket Vaibhav Mehta 1, Saurabh Garg 1,2, Emma Strubell 1, Michael Oberst 3, Bryan Wilder 1, Zachary C. Lipton 1 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-17009.txt.
Introduction
Byun and co-authors propose Generate to Discriminate, a continual learning method that routes data to expert models. Rules often let institutions share models but not the data behind them. The goal is adapting to new domains without forgetting old ones, the classic catastrophic forgetting trap.
An ensemble of domain experts can fit each site closely, if test samples reach the right expert. G2D trains a domain discriminator on synthetic data to do that routing. Oddly, routing with synthetic data beat training the main classifier on it directly. G2D topped rival domain-incremental methods on both vision and language tasks.
Catalog record
| CREATOR | Yewon Byun 1, Sanket Vaibhav Mehta 1, Saurabh Garg 1,2, Emma Strubell 1, Michael Oberst 3, Bryan Wilder 1, Zachary C. Lipton 1 |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-17009
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generate to Discriminate: Expert Routing for Continual Learning Yewon Byun 1, Sanket Vaibhav Mehta 1, Saurabh Garg 1,2, Emma Strubell 1, Michael Oberst 3, Bryan Wilder 1, Zachary C. Lipton 1 Abstract In many real-world settings, regulations and economic incentives permit the sharing of models but not data across institutional boundaries. In such scenarios, practitioners might hope to adapt models to new domains, without losing performance on previous domains (so-called catastrophic forgetting). While any single model may struggle to achieve this goal, learning an ensemble of domain-specific experts offers the potential to adapt more closely to each individual institution. However, a core challenge in this context is determining which expert to deploy at test time. In this paper, we propose Generate to Discriminate (G2D), a domain-incremental continual learning method that leverages synthetic data to train a domain-discriminator that routes samples at inference time to the appropriate expert. Surprisingly, we find that leveraging synthetic data in this capacity is more effective than using the samples to directly train the downstream classifier (the more common approach to leveraging synthetic data in the lifelong learning literature). We observe that G2D outperforms competitive domain-incremental learning methods on tasks in both vision and language modalities, providing a new perspective on the use of synthetic data in the lifelong learning literature. 1 Introduction To deploy machine learning reliably, it is important to develop methods for adapting models as we encounter environments sequentially in the wild. In medical imaging and risk prediction tasks, practitioners often apply models trained on one hospital’s data to make predictions on samples from other institutions [ 73 , 48 , 23 ] . Autonomous vehicle navigation systems must safely operate on different terrains, in different states, and even different countries. Moreover as we adapt to each new environment, it is desirable (whenever possible) that this adaptation comes without loss of accuracy in previously seen environments. In such continual learning problems, some of the most effective methods rely on a rehearsal buffer to re-train on a portion of past samples [ 7 , 39 ] . However, these solutions are often not viable in real-world prediction settings, where the landscape of regulatory and industry norms place stringent constraints on sharing data across institutional boundaries. We focus on the domain-incremental learning setting where the set of classes is fixed across domains and explicit data sharing across domains is prohibited. Crucially, domain identifiers are not given during inference time. The goal, after each round, is to produce a system that performs well on test examples drawn at random among all previously seen domains. At any round, our model only has access to the current data and, thus, simply performing gradient updates is liable to cause catastrophic forgetting , where performance decays on previously seen domains, even when the tasks are not fundamentally in conflict [ 43 , 22 ] . To enable lifelong learning in scenarios characterized by stringent constraints on data sharing, several settings for this problem have been proposed [ 58 ] . For example, generative replay methods utilize generative models to create synthetic samples for experience rehearsal [ 55 , 59 , 49 ] . However, these approaches have largely under-performed state-of-the-art discriminative approaches [ 59 ] , due at least in part to the introduction of noise in the form of low-quality synthetic data. Figure 1: Generate to Discriminate (G2D) ; During training (black text and arrows), we (i) finetune the generator and expert classifier and (ii) finetune a domain discriminator on synthetic images produced by our generator. At inference time (blue text and arrows), we route test samples to the corresponding expert, based on our discriminator’s prediction. Without access to a rehearsal buffer, rehearsal-free methods often tackle forgetting with dynamically expandable systems with domain-specific parameters, such as expert models [ 2 , 53 ] or parameter-efficient prompts [ 69 , 68 , 67 ] . As widely demonstrated in the literature [ 65 , 67 , 69 ] , such modular approaches allow learning parameters independently across domains to potential negative interference, leading to less/no forgetting and better transferring, avoiding tug-of-war scenarios that are exhibited in single, generalist models. However, a core remaining challenge is determining which expert or module to invoke at test time. Some recent methods rely on the implicit domain discriminative (zero-shot) capabilities of foundation models [ 1 ] for an inference-time routing strategy [ 67 ] . However, as we will show, in settings where the zero-shot discriminative performance of foundation models is limited, these methods underperform (see Table 1 ). The limitations and above-discussed challenges raise an interesting question: Rather than employing synthetic samples to train the downstream classifier, can synthetic samples serve a more effective purpose in domain discrimination? Can we leverage synthetically generated samples to develop an inference-time routing mechanism? In this paper, we propose a simple yet effective method for expert routing in domain-incremental continual learning, Generate to Discriminate (G2D) . G2D leverages synthetic data to train a domain discriminator that routes inference-time samples to a set of domain-specific expert classifiers, each checkpointed after each domain is encountered. Surprisingly, we observe that using synthetic data for training a domain router is more effective than using the same samples to augment training data for training the label classifier (as in generative replay methods). We also find that our approach outperforms other competitive domain-incremental learning methods, such as replay, regularization-based, and rehearsal-free (prompt-based) methods across benchmarks in both vision and text modalities. To further stress test current methods in more realistic settings, we introduce a new benchmark, DermCL, which consists of a training sequence of four publicly available dermatology medical image classification tasks, each previously utilized independently to evaluate transfer learning or robustness to distribution shifts [ 71 ] . We empirically show that state-of-the-art prompt-based methods that rely on the implicit (zero-shot) capabilities of foundation models [ 1 , 67 ] underperform (in comparison with our method) in these settings, where the downstream task (i.e., skin lesion prediction in minority populations) might diverge from tasks seen in pretraining corpora (see Table 1 ), emphasizing the need for more realistic benchmarks for a holistic evaluation of continual learning methods. To summarize, our contributions are as follows: • We empirically observe that with the same synthetic samples, training a domain identifier outperforms augmenting training data for the downstream label classifier—the common approach to leveraging synthetic data in the lifelong learning literature. • Leveraging this observation, we develop a simple method for expert routing in the domain-incremental learning setting, without any access to real data from previously seen tasks, that outperforms considered baselines in both vision and text modalities. • Towards evaluation of current methods in more realistic settings, we propose a new continual learning benchmark consisting of a sequence of four real-world, dermatology classification tasks. 2 Related Work The majority of continual learning methods fall into (i) parameter-based and (ii) data-based regularization techniques [ 58 ] . We also touch upon more recent approaches, dubbed (iii) prompt-based techniques that leverage pretrained models. Parameter-based Regularization Approaches. Most notable works in this category, namely Elastic Weight Consolidation (EWC; Kirkpatrick et al. 2017 ) and Synaptic Intelligence (SI; Zenke et al. 2017 ), assess the importance of parameters