Drift-aware Collaborative Assistance Mixture of Experts for Heterogeneous Multistream Learning

En Yu, Jie Lu, Kun Wang, Xiaoyu Yang, Guangquan Zhang · 2025

Read Drift-aware Collaborative Assistance Mixture of Experts for Heterogeneous Multistream Learning by En Yu, Jie Lu, Kun Wang, Xiaoyu Yang, Guangquan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01598.txt.

Introduction

En Yu and co-authors present CAMEL in 2025, a learning system for data that arrives as many live streams at once. Streams differ from each other and their patterns drift over time, which breaks methods built for uniform, steady data.

Each stream gets its own feature extractor and task head, plus a pool of private experts for its quirks. A shared assistance expert watches all streams and pulls in only helpful context, using attention to weigh each source. A tuner adds new experts when patterns shift and retires stale ones. Tests across varied stream sets showed strong accuracy and unusual toughness against drift, the authors report.

Catalog record

CREATOREn Yu, Jie Lu, Kun Wang, Xiaoyu Yang, Guangquan Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01598

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Drift-aware Collaborative Assistance Mixture of Experts for Heterogeneous Multistream Learning En Yu, Jie Lu, Kun Wang, Xiaoyu Yang, Guangquan Zhang Abstract Learning from multiple data streams in real-world scenarios is fundamentally challenging due to intrinsic heterogeneity and unpredictable concept drifts. Existing methods typically assume homogeneous streams and employ static architectures with indiscriminate knowledge fusion, limiting generalizability in complex dynamic environments. To tackle this gap, we propose CAMEL , a dynamic C ollaborative A ssistance M ixture of E xperts L earning framework. It addresses heterogeneity by assigning each stream an independent system with a dedicated feature extractor and task-specific head. Meanwhile, a dynamic pool of specialized private experts captures stream-specific idiosyncratic patterns. Crucially, collaboration across these heterogeneous streams is enabled by a dedicated assistance expert. This expert employs a multi-head attention mechanism to distill and integrate relevant context autonomously from all other concurrent streams. It facilitates targeted knowledge transfer while inherently mitigating negative transfer from irrelevant sources. Furthermore, we propose an Autonomous Expert Tuner (AET) strategy, which dynamically manages expert lifecycles in response to drift. It instantiates new experts for emerging concepts (freezing prior ones to prevent catastrophic forgetting) and prunes obsolete ones. This expert-level plasticity provides a robust and efficient mechanism for online model capacity adaptation. Extensive experiments demonstrate CAMEL ’s superior generalizability across diverse multistreams and exceptional resilience against complex concept drifts. Introduction Learning from streaming data has become fundamental to modern intelligent systems, enabling real-time decision-making in dynamic and continuously evolving environments ( Cacciarelli and Kulahci 2024 ; Marcu and Bouvry 2024 ; Agrahari and Singh 2022 ) . A central challenge in streaming learning is concept drift—the phenomenon where the underlying data distribution changes over time—requiring models to continuously adapt in order to maintain predictive performance ( Lu et al. 2018 ) . While most streaming learning studies focus on single-stream settings ( Jiao et al. 2024 ; Wen et al. 2023 ) , many real-world applications inherently involve multiple concurrent data streams. For example, a smart city platform integrates traffic sensor feeds, weather reports, public transportation logs, and social media sentiment streams. These streams evolve independently yet often carry latent correlations that, if exploited effectively, can provide complementary information for more accurate and robust decision-making ( Xiang et al. 2023 ; Read and Zliobaite 2025 ; Ma et al. 2024 ) . Capturing such dynamic inter-stream relationships while adapting to concept drift is crucial for advancing streaming learning toward practical deployment ( Yang, Lu, and Yu 2025 ; Xu, Chen, and Wang 2025a ) . Despite recent progress, existing multistream learning methods are caught in a critical dilemma. On the one hand, most approaches operate under a homogeneous space assumption, which presumes that all streams share the same feature and label spaces ( Yu et al. 2024 ; Jiao et al. 2023 ) . This assumption fails to deal with the intrinsic heterogeneity commonly present in practical applications, where streams may originate from distinct feature spaces or predictive objectives due to different data sources ( Korycki and Krawczyk 2021 ; Panchal et al. 2023 ) . On the other hand, prevailing methods typically employ a monolithic and static architecture, either retrained or incrementally fine-tuned ( Xu, Chen, and Wang 2025b ; Wang et al. 2021 ) . This design suffers from critical limitations in multistream environments, e.g., retraining induces catastrophic forgetting of prior knowledge, while fine-tuning becomes fragile under asynchronous drifts, where adapting to one stream’s evolution can degrade performance on others. The lack of structural flexibility and targeted adaptation thus prevents robust performance across heterogeneous evolving streams. To bridge this gap, we formalize the problem as Heterogeneous Multistream Learning (HML), where multiple concurrent data streams exhibit intrinsic heterogeneity, latent inter-stream correlations, and asynchronous concept drifts. Specifically, 1) Intrinsic Heterogeneity : feature and label spaces across streams differ in dimensionality and semantics, precluding direct application of homogeneous models; 2) Knowledge Fusion : while streams may contain useful correlations, such relationships are dynamic and selective, requiring mechanisms that can leverage relevant information while avoiding negative transfer from irrelevant streams; and 3) Asynchronous Concept Drifts : streams evolve independently with diverse drift patterns, demanding flexible and stream-specific adaptation. These challenges necessitate a generalized and drift-aware learning framework that can handle stream-wise specialization while enabling intelligent knowledge fusion across heterogeneous drifting streams. To address these challenges, we propose CAMEL , a dynamic Collaborative Assistance Mixture of Experts Learning framework tailored for heterogeneous data streams. It introduces a modular drift-aware architecture that explicitly addresses the three core challenges. First, to handle intrinsic heterogeneity, we assign each stream a specific learning system comprising a dedicated feature extractor, a private expert pool, and a task-specific prediction head, ensuring stream-specific specialization. Second, to enable adaptive and selective knowledge fusion, CAMEL incorporates a novel collaborative assistance mechanism. It employs a dedicated attention-based expert per stream dynamically distills relevant contextual information from all other concurrent streams on demand, effectively capturing latent inter-stream correlations while inherently mitigating negative transfer ( Vaswani et al. 2017 ) . Third, to cope with asynchronous concept drifts, an Autonomous Expert Tuner (AET) is proposed, which monitors drift signals by a distribution-based drift detector and performance indicators per stream, dynamically adding new experts for emerging concepts and pruning obsolete ones. This expert-level plasticity allows our method to autonomously restructure its capacity and specialization over time. Extensive experiments on diverse synthetic and real-world multistream scenarios demonstrate the superior adaptability and generalization ability of our method compared to existing state-of-the-art methods. In summary, our main contributions are: • We propose CAMEL , a generalized and dynamic MoE framework that learns from multiple data streams characterized by heterogeneous features, diverse label spaces and asynchronous concept drifts. • We introduce a collaborative assistance mechanism, where dedicated attention-based experts perform targeted knowledge fusion, providing a effective and adaptive solution to the challenge of positive knowledge transfer. • We design an autonomous tuning strategy that manages the expert lifecycle at a modular level (adding/pruning experts), offering a more robust and interpretable way for drift adaptation. • Comprehensive experiments and theoretical analysis validate the generalizability and robustness of our method across complex synthetic and real-world HML scenarios. Figure 1: The overall framework of CAMEL . Concretely, each stream’s MoE module leverages a dynamic pool of private experts and a dedicated assistance expert that performs collaborative fusion via multi-head attention. The entire system follows a Test-Diagnose-Adapt cycle where an Autonomous Expert Tuner (AET) dynamically manages the expert lifecycle (adding/freezing/pruning) in response to drift and performance signals, ensuring continuous adaptation in the HML environment. Related Works Stream Learning. Early research in streaming learning primarily addressed single-stream scenarios with concept drift ( Wan, Liang, and Yoon 2024 ; Li et al. 2022 ; Kim, Hwang, and Whang 2024 ; Yu et al. 2025 ) , broadly falling into two paradigms: 1) informed methods integrate explicit drift detection mechanisms to trigger model adaptation based on distribution variations or error signals ( Bifet and Gavalda 2007 ; Gomes, Read, and Bifet 2019 ; Lu et al. 2025 ) , while 2) adaptive approaches employ detection-free strategies that continuously adjust model parameters in response to evolving data dynamics ( Guo, Zhang, and Wang 2021 ; Brzezinski and Stefanowski 2013 ; Jiao et al.

Read the complete text