CapyMOA: Efficient Machine Learning for Data Streams in Python
Heitor Murilo Gomes [email protected], Anton Lee [email protected], Nuwan Gunasekara [email protected], Yibin Sun [email protected], Guilherme Weigert Cassales [email protected], Justin Liu [email protected], Marco Heyden [email protected], Vitor Cerquei · 2025
Read CapyMOA: Efficient Machine Learning for Data Streams in Python by Heitor Murilo Gomes [email protected], Anton Lee [email protected], Nuwan Gunasekara [email protected], Yibin Sun [email protected], Guilherme Weigert Cassales [email protected], Justin Liu [email protected], Marco Heyden [email protected], Vitor Cerquei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07432.txt.
Introduction
Heitor Murilo Gomes and co-authors present CapyMOA, a Python library for learning from data streams, in a 2025 paper. Streaming data never ends and never sits still, so models must learn online and adapt as patterns drift. Python is where practitioners live, but the classic stream tools grew up in other languages.
CapyMOA offers a structured framework for real-time, continually adapting models. It plugs into MOA's fast algorithms, scikit-learn's familiar tools, and PyTorch's deep networks, mixing them in one workflow. The design stresses speed, scale, and plain usability. The open-source release, with its website and code repository, targets both researchers benchmarking stream learners and engineers shipping them into live systems.
Catalog record
| CREATOR | Heitor Murilo Gomes [email protected], Anton Lee [email protected], Nuwan Gunasekara [email protected], Yibin Sun [email protected], Guilherme Weigert Cassales [email protected], Justin Liu [email protected], Marco Heyden [email protected], Vitor Cerquei |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-07432
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CapyMOA: Efficient Machine Learning for Data Streams in Python Heitor Murilo Gomes [email protected], Anton Lee [email protected], Nuwan Gunasekara [email protected], Yibin Sun [email protected], Guilherme Weigert Cassales [email protected], Justin Liu [email protected], Marco Heyden [email protected], Vitor Cerqueira [email protected], Maroua Bahri [email protected], Yun Sing Koh [email protected], Bernhard Pfahringer [email protected], Albert Bifet [email protected] Abstract CapyMOA is an open-source library designed for efficient machine learning on streaming data. It provides a structured framework for real-time learning and evaluation, featuring a flexible data representation. CapyMOA includes an extensible architecture that allows integration with external frameworks such as MOA and PyTorch, facilitating hybrid learning approaches that combine traditional online algorithms with deep learning techniques. By emphasizing adaptability, scalability, and usability, CapyMOA allows researchers and practitioners to tackle dynamic learning challenges across various domains. † † shortheadings: Gomes et al. / Gomes et al. † † firstpage: 1 keywords Data Streams, Stream Learning, Machine Learning, Python, Java, Online Learning, Data Stream Mining 1 Introduction CapyMOA is an open-source library (BSD 3-Clause License) designed for efficient machine learning on streaming data. It provides a structured and extensible framework that enables real-time model updates while ensuring efficiency , interoperability , and accessibility . By offering a diverse set of algorithms and tools tailored for streaming data, CapyMOA facilitates research and practical applications. CapyMOA supports a wide range of online use cases, including classification, regression, anomaly detection, concept drift detection, clusering, semi-supervised learning, AutoML, online pipelines, and more. CapyMOA goes beyond existing libraries by offering a high-level modern API in Python coupled with efficient implementations of algorithms in Java. 2 Background and Related Work Machine learning for data streams presents unique challenges that distinguish it from traditional batch learning. Unlike static datasets, streaming data arrives continuously and must be processed incrementally without storing the entire dataset. Two key challenges in such scenarios are concept drift , where the statistical properties of the data change over time, requiring models to adjust dynamically to maintain predictive performance, and computational constraints , as models must process each instance upon arrival while operating under strict memory and processing limitations. To effectively address these challenges, practitioners and researchers need specialized tools for evaluating, analyzing, and experimenting with various algorithms. Several frameworks have been developed to address online learning 1 1 1 The terms online learning and stream learning are often used interchangeably, but they are not strictly identical. Online learning generally refers to models that update incrementally, instance by instance, without storing past data. Stream learning further emphasizes the dynamic nature of data streams, including challenges such as concept drift and strict computational constraints. In this paper, we use the terms contextually depending on the focus of discussion. , the most popular ones being MOA ( Bifet et al. 2010 ) , River ( Montiel et al. 2021 ) and Scikit-Multiflow ( Montiel et al. 2018 ) . MOA is a well-established Java-based framework with a large range of efficient implementations, but its Java-based code limits accessibility for new users that often prefer Python. River is a fully Python-based framework that focuses on usability and flexibility but sacrifices computational efficiency due to its pure Python implementation. Scikit-Multiflow was initially developed to provide Python implementations of MOA algorithms but has since been largely replaced by River. In summary, while MOA remains the most efficient choice for large-scale stream learning, it lacks the accessibility of a Python-native tool, whereas River prioritizes usability but struggles with performance, highlighting a trade-off between efficiency and accessibility in existing frameworks. CapyMOA addresses these limitations by providing an efficient, extensible, and user-friendly framework for stream learning. It is built upon three core pillars: Efficiency , leveraging optimized Java implementations with minimal memory overhead in a modern Python framework; Interoperability , supporting both native Python implementations and seamless integration with MOA and PyTorch for hybrid learning; and Accessibility , providing an intuitive Python API that simplifies complex stream learning tasks, such as concept drift detection, simulation, and evaluation. 3 Core Features of CapyMOA 3.1 Data Representation CapyMOA provides a structured approach to handling data streams through its Schema and Instance representations. A Schema defines the structure of a data stream, specifying the number of attributes, their types (nominal or numeric), and the target variable. This maintains consistency across streams and prevents runtime errors. An Instance represents a single data point within a stream, adhering to a predefined Schema that may evolve during execution. Unlike River , which uses a flexible but potentially ambiguous dictionary-based representation, CapyMOA enforces a structured Schema to minimize silent errors caused by inconsistent feature representations. This explicit design enhances stability, making CapyMOA particularly well-suited for long-term stream learning tasks. CapyMOA supports both snapshots of streams , i.e., datasets in either CSV or ARFF formats, and synthetic data streams generated in real-time. Synthetic streams enable controlled experimentation, particularly for evaluating concept drift adaptation. Finally, when processing data with MOA, CapyMOA fully abstracts MOA’s internal representations, eliminating the need for users to interact with Java-based structures. This ensures seamless integration with Python while maintaining a consistent evaluation framework and interoperability across implementations. 3.2 Pipelines Pipelines systematically prepare and process input data for machine learning models, ensuring smooth data flow in streaming environments. In the context of streaming data, pipelines are particularly challenging to design and implement due to the stateful nature of some transformations ( Gomes et al. 2019b ) , which require continuous updates as new data arrives. Streaming pipelines often involve complex interactions between components. For instance, drift detection mechanisms in a pipeline may broadcast information to multiple downstream components, requiring synchronization and efficient communication. Pipeline elements represent single processing steps, including transformers (such as feature scaling), classifiers, regressors, and drift detectors. An essential step in any machine learning pipeline is feature transformation or data preprocessing, which encompasses tasks such as the creation of new features, dimensionality reduction, and feature selection ( Bahri et al. 2021 ) . In CapyMOA, pipelines are designed to handle these challenges effectively, providing a modular and extensible framework. Users can build pipelines with components like incremental feature scalers, drift detectors, and classifiers that adapt in real time. The framework supports advanced configurations, such as setting up listeners for drift detection events and dynamically altering pipeline behavior based on these signals. Furthermore, CapyMOA provides high flexibility in pipeline design. Users can define custom pipeline elements by implementing the required methods in the abstract “PipelineElement” class, or compose complex integrations such as multiple drift detectors at different points (e.g., ABCD for input distribution monitoring and ADWIN for accuracy tracking). 3.3 Handling Concept Drifts CapyMOA provides robust tools for simulating, detecting, and adapting to concept drifts, a crucial challenge in stream learning. Concept drift refers to changes in the data distribution over time, which can degrade model performance if not handled appropriately. CapyMOA’s DriftStream module enables users to simulate various types of concept drifts, such as abrupt and gradual shifts. This allows testing adaptive algorithms like Adaptive Random Forest in realistic dynamic environments. For example, the following code configures a stream with multiple drifts: ⬇ from capymoa . stream . generator import SEA from capymoa . stream . drift import DriftStream , AbruptDrift , GradualDrift stream = DriftStream ( stream =[ SEA (1), AbruptDrift (10000), SEA (2), GradualDrift