FastODT: A tree-based framework for efficient continual learning

Daniel BretskoMasaryk UniversityBrno, Czech Republic [email protected], Piotr WalasMasaryk University, Brno, Czech RepublicWarsaw University of Technology, Warsaw, Poland [email protected], Devashish KhulbeMasaryk UniversityBrno, Czech Republic [email protected] Sebastian StrosMycroft MindBrno, Czech Republic [email protected] Stanislav SobolevskyMasaryk University, Brno, Czech RepublicNew York University, New York, NY, USAAnalog AI, Abu Dhabi, UAE [email protected] Tomas SaturaMycroft MindBrno, Czech Republic [email protected] · 2026

Read FastODT: A tree-based framework for efficient continual learning by Daniel BretskoMasaryk UniversityBrno, Czech Republic [email protected], Piotr WalasMasaryk University, Brno, Czech RepublicWarsaw University of Technology, Warsaw, Poland [email protected], Devashish KhulbeMasaryk UniversityBrno, Czech Republic [email protected] Sebastian StrosMycroft MindBrno, Czech Republic [email protected] Stanislav SobolevskyMasaryk University, Brno, Czech RepublicNew York University, New York, NY, USAAnalog AI, Abu Dhabi, UAE [email protected] Tomas SaturaMycroft MindBrno, Czech Republic [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13276.txt.

Introduction

Daniel Bretsko and co-authors introduce FastODT, a tree-based framework for efficient continual learning. Models in energy, weather, and sensing work face shifting data and tight compute budgets. Their oblivious tree model uses a Hoeffding bound to control its growth. Rapid learning and inference sit alongside efficient memory use and robust knowledge retention. Tests on energy and environmental time-series benchmarks matched or beat online and batch rivals. The authors present it as a scalable base for non-stationary settings with limited resources.

Catalog record

CREATORDaniel BretskoMasaryk UniversityBrno, Czech Republic [email protected], Piotr WalasMasaryk University, Brno, Czech RepublicWarsaw University of Technology, Warsaw, Poland [email protected], Devashish KhulbeMasaryk UniversityBrno, Czech Republic [email protected] Sebastian StrosMycroft MindBrno, Czech Republic [email protected] Stanislav SobolevskyMasaryk University, Brno, Czech RepublicNew York University, New York, NY, USAAnalog AI, Abu Dhabi, UAE [email protected] Tomas SaturaMycroft MindBrno, Czech Republic [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35866
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13276

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FastODT: A tree-based framework for efficient continual learning Daniel BretskoMasaryk UniversityBrno, Czech Republic [email protected], Piotr WalasMasaryk University, Brno, Czech RepublicWarsaw University of Technology, Warsaw, Poland [email protected], Devashish KhulbeMasaryk UniversityBrno, Czech Republic [email protected] Sebastian StrosMycroft MindBrno, Czech Republic [email protected] Stanislav SobolevskyMasaryk University, Brno, Czech RepublicNew York University, New York, NY, USAAnalog AI, Abu Dhabi, UAE [email protected] Tomas SaturaMycroft MindBrno, Czech Republic [email protected] Abstract Machine learning models deployed in real-world settings must operate under evolving data distributions and constrained computational resources. This challenge is particularly acute in non-stationary domains such as energy time series, weather monitoring, and environmental sensing. To remain effective, models must support adaptability, continuous learning, and long-term knowledge retention. This paper introduces a oblivious tree-based model with Hoeffding bound controlling its growth. It seamlessly integrates rapid learning and inference with efficient memory management and robust knowledge preservation, thus allowing for online learning. Extensive experiments across energy and environmental sensing time-series benchmarks demonstrate that the proposed framework achieves performance competitive with, and in several cases surpassing, existing online and batch learning methods, while maintaining superior computational efficiency. Collectively, these results demonstrate that the proposed approach fulfills the core objectives of adaptability, continual updating, and efficient retraining without full model retraining. The framework provides a scalable and resource-aware foundation for deployment in real-world non-stationary environments where resources are constrained and sustained adaptation is essential. 1 1 footnotetext: Equal contribution K eywords Continual Learning \cdot Sequential data modeling \cdot Tree-based models \cdot Hoeffding bound \cdot Obliviousness 1 Introduction 1.1 Background Continual and incremental learning (IL) remains a key frontier in machine learning, enabling models to adapt to evolving data distributions without requiring complete retraining. This capability is essential in real-world settings where data arrives sequentially or the underlying data distribution drifts over time. Incremental learning supports efficient adaptation and long-term retention, making it particularly valuable in domains characterized by non-stationary and streaming data, such as energy demand forecasting, real-time traffic prediction, weather monitoring, and environmental sensing [ 1 ] . Beyond these traditional applications, continual learning has recently gained prominence in large-scale foundational models, where the goal is to incrementally incorporate new knowledge while preventing catastrophic forgetting [ 2 , 3 ] . Incremental fine-tuning and online adaptation strategies are researched to keep ML models up to date with evolving facts, domains, and user preferences without costly retraining on massive corpora. Broadly, supervised incremental learning algorithms can be categorized into three main scenarios: Task-incremental, Domain-incremental, and Class-incremental learning [ 4 ] . In this work, we focus on Domain-incremental learning methods, specifically on the short-term time-series forecasting task, where the predictive model adapts to conceptually drifted distributions of lagged target variables. 1.2 Related Work Although most high-performing approaches to incremental learning are combinations of past approaches, there are at least two relevant ways to categorize IL approaches. The first categorization states that IL algorithms in the non-neural network part of machine learning can be divided into tree-based and non-tree-based approaches. Tree-based IL extends decision and regression trees to streaming contexts by implementing an update mechanism into how the trees are constructed. The most famous example is the Very Fast Decision Tree (VFDT) or Hoeffding Tree [ 5 ] . Unlike conventional approaches to splitting, which require full data, VFDT and its offshoots use the Hoeffding bound to determine statistically reliable splits from an incomplete dataset. In comparison, non-tree-based approaches have an updating mechanism implemented on a higher level than that of tree growing. This could involve tree deletion, weighting, instance weighting, forgetting, or even retraining the entire model. There are so many non-tree-based approaches that a second more descriptive categorization approach is needed. The second categorization approach introduces three categories windowing , performance-based methods , and ensemble methods . Windowing selects and weights instances into a window (or windows) on which the ML model is retrained. One of the oldest algorithms using this approach is STAGGER [ 6 ] . Currently, the most popular algorithm that retrains on a dynamically expanding and contracting window of the most relevant instances is ADWIN [ 7 ] . In contrast to picking the window of the most relevant instances irrespective of model performance, performance-based methods initiate retraining of the machine learning model when its accuracy drops or significantly deteriorates. State of the art algorithm of the approach is the Drift Detection Method (DDM), which treats deteriorated model performance as a rejection of the null hypothesis that the current data distribution has not changed [ 8 ] . The model’s bad performance is a temporal threshold from which point on window of new data is collected. After the performance worsens by 3 \sigma , the model is retrained on the data collected since the temporal threshold. Although both performance-based and windowing approaches generated burgeoning literature from the 1990s until the 2010s, the focus and highest performance on conceptually drifted data has shifted toward the ensemble methods. Our approach combines tree-based and ensemble methods. While building on previous incremental learning approaches and static machine learning ensemble models, incrementally learning ensemble methods introduce tree removal, retraining, and weighting. Ensemble methods such as Online Bagging [ 9 ] , Learn++.NSE [ 10 ] , and Adaptive Random Forest (ARF) [ 11 ] combine dynamic tree replacement, drift detection, and instance weighting to maintain accuracy under concept drift. More recently, incremental gradient boosting variants have emerged, aiming to retain the predictive power of gradient-boosted ensembles in online environments. iGBDT [ 12 ] incrementally updates gradient boosting trees by merging new batches with existing data, selectively rebuilding sub-tree structures when split attributes change without explicit drift detection thresholds. In contrast, eGBDT [ 13 ] , and GBDT-IL [ 14 ] extend the framework with pruning and rollback mechanisms that remove or revert under-performing trees when drift occurs, using ensemble-level residual monitoring and, in the case of GBDT-IL, Fisher Score–based feature selection. These methods improve the balance between adaptability and stability by integrating additive and subtractive updates, though they still face computational challenges due to the sequential nature of boosting. Overall, tree-based incremental ensemble methods offer interpretable, modular, and resource-efficient solutions, with ARF and incremental GBDTs representing two complementary directions, adaptive bagging versus boosting, for concept-drift–aware learning. In contrast, non-tree-based IL methods encompass neural, kernel, and probabilistic models—such as incremental SVMs [ 15 ] , evolving neural networks [ 16 ] , Bayesian learners [ 17 ] , and prototype-based models [ 18 ] , which incrementally update parameters, support vectors, or prototypes. These methods often achieve strong flexibility and generalization but at the cost of higher computational demands and reduced interpretability, making tree-based approaches particularly attractive for real-time, large-scale streaming applications. 1.3 System Proposal Our work proposes FastODT, a novel incremental learning framework that combines the efficiency of Oblivious Decision Trees (ODTs) with the adaptive growth of VFDTs. FastODT employs the Hoeffding Bound to incrementally expand the tree structure while maintaining a compact ODT-like form that eliminates traversal during inference. This results in low memory usage and high-speed predictions. However, once a tree reaches its maximum depth, learning halts to prevent an infinite increase in memory demand. End of growing makes the model susceptible to concept drift, which can be prevented by employing FastODTs in ensemble method. Particularily, two ensemble strategies

Read the complete text