DeepBooTS: Dual‑Stream Residual Boosting for Drift‑Resilient Time‑Series Forecasting
Daojun Liang, Jing Chen, Xiao Wang, Yinglong Wang, Suo Li · 2025
Read DeepBooTS: Dual‑Stream Residual Boosting for Drift‑Resilient Time‑Series Forecasting by Daojun Liang, Jing Chen, Xiao Wang, Yinglong Wang, Suo Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06893.txt.
Introduction
Daojun Liang and four co-authors design a forecasting method for time series whose patterns drift over time. Concept drift defeats many forecasters even when they normalize each window of data. Analyzing drift through a bias-variance lens, the team first proves that weighted ensembles reduce variance without adding bias, and builds their method on that insight.
DeepBooTS is a dual-stream boosting design in which each block of a deep network acts as an ensemble learner, with an auxiliary output branch forming a highway to the final prediction. Block outputs progressively correct the residuals of earlier blocks, decomposing both inputs and targets in a learned way. Across large-scale benchmarks it beat current methods by an average of about sixteen percent, setting a new mark for drift-resilient forecasting.
Catalog record
| CREATOR | Daojun Liang, Jing Chen, Xiao Wang, Yinglong Wang, Suo Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 108 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-06893
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DeepBooTS: Dual‑Stream Residual Boosting for Drift‑Resilient Time‑Series Forecasting Daojun Liang, Jing Chen, Xiao Wang, Yinglong Wang, Suo Li Abstract Time-Series (TS) exhibits pronounced non-stationarity. Consequently, most forecasting methods display compromised robustness to concept drift, despite the prevalent application of instance normalization. We tackle this challenge by first analysing concept drift through a bias-variance lens and proving that weighted ensemble reduces variance without increasing bias. These insights motivate DeepBooTS, a novel end-to-end dual-stream residual-decreasing boosting method that progressively reconstructs the intrinsic signal. In our design, each block of a deep model becomes an ensemble of learners with an auxiliary output branch forming a highway to the final prediction. The block‑wise outputs correct the residuals of previous blocks, leading to a learning‑driven decomposition of both inputs and targets. This method enhances versatility and interpretability while substantially improving robustness to concept drift. Extensive experiments, including those on large-scale datasets, show that the proposed method outperforms existing methods by a large margin, yielding an average performance improvement of 15.8% across various datasets, establishing a new benchmark for TS forecasting. Code — https://github.com/Anoise/DeepBooTS 1 Introduction Time Series (TS) from natural and engineered systems often evolve under transient conditions ( 2 ) and therefore violate stationarity assumptions ( 16 ) . Classical statistical models such as ARIMA ( 43 ) and exponential smoothing ( 11 ) , which depend on fixed statistical properties, struggle to track these shifts ( 10 ) . Deep neural networks have become attractive alternatives because of their powerful nonlinear modeling capabilities ( 15 ) , yet even the latest attention‑based ( 67 ; 40 ; 35 ; 57 ; 49 ; 50 ) and graph‑neural approaches ( 55 ; 59 ) provide only marginal gains over simple multilayer perceptrons and incur substantial inference overhead ( 46 ; 63 ; 58 ; 61 ; 62 ) . Figure 1: TS forecasting models suffer from concept drift problems, more experiments are provided in Appendix H . A deeper issue is that leading architectures like Transformers ( 48 ) and their variants are highly susceptible to concept drift—a mismatch between training and testing distributions. Empirical evidence shows that validation error rises early in training even as training error falls, as shown in Fig. 1 , a sign that the learned model fails to generalize as the underlying data distribution shifts. Prevailing solutions, including reversible instance normalization (RevIN) ( 19 ) and temporal re‑aggregation, e.g., DLinear ( 63 ) and iTransformer ( 35 ) , alleviate mean shifts but leave the variance unstable, which leads to large fluctuations and persistent concept drift. This phenomenon occurs across multiple TS forecasting models and datasets, as shown in Fig. 2 (a), underscoring the need for an approach that directly targets the high‑variance component. Figure 2: (a) Concept drift exists in current TS datasets. (b) Concept drift causes the model training error to decrease while the validation error increases. (c) The performance of DeepBooTS after reducing concept drift. This work addresses the problem by framing concept drift through the bias-variance decomposition. We theoretically show that when the mean bias and noise are fixed, the degree of drift is controlled by the variance of the model predictions. We then prove that ensemble averaging reduces the prediction variance while preserving bias and derive new bounds showing that weighted ensembles achieve a strictly lower error under distribution shift. These results suggest that a properly designed ensemble can substantially mitigate concept drift. Building on this insight, we introduce DeepBooTS, a novel dual-stream residual-decreasing boosting architecture that progressively reinstates the intrinsic values of a non‑stationary TS. The key innovations are: • We rigorously analyze concept drift through the lens of the bias-variance trade-off and prove that even simple deep ensembles can substantially reduce prediction variance without increasing bias. • An efficient implementation of DeepBooTS is presented. Specifically, the outputs of subsequent blocks subtract the predictions of previous blocks, causing the network to explicitly model and reduce residual errors layer by layer. This residual‑learning mechanism is analogous to gradient boosting, but implemented within a deep network, enhancing robustness to distributional shifts. • A dual‑stream decomposition is designed that decomposes both the input and labels, enabling the model to learn complementary representations while enhancing model versatility and interpretability. We validate DeepBooTS on a wide range of TS benchmarks, including large‑scale TS datasets. As shown in Fig. 2 (c), our method consistently outperforms state‑of‑the‑art baselines, achieving an average 15.8% improvement. These results, together with our theoretical analysis, demonstrate that DeepBooTS not only advances the accuracy of TS forecasting but also provides a principled solution to the longstanding problem of concept drift. The code and datasets can be obtained from the appendices we uploaded. 2 Methodology Figure 3: (a) Relationship among model’s bias, variance, and loss. (b) Deep boosting ensemble learning process. 2.1 Deep Ensemble helps Alleviate Concept Drift As analyzed above, RevIN-processed (Z-Score) TS data maintains mean stationarity while exhibiting substantial variance instability. We consider a forecasting task where the data distribution shifts from an initial distribution P_{0} to a new distribution P_{t} at time t . Formally, let Y=f_{0}(X)+\varepsilon under P_{0} (with true regression function f_{0} ), and at time t let Y=f_{t}(X)+\varepsilon under P_{t} , where f_{t} is the new underlying function after drift, \varepsilon\sim\mathcal{N}(0,\sigma) . For forecasting results \hat{Y} , the estimation error (MSE) of the model is \underbrace{\text{Var}(\hat{Y})+(\text{Bias}(\hat{Y}))^{2}+\sigma^{2}}_{\text{Test Error}}=\underbrace{\mathbb{E}[(\hat{Y}-Y)^{2}]+2\mathbb{E}(\varepsilon(\hat{Y}-\mathcal{Y}))}_{\text{Training Error}}. (1) The proof is given in Appendix B.1 . Eq. 1 , which is illustrated in Fig. 3 (a), shows that when the mean \text{Bias}(\hat{Y}) and noise level \sigma^{2} are fixed, the variance of the data governs the extent of concept drift. Thus, models incapable of adapting to large data distribution shifts exhibit high prediction variance. Now, we show that ensemble reduces the variance. Without loss of generality, using MSE as the metric and applying the simple average ensemble method, we have Theorem 1 . Let \hat{Y}_{1},\cdots,\hat{Y}_{N} be N i.i.d random variables drawn from some unknown distribution, and the ensemble method is \bar{Y}=\frac{1}{N}\sum_{t}\hat{Y}^{t} . Then, \displaystyle\text{Bias}(\bar{Y})=\text{Bias}(\hat{Y}),\quad\text{Var}(\bar{Y})\leq\text{Var}(\hat{Y}). (2) The proof is given in Appendix B.2 . Although it adopts the simplest ensemble form, it proves that ensembling reduces variance while conserving bias. If substituting \hat{f}_{0}(X) (which was trained to approximate f_{0} on P_{0} ) for \hat{Y} in Eq. 1 , we have \text{MSE}(f_{t}(X)-\hat{f}_{0}(X))=\text{Var}(\hat{f}_{0}(X))+(\text{Bias}(\hat{f}_{0}(X)))^{2}+\sigma^{2}. (3) After the distribution shifts to P_{t} , the forecasting error is \text{MSE}_{P_{t}}(f_{0}(X))=\mathbb{E}_{P_{t}}[\text{Bias}_{P_{0}}(X)^{2}]+\mathbb{E}_{P_{t}}[\text{Var}_{P_{0}}(f_{0}(X))]+\sigma^{2}. (4) Let \mathrm{Var}_{P_{t}}[X]=\sigma_{t}^{2} with c^{2}=\sigma^{2}_{t}/\sigma^{2} to quantify the change in input variance, thus we have Theorem 2 . Given a weighted ensemble models \hat{f}_{\text{ens}}(X)=\frac{1}{L}\sum_{l=1}^{L}\alpha_{l}g_{l}(x) , where \sum_{l=1}^{L}\alpha_{l}=1 , we have \displaystyle\text{MSE}_{P_{t}}(\hat{f}_{0})-\text{MSE}_{P_{t}}(\hat{f}_{\text{ens}})\approx\frac{(L-1)(1-\alpha_{l})}{L}c^{2}\sigma^{2}_{t}\geq 0. (5) The proof is given in Appendix B.3 . It indicates that the ensemble’s MSE under P_{t} is lower than that of the single one. 2.2 Deep Ensemble Model for TS Forecasting Based on Theorem 2 , we consider a ensemble learner set \{g_{l}\}_{l=1}^{L} to minimize \displaystyle\ \ \mathbb{E}(Y-\hat{f}_{\text{ens}}(X))^{2}=\mathbb{E}(Y-\sum_{l=1}^{L}\alpha_{l}g_{l}(X))^{2}, (6) \displaystyle s.t.\ \ \sum_{l=1}^{L}\alpha_{l}=1,\text{and}\ \ \alpha_{l}\geq 0,\ \ l=1,\cdots,L. Here, the objective in Eq. 6 implies that (Y-\hat{f}_{\text{ens}}(X))^{2}=\sum_{l=1}^{L}\alpha_{l}(Y-g_{l}(X))^{2}-\sum_{l=1}^{L}\alpha_{l}(\hat{f}_{\text{ens}}(X)-g_{l}(X))^{2}. (7) The proof is given in Appendix B.4 . Eq. 7 proves that the ensemble generalization error can be reduced by increasing the ambiguity without increasing the bias, since the ambiguity item (the 2nd term on the right), which measures the disagreement among the base predictors, is always