Binary Cumulative Encoding meets Time Series Forecasting
Andrei Chernov, Vitaliy Pozdnyakov, Ilya Makarov · 2025
Read Binary Cumulative Encoding meets Time Series Forecasting by Andrei Chernov, Vitaliy Pozdnyakov, Ilya Makarov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24595.txt.
Introduction
Andrei Chernov, Vitaliy Pozdnyakov, and Ilya Makarov introduced BinConv in 2025, a forecasting design that treats future values as ordered categories. Turning a forecasting task into classification gives stable training and better uncertainty handling. The usual one-hot encoding, though, throws away the fact that the categories are ordered by size.
Their encoding writes each value as a cumulative binary code, which keeps order and distance information inside a classification setup. Their network is fully convolutional, which they show beats standard dense layers in both accuracy and cost for this encoding. On standard benchmarks, BinConv beat widely used baselines on point and probabilistic forecasts with fewer parameters and faster training.
Catalog record
| CREATOR | Andrei Chernov, Vitaliy Pozdnyakov, Ilya Makarov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24595
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Binary Cumulative Encoding meets Time Series Forecasting Andrei Chernov, Vitaliy Pozdnyakov, Ilya Makarov Abstract Recent studies in time series forecasting have explored formulating regression via classification task. By discretizing the continuous target space into bins and predicting over a fixed set of classes, these approaches benefit from stable training, robust uncertainty modeling, and compatibility with modern deep learning architectures. However, most existing methods rely on one-hot encoding that ignores the inherent ordinal structure of the underlying values. As a result, they fail to provide information about the relative distance between predicted and true values during training. In this paper, we propose to address this limitation by introducing binary cumulative encoding (BCE), that represents scalar targets into monotonic binary vectors. This encoding implicitly preserves order and magnitude information, allowing the model to learn distance-aware representations while still operating within a classification framework. We propose a convolutional neural network architecture specifically designed for BCE, incorporating residual and dilated convolutions to enable fast and expressive temporal modeling. Through extensive experiments on benchmark forecasting datasets, we show that our approach outperforms widely used methods in both point and probabilistic forecasting, while requiring fewer parameters and enabling faster training. 1 Introduction Time series forecasting plays a key role in decision-making across domains such as finance [ 1 ] , healthcare [ 30 ] , retail [ 5 ] , and climate science [ 33 ] . While classical methods like ARIMA and Exponential Smoothing remain widely used in practice [ 14 ] , recent advances increasingly favor deep learning approaches [ 18 , 16 ] , particularly in scenarios requiring high-precision forecasting [ 35 ] . A critical component of deep learning-based forecasting is data preprocessing. This typically includes normalization, min-max or mean scaling [ 29 ] , Box-Cox transformation [ 14 ] , and short-time Fourier transforms [ 10 ] . Recent studies [ 2 , 20 ] have explored quantization, discretizing the continuous target space into fixed-size bins. Values are then represented as one-hot vectors, enabling a tokenized view of time series aligned with Transformer-style architectures. Empirical results suggest that such representations enhance performance, especially within Time Series Foundation Models [ 35 ] . However, one-hot encoding lacks ordinal awareness, treating bins as unrelated categories and ignoring relative distances–important cues for modeling temporal dynamics. To address this, we propose Binary Cumulative Encoding (BCE): a representation where all bins below (or equal to) the current value are activated. This monotonic encoding, previously applied to tabular data [ 11 ] , preserves global ordinal structure and yields richer inputs for neural models. Figure 1 shows an example of BCE applied to a time series. Figure 1 : Visualization of different representations of time series data. (left) Original time series showing monthly airline passenger counts. (center) One-hot encoding of discretized values. (right) Binary Cumulative Encoding (BCE), where all bins less than or equal to the active bin are highlighted, preserving ordinal structure. Choosing an appropriate neural architecture is another central design decision. RNNs, CNNs, and Transformers are commonly used, with Transformer-based models [ 35 ] excelling due to their capacity for long-range temporal modeling [ 32 ] . Yet, their nearly quadratic complexity with sequence length [ 31 ] poses practical challenges. To overcome this, we introduce a CNN-based model that combines 2D and 1D convolutions to process BCE representations efficiently. This design reduces computational cost while preserving representational expressiveness. Moreover, BCE naturally supports probabilistic output by modeling distributions over bins–an ability often missing in standard CNN forecasting models. We validate our method on the ProbTS benchmark [ 35 ] , which includes diverse datasets from multiple domains. Our model consistently outperforms strong baselines in both point and probabilistic forecasting tasks. The main contributions of this work are: • A novel discrete representation, Binary Cumulative Encoding (BCE) , which captures ordinal and metric structure in time series. • A lightweight convolutional model, BinConv , combining 2D and 1D convolutions to process BCE sequences efficiently. • Extensive evaluation on the ProbTS benchmark, showing that our method achieves state-of-the-art results across multiple forecasting datasets. 2 Related Work Classical and Deep Learning Methods for Time Series Forecasting. Classical models such as ARIMA, exponential smoothing, and vector autoregression are widely used for their simplicity and efficiency [ 14 ] . However, they often underperform on large-scale datasets exhibiting complex and nonlinear patterns, as shown in M4, M5, M6 forecasting competitions [ 24 , 22 , 25 ] . Deep learning approaches improve upon these by framing forecasting as a sequence-to-sequence problem using fixed context windows [ 18 ] . Recent developments like Time Series Foundation Models [ 17 , 9 , 8 ] leverage large-scale pretraining to generalize across domains, achieving strong results in both point and probabilistic forecasting [ 35 ] . Neural Network Architectures for Time Series Forecasting. Neural models vary in their trade-offs between accuracy, scalability, and speed. Transformer-based models such as PatchTST [ 26 ] , TFT [ 19 ] , and Informer [ 36 ] capture long-range dependencies well, but incur high computational costs due to their quadratic scaling with sequence length. Earlier RNN-based models like LSTM [ 13 ] , GRU [ 6 ] , and DeepAR [ 29 ] are more efficient at inference, though limited by their sequential nature, which hinders training parallelism. CNN-based architectures such as TCN [ 4 ] , DeepTCN [ 7 ] , and ModernTCN [ 21 ] offer fast, parallel training and strong forecasting performance. Following this line, our CNN-based BinConv model balances accuracy and efficiency when used with BCE inputs. Token-Based Representations of Time Series. Inspired by NLP, recent work reformulates time series forecasting as next-token prediction using discretization. Models like LLMTime [ 12 ] , TiMER [ 20 ] , and Chronos [ 2 ] encode values as digits, patches, or bins. Gorishniy et al. [ 11 ] proposed piecewise linear encoding (PLE) for tabular data, preserving ordinal structure. Most models rely on one-hot encoding, which discards distance and order information between bins. This limits the ability to learn value relationships. In contrast, our BCE encoding captures both order and magnitude, improving learning while remaining compatible with token-based models. Probabilistic Forecasting with Deep Learning Methods. Probabilistic forecasting offers both predictions and uncertainty. Deep learning models approach this via quantile regression (e.g., TFT, DeepAR), parametric output distributions (e.g., DeepTCN), or generative models like TimeGrad [ 27 ] and GRU-NVP [ 28 ] . While effective, generative approaches can be unstable and costly to train. Token-based models such as Chronos and TiMER enable probabilistic inference via sampling in discrete space. Our model adopts this approach using BCE, supporting both point and probabilistic forecasts through structured sampling. 3 Binary Cumulative Encoding b_{0} b_{1} b_{2} b_{3} b_{4} \mathbb{R} \tilde{x} \Downarrow \mathrm{BCE}(x)= 1 v_{1} 1 v_{2} 1 v_{3} 0 v_{4} Figure 2 : Example of BCE encoding for 4 bins. The scale of time series can vary significantly, even within a single dataset. Therefore, before converting time series values into an encoding space, we apply normalization to map values into a quantized range. Similar to the Chronos model [ 2 ] , we use the mean scaling technique [ 29 ] , where each value is divided by the mean