Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series

Azimov Sherkhon, Susana López-Moreno, Eric Dolores-Cuenca, Sieun Lee, Sangil Kim · 2025

Read Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series by Azimov Sherkhon, Susana López-Moreno, Eric Dolores-Cuenca, Sieun Lee, Sangil Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08738.txt.

Introduction

Sherkhon Azimov and a team of co-authors built a self tuning forecaster for noisy chaotic series. Their paper, from twenty twenty-five, deals with nonlinear vector autoregression and its kin, reservoir computing. Both have done well on test systems like the Lorenz flow and on climate swings. But both rest on fixed bends of the data, which caps their fit to noise or wild real fields. High sizes also bring dear matrix costs.

Their model joins delay stacked linear inputs with bends from a small trainable net. The net and the read out train as one by gradient steps. Readers who forecast storms, seas, or markets get a model that shapes its own bends to the data.

Catalog record

CREATORAzimov Sherkhon, Susana López-Moreno, Eric Dolores-Cuenca, Sieun Lee, Sangil Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08738

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series Azimov Sherkhon, Susana López-Moreno, Eric Dolores-Cuenca, Sieun Lee, Sangil Kim Abstract Nonlinear vector autoregression (NVAR) and reservoir computing (RC) have shown promise in forecasting chaotic dynamical systems, such as the Lorenz-63 model and El Niño–Southern Oscillation. However, their reliance on fixed nonlinearities—polynomial expansions in NVAR or random feature maps in RC—limits their adaptability to high noise or real-world data. These methods also scale poorly in high-dimensional settings due to costly matrix inversion during readout computation. We propose an adaptive NVAR model that combines delay-embedded linear inputs with features generated by a shallow, learnable multi-layer perceptron (MLP). The MLP and linear readout are jointly trained using gradient-based optimization, enabling the model to learn data-driven nonlinearities while preserving a simple readout structure. Unlike standard NVAR, our approach avoids the need for an exhaustive and sensitive grid search over ridge and delay parameters. Instead, tuning is restricted to neural network hyperparameters, improving scalability. Initial experiments on chaotic systems tested under noise-free and synthetically noisy conditions showed that the adaptive model outperformed the standard NVAR in predictive accuracy and showed robust forecasting under noisy conditions with a lower observation frequency. 1 Introduction 1.1 Motivation Time series analysis and forecasting have become essential tools in many fields, such as climate science [ 2 ] , [ 6 ] , [ 43 ] , [ 11 ] , finance [ 4 ] , [ 10 ] , [ 38 ] , healthcare and medicine [ 24 ] , [ 5 ] , [ 34 ] and transportation [ 27 ] . Numerous cutting-edge methods have been proposed to address the difficulties in modeling complex temporal dependencies in these domains. Reservoir computing (RC) is a machine learning technique developed in the early 2000s that uses reservoirs to learn spatiotemporal features in time series data [ 28 ] , [ 19 ] , [ 30 ] . An optimized RC can handle even the most difficult problems, such as chaotic or complex spatiotemporal behaviors [ 33 ] , [ 32 ] . In echo state networks (ESN) [ 18 ] , the most commonly used RC model, the reservoir consists of a recurrent neural network [ 15 ] , where the weights are randomly sampled and fixed (i.e., they are untrained). By fixing the reservoir weights, training is reduced to a simple and efficient least squares estimation, rather than the costly, fully nonlinear optimization required for standard recurrent neural networks. Additionally, a universal approximation theorem was established in [ 8 ] demonstrating that reservoir computers can approximate causal, time-invariant functionals of stationary stochastic processes. Although RC offers the fast computation and lightweight design, it presents several challenges, as described in [ 44 ] . Next-generation reservoir computing (NG-RC) was introduced in [ 7 ] to address the lack of interpretability caused by the reservoir acting as a black box, the large amount of memory required by the randomly sampled matrices and the presence of numerous hyperparameters that need to be tuned-such as the spectral radius, input scaling and leaking rate. NG-RC formalizes the nonlinear vector autoregression (NVAR) framework. It was also shown in [ 3 ] that RC was mathematically comparable to NVAR when the activation function of the reservoir is set to be the identity function. A comparative study in [ 36 ] showed how NVAR outperforms long short-term memory (LSTM) networks, gated recurrent unit, and several echo state network architectures in the prediction of chaotic time series. In [ 7 ] , it was demonstrated that NVAR performs well on three challenging RC benchmark problems. However, to the best of our knowledge no studies have conducted experiments on noisy data. Standard NVAR uses fixed polynomial basis functions to incorporate nonlinearity, thereby limiting its adaptability in handling chaotic, high-dimensional or noisy data. Moreover, the linear readout matrix in both RC and NVAR depends on the nonlinear feature vector and is therefore susceptible to accumulating errors originating from this vector. This linear readout matrix calculation, which we introduce in Section 2.1 , also involves computing a matrix inverse, which can become a computational bottleneck when the reservoir—or feature vector, in the case of NVAR— is large. A kernel ridge regression approach for the NVAR with polynomial basis functions was introduced in [ 9 ] . That study also explored replacing the polynomial kernel by a Volterra kernel. However, kernel methods suffer from scalability issues when applied to large-scale data. We address the limitations of standard NVAR by replacing the handcrafted nonlinear feature vector with a learnable, shallow multilayer perceptron (MLP) that transforms delay-embedded inputs in a data-adaptive manner. The MLP is equipped with a dropout layer [ 37 ] , which helps the model generalize better under noisy conditions, thereby enhancing prediction accuracy. By using an MLP, we trade the interpretability offered by the NVAR technique for improved performance on noisy data in chaotic systems, an essential trade-off for forecasting real-life scenarios. A key innovation of our method is the joint optimization of both the nonlinear feature vector and the linear readout matrix through a gradient-based method, within a skip-connection architecture [ 42 ] , [ 13 ] . This end-to-end training enables the linear readout matrix and nonlinear features to be learned simultaneously, yielding an optimal joint configuration and decoupling the readout from any fixed feature representation. As a result, the learned features are explicitly tuned to the data, and the model’s adaptability to chaotic and noise-perturbed systems is enhanced, making the approach well suited for forecasting geophysical processes such as ocean salinity, sea surface temperature and climate indices. The work in [ 29 ] is closely related to our approach in the sense that it also considers skip connection layers and neural networks to replace the nonlinear feature vector of a standard NVAR. However, the architecture described in that work uses fixed weights, and the readout matrix is still computed through least squares regression, which limits their method to noise-free observations. This paper is organized as follows. In Section 2.1 , we provide preliminary concepts on RC, followed by Section 2.2 , where NVAR is introduced. Section 5.1 presents the proposed data-adaptive model. In Section 3 , we showcase the results of our experiments, starting with noise-free data from the Lorenz-63 chaotic system, present the performance of both standard and our adaptive NVAR, introduce varying levels of synthetic noise, and compare the forecasting performance of the two models. Finally, in Sections 4 and 5.2 , we discuss the experimental results and methods used. 2 Background 2.1 Reservoir Computing In this section, we briefly introduce traditional RC, extended to include leaky integrator neurons. Let X_{i}=[x_{1,i},x_{2,i},\dots,x_{d,i}]\in\mathbb{R}^{d} be a time series training data for time i\in\{1,\dots,T\} . This linear vector is introduced into the reservoir through the input layer with a fixed, randomly sampled matrix W_{\text{in}} . The dynamic reservoir system can then be represented by the following iterations \displaystyle r_{i+1} \displaystyle=(1-\alpha)r_{i}+\alpha f(Ar_{i}+W_{\text{in}}X_{i}+b), (1) \displaystyle\hat{X}_{i+1} \displaystyle=W_{\text{out}}r_{i+1}, where r_{i} is an m -dimensional vector whose dimension is generally taken to be of a much higher dimension m>d . The parameter \alpha\in[0,1] is the leaking (or decay) rate of the nodes, f is an activation function (commonly, f(x)=tanh(x) ), A

Read the complete text