Reservoir computing with large valid prediction time for the Lorenz system

L A Hurley, S E Shaheen · 2025

Read Reservoir computing with large valid prediction time for the Lorenz system by L A Hurley, S E Shaheen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06730.txt.

Introduction

Lauren Hurley and Sean Shaheen study, in a 2025 paper, how long a reservoir computer can predict a chaotic system before its forecast breaks down. The test bed is the Lorenz system, a classic model of chaos. How far ahead the forecast stays valid depends on the machine's settings, and the pair map that dependence with care.

With well-chosen settings, their machine reached about seventy percent of a strong benchmark score. Its valid forecasts ran past thirty Lyapunov times, helped by the fact that the data are noiseless. They show the early-step error can predict the valid span, which gives a cheap way to grade settings. They also warn that solver-made data carry their own trust limit, and they find two size regimes that forecast far.

Catalog record

CREATORL A Hurley, S E Shaheen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-06730

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reservoir computing with large valid prediction time for the Lorenz system L A Hurley, S E Shaheen Abstract We study the dependence of the Valid Prediction Time (VPT) of Reservoir Computers (RCs) on hyperparameters including the regularization coefficient, reservoir size, and spectral radius. Under carefully chosen conditions, the RC can achieve approximately 70% of a benchmark performance, based on the output of a single prediction step used as initial conditions for the Lorenz equations. We report high VPT values ( > > 30 Lyapunov times), as we are predicting a noiseless system where overfitting can be beneficial. While these conditions may not hold for noisy systems, they could still be useful for real-world applications with limited noise. Furthermore, utilizing knowledge of the Lyapunov exponent, we find that the VPT can be predicted by the error in the first few prediction steps, offering a computationally efficient evaluation method. We emphasize the importance of the numerical solver used to generate the Lorenz dataset and define a Valid Ground Truth Time (VGTT), during which the outputs of several common solvers agree. A VPT exceeding the VGTT is not meaningful, as a different solver could produce a different result. Lastly, we identify two spectral radius regimes that achieve large VPT: a small radius near zero, resulting in simple but stable operation, and a larger radius operating at the “edge of chaos.” August 2025 Keywords : echo state network (ESN), reservoir computer, ridge regression, spectral radius, valid prediction time 1 Introduction Reservoir Computing (RC) is a type of recurrent neural network consisting of fixed input layer weights, one hidden layer (the reservoir) with fixed weights, and an output layer whose weights are trained. One advantage of RC over other neural networks is that only the output layer weights are trained, resulting in fast and efficient training [ 1 , 2 , 3 ] . Although the output layer weights are the only parameter, there is a host of hyperparameters which must be carefully selected by the user. This includes the choice of activation function, input weight scaling, leakage rate, regularization parameter, reservoir size, sparsity, spectral radius, and weight distribution, among others [ 4 , 5 ] . The literature offers some guidance on choosing hyperparameters, such as the following publications: [ 6 , 7 , 8 , 9 ] . Still, there can be exceptions to these general guidelines depending on the task, and in those cases the choice of hyperparameters is often determined by trial and error. As this can be a time-consuming process, it may be neglected , and the operating conditions of the RC would not be fully optimized. Here, we investigate two hyperparameters in particular, the Ridge Regression coefficient \lambda and the spectral radius, to optimize the Valid Prediction Time (VPT). A comprehensive search through radius and reservoir size at low \lambda allows us to achieve higher VPTs than have been previously reported. We chose to investigate these two hyperparameters since they offer direct control over fundamental properties of the RC, such as dynamics, memory, and generalization [ 6 , 7 , 10 ] . We find that the optimal \lambda to optimize VPT is very small ( <10^{-20} ), which is nearly identical to a simple least squares regression. This trend agrees with existing literature, where it is generally considered that \lambda is necessary for noisy systems but not for noiseless ones [ 11 ] . Moreover, preliminary work by dos Santos has noted that regularization may not be necessary depending on how the least squares problem is solved [ 12 ] . They recommend that a good solver should not be sensitive to perturbations, have high numerical accuracy, be stable, and be robust to ill-conditioning. Yet, to our knowledge, no publications on RC predicting the Lorenz system have used this \lambda dependence of VPT to their advantage, only reporting VPTs of around 15 [ 7 , 13 ] , while we report VPTs exceeding 30 Lyapunov times. Moreover, we find two regimes of spectral radius that lead to high VPT: one peak at small radii ( 10^{-4} ) that does not depend on N, and another at higher radius which does depend on N, which we refer to as the edge of chaos (this may also be referred to as the edge of stability, see [ 14 ] ). The peak at the edge of chaos is slightly higher, but requires more precision in choice of N to attain, while the peak at low radius is an option that works for any N. Although, we note that the values slightly off-peak are still well above what has been reported in the literature. A spectral radius of zero means that the RC has no memory of past inputs and is an extreme learning machine [ 15 ] . As the spectral radius is increased, the RC retains a memory of increasingly distant past inputs, but this comes at the cost of decreasing stability. Our findings agree with the literature, such as work by Jaurigue which found that that the highest VPT for a small reservoir (20 nodes) occurred with a radius close to zero [ 16 ] , and another by Platt et al. which found the best VPT for a large reservoir (2000 nodes) [ 7 ] . Still, these are just individual cases, while the more thorough study of spectral radius and reservoir size presented here gives insight into previously hidden trends and regimes of performance. Lastly, we emphasize the importance of careful choice of numerical solver methods. We find that the performance of the RC depends on the solver and tolerances used to generate the dataset. While there is some discussion of this in the literature (e.g. [ 17 ] notes that the sampling time of the solver used to generate the Lorenz system impacts the difficulty of the prediction task), we find that this tends to be overlooked, and we encourage the community to be specific in reporting the numerical methods used to generate chaotic time series datasets for RC and other neural network prediction tasks. 2 Methods 2.1 Time Series Prediction We simulate an echo state network (ESN) reservoir computer to predict a chaotic time series by training it on the history of that system’s coordinates {\bf z} sampled every \tau time units during a training time T , \{{\bf z}(n\tau)\}_{n=1}^{T/\tau} . The system drives the internal states of the reservoir, {\bf r} , by using a fixed input matrix W_{\text{in}} as \displaystyle{\bf r}((n+1)\tau)=f(A{\bf r}(n\tau)+W_{\text{in}}{\bf z}(n\tau)), (1) where A is the reservoir connectivity matrix with a spectral radius \rho and f(\cdot) is the activation function. The reservoir output is given by \hat{\bf z}((n+1)\tau)=W_{\text{out}}{\bf r}((n+1)\tau) , where the output matrix is chosen so that the output predicts as closely as possible the future state of the reservoir, i.e., \textbf{z}((n+1)\tau)\approx\hat{\textbf{z}}((n+1)\tau) . Here, we use a single least-squares regression that minimizes the cost function \displaystyle\sum_{n=1}^{T/\tau}\|\hat{\bf z}(n\tau)-{\bf z}(n\tau)\|^{2}+\lambda\text{Tr}(W_{\text{out}}W_{\text{out}}^{T}), (2) where \lambda is a regularization term to prevent overfitting. The Lorenz System, a system of three coupled differential equations, is a common benchmark for chaotic time series prediction and is given by [ 18 ] \begin{array}[]{l}\frac{dx}{dt}=\sigma(y-x),\\ \frac{dy}{dt}=x(\rho-z)-y,\\ \frac{dz}{dt}=xy-\beta z,\end{array} (3) with \sigma=10 , \rho=28 , and \beta=8/3 (a.u.). For prediction,

Read the complete text