Optical Echo State Network Reservoir Computing

Ishwar S. Kaushik, Peter J. Ehlers, Daniel Soh · 2025

Read Optical Echo State Network Reservoir Computing by Ishwar S. Kaushik, Peter J. Ehlers, Daniel Soh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08224.txt.

Introduction

Ishwar S. Kaushik, Peter J. Ehlers, and Daniel Soh designed a computer made of light in 2025, built on a scheme called an echo state network. Reservoir computers like these let a fixed, tangled system do the hard processing while only a simple output layer is trained. Software versions are capable but spend energy on arithmetic that optics might do almost for free.

Their design performs the needed matrix multiplication with optical components and gets its nonlinear response from stimulated Brillouin scattering, a light-sound interaction in optical media. Because the nonlinearity happens inside the physics, no measurement step interrupts the computation. Simulations checked the system's memory capacity, its nonlinear processing strength, and its ability at polynomial algebra. Performance matched software echo state networks on the benchmark tasks, suggesting a feasible, scalable route to optical machine intelligence hardware.

Catalog record

CREATORIshwar S. Kaushik, Peter J. Ehlers, Daniel Soh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-08224

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Optical Echo State Network Reservoir Computing Ishwar S. Kaushik, Peter J. Ehlers, Daniel Soh Abstract We propose an innovative design for an optical Echo State Network (ESN), an advanced type of reservoir computer known for its universal computational capabilities. Our design enables an optical implementation of arbitrary ESNs, featuring flexibility in optical matrix multiplication and nonlinear activation. Leveraging the nonlinear characteristics of stimulated Brillouin scattering (SBS), the architecture efficiently realizes measurement-free nonlinear activation. The approach significantly reduces computational overhead and energy consumption compared to traditional software-based methods. Comprehensive simulations validate the system’s memory capacity, nonlinear processing strength, and polynomial algebra capabilities, showcasing performance comparable to software ESNs across key benchmark tasks. Our design establishes a feasible, scalable, and universally applicable framework for optical reservoir computing, suitable for diverse machine learning applications. I Introduction Recurrent neural networks (RNNs) have inherent memory capabilities, and are trained on sequential or time series data to create a machine learning model that attempts to make accurate time series predictions [ 1 ] . However, their reliance on multiple layers of tunable weights leads to high computational and training costs. A potential way to overcome this cost is with a recent paradigm of RNNs with fixed internal weights, known as reservoir computers (RCs). Reservoir computing has been of recent interest, with applications to time-series prediction tasks, classification tasks, and much more [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ] . Unlike RNNs, which require large amounts of time and resources, RCs offer a much less computationally expensive method for machine learning. An RC is a nonlinear dynamical system driven by time-dependent inputs and a single trainable output layer. Unlike RNNs or deep learning architectures, the inner layers of the computing reservoir are fixed and all of the training takes place in the final output layer. This output is usually a simple linear combination of the values of the state-vector of the reservoir, which allows for the weights to be determined by simple linear regression. This property makes reservoir computing an attractive choice for hardware designs; manipulating the weights in multiple layers for a typical RNN in hardware is exceedingly difficult in comparison to this simpler approach. Moreover, by adopting a lean, hardware-centric design in RCs, the overall hardware overhead is significantly reduced compared to software-based systems that depend on traditional computing layers—such as CPUs, long-term and short-term memories, and their interconnects. Operating on specialized hardware further increases speed by eliminating unnecessary data processing steps, thus improving overall efficiency. Optical platforms for computation and machine learning can, in principle, achieve more efficient computation in terms of energy and time [ 13 , 14 ] . One important challenge in optical approaches is how to realize the nonlinear activation. For this, various nonlinear optical processes have been studied including the nonlinear behavior of SBS [ 15 ] . It is important to note that, while the implementations of optical reservoir computers have been reported [ 16 ] , the realization of an optical RC with universal computation capability (i.e., universality) has not yet been demonstrated to our best knowledge. Universality refers to a computing machine’s ability to perform any computational task with any desired accuracy. In a supervised learning context, it specifically denotes the capacity to approximate any input-output relationship with arbitrary precision. Consequently, having a mathematically rigorous proof of universality is very desirable, as it ensures that the computing scheme can robustly handle a wide range of tasks. Arbitrary RCs are not guaranteed to obey the conditions necessary to be capable of universal computations. We present, for the first time to our best knowledge, a new approach representing a novel optical implementation of Echo State Networks (ESNs), which have recently been shown to form a universal approximating class of RCs [ 17 ] . ESNs are a family of RCs that are formed from iteratively applying a nonlinear transformation after a fixed linear transformation applied at each input time step [ 18 ] . ESNs are similar to RNNs except that the linear weights given to the state of the system at each time step are fixed for the entire run. The family of ESNs as a whole is universal [ 17 ] which means that for a given target sequence and input, we can find a specific ESN configuration that can approximate the target function to any desired accuracy. Here, we propose an innovative way to realize an optical implementation of any desired ESN configuration. The advantages provided by an optical reservoir architecture, as stated earlier, are fast computation and better energy efficiency than large software alternatives. For this reason, there has been growing interest in recent years in optical implementations of the matrix multiplication stage of machine learning architectures [ 19 , 20 , 21 ] . Here, we propose an in-fiber method for optical matrix-vector multiplication, which is followed by an in-fiber solution for optical nonlinear activation inspired by Slinkov et. al [ 22 ] using SBS. This unique combination of optical matrix multiplication and nonlinear activation allows for a feasible design of the optical ESN. The paper is organized as follows: Section II covers the basic concepts of ESNs and the theory behind the SBS linear amplifiers and activation function, Section III addresses our proposed design for the optical ESN, and Section IV compiles the performance of the ESN on various tasks that evaluate its universality, and compares it to a software ESN of the same size. The tunability of the nonlinearity of SBS interaction is also analyzed in this section. Finally, Section V contains the concluding remarks. II Optical Reservoir Computing Concepts II.1 Echo State Network An ESN is a form of reservoir computing, responding to time-series inputs reflected in its state vector after each time step. The state of the reservoir at each time step is given by x_{k+1}=g(x_{k},u_{k}). (1) Where the vector x_{k} represents the state of the reservoir at time step k . The number of elements of the state vector is equal to the number of nodes N of the reservoir computer. The function g(x_{k},u_{k}) is a fixed nonlinear function of the reservoir that provides the updated state of the reservoir dependent on the k th element of the input sequence u_{k} . The predicted output of the network is given by \hat{y}_{k}=W^{T}x_{k}+C, (2) and the cost function is S=\frac{1}{2N}\sum^{N-1}_{k=0}(y_{k}-\hat{y}_{k})^{2}. (3) The parameters W and C given in Equation ( 2 ) are chosen to minimize the cost function S given in Equation ( 3 ). Here, y_{k} denotes the target sequence the network output is fit to. So, the fitting of the output function to the target data is a linear regression problem. The reservoir dynamics are a nonlinear function of the inputs u_{k} . The readouts from the reservoir can be used to approximate the function y_{k} with linear regression. A reservoir computer is required to satisfy the uniform convergence property [ 17 ] . This property asserts that the reservoir state converges to a unique sequence after many time steps, which means that the initial state of the reservoir from the distant past has no effect on the current state.

Read the complete text