A Random Matrix Perspective of Echo State Networks: From Precise Bias–Variance Characterization to Optimal Regularization

Yessin Moahker, Malik Tiomoko, Cosme Louart, Zhenyu Liao · 2025

Read A Random Matrix Perspective of Echo State Networks: From Precise Bias–Variance Characterization to Optimal Regularization by Yessin Moahker, Malik Tiomoko, Cosme Louart, Zhenyu Liao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22011.txt.

Description

Echo state networks are simple recurrent models whose inner weights stay fixed while only the output is trained. Yessin Moakher, Malik Tiomoko, Cosme Louart, and Zhenyu Liao analyze how well they learn in this 2025 paper. They use random matrix theory, the study of large random matrices, in a clean setup where a student network learns from a linear teacher with known weights.

The analysis gives closed-form formulas for the network's bias, variance, and total error in the large-size limit. Two findings depart from classic regression. These networks show no double descent, the odd second improvement seen in some models. They also err less when training data and teacher memory are both limited. The authors also give a formula for the best regularization strength in a simple case, and a numerical method for the general case, as practical tuning guidance.

Catalog record

CREATORYessin Moahker, Malik Tiomoko, Cosme Louart, Zhenyu Liao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-22011

Browse the full library · About the Tower of Babel