A Random Matrix Perspective of Echo State Networks: From Precise Bias–Variance Characterization to Optimal Regularization
Yessin Moahker, Malik Tiomoko, Cosme Louart, Zhenyu Liao · 2025
Read A Random Matrix Perspective of Echo State Networks: From Precise Bias–Variance Characterization to Optimal Regularization by Yessin Moahker, Malik Tiomoko, Cosme Louart, Zhenyu Liao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22011.txt.
Description
Echo state networks are simple recurrent models whose inner weights stay fixed while only the output is trained. Yessin Moakher, Malik Tiomoko, Cosme Louart, and Zhenyu Liao analyze how well they learn in this 2025 paper. They use random matrix theory, the study of large random matrices, in a clean setup where a student network learns from a linear teacher with known weights.
The analysis gives closed-form formulas for the network's bias, variance, and total error in the large-size limit. Two findings depart from classic regression. These networks show no double descent, the odd second improvement seen in some models. They also err less when training data and teacher memory are both limited. The authors also give a formula for the best regularization strength in a simple case, and a numerical method for the general case, as practical tuning guidance.
Catalog record
| CREATOR | Yessin Moahker, Malik Tiomoko, Cosme Louart, Zhenyu Liao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22011