Multiscale replay: A robust algorithm for stochastic variational inequalities with a Markovian buffer
2026
Read Multiscale replay: A robust algorithm for stochastic variational inequalities with a Markovian buffer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01502.txt.
Description
Milind Nakul, Tianjiao Li, and Ashwin Pananjady speed up a class of algorithms that learn from a stream of dependent data. Many training methods assume each sample is fresh and independent. Reinforcement learning breaks that: samples arrive from a Markov chain, correlated in time, and reusing them from a memory buffer is standard practice. Uniform replay from the buffer still carries bias.
Their Multiscale Experience Replay samples the buffer at several scales at once, mimicking how the algorithms behave on independent data. Unlike skipping-based fixes, it needs no knowledge of how fast the chain mixes, and it speeds convergence whenever speed is available. The theory covers stochastic variational inequalities, a broad family that includes the target problems. Two applications stand out: policy evaluation with temporal difference learning, and fitting generalized linear models on dependent data. Robust acceleration without tuning knobs is a practical prize for reinforcement learning systems.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 128 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-01502