Periodic Regularized Q-Learning
Hyukjun Yang, Han-Dong Lim, Donghwan Lee · 2026
Read Periodic Regularized Q-Learning by Hyukjun Yang, Han-Dong Lim, Donghwan Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03301.txt.
Description
Q-learning that stays stable under function approximation is pursued by Hyukjun Yang, with 2 co-authors, through periodic regularization. In tabular settings Q-learning is guaranteed to converge, but that guarantee fails with linear function fitting, where updates can diverge. Such techniques have been introduced to restore stability, and this work regularizes at the level of the projection operator itself in the value iteration behind the method.
The authors construct a regularized projected value iteration and show it becomes a contraction with apt regularization. Extending that projection into the stochastic, sample-based setting yields their method, Periodic Regularized Q-Learning. A rigorous theory-based analysis gives finite-time settling guarantees under linear function fitting. The contribution is mainly theoretical, giving practitioners a provably stable variant.
Catalog record
| CREATOR | Hyukjun Yang, Han-Dong Lim, Donghwan Lee |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 153 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03301