Policy Gradient Algorithms in Average-Reward Multichain MDPs
Jongmin Lee, Ernest K. Ryu · 2026
Read Policy Gradient Algorithms in Average-Reward Multichain MDPs by Jongmin Lee, Ernest K. Ryu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18003.txt.
Description
Jongmin Lee studies policy gradient methods for average-reward Markov decision processes, a setting far less developed than the discounted-reward case. Most prior results cover only ergodic or unichain processes, leaving the multichain case open. Lee builds on the fact that the recurrent-or-transient classification of states stays invariant.
From that invariant, he establishes a policy gradient theorem for average-reward multichain processes and develops refined analyses with convergence and sample-complexity results. In particular, an alpha-clipped policy mirror ascent algorithm reaches a near-optimal policy among positive policies. The work extends the theoretical footing for policy gradient methods beyond the ergodic and unichain settings into the broader multichain world.
Catalog record
| CREATOR | Jongmin Lee, Ernest K. Ryu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-18003