Policy Gradient Algorithms in Average-Reward Multichain MDPs

Jongmin Lee, Ernest K. Ryu · 2026

Read Policy Gradient Algorithms in Average-Reward Multichain MDPs by Jongmin Lee, Ernest K. Ryu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18003.txt.

Description

Jongmin Lee studies policy gradient methods for average-reward Markov decision processes, a setting far less developed than the discounted-reward case. Most prior results cover only ergodic or unichain processes, leaving the multichain case open. Lee builds on the fact that the recurrent-or-transient classification of states stays invariant.

From that invariant, he establishes a policy gradient theorem for average-reward multichain processes and develops refined analyses with convergence and sample-complexity results. In particular, an alpha-clipped policy mirror ascent algorithm reaches a near-optimal policy among positive policies. The work extends the theoretical footing for policy gradient methods beyond the ergodic and unichain settings into the broader multichain world.

Catalog record

CREATORJongmin Lee, Ernest K. Ryu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-18003

Browse the full library · About the Tower of Babel