Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework

Phalguni Nanda, Zaiwei Chen Edwardson School of Industrial Engineering, Purdue University · 2026

Read Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework by Phalguni Nanda, Zaiwei Chen Edwardson School of Industrial Engineering, Purdue University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10671.txt.

Description

Phalguni Nanda and Zaiwei Chen show that natural policy gradient, a core reinforcement-learning method, is exactly a smoothed and averaged form of policy iteration. Their doubly smoothed policy iteration framework builds each policy from a regularized greedy step over a weighted average of past value functions. Using only monotonicity and contraction of smoothed Bellman operators, they prove distribution-free global geometric convergence, and extend the framework to linear function approximation and stochastic shortest paths.

Catalog record

CREATORPhalguni Nanda, Zaiwei Chen Edwardson School of Industrial Engineering, Purdue University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122705
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10671

Browse the full library · About the Tower of Babel