MDP Planning as Policy Inference

David Tolpin · 2026

Read MDP Planning as Policy Inference by David Tolpin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17375.txt.

Description

David Tolpin, writing alone, recasts episodic Markov decision process planning as Bayesian inference over policies. The main point is new: the policy itself becomes the latent variable, and expected return defines an unnormalized posterior density over policies. That preserves the standard return goal, unlike path-centric planning schemes with extra optimality variables and unlike entropy-regularized policy methods, which solve a different goal.

In the exact form, the spread over fixed policies induces an optimal stochastic policy under preference uncertainty. For discrete MDPs with random transitions, the paper studies variational sequential Monte Carlo as one approximate method, introducing policy consistency under state revisitation and coupled transition randomness across particles. Tests on grid worlds, Blackjack, Triangle Tireworld, and Academic Advising examine behavior of that approach against entropy methods, supporting that view of MDP planning.

Catalog record

CREATORDavid Tolpin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17375

Browse the full library · About the Tower of Babel