MDP Planning as Policy Inference
David Tolpin · 2026
Read MDP Planning as Policy Inference by David Tolpin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17375.txt.
Description
David Tolpin, writing alone, recasts episodic Markov decision process planning as Bayesian inference over policies. The main point is new: the policy itself becomes the latent variable, and expected return defines an unnormalized posterior density over policies. That preserves the standard return goal, unlike path-centric planning schemes with extra optimality variables and unlike entropy-regularized policy methods, which solve a different goal.
In the exact form, the spread over fixed policies induces an optimal stochastic policy under preference uncertainty. For discrete MDPs with random transitions, the paper studies variational sequential Monte Carlo as one approximate method, introducing policy consistency under state revisitation and coupled transition randomness across particles. Tests on grid worlds, Blackjack, Triangle Tireworld, and Academic Advising examine behavior of that approach against entropy methods, supporting that view of MDP planning.
Catalog record
| CREATOR | David Tolpin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-17375