Dynamic Programming for Epistemic Uncertainty in Markov Decision Processes

Axel Benyamine, Julien Grand-Clément, Marek Petrik, Michael I. Jordan, Alain Durmus · 2026

Read Dynamic Programming for Epistemic Uncertainty in Markov Decision Processes by Axel Benyamine, Julien Grand-Clément, Marek Petrik, Michael I. Jordan, Alain Durmus (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03381.txt.

Description

Axel Benyamine and 4 co-authors build a general theory for decision problems where the transition chances themselves are uncertain. Their 2026 paper treats those chances as random variables and scores a policy by applying a risk measure to the return it randomly produces. The aim is a single setup that covers many existing models of Markov decision processes under epistemic uncertainty.

Within it, the authors extend value functions and Bellman operators, the standard tools for computing good policies. They show when dynamic programming still works, covering stationary policies plus value and policy iteration methods. They also fully pin down which law-invariant risk measures remain in line with dynamic programming, marking the boundary where a planner must leave that approach behind.

Catalog record

CREATORAxel Benyamine, Julien Grand-Clément, Marek Petrik, Michael I. Jordan, Alain Durmus
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE160 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03381

Browse the full library · About the Tower of Babel