MDP modeling for multi-stage stochastic programs
David P. Morton, Oscar Dowson, Bernardo K. Pagnoncelli · 2026
Read MDP modeling for multi-stage stochastic programs by David P. Morton, Oscar Dowson, Bernardo K. Pagnoncelli (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13473183.txt.
Description
Morton and co-authors merge two traditions for deciding under uncertainty, multi-stage stochastic programming and Markov decision processes, into one modeling language built on policy graphs. Their 2026 paper extends those graphs so that today's decisions can change tomorrow's probabilities and can even buy information, then develops matching variants of stochastic dual dynamic programming to solve the models.
Worked examples climb in complexity to the classic tiger problem, where an agent listens at two doors. A muffled roar carries a 15% false-positive rate in their parameterization, with a discount factor of 0.95 applied to future rewards. Across 100 simulations of the listening policy, only one run ended with the agent wrongly opening the tiger's door after three straight false alarms. The machinery also handles continuous states and actions, which classic decision-process tools struggle to scale.
Catalog record
| CREATOR | David P. Morton, Oscar Dowson, Bernardo K. Pagnoncelli |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 204 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |