Optimizing Return Distributions with Distributional Dynamic Programming
Bernardo Ávila Pires [email protected], Mark Rowland, Diana Borsa, Zhaohan Daniel Guo, Khimya Khetarpal, André Barreto, David Abel, Rémi Munos, Will Dabney · 2025
Read Optimizing Return Distributions with Distributional Dynamic Programming by Bernardo Ávila Pires [email protected], Mark Rowland, Diana Borsa, Zhaohan Daniel Guo, Khimya Khetarpal, André Barreto, David Abel, Rémi Munos, Will Dabney (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13028.txt.
Description
A team of nine authors extends dynamic programming to whole return distributions in this 2025 paper. Reinforcement learning usually chases average reward, but many goals care about the spread of outcomes, such as avoiding rare disasters.
Their distributional methods optimize statistical measures of the return distribution, with standard learning as a special case. To go past what older methods could reach, they augment the state with a running statistic of rewards so far. The paper gives value and policy iteration versions, proves bounds, and maps which goals the methods can and cannot serve. Uses include maximizing a risk measure and keeping an agent in homeostatic balance.
Catalog record
| CREATOR | Bernardo Ávila Pires [email protected], Mark Rowland, Diana Borsa, Zhaohan Daniel Guo, Khimya Khetarpal, André Barreto, David Abel, Rémi Munos, Will Dabney |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 235 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-13028