Optimizing Return Distributions with Distributional Dynamic Programming

Bernardo Ávila Pires [email protected], Mark Rowland, Diana Borsa, Zhaohan Daniel Guo, Khimya Khetarpal, André Barreto, David Abel, Rémi Munos, Will Dabney · 2025

Read Optimizing Return Distributions with Distributional Dynamic Programming by Bernardo Ávila Pires [email protected], Mark Rowland, Diana Borsa, Zhaohan Daniel Guo, Khimya Khetarpal, André Barreto, David Abel, Rémi Munos, Will Dabney (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13028.txt.

Description

A team of nine authors extends dynamic programming to whole return distributions in this 2025 paper. Reinforcement learning usually chases average reward, but many goals care about the spread of outcomes, such as avoiding rare disasters.

Their distributional methods optimize statistical measures of the return distribution, with standard learning as a special case. To go past what older methods could reach, they augment the state with a running statistic of rewards so far. The paper gives value and policy iteration versions, proves bounds, and maps which goals the methods can and cannot serve. Uses include maximizing a risk measure and keeping an agent in homeostatic balance.

Catalog record

CREATORBernardo Ávila Pires [email protected], Mark Rowland, Diana Borsa, Zhaohan Daniel Guo, Khimya Khetarpal, André Barreto, David Abel, Rémi Munos, Will Dabney
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE235 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13028

Browse the full library · About the Tower of Babel