Multivariate Distributional Reinforcement Learning Using Sliced Divergences
Baptiste Debes, Tinne Tuytelaars · 2026
Read Multivariate Distributional Reinforcement Learning Using Sliced Divergences by Baptiste Debes, Tinne Tuytelaars (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31222.txt.
Introduction
Debes and Tuytelaars extend distributional reinforcement learning to multivariate returns. It models the full return spread, not just averages. Most metrics fail beyond one dimension, and general matrix discounting lacks contraction results. Their sliced method lifts one-dimensional gaps to multi-dimensional returns via projections. They prove Bellman contraction for uniform slicing under scalar discounting, and a maximum-slicing variant for dense discount matrices. They analyse Wasserstein, Cramér and Maximum Mean Discrepancy gaps for single-sample Bellman updates. Tests cover a toy chain, a gridworld environment and Atari games.
Catalog record
| CREATOR | Baptiste Debes, Tinne Tuytelaars |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 216198 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.31222