Multivariate Distributional Reinforcement Learning Using Sliced Divergences

Baptiste Debes, Tinne Tuytelaars · 2026

Read Multivariate Distributional Reinforcement Learning Using Sliced Divergences by Baptiste Debes, Tinne Tuytelaars (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31222.txt.

Introduction

Debes and Tuytelaars extend distributional reinforcement learning to multivariate returns. It models the full return spread, not just averages. Most metrics fail beyond one dimension, and general matrix discounting lacks contraction results. Their sliced method lifts one-dimensional gaps to multi-dimensional returns via projections. They prove Bellman contraction for uniform slicing under scalar discounting, and a maximum-slicing variant for dense discount matrices. They analyse Wasserstein, Cramér and Maximum Mean Discrepancy gaps for single-sample Bellman updates. Tests cover a toy chain, a gridworld environment and Atari games.

Catalog record

CREATORBaptiste Debes, Tinne Tuytelaars
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE216198
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.31222

Browse the full library · About the Tower of Babel