Moments Matter: Stabilizing Policy Optimization using Return Distributions

Dennis Jabs, Aditya Mohan, Marius Lindauer · 2026

Read Moments Matter: Stabilizing Policy Optimization using Return Distributions by Dennis Jabs, Aditya Mohan, Marius Lindauer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01803.txt.

Description

Reinforcement learning agents can reach the same average score with wildly different behavior, and small training updates sometimes wreck a walking robot's gait. Dennis Jabs, Aditya Mohan, and Marius Lindauer trace part of the problem to the spread of returns an update produces, and propose correcting policy learning using the shape of the return distribution.

Rather than estimating that spread directly, which is expensive, their method reads skewness and kurtosis from a distributional critic and biases the advantage in the PPO algorithm, penalizing extreme tail behavior. In the Walker2D task the correction improved stability by up to seventy-five percent while keeping evaluation returns comparable. The idea offers a cheaper route to steadier, more transferable control policies.

Catalog record

CREATORDennis Jabs, Aditya Mohan, Marius Lindauer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-01803

Browse the full library · About the Tower of Babel