Moments Matter: Stabilizing Policy Optimization using Return Distributions
Dennis Jabs, Aditya Mohan, Marius Lindauer · 2026
Read Moments Matter: Stabilizing Policy Optimization using Return Distributions by Dennis Jabs, Aditya Mohan, Marius Lindauer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01803.txt.
Description
Reinforcement learning agents can reach the same average score with wildly different behavior, and small training updates sometimes wreck a walking robot's gait. Dennis Jabs, Aditya Mohan, and Marius Lindauer trace part of the problem to the spread of returns an update produces, and propose correcting policy learning using the shape of the return distribution.
Rather than estimating that spread directly, which is expensive, their method reads skewness and kurtosis from a distributional critic and biases the advantage in the PPO algorithm, penalizing extreme tail behavior. In the Walker2D task the correction improved stability by up to seventy-five percent while keeping evaluation returns comparable. The idea offers a cheaper route to steadier, more transferable control policies.
Catalog record
| CREATOR | Dennis Jabs, Aditya Mohan, Marius Lindauer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-01803