Beyond Softmax and Entropy: Improving Convergence Guarantees of Policy Gradients by ๐’‡ \boldsymbol{f} -SoftArgmax Parameterization with Coupled Regularization

Safwan Labbi, Daniil Tiapkin, Paul Mangold, Eric Moulines ยท 2026

Read Beyond Softmax and Entropy: Improving Convergence Guarantees of Policy Gradients by ๐’‡ \boldsymbol{f} -SoftArgmax Parameterization with Coupled Regularization by Safwan Labbi, Daniil Tiapkin, Paul Mangold, Eric Moulines (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12604.txt.

Description

Safwan Labbi, Daniil Tiapkin, Paul Mangold, and Eric Moulines work on policy gradient methods in reinforcement learning. These methods tune a policy, a rule for picking actions, by following a gradient. They are known to be sensitive to how the policy is parameterized. The standard softmax choice can make the optimization landscape ill conditioned.

That bad shape can slow convergence to an exponential crawl. Preconditioning can fix the shape, but it often costs too much compute. The team propose a new parameterization based on an f SoftArgmax form, with coupled regularization. Their title claims better convergence guarantees from this design. Stronger guarantees help users pick step sizes and trust training runs. The work targets a core pain point in modern reinforcement learning.

Catalog record

CREATORSafwan Labbi, Daniil Tiapkin, Paul Mangold, Eric Moulines
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE270 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12604

Browse the full library ยท About the Tower of Babel