Beyond Softmax and Entropy: Improving Convergence Guarantees of Policy Gradients by ๐ \boldsymbol{f} -SoftArgmax Parameterization with Coupled Regularization
Safwan Labbi, Daniil Tiapkin, Paul Mangold, Eric Moulines ยท 2026
Read Beyond Softmax and Entropy: Improving Convergence Guarantees of Policy Gradients by ๐ \boldsymbol{f} -SoftArgmax Parameterization with Coupled Regularization by Safwan Labbi, Daniil Tiapkin, Paul Mangold, Eric Moulines (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12604.txt.
Description
Safwan Labbi, Daniil Tiapkin, Paul Mangold, and Eric Moulines work on policy gradient methods in reinforcement learning. These methods tune a policy, a rule for picking actions, by following a gradient. They are known to be sensitive to how the policy is parameterized. The standard softmax choice can make the optimization landscape ill conditioned.
That bad shape can slow convergence to an exponential crawl. Preconditioning can fix the shape, but it often costs too much compute. The team propose a new parameterization based on an f SoftArgmax form, with coupled regularization. Their title claims better convergence guarantees from this design. Stronger guarantees help users pick step sizes and trust training runs. The work targets a core pain point in modern reinforcement learning.
Catalog record
| CREATOR | Safwan Labbi, Daniil Tiapkin, Paul Mangold, Eric Moulines |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 270 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12604