Ratio-Variance Regularized Policy Optimization
Yu Luo, Shuo Han, Yihan Hu, Lei Lv, Huaping Liu, Fuchun Sun, Jianye Hao, Dong Li · 2026
Read Ratio-Variance Regularized Policy Optimization by Yu Luo, Shuo Han, Yihan Hu, Lei Lv, Huaping Liu, Fuchun Sun, Jianye Hao, Dong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26784.txt.
Description
Luo and seven colleagues replace the hard clipping used in on-policy reinforcement learning with a softer constraint on policy ratio variance. Hard clipping truncates high-return, high-divergence updates wholesale. Their ratio-variance limit instead acts as a soft brake, keeping useful gradient signals and reusing stale off-policy data. Their R2VPO method, built on primal-dual optimization, beats PPO baselines on math reasoning and robotic control, with the largest gains on smaller models.
Catalog record
| CREATOR | Yu Luo, Shuo Han, Yihan Hu, Lei Lv, Huaping Liu, Fuchun Sun, Jianye Hao, Dong Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62414 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.26784