Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards
Bizhe Bai, Xinyue Wang, Peng Ye, Tao Chen · 2026
Read Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards by Bizhe Bai, Xinyue Wang, Peng Ye, Tao Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02555.txt.
Description
Bizhe Bai and 3 co-authors push language model exploration by shaking the model's parameters instead of its outputs, in a 2026 arXiv paper. Reinforcement learning with verifiable rewards often reweights solution traces the model already produces, hitting a ceiling at large sampling budgets. Action-space noise breaks the flow of long reasoning chains.
Their method perturbs policy parameters before each rollout, yielding trajectory-level exploration that stays steady within an attempt. Truncated importance sampling corrects the mismatch this creates between sampling and updates. A lightweight scheduler, driven by semantic diversity and normalized self-certainty, replaces costly divergence-based noise control. Built on a widely used group tuning method, the approach widened the effective reasoning frontier across math benchmarks and model families. Gains were largest at big sampling budgets, and the method stacks with other exploration methods.
Catalog record
| CREATOR | Bizhe Bai, Xinyue Wang, Peng Ye, Tao Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02555