Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards

Bizhe Bai, Xinyue Wang, Peng Ye, Tao Chen · 2026

Read Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards by Bizhe Bai, Xinyue Wang, Peng Ye, Tao Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02555.txt.

Description

Bizhe Bai and 3 co-authors push language model exploration by shaking the model's parameters instead of its outputs, in a 2026 arXiv paper. Reinforcement learning with verifiable rewards often reweights solution traces the model already produces, hitting a ceiling at large sampling budgets. Action-space noise breaks the flow of long reasoning chains.

Their method perturbs policy parameters before each rollout, yielding trajectory-level exploration that stays steady within an attempt. Truncated importance sampling corrects the mismatch this creates between sampling and updates. A lightweight scheduler, driven by semantic diversity and normalized self-certainty, replaces costly divergence-based noise control. Built on a widely used group tuning method, the approach widened the effective reasoning frontier across math benchmarks and model families. Gains were largest at big sampling budgets, and the method stacks with other exploration methods.

Catalog record

CREATORBizhe Bai, Xinyue Wang, Peng Ye, Tao Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02555

Browse the full library · About the Tower of Babel