Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime

Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen · 2026

Read Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime by Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05112.txt.

Description

Tianshu Zhu and ten co-authors steer reinforcement learning for software engineering agents toward training groups whose pass rate sits near one half. Groups of attempts earn binary rewards, and when almost all pass or fail, the learning signal is weak and skewed. The authors show the reward-side signal is strongest near an even pass rate under several criteria.

Their Prefix Sampling method replays pieces of the model's own earlier trajectories to rebalance skewed groups: good prefixes help mostly-failing groups, and bad prefixes handicap mostly-passing ones. Replayed tokens are masked from the loss, so learning applies only to fresh continuations. On a verified software benchmark the method matched baseline peak scores with large time savings.

Catalog record

CREATORTianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.05112

Browse the full library · About the Tower of Babel