Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime
Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen · 2026
Read Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime by Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05112.txt.
Description
Tianshu Zhu and ten co-authors steer reinforcement learning for software engineering agents toward training groups whose pass rate sits near one half. Groups of attempts earn binary rewards, and when almost all pass or fail, the learning signal is weak and skewed. The authors show the reward-side signal is strongest near an even pass rate under several criteria.
Their Prefix Sampling method replays pieces of the model's own earlier trajectories to rebalance skewed groups: good prefixes help mostly-failing groups, and bad prefixes handicap mostly-passing ones. Replayed tokens are masked from the loss, so learning applies only to fresh continuations. On a verified software benchmark the method matched baseline peak scores with large time savings.
Catalog record
| CREATOR | Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo, Lun Tian, Haotian Zhao, Yucheng Zeng, Jingnan Gu, Daxiang Dong, Jianmin Wu, Dawei Yin, Dou Shen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.05112