APRIL: Active Partial Rollouts in Reinforcement Learning to tame long-tail generation
———– Yuzhen Zhou, Jiajun Li*, Yusheng Su*, Gowtham Ramesh*, Zilin Zhu, Xiang Long, Chenyang Zhao, Jin Pan, Xiaodong Yu, Ze Wang, Kangrui Du, Jialian Wu, Ximeng Sun, Jiang Liu, Qiaolin Yu, Hao Chen, ———————————– Zicheng Liu, Emad Barsoum, ————————— University of California, Los Angeles (UCLA) ————— z · 2025
Read APRIL: Active Partial Rollouts in Reinforcement Learning to tame long-tail generation by ———– Yuzhen Zhou, Jiajun Li*, Yusheng Su*, Gowtham Ramesh*, Zilin Zhu, Xiang Long, Chenyang Zhao, Jin Pan, Xiaodong Yu, Ze Wang, Kangrui Du, Jialian Wu, Ximeng Sun, Jiang Liu, Qiaolin Yu, Hao Chen, ———————————– Zicheng Liu, Emad Barsoum, ————————— University of California, Los Angeles (UCLA) ————— z (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18521.txt.
Description
Training large language models with reinforcement learning spends most of its time generating sample answers. Yuzhen Zhou and 17 co-authors present APRIL, a system that cuts the waste in that step. Their 2025 paper names the core problem: a few very long answers stall whole batches, leaving graphics chips idle. Rollouts, as these samples are called, take over ninety percent of training time.
APRIL asks for more samples than needed and stops once the target count is reached. Partial answers are not thrown away; they are recycled and finished in later steps. That rule keeps every chip busy and wastes no work. Across three common training algorithms, rollout throughput rose by about a fifth on average. Final accuracy also edged up slightly. The system works across frameworks and chip brands. It is already built into an open training framework.
Catalog record
| CREATOR | ———– Yuzhen Zhou, Jiajun Li*, Yusheng Su*, Gowtham Ramesh*, Zilin Zhu, Xiang Long, Chenyang Zhao, Jin Pan, Xiaodong Yu, Ze Wang, Kangrui Du, Jialian Wu, Ximeng Sun, Jiang Liu, Qiaolin Yu, Hao Chen, ———————————– Zicheng Liu, Emad Barsoum, ————————— University of California, Los Angeles (UCLA) ————— z |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-18521