Where Rollouts Begin: Low-Load, High-Leverage First-Token Diversification for RLVR

Soeun Kim, Albert No · 2026

Read Where Rollouts Begin: Low-Load, High-Leverage First-Token Diversification for RLVR by Soeun Kim, Albert No (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28295.txt.

Description

Soeun Kim and Albert No diversify reasoning-model training by varying the very first token of each rollout. Since first-token probabilities concentrate sharply yet relate weakly to final correctness, sampling first tokens uniformly from the top candidates broadens explored paths cheaply. Their REFT method improved pass rates across eight models and beat rival diversification schemes at lowest rollout cost.

Catalog record

CREATORSoeun Kim, Albert No
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58014
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28295

Browse the full library · About the Tower of Babel