Length-Unbiased Sequence Policy Optimization: Revealing and Controlling Response Length Variation in RLVR

Fanfan Liu, Youyang Yin, Peng Shi, Siqi Yang, Zhixiong Zeng, Haibo Qiu · 2026

Read Length-Unbiased Sequence Policy Optimization: Revealing and Controlling Response Length Variation in RLVR by Fanfan Liu, Youyang Yin, Peng Shi, Siqi Yang, Zhixiong Zeng, Haibo Qiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05261.txt.

Description

Reinforcement learning with verifiable rewards makes models reason longer. Fanfan Liu and 5 co-authors show in this 2026 paper that response length is partly an artifact of the algorithm, not purely a sign of deeper thought. Varied training methods inflate or collapse lengths in characteristic ways during training.

The authors dissect the parts of mainstream algorithms and derive which factors push length around, confirming the theory with extensive tests. Building on that study, they propose LUSPO, a length-unbiased sequence policy optimization method. It repairs a bias inside a recent group sequence method, making the loss truly neutral about response length and fixing a collapse failure. On mathematical reasoning benchmarks and multimodal reasoning tasks, LUSPO consistently beat established methods such as GRPO and GSPO. The paper reframes length from a mysterious emergent trait into a knob the algorithm designer controls.

Catalog record

CREATORFanfan Liu, Youyang Yin, Peng Shi, Siqi Yang, Zhixiong Zeng, Haibo Qiu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05261

Browse the full library · About the Tower of Babel