FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization

Qwen Pilot Team, Alibaba Group · 2026

Read FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization by Qwen Pilot Team, Alibaba Group (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19835.txt.

Description

Ma and co-authors present FIPO, a reinforcement learning method for deep reasoning in language models. Standard GRPO-style training spreads one outcome reward over every token. That coarse credit cannot tell key logic steps from filler tokens. FIPO adds discounted future-KL terms into the policy update. Tokens are weighted by their sway over what follows. On Qwen2.5-32B, chains of thought grew from about 4,000 to over 10,000 tokens. AIME 2024 accuracy rose from 50.0 to a peak of 58.0 percent. That beat DeepSeek-R1-Zero-Math-32B and roughly matched o1-mini.

Catalog record

CREATORQwen Pilot Team, Alibaba Group
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83018
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19835

Browse the full library · About the Tower of Babel