FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization
Qwen Pilot Team, Alibaba Group · 2026
Read FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization by Qwen Pilot Team, Alibaba Group (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19835.txt.
Description
Ma and co-authors present FIPO, a reinforcement learning method for deep reasoning in language models. Standard GRPO-style training spreads one outcome reward over every token. That coarse credit cannot tell key logic steps from filler tokens. FIPO adds discounted future-KL terms into the policy update. Tokens are weighted by their sway over what follows. On Qwen2.5-32B, chains of thought grew from about 4,000 to over 10,000 tokens. AIME 2024 accuracy rose from 50.0 to a peak of 58.0 percent. That beat DeepSeek-R1-Zero-Math-32B and roughly matched o1-mini.
Catalog record
| CREATOR | Qwen Pilot Team, Alibaba Group |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83018 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19835