P 2 O: Joint Policy and Prompt Optimization

Xinyu Lu, Kaiqi Zhang, Jinglin Yang, Boxi Cao, Yaojie Lu, Hongyu Lin, Min He, Xianpei Han, Le Sun · 2026

Read P 2 O: Joint Policy and Prompt Optimization by Xinyu Lu, Kaiqi Zhang, Jinglin Yang, Boxi Cao, Yaojie Lu, Hongyu Lin, Min He, Xianpei Han, Le Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21877.txt.

Description

Lu and co-authors improve reinforcement learning for reasoning models by tuning prompts alongside the policy. When every rollout of a hard question scores the same, the learning signal collapses, worst on hard items. Their P2O method mines those hard samples and evolves reasoning prompts for them with GEPA. Good trajectories are then distilled into the model under the plain question, so no prompt is needed at test time. A mask filters extreme ratios during distillation. Gains reached 8 points over GRPO on average, and 16 on hard contest sets.

Catalog record

CREATORXinyu Lu, Kaiqi Zhang, Jinglin Yang, Boxi Cao, Yaojie Lu, Hongyu Lin, Min He, Xianpei Han, Le Sun
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40822
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21877

Browse the full library · About the Tower of Babel