rePIRL: Learn PRM with Inverse RL for LLM Reasoning

Xian Wu, Kaijie Zhu, Ying Zhang, Lun Wang, Wenbo Guo · 2026

Read rePIRL: Learn PRM with Inverse RL for LLM Reasoning by Xian Wu, Kaijie Zhu, Ying Zhang, Lun Wang, Wenbo Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07832.txt.

Description

Process reward models score each step of a model's reasoning, improving training, but learning them well usually demands strong assumptions about an expert policy. Xian Wu and four co-authors propose rePIRL, an inverse-reinforcement-learning framework that learns process rewards with minimal assumptions about experts.

A dual process alternates between updating the policy and updating the reward model, with techniques adapted to scale inverse reinforcement learning to large language models. Theory shows the framework unifies online and offline process-reward learning, and experiments support its effectiveness. Researchers training reasoning models gain a more general, less brittle route to step-by-step supervision.

Catalog record

CREATORXian Wu, Kaijie Zhu, Ying Zhang, Lun Wang, Wenbo Guo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07832

Browse the full library · About the Tower of Babel