Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards

Xiaoyuan Liu ,1,2, Tian Liang 1, Zhiwei He 1, Jiahao Xu 1, Wenxuan Wang 1, Pinjia He †,2, Zhaopeng Tu ,1, Haitao Mi 1, Dong Yu 1 · 2025

Read Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards by Xiaoyuan Liu ,1,2, Tian Liang 1, Zhiwei He 1, Jiahao Xu 1, Wenxuan Wang 1, Pinjia He †,2, Zhaopeng Tu ,1, Haitao Mi 1, Dong Yu 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13445.txt.

Description

Xiaoyuan Liu and eight co-authors introduce RISE, a reinforcement learning framework that trains language models to solve and to verify at the same time. Models trained with verifiable rewards often show superficial self reflection and fail to check their own outputs robustly. RISE targets that weakness directly inside a single online training loop.

Each round, the model generates solutions and then critiques its own fresh solutions. Verifiable rewards give on the fly feedback for both tasks, and both kinds of trajectories update the policy. Tests on math reasoning benchmarks show steady gains in solving accuracy plus strong self verification skill. More verification compute helps, and trained models verify more often and more accurately during reasoning.

Catalog record

CREATORXiaoyuan Liu ,1,2, Tian Liang 1, Zhiwei He 1, Jiahao Xu 1, Wenxuan Wang 1, Pinjia He †,2, Zhaopeng Tu ,1, Haitao Mi 1, Dong Yu 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-13445

Browse the full library · About the Tower of Babel