Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards
Xiaoyuan Liu ,1,2, Tian Liang 1, Zhiwei He 1, Jiahao Xu 1, Wenxuan Wang 1, Pinjia He †,2, Zhaopeng Tu ,1, Haitao Mi 1, Dong Yu 1 · 2025
Read Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards by Xiaoyuan Liu ,1,2, Tian Liang 1, Zhiwei He 1, Jiahao Xu 1, Wenxuan Wang 1, Pinjia He †,2, Zhaopeng Tu ,1, Haitao Mi 1, Dong Yu 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13445.txt.
Description
Xiaoyuan Liu and eight co-authors introduce RISE, a reinforcement learning framework that trains language models to solve and to verify at the same time. Models trained with verifiable rewards often show superficial self reflection and fail to check their own outputs robustly. RISE targets that weakness directly inside a single online training loop.
Each round, the model generates solutions and then critiques its own fresh solutions. Verifiable rewards give on the fly feedback for both tasks, and both kinds of trajectories update the policy. Tests on math reasoning benchmarks show steady gains in solving accuracy plus strong self verification skill. More verification compute helps, and trained models verify more often and more accurately during reasoning.
Catalog record
| CREATOR | Xiaoyuan Liu ,1,2, Tian Liang 1, Zhiwei He 1, Jiahao Xu 1, Wenxuan Wang 1, Pinjia He †,2, Zhaopeng Tu ,1, Haitao Mi 1, Dong Yu 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13445