Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification

Jiahui Li *, Jianfeng Shan *, Wenpei Chen, Shunyu Wu, Jian Lou, Wenjie Feng, Dan Li, See-Kiong Ng · 2026

Read Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification by Jiahui Li *, Jianfeng Shan *, Wenpei Chen, Shunyu Wu, Jian Lou, Wenjie Feng, Dan Li, See-Kiong Ng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03608.txt.

Description

Li and co-authors improve label-free reinforcement learning for reasoning models at test time. Test-time reinforcement learning trains without labels, using the model's own outputs. Past work chased Pass at 1, but Pass at k coverage matters too. Direct Pass at k designs from supervised settings perform poorly here. The team diagnoses two root causes of the failure.

Low-confidence samples get pseudo-labels that are often wrong. High-confidence samples collapse into narrow answer diversity. Their framework, TTRL-CoCoV, adapts verification by confidence. High-confidence samples get an exploration reward to keep diversity. Low-confidence samples let a verifier pick the pseudo-label. Medium-confidence samples skip verification entirely. Six benchmarks show gains over the best rivals. Pass at 1 rises by about 10 percent on average, and Pass at 16 by about 19 percent. Even fully supervised RL methods are beaten on some benchmarks.

Catalog record

CREATORJiahui Li *, Jianfeng Shan *, Wenpei Chen, Shunyu Wu, Jian Lou, Wenjie Feng, Dan Li, See-Kiong Ng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03608

Browse the full library · About the Tower of Babel