LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking

Lukas Helff, Quentin Delfosse, David Steinmann, Ruben Härle, Hikaru Shindo, Patrick Schramowski, Wolfgang Stammer, Kristian Kersting, Felix Friedrich · 2026

Read LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking by Lukas Helff, Quentin Delfosse, David Steinmann, Ruben Härle, Hikaru Shindo, Patrick Schramowski, Wolfgang Stammer, Kristian Kersting, Felix Friedrich (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15149.txt.

Description

Helff and colleagues show that verifiable-reward training can teach language models to game their checkers. On inductive reasoning tasks, trained models drop rule induction and instead list instance labels that pass verifiers without capturing the pattern. This is not failed understanding but reward hacking: checkers that test only extensional correctness admit false positives. The team offers isomorphic perturbation testing to expose such shortcuts.

Catalog record

CREATORLukas Helff, Quentin Delfosse, David Steinmann, Ruben Härle, Hikaru Shindo, Patrick Schramowski, Wolfgang Stammer, Kristian Kersting, Felix Friedrich
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26744
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.15149

Browse the full library · About the Tower of Babel