Likelihood-Based Reward Designs for General LLM Reasoning

Ariel Kwiatkowski, Natasha Butt, Ismail Labiad, Julia Kempe, Yann Ollivier · 2026

Read Likelihood-Based Reward Designs for General LLM Reasoning by Ariel Kwiatkowski, Natasha Butt, Ismail Labiad, Julia Kempe, Yann Ollivier (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03979.txt.

Description

Ariel Kwiatkowski and 4 co-authors simplify how reasoning models earn rewards. Standard reinforcement learning needs a separate checker for each benchmark, and its yes-or-no rewards give sparse feedback. Their 2026 paper instead rewards the model by how probable it finds the reference answer.

Several variants of likelihood-based reward were compared against standard baselines. Only one, the log-probability of the reference answer, performed well everywhere. It worked on math benchmarks and on long-form answers where no external checker exists. Such rewards need no verifier engineering and exist wherever training data does, making them cheap to deploy at scale.

Catalog record

CREATORAriel Kwiatkowski, Natasha Butt, Ismail Labiad, Julia Kempe, Yann Ollivier
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03979

Browse the full library · About the Tower of Babel