REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge

Yasi Zhang, Tianyu Chen, Mingyuan Zhou, Oscar Leong, Ying Nian Wu, Michal Lukasik · 2026

Read REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge by Yasi Zhang, Tianyu Chen, Mingyuan Zhou, Oscar Leong, Ying Nian Wu, Michal Lukasik (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17145.txt.

Description

Zhang and co-authors present REAL, a reinforcement learning method for training language models that judge with numeric scores. Standard RL uses binary rewards, which ignore how close a score is. A guess of 4 beats a guess of 1 when the truth is 5, and binary rewards miss that. REAL optimizes regression rewards directly, using a generalized policy gradient estimator. Training splits into exploring reasoning paths and refining the final score. Across model scales, REAL beat supervised and standard RL baselines.

Catalog record

CREATORYasi Zhang, Tianyu Chen, Mingyuan Zhou, Oscar Leong, Ying Nian Wu, Michal Lukasik
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69514
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.17145

Browse the full library · About the Tower of Babel