REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge
Yasi Zhang, Tianyu Chen, Mingyuan Zhou, Oscar Leong, Ying Nian Wu, Michal Lukasik · 2026
Read REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge by Yasi Zhang, Tianyu Chen, Mingyuan Zhou, Oscar Leong, Ying Nian Wu, Michal Lukasik (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17145.txt.
Description
Zhang and co-authors present REAL, a reinforcement learning method for training language models that judge with numeric scores. Standard RL uses binary rewards, which ignore how close a score is. A guess of 4 beats a guess of 1 when the truth is 5, and binary rewards miss that. REAL optimizes regression rewards directly, using a generalized policy gradient estimator. Training splits into exploring reasoning paths and refining the final score. Across model scales, REAL beat supervised and standard RL baselines.
Catalog record
| CREATOR | Yasi Zhang, Tianyu Chen, Mingyuan Zhou, Oscar Leong, Ying Nian Wu, Michal Lukasik |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69514 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.17145