Generating Data-Driven Reasoning Rubrics for Domain-Adaptive Reward Modeling

Kate Sanders, Nathaniel Weir, Sapana Chaudhary, Kaj Bostrom, Huzefa Rangwala · 2026

Read Generating Data-Driven Reasoning Rubrics for Domain-Adaptive Reward Modeling by Kate Sanders, Nathaniel Weir, Sapana Chaudhary, Kaj Bostrom, Huzefa Rangwala (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06795.txt.

Description

Kate Sanders, Nathaniel Weir, Sapana Chaudhary, Kaj Bostrom, and Huzefa Rangwala improve how language models check reasoning. Models struggle to spot errors in long chains of thought, especially in expert domains or problems without checkable answers. The paper builds detailed error taxonomies, or rubrics, directly from data, and uses them to sharpen error detection on unseen traces.

Rubric-based classifiers beat baseline methods in coding, math, and chemical engineering. The rubrics then serve as reward functions for training reasoning models. Compared with models trained under general model-as-judge rewards, task accuracy rose by up to 45 percent. That approaches the performance of training with fully verifiable rewards while using as little as a fifth of the gold labels. The work extends rubric methods from judging style to judging correctness.

Catalog record

CREATORKate Sanders, Nathaniel Weir, Sapana Chaudhary, Kaj Bostrom, Huzefa Rangwala
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06795

Browse the full library · About the Tower of Babel