A Matched Holistic Rubric Rivals Self-Decomposing Atomic Judges for Benchmark-Style Reference-Support Classification

Xinran Zhang · 2026

Read A Matched Holistic Rubric Rivals Self-Decomposing Atomic Judges for Benchmark-Style Reference-Support Classification by Xinran Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28005.txt.

Description

Zhang compares four ways for a language model judge to label whether an answer is supported by a reference. One design splits the answer into atomic claims; others use a holistic rubric, split the reference instead, or combine both. Labels come from TruthfulQA, ASQA, and QAMPARI. Splitting the reference proved 12-21 points more accurate on ASQA, at modest extra token cost. Splitting the answer helped mainly where labels hinge on false beliefs rather than completeness.

Catalog record

CREATORXinran Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43372
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28005

Browse the full library · About the Tower of Babel