SedarEval: Automated Evaluation using Self-Adaptive Rubrics
Zhiyuan Fan, Weinong Wang, Xing Wu, Debing Zhang · 2025
Read SedarEval: Automated Evaluation using Self-Adaptive Rubrics by Zhiyuan Fan, Weinong Wang, Xing Wu, Debing Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15595.txt.
Description
Zhiyuan Fan, Weinong Wang, Xing Wu, and Debing Zhang let language models grade other language models with rubrics that adapt to each question. Fixed rubrics miss what makes one answer right and another wrong. Human graders, by contrast, write question-specific scoring guides.
Their method builds a tailored rubric per question, listing main and minor criteria with scoring and deduction points. A benchmark then tests the graders across knowledge, math, coding, and logic tasks. The aim is grading that is steadier and closer to human judgment. Automated grading decides leaderboards and training signals alike. Rubrics that fit the question should cut grading noise. The work borrows the examiner's craft, not just the examiner's chair.
Catalog record
| CREATOR | Zhiyuan Fan, Weinong Wang, Xing Wu, Debing Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-15595