RubricRAG : Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generation
Kaustubh D. Dhole, Eugene Agichtein · 2026
Read RubricRAG : Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generation by Kaustubh D. Dhole, Eugene Agichtein (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20882.txt.
Description
Dhole and Agichtein study how language models can write clear scoring rubrics for other model answers. Simple numeric scores do not show what an answer did right or wrong. Hand written rubrics are clearer but too slow to make at scale. They find that off the shelf models write rubrics that align poorly with human ones. Their RubricRAG method pulls domain knowledge from rubrics of related questions at run time. It yields rubrics closer to human ones. It also improves how well good responses are picked out downstream.
Catalog record
| CREATOR | Kaustubh D. Dhole, Eugene Agichtein |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37932 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20882