LMUnit : Fine-grained Evaluation with Natural Language Unit Tests

Jon Saad-Falcon 1,2 , Rajan Vivek 1* , William Berrios 1* Nandita Shankar Naik 1 , Matija Franklin 1 , Bertie Vidgen 1 Amanpreet Singh 1 , Douwe Kiela 1,2 , Shikib Mehri 1 1 Contextual AI 2 Stanford University, Department of Computer Science [email protected] , [email protected] · 2024

Read LMUnit : Fine-grained Evaluation with Natural Language Unit Tests by Jon Saad-Falcon 1,2 , Rajan Vivek 1* , William Berrios 1* Nandita Shankar Naik 1 , Matija Franklin 1 , Bertie Vidgen 1 Amanpreet Singh 1 , Douwe Kiela 1,2 , Shikib Mehri 1 1 Contextual AI 2 Stanford University, Department of Computer Science [email protected] , [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-13091.txt.

Introduction

Jon Saad-Falcon and eight co-authors present a new way to grade language model answers in a paper from two thousand twenty-four. Human grading is costly and noisy, while automated metrics give coarse signals that are hard to interpret. Their idea breaks response quality into explicit, testable criteria stated in plain language.

They call these criteria natural language unit tests, by analogy with software tests. A scoring model, LMUnit, is trained on preferences, direct ratings, and written rationales at once. Human studies showed the tests raise agreement between annotators. The model reached top scores on two evaluation benchmarks and competitive results on a third. The authors present the approach as a path to better model development.

Catalog record

CREATORJon Saad-Falcon 1,2 , Rajan Vivek 1* , William Berrios 1* Nandita Shankar Naik 1 , Matija Franklin 1 , Bertie Vidgen 1 Amanpreet Singh 1 , Douwe Kiela 1,2 , Shikib Mehri 1 1 Contextual AI 2 Stanford University, Department of Computer Science [email protected] , [email protected]
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-13091

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LMUnit : Fine-grained Evaluation with Natural Language Unit Tests Jon Saad-Falcon 1,2 , Rajan Vivek 1* , William Berrios 1* Nandita Shankar Naik 1 , Matija Franklin 1 , Bertie Vidgen 1 Amanpreet Singh 1 , Douwe Kiela 1,2 , Shikib Mehri 1 1 Contextual AI 2 Stanford University, Department of Computer Science [email protected] , [email protected] Abstract As language models become integral to critical workflows, assessing their behavior remains a fundamental challenge – human evaluation is costly and noisy, while automated metrics provide only coarse, difficult-to-interpret signals. We introduce natural language unit tests , a paradigm that decomposes response quality into explicit, testable criteria, along with a unified scoring model, LMUnit , which combines multi-objective training across preferences, direct ratings, and natural language rationales. Through controlled human studies, we show this paradigm significantly improves inter-annotator agreement and enables more effective LLM development workflows. LMUnit achieves state-of-the-art performance on evaluation benchmarks (FLASK, BigGenBench) and competitive results on RewardBench. These results validate both our proposed paradigm and scoring model, suggesting a promising path forward for language model evaluation and development. 1 Introduction Figure 1: Natural Language Unit Tests: Overview of the three-step process: (1) unit test creation, (2) LMUnit-based scoring with natural language rationales, and (3) score aggregation for overall quality assessment. The evaluation of generative language models remains one of the most fundamental challenges in natural language processing Jones and Galliers 1995 ; Deriu et al. 2021 ; Smith et al. 2022 ; Chang et al. 2024 – it determines how we measure progress and validate improvements, and it ultimately shapes the field’s trajectory. As these models transition from research prototypes to production systems, real users increasingly rely on them for critical workflows ( Lin et al. 2024 ) , creating an urgent need for evaluation methods that identify response strengths/weaknesses, ensure reliability, detect context-dependent failures, and prevent costly regressions. Yet current approaches fall short: human evaluation is expensive, noisy, and struggles to discern subtle differences among top-performing models ( Hosking et al. 2023 ; Clark et al. 2021 ; Karpinska et al. 2021 ) , while automated metrics compress the rich, multi-dimensional nature of response quality into coarse, difficult-to-interpret scores ( Stent et al. 2005 ; Liu et al. 2016 ) that rely on implicitly learned, often biased evaluation criteria ( Dubois et al. 2024a ; Shankar et al. 2024 ; Zhang et al. 2024a ) . As models become more capable and deeply integrated into essential workflows, it is imperative that our evaluation methodologies evolve in tandem, empowering LLM practitioners to reliably detect subtle failures , meaningfully distinguish among top-performing systems , and generate actionable insights that drive sustained improvements in capabilities. We focus on one of the most critical challenges in evaluating language models: measuring response quality. Defining “response quality” is inherently complex – what constitutes a good response depends on multiple factors, including factual accuracy, logical coherence, and alignment with user-specific objectives, among other dimensions, all of which vary by domain, application, style, and context ( Mehri and Eskenazi 2020a ; Ye et al. 2023 ; Krishna et al. 2023 ) . Existing evaluation approaches struggle with this complexity, which results in an unreliable evaluation signal: (1) reference-based comparisons are ill-suited to open-ended scenarios and subtle failure modes where no single “correct” response exists ( Liu et al. 2016 ; Lowe et al. 2017 ) , (2) human evaluations become inconsistent and costly as models grow more capable and errors subtler ( Walker et al. 2007 ; Pan et al. 2024 ; Christiano et al. 2023 ) , and (3) preference models or prompted LLM judges compress nuanced assessments into opaque, coarse-grained metrics that are difficult to interpret or steer ( Dubois et al. 2024b ; D’Oosterlinck et al. 2024 ; Singhal et al. 2023 ) . To address these limitations, we propose natural language unit tests , a paradigm that decomposes response quality into explicit, testable criteria that humans can define, refine, and guide over time (Figure 1 ). While this approach enhances transparency and adaptability, it introduces a key technical challenge: how to reliably score and integrate these fine-grained assessments in a manner that remains aligned with human values. Building an effective scoring model for unit tests presents a significant challenge: it must accurately evaluate a wide range of criteria – ranging from broad notions of quality to detailed rubrics that capture intricate, context-specific requirements. Existing approaches each address part of the problem: prompted LLM judges can be instructed to consider certain criteria ( Liu et al. 2023 ) , but their accuracy is limited by generic instruction-following abilities and the inability to learn directly from preference data ( Wang et al. 2024b ; Zhong et al. 2022 ) ; preference models, while closely aligned with human judgments, lack promptability and struggle to handle more granular, human-defined criteria ( Singhal et al. 2023 ; Lambert and Calandra 2023 ) . To address these challenges, we propose LMUnit 1 1 1 https://lmunit.contextual.ai/ , a unified modeling approach that optimizes large language models as preference models while supporting flexible, user-defined evaluation criteria. By combining diverse training signals with natural language rationales, LMUnit achieves strong results across preference modeling, direct scoring, and fine-grained unit test evaluations, laying a robust foundation for more adaptive and transparent evaluation methodologies. To demonstrate how our paradigm enables human stakeholders to effectively intervene and steer the assessment and LLM development process, we assess its real-world impact beyond standard offline benchmarks through human studies. First, in a controlled annotation study, expert raters achieved substantially higher inter-annotator agreement when evaluating outputs against explicit unit tests compared to standard preference annotations. Second, in a case study with LLM developers, we found that LMUnit ’s transparent, test-driven evaluations enabled the identification of more errors than conventional LLM judges, demonstrating the value of the proposed paradigm. Our key contributions include: (1) proposing the paradigm of natural language unit tests, and validating it at scale, (2) developing LMUnit as a unified scoring model that achieves state-of-the-art performance, (3) demonstrating improved performance with post-training of rationales, (4) showing the benefits and challenges of effective unit test creation and weighting strategies, and (5) validating our approach through human studies that demonstrate improved inter-annotator agreement and more effective LLM development workflows. 2 Related Work 2.1 Evaluation of Generative Language Models As LLM development continues to progress, robust and interpretable evaluation methodologies become increasingly critical. While human evaluation is often considered the gold standard ( Ouyang et al. 2022 ; Touvron et al. 2023 ) , its cost and scalability limitations Hosking et al. 2023 ; Schoch et al. 2020 have driven the development of automated approaches. These include word overlap metric ( Papineni et al. 2002 ; Lin 2004 ) , embedding-based scoring methods ( Yuan et al. 2021 ; Zhang et al. 2019 ) , model-based evaluations ( Lowe et al. 2017 ; Mehri and Eskenazi 2020b ; Zhong et al. 2022 ; Saad-Falcon et al. 2023 ) , reward modeling ( Christiano et al. 2017 ; Askell et al. 2021 ; Kim et al. 2023 ) , and LM judges ( Zheng et al. 2023 ; Liu et al. 2023 ; Es et al. 2023 ; Ravi et al. 2024

Read the complete text