Rethinking Verification for LLM Code Generation: From Generation to Testing

Zihan Ma, Taolin Zhang, Maosong Cao, Junnan Liu, Wenwei Zhang, Minnan Luo, Songyang Zhang, Kai Chen · 2025

Read Rethinking Verification for LLM Code Generation: From Generation to Testing by Zihan Ma, Taolin Zhang, Maosong Cao, Junnan Liu, Wenwei Zhang, Minnan Luo, Songyang Zhang, Kai Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06920.txt.

Introduction

Code benchmarks grade AI programmers with a handful of tests, and Zihan Ma and co-authors show how much slips through. Suites like HumanEval use few, similar test cases, so subtle bugs pass as correct, scores inflate, and reinforcement learning trained on those scores chases a mirage. The team proposes metrics that grade the test suites themselves for thoroughness, and a benchmark, TCGBench, for the test-writing task.

Their method, SAGA, pairs human programmers with a model to write sharper tests. On TCGBench it caught 90.62 percent of planted faults, and benchmarks it synthesized judged code verifiers far more accurately than a leading rival. The authors pitch test generation as the foundation trustworthy code evaluation stands on.

Catalog record

CREATORZihan Ma, Taolin Zhang, Maosong Cao, Junnan Liu, Wenwei Zhang, Minnan Luo, Songyang Zhang, Kai Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-06920

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Rethinking Verification for LLM Code Generation: From Generation to Testing Zihan Ma, Taolin Zhang, Maosong Cao, Junnan Liu, Wenwei Zhang, Minnan Luo, Songyang Zhang, Kai Chen Abstract Large language models (LLMs) have recently achieved notable success in code - generation benchmarks such as HumanEval and LiveCodeBench. However, a detailed examination reveals that these evaluation suites often comprise only a limited number of homogeneous test cases, resulting in subtle faults going undetected. This not only artificially inflates measured performance but also compromises accurate reward estimation in reinforcement learning frameworks utilizing verifiable rewards (RLVR). To address these critical shortcomings, we systematically investigate the test-case generation (TCG) task by proposing multi-dimensional metrics designed to rigorously quantify test-suite thoroughness. Furthermore, we introduce a human-LLM collaborative method (SAGA), leveraging human programming expertise with LLM reasoning capability, aimed at significantly enhancing both the coverage and the quality of generated test cases. In addition, we develop a TCGBench to facilitate the study of the TCG task. Experiments show that SAGA achieves a detection rate of 90.62% and a verifier accuracy of 32.58% on TCGBench. The Verifier Accuracy (Verifier Acc) of the code generation evaluation benchmark synthesized by SAGA is 10.78% higher than that of LiveCodeBench-v6. These results demonstrate the effectiveness of our proposed method. We hope this work contributes to building a scalable foundation for reliable LLM code evaluation, further advancing RLVR in code generation, and paving the way for automated adversarial test synthesis and adaptive benchmark integration. 1 1 1 The data demo and prompts can be accessed via https://github.com/open-compass/SAGA † † \dagger means corresponding authors, ‡ means project, * means authors contributed equally. 1 Introduction Large Language Models (LLMs) have triggered a paradigm shift in automatic code generation, demonstrating capabilities on par with or even exceeding human programmers on numerous benchmark tasks. As LLMs become increasingly integrated into software development workflows, ensuring the quality and reliability of the code they produce is paramount. This necessitates reliable evaluation methodologies, where code verifiers—typically powered by test suites—play a critical role. This raises a crucial question: Are the test cases of current benchmarks for evaluating models’ code capabilities robust enough? Initial analyses of benchmarks like HumanEval [ 6 ] (avg. 7.7 tests/problem), MBPP [ 3 ] (3 tests/problem), and EvalPlus [ 27 , 28 ] (which saw a 15% pass rate drop with 80× more tests) indicate the fragility of current evaluation setups due to sparse test coverage. While methods like TestEval [ 41 ] tailor tests to specific solutions, they are inefficient for large-scale evaluation and impractical for dynamic integration into RL training loops, thereby hindering the development of models robust against diverse failures. LiveCodeBench [ 19 ] employs LLMs to generate numerous tests from golden solutions and synthetic inputs, aiming to enhance robustness. Figure 1 : (a) Verifiers synthesized primarily from LLM-generated data exhibit a high failure rate when testing human-written bugs. (b) PCA analysis reveals that LLM-induced errors are highly clustered, indicating systematic weaknesses, whereas human errors are diverse and dispersed, posing greater challenges to existing verifiers. However, a fundamental concern is that such methods may inadvertently create tests biased towards typical, often homogenized, LLM error patterns, which starkly contrast with diverse human reasoning errors. Conversely, competitive programming platforms (Online Judges) possess extensive, rigorously curated test suites for assessing code robustness, but these are often private and inaccessible. This underscores the urgent need for accessible and robust code verifiers to enable reliable performance evaluation and reward estimation. In this work, we first identify limitations in current benchmarks through preliminary experiments. Our investigation reveals critical weaknesses in existing verifier suites. Specifically, when we took LLM-generated solutions that had passed LiveCodeBench’s private tests and re-evaluated them on LeetCode’s online judge, we found that for a significant portion of these solutions—20% for medium and 40% for hard problems, respectively—LeetCode identified errors that LiveCodeBench’s verifier had missed. This demonstrates that LiveCodeBench’s verifiers can be flawed, failing to achieve comprehensive error detection and thereby overestimating the true quality of the LLM-generated solutions. Furthermore, these LLM-centric verifiers themselves exhibit a high failure rate (i.e., they fail to detect existing bugs) when evaluating human-written faulty code, a rate much higher than any apparent failure rate on LLM-generated errors (Figure 1 (a)). PCA analysis of error patterns (Figure 1 (b)) reveals that LLM errors cluster tightly, indicating shared systematic biases, while human errors are widely distributed across a complex error landscape. These findings underscore the fundamental inadequacy of constructing verifiers solely from LLM-generated data, as such approaches misassess LLMs’ coding capabilities and provide flawed training feedback, manifesting two critical challenges: (1) Test Case Homogenization and LLM-Centric Bias : LLM-based TCG methods produce test suites that mirror the generating models’ error patterns and cognitive biases, creating a “homogenization trap” where tests focus on LLM-like failures while neglecting diverse human programming errors (e.g., logical flaws, integer overflows). (2) Verifier Ineffectiveness and Persistent Blind Spots : Verifiers built on such test suites exhibit blind spots for human-like errors, failing to rigorously evaluate code due to challenges in generating tests for complex boundary conditions and interaction scenarios, compounded by LLM-centric test design. These challenges impede reinforcement learning frameworks (e.g., DeepSeek-R1 [ 8 ] , O1-IOI [ 10 ] ) from leveraging verifiable rewards, leading to optimization misdirection via reward hacking 2 2 2 Models exploit verifier weaknesses instead of achieving genuine correctness, as they are insufficiently penalized for diverse errors. . To address these challenges, we develop a comprehensive measurement framework for code verifiers, introducing multi-dimensional evaluation metrics (detection rate, verifier accuracy) and identifying test case diversity and per-case strength as critical quality factors. We derive an upper bound for detection rate and validate it empirically across 1,500+ coding problems. Using this framework, we uncover systemic test quality issues in leading benchmarks, such as CodeForce-CoTs [ 37 ] , where 50% of problems had tests failing to detect known errors and 84% of verifiers were flawed, further underscoring these limitations. To systematically improve the quality of code verifiers, we formally define the Test Case Generation (TCG) task, comprising three core components: task formulation , benchmark construction , and method exploration . We introduce TCGBench, a benchmark curated by aggregating representative problems from three leading competitive programming platforms—Atcoder, Codeforces, and Nowcoder. TCGBench curates human-verified adversarial examples spanning diverse error patterns (e.g., logical flaws, edge cases) and supports comprehensive evaluation of TCG methods. We further investigate current TCG methods and propose SAGA ( S trategic A dversarial & Constraint-differential G ener A tive workflow), a novel human-LLM collaborative framework. SAGA is designed to systematically generate high-coverage, highly discriminative test cases by leveraging both human-derived constraints from correct solutions and insights from failure modes in incorrect solutions. This dual-pronged analytical approach allows SAGA to achieve improvements over current state-of-the-art TCG methods, with the Detection Rate increasing by 9.55% and Verifier Accuracy by 12.14%. Additionally, we leverage SAGA to enhance the quality of the popular code generation benchmark LiveCodeBench-v6 (subset) 3 3 3 The subset used in our study comprises 101 problems from AtCoder, specifically from contests ABC387 through ABC400 and ARC190 through ARC196 (problems A to F). This selection facilitates a consistent evaluation scope when comparing with benchmarks like our proposed TCGBench, which also incorporates these recent algorithmic challenges. ,

Read the complete text