DaJ : Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation

Peijia Qin, Ruiyi Zhang, Qi Cao, Pengtao Xie · 2026

Read DaJ : Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation by Peijia Qin, Ruiyi Zhang, Qi Cao, Pengtao Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22230.txt.

Introduction

Peijia Qin and co-authors improve the judges that pick the best generated code. Test-time scaling for code generation often samples many candidate solutions and lets a language model judge select the best. Training a reliable judge is hard because the candidates shift in distribution, with severe imbalances between good and bad samples.

Their 2026 preprint presents DaJ, a data-reweighted language model judge for test-time scaling in code generation. Reweighting the training data counters those imbalances, so the judge learns to spot the faults that actually appear at test time. A better judge multiplies the value of every extra sample, making test-time scaling cheaper for coding assistants.

Catalog record

CREATORPeijia Qin, Ruiyi Zhang, Qi Cao, Pengtao Xie
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22230

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DaJ : Data-Reweighted LLM Judge for Test-Time Scaling in Code Generation Peijia Qin, Ruiyi Zhang, Qi Cao, Pengtao Xie Abstract Test-time scaling for code generation commonly relies on Best-of-N selection, in which multiple candidate solutions are sampled from a base model, and the best one is selected by an LLM judge. However, training reliable LLM judges is challenging due to severe distribution shifts, including imbalances between easy and hard problems, mismatches between training tasks and evaluation benchmarks, and trajectory mismatch arising from training data generated by cheaper models whose behavior differs from that of inference-time models. We propose DaJ , a reasoning-based LLM judge trained with verifiable rewards under a bi-level data-reweighted learning framework. The proposed framework learns data-importance weights (either domain-level or instance-level) to optimize generalization performance on a held-out meta set aligned with target benchmarks. To the best of our knowledge, this is the first application of data reweighting to LLM-as-a-Judge training for test-time scaling. Our approach automatically emphasizes hard problems, in-distribution samples, and trajectory-aligned data, without relying on hand-crafted heuristics. Empirically, DaJ achieves state-of-the-art performance on LiveCodeBench and BigCodeBench, outperforming strong test-time scaling baselines as well as leading proprietary models. Figure 1 : DaJ achieves first place on both LiveCodeBench and BigCodeBench, outperforming other top models. Results are taken from the official leaderboard. Figure 2 : Bi-level optimization-based data-reweighted training. Top left: Three reweighting designs. Domain reweighting assigns weights to sample groups (e.g., different coding domains); the instance table assigns explicit learnable weights to each sample, while the instance net parameterizes these weights via a lightweight MLP. Bottom left: The bi-level optimization framework. Lower-level optimization updates the judge’s parameters using weighted training data, whereas upper-level optimization evaluates the judge on a held-out meta dataset and updates data weights to maximize generalization. Right: A detailed comparison between the lower-level low-quality dataset and the upper-level high-quality dataset. Figure 3 : Overview of DaJ training and inference. (a) Top left: Given a coding problem, we sample n candidate solutions from the policy model for parallel test-time scaling. Top right: During training, DaJ performs step-by-step reasoning (“Let’s think step by step…”) before outputting a selection. The model receives a verifiable reward ( Equation 1 ), enabling preference optimization without human-annotated reasoning traces. Bottom: At inference time, multi-round pairwise voting selects the final output ( Section 3 ). (b) A simplified example of inference process, where the judge is asked to select the preferred solution from two candidates, based on a 5-step reasoning pattern ( Appendix D ). 1 Introduction Test-time scaling (TTS) refers to the use of additional inference-time computation to improve model outputs. It has recently emerged as an effective approach for enhancing large language model (LLM) performance on coding tasks ( 49 ; 38 ; 16 ; 10 ) . Among existing TTS approaches, selection-based methods form an important class that generates multiple candidate outputs at test time and selects the best-performing sample, thereby improving prediction quality without modifying model parameters. A representative selection-based strategy is Best-of-N ( 15 ) , which selects among multiple sampled candidates based on a scalar score produced by a separate judge. Judge models are therefore a core component of selection-based TTS methods, and are typically implemented using listwise or pairwise comparisons to evaluate generated responses and assign scores. More recently, LLM-as-a-Judge approaches have leveraged large language models themselves as judges to evaluate candidate solutions ( 52 ; 43 ) . By leveraging LLMs’ reasoning capabilities, these methods can yield more interpretable evaluations and have demonstrated strong promise for selection-based test-time scaling. Training LLM judges for code generation is challenging due to multiple forms of mismatch between training and target distributions. First , an easy–hard imbalance is common in training data, where easy problems are overrepresented relative to hard ones, leading to degraded judge performance on challenging test-time instances. Second , task distribution mismatch arises when the tasks used for training differ from those encountered during evaluation, limiting the ability of judges to generalize to unseen or differently distributed benchmarks. Third , trajectory mismatch occurs because training trajectories are often generated by cheaper or weaker models to reduce cost, whereas inference-time candidate solutions typically originate from stronger models, resulting in systematic misalignment between training and deployment settings. These challenges highlight the need for a principled, distribution-shift-aware training methodology that explicitly accounts for such discrepancies to achieve strong generalization to test-time tasks. We propose DaJ , a Da ta-Reweighted LLM J udge, to address the challenges of training LLM judges for code generation through principled data reweighting ( Figure 2 ). In DaJ , each training sample is assigned an importance weight with clear task-specific semantics that captures three complementary factors. First , instance difficulty is reweighted, where hard problems are upweighted while easy ones are downweighted. Second , task similarity is reweighted, where samples whose tasks are more similar to test-time scenarios are upweighted while those that are more different are downweighted. Third , trajectory alignment is reweighted, where samples whose solution candidates resemble inference-time candidates generated by stronger models are upweighted while those whose solution candidates rarely appear at test time are downweighted. DaJ is trained under a bi-level optimization framework that automatically learns these data weights. The upper-level objective optimizes generalization performance on a held-out meta dataset that more closely matches the test-time distribution, while the lower-level objective trains the judge on reweighted training data. This design enables adaptability to distribution shifts without relying on hand-crafted heuristics. For the judge architecture, we adopt a reasoning-based LLM-as-a-Judge paradigm ( 40 ; 18 ; 23 ) that is particularly well suited for code evaluation ( Section 3 ). The judge is trained using preference optimization or reinforcement learning objectives, where rewards and preference pairs are generated in a verifiable manner without human annotations: correct selections are assigned higher scores than incorrect selections or format errors. The overall training and inference pipelines are illustrated in Figure 3 . Empirically, on LiveCodeBench ( 17 ) and BigCodeBench ( 54 ) , DaJ achieves state-of-the-art performance ( Figure 1 ), outperforming learning zero-shot methods as well as strong test-time scaling baselines ( 21 ; 11 ; 27 ) . Our main contributions are: • We introduce a novel bi-level optimization-based data-reweighted training framework for LLM-as-a-Judge, which improves Best-of-N selection in test-time scaling by automatically emphasizing important training examples across three complementary aspects. • We conduct comprehensive experimental studies on data reweighting strategies and training paradigms, including preference optimization and reinforcement learning, evaluated across a diverse set of base policy models. • We achieve state-of-the-art performance on LiveCodeBench and BigCodeBench, outperforming leading proprietary models and existing test-time scaling methods. 2 Related Work LLM-as-a-Judge. Large language models are increasingly used as judges to evaluate open-ended generation, motivated by the scalability and flexibility of natural-language evaluation compared with traditional reference-based metrics. Early work on LLM-as-a-Judge demonstrated strong agreement between LLM judgments and human preferences in conversational settings ( 52 ) , establishing the feasibility of using LLMs as evaluators. Subsequent studies have shown that LLM judges are particularly effective for selection and reranking, where the goal is to select the best solution among multiple candidates. A notable application is large-scale sampling combined with filtering or selection in code generation, as

Read the complete text