When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On
Mehmet Saygin Seyfioglu, Karim Bouyarmane, Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna, Karim Bouyarmane · 2026
Read When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On by Mehmet Saygin Seyfioglu, Karim Bouyarmane, Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna, Karim Bouyarmane (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05659.txt.
Introduction
Ikezogwo and colleagues identify a reference-free gap in post-training methods, where many tasks admit multiple valid outputs and lack a single ideal answer. They propose Implicit Error Counting, which enumerates what a response gets wrong instead of what it gets right. As a case study, they validate Implicit Error Counting on virtual try-on. They introduce Cascaded Error Counting and curate Mismatch-DressCode, a benchmark with maximal attribute mismatch. On that benchmark, Implicit Error Counting outperforms Rubrics as Rewards across all metrics.
Catalog record
| CREATOR | Mehmet Saygin Seyfioglu, Karim Bouyarmane, Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna, Karim Bouyarmane |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68530 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.05659
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On Thanks: Work done during internship at Amazon. Affiliation: Affiliation: University of Washington Mehmet Saygin Seyfioglu, Karim Bouyarmane, Wisdom Ikezogwo, Mehmet Saygin Seyfioglu, Ranjay Krishna, Karim Bouyarmane Abstract Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers. But many real-world tasks admit multiple valid outputs and lack the single ideal answer that rubric generation depends on. We identify this reference-free setting as a gap in current post-training methods and propose Implicit Error Counting (IEC) to fill it. Instead of checking what a response gets right against a rubric, IEC enumerates what it gets wrong, applying severity-weighted scores across task-relevant axes and converting them into calibrated per-aspect rewards. We show that naïve explicit enumeration is too noisy for stable optimization, and that two design choices: implicit score emission and group calibration are necessary to make error counting a reliable reward. As a case study, we validate IEC on virtual try-on (VTO), a domain that is simultaneously too constrained for holistic scoring and too permissive for rubric-based evaluation: subtle garment errors are unacceptable, yet many output variations are correct. We introduce Cascaded Error Counting (CEC) as an evaluation metric, which tracks human preferences well (60% top-1 vs. 30% others), and curate Mismatch-DressCode (MDressBench), a benchmark with maximal attribute mismatch to stress-test reward designs. On MDressBench, IEC outperforms RaR across all metrics (CEC: 5.31 vs. 5.60 on flat references; 5.20 vs. 5.53 on non-flat). On VITON-HD and DressCode, IEC matches or surpasses six baselines on 6 of 8 perceptual metrics. These results suggest that when ideal answers are unavailable, counting errors provide a stronger signal than constructing rubrics. Figure 1 : Qualitative comparison of reward designs on non-flat references : Given the input, IEC avoids attribute preservation errors across 3 examples, garment transfer failures in example C, source integrity issues in A and D, and realism errors in examples B and C, over all other reward designs. Per-category radar plots are in Appendix B.3 . 1 Introduction Reinforcement learning has become a standard post-training step for aligning generative models with human expectations. In domains with verifiable outcomes—mathematics, code generation, game playing — reward is straightforward: a scoring function or test environment confirms correctness ( 29 ; 25 ) . Reinforcement Learning with Verifiable Rewards (RLVR) ( 29 ; 27 ; 31 ; 1 ; 9 ; 43 ) produces dramatic gains precisely because the reward signal is cheap, unambiguous, and dense. Real-world tasks are rarely so clean. Medical diagnosis, scientific reasoning, and creative generation require nuanced, multi-criteria evaluation with no single correct answer. Rubrics as Rewards (RaR) ( 12 ) addresses this gap: given a prompt and an ideal reference answer , it synthesizes instance-specific rubric criteria and scores responses against them. RaR provides structured, interpretable supervision that outperforms coarse Likert scoring. Its core assumption, however, is that an ideal answer exists and can ground rubric generation. When it does, as in medical QA with expert-written references, rubrics capture subtle quality dimensions that binary correctness cannot. But many domains lack ideal answers altogether. In embodied control, multiple action sequences can achieve the same goal. In open-ended design tasks, “correctness” is defined not by what the output matches, but by what it avoids—wrong colors, broken geometry, identity drift. We refer to this as the reference-free setting: the task is subjective enough that rubrics are needed, yet no ideal output exists to generate them from. In this setting, the nature of quality assessment inverts. Rather than asking “does this response satisfy a checklist derived from an ideal answer?” one asks “what specific errors does this response exhibit?” When multiple outputs are plausible, enumerating failures is more stable than enumerating successes, because the space of errors is typically smaller and more structured than the space of valid completions. We test this hypothesis with Implicit Error Counting (IEC) , a reward framework for reinforcement post-training in reference-free domains. Given a candidate output and its conditioning inputs, a multimodal judge enumerates errors across task-relevant evaluation axes, weights them by severity, and emits calibrated per-aspect scores. The “implicit” qualifier is critical: asking the judge to expose raw error lists (Explicit Error Counting, or EEC) introduces high variance from inconsistent enumeration, where near-identical outputs receive different error counts due to surface-form variation in the judge’s language. IEC instead asks the judge to internalize the counting and output only scores, preserving the conceptual grounding of error enumeration while stabilizing the reward signal. A lightweight group calibration step further reduces prompt-to-prompt scale drift. Notably, IEC is also more efficient: it requires a single judge call per candidate, whereas RaR requires two (rubric generation followed by evaluation). We validate IEC on virtual try-on (VTO) as a case study that exemplifies the reference-free challenge. VTO synthesizes images of users wearing target garments ( 4 ; 24 ; 49 ; 19 ; 28 ) , and its failure modes are simultaneously subtle and consequential: a slightly wrong sleeve length, a discontinuous pattern, or a shifted neckline can render a result unusable. Crucially, many distinct outputs are correct - the garment can drape differently, lighting can vary, and fine details can differ - so no single ideal answer grounds rubric generation. At the same time, errors are enumerable and localizable : they fall along well-defined axes (garment transfer, attribute preservation, realism, lighting, source integrity) and can be detected by multimodal judges. Supervised objectives on narrow datasets ( 24 ; 4 ) cannot cover the combinatorial space of poses, body shapes, textures, and garment types, and tend toward shortcut learning ( 17 ; 37 ; 30 ) , especially copy-paste behavior in masked-image editing. This motivates on-policy post-training that improves behavior without new paired supervision. To stress-test reward designs under distribution shift, we curate Mismatch-DressCode (MDressBench) , a benchmark of 700 source-reference pairs selected for maximal attribute disagreement (e.g., short-sleeve source with long-sleeve reference). We also introduce Cascaded Error Counting (CEC) , an evaluation metric that builds a shared error vocabulary across candidates, enabling consistent measurement without ground truth. In a human study validating CEC as a selection metric, it achieves 60% top-1 accuracy in recovering annotator preferences, compared to 30% for both direct scoring and RaR-based selection. Across MDressBench, IEC consistently outperforms both direct scoring and RaR on all metrics, achieving relative improvements of 6.5% (CEC), 2.3% (garment transfer), 3.4% (attribute preservation), 4.3% (realism), 3.3% (lighting), and 0.6% (source integrity) over RaR on non-flat references. On VITON-HD and DressCode, IEC outperforms all supervised and RL baselines on 6 of 8 perceptual metrics. Our ablations confirm that EEC underperforms IEC, and that group calibration provides consistent gains. 2 From Rubrics to Error Counting We first formalize the gap between rubric-based and reference-free reward settings (§ 2.1 ), then present error counting as a general reward framework (§ 2.2 ), and finally instantiate it for virtual try-on (§ 2.3 ). 2.1 The Reference-Free Gap Rubrics as Rewards ( 12 ) defines a structured reward function for each prompt x via a set of k rubric items \{(w_{j},c_{j})\}_{j=1}^{k} , where w_{j}