AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment

Kuei-Chun Kao 1 Daixuan Huo 1 Yuanhao Ban 1,2 Cho-Jui Hsieh 1,2 1 University of California, Los Angeles 2 Arena [email protected], [email protected] Project Page: AutoRubric-T2I Code · 2026

Read AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment by Kuei-Chun Kao 1 Daixuan Huo 1 Yuanhao Ban 1,2 Cho-Jui Hsieh 1,2 1 University of California, Los Angeles 2 Arena [email protected], [email protected] Project Page: AutoRubric-T2I Code (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17602.txt.

Introduction

Kuei-Chun Kao and colleagues present AutoRubric-T2I, which auto-builds scoring rubrics for vision-language judges of text-to-image generation. Existing reward models train as Bradley-Terry models on huge corpora, costing much and hiding their criteria, while hand-made rubrics may miss human taste. Their system turns reasoning traces into candidate rubrics and refines the set with L1-regularized logistic regression. Using a tiny share of annotated data, it beats strong baselines on MMRB2 and lifts downstream generation quality under Flow-GRPO.

Catalog record

CREATORKuei-Chun Kao 1 Daixuan Huo 1 Yuanhao Ban 1,2 Cho-Jui Hsieh 1,2 1 University of California, Los Angeles 2 Arena [email protected], [email protected] Project Page: AutoRubric-T2I Code
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62884
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17602

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment Kuei-Chun Kao 1 Daixuan Huo 1 Yuanhao Ban 1,2 Cho-Jui Hsieh 1,2 1 University of California, Los Angeles 2 Arena [email protected], [email protected] Project Page: AutoRubric-T2I Code Abstract Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment and perceptual quality. Existing reward models are commonly trained as Bradley-Terry (BT) preference models on large-scale human preference corpora, making them costly to train, difficult to adapt, and opaque in their evaluation criteria. Meanwhile, Vision-Language Model (VLM) judges can provide more fine-grained assessments through textual rubrics, but their manually designed or heuristically generated scoring rules may fail to reliably reflect human preferences. In this paper, we propose AutoRubric-T2I , the first rubric learning framework in T2I that automatically synthesizes and selects explicit rubrics for guiding VLM judges. AutoRubric-T2I first synthesizes reasoning traces from preference pairs into candidate rubrics, then uses a VLM judge to score paired images under each rubric, producing pairwise rubric-score differences for preference learning. To remove noisy and redundant rules, we further employ a ℓ 1 \ell_{1} -Regularized Logistic Regression Refiner , which selects the Top- N N most discriminative rubrics. Extensive evaluations show that AutoRubric-T2I produces high-quality, interpretable reward signals using less than 0.01% of the annotated preference data, substantially reducing the need for large-scale reward-model training. On image reward benchmarks such as MMRB2, AutoRubric-T2I outperforms strong reward model baselines. We further validate AutoRubric-T2I as an RL reward on downstream T2I tasks, including TIIF and UniGenBench++, where it improves generation quality over scalar reward models using the Flow-GRPO pipeline on diffusion models. 1 Introduction Recent advancements in T2I generation have made human preference alignment a central objective. Image reward models provide a practical mechanism for this alignment by learning to predict human judgments over generated images. In the T2I setting, an image reward model typically takes a text prompt together with one or more generated images as input and outputs either scalar reward scores or pairwise preferences indicating which image better satisfies the prompt and human quality expectations. These models are widely used for candidate ranking, best-of- N selection, data filtering, reinforcement fine-tuning (RFT), and automatic evaluation of T2I systems [ 13 , 18 , 14 ] . Existing image reward models mainly fall into two categories. The first category consists of learned BT preference models trained on large-scale human preference corpora, such as ImageReward [ 32 ] , PickScore [ 15 ] , HPSv2 [ 29 ] , and HPSv3 [ 20 ] . These models can capture real-world human preferences and are effective for global image ranking, but they require massive annotated datasets and expensive fine-tuning. Moreover, because they usually compress multiple evaluation dimensions into a single scalar score, they provide limited transparency and may overlook fine-grained visual errors such as incorrect object counts, missing attributes, distorted anatomy, or violated spatial relations [ 10 , 16 , 8 ] . The second category consists of VLM-based judges and question-answering-based evaluators, which evaluate images through textual prompts, visual questions, or rubrics [ 10 , 16 , 9 , 26 ] . These judges can assess fine-grained visual correctness when properly instructed, but their criteria are typically manually specified or heuristically generated rather than learned from human preferences. As a result, their judgments may not reliably correlate with actual human preference; for example, prompted VLM judges can be up to 10% worse than learned BT reward models on HPS or PickScore preference datasets in Table 1 . These limitations motivate rubric-based reward modeling , which replaces implicit scalar rewards with multi-dimensional evaluation criteria [ 31 , 24 , 6 , 38 , 19 ] . Rubrics make the reward signal more interpretable by decomposing human preferences into explicit rules. Recent works have explored rubrics as reward signals for LLM alignment and post-training [ 31 , 24 , 6 , 11 , 33 , 17 ] , and emerging T2I work has begun to study rubric rewards for image generation [ 4 ] . However, existing rubric-based approaches often rely on manually designed or heuristically generated rubrics, leaving open the question of how to automatically derive, select, and refine rubrics that better align with human preferences. To address this gap, we propose AutoRubric-T2I , the first rubric learning framework that automatically derives and refines an explicit rubric set for guiding off-the-shelf VLM judges in T2I reward modeling. Instead of fine-tuning a reward model, AutoRubric-T2I learns which rubrics are most predictive of human preferences and iteratively improves them through failure analysis. This design preserves the interpretability of rubric-based evaluation while avoiding the cost and opacity of training a dense scalar reward model. To achieve this, we formulate automated rubric learning as a sparse logistic regression problem within an infinite-dimensional space. We then introduce an iterative block coordinate descent method that dynamically adds new coordinates to the working set and employs \ell_{1} -regularization to assign weights and prune redundant coordinates (rubrics). This formulation is related to sparse function approximation and coordinate-selection methods such as orthogonal matching pursuit and sparse random features [ 21 , 12 , 37 ] . To enhance efficiency, we integrate a hard-pair mining algorithm for rubric refinement, ensuring that only the most informative coordinates are prioritized during the learning process. Our main contributions are as follows: • Sparse Rubric Learning for T2I Reward Modeling: We introduce AutoRubric-T2I, a framework that learns a compact, weighted set of natural-language rubrics from image preference data, enabling interpretable VLM-based reward modeling without fine-tuning. • Failure-Driven Rubric Refinement: We formulate rubric selection as an \ell_{1} -regularized logistic regression problem over VLM-scored rubric features and iteratively expand the rubric pool through curriculum-bucketed hard-pair mining. • Strong Preference Prediction and Downstream Alignment: AutoRubric-T2I achieves strong preference prediction on MMRB2 among open-source reward models and improves downstream RFT on T2I tasks such as TIIF and UniGenBench++ using Flow-GRPO. 2 Related Work 2.1 Text-to-Image Preference Alignment and Reward Modeling Aligning text-to-image (T2I) models with human preferences commonly relies on reward models trained from human preference data. Many image reward models are trained from pairwise comparisons, often with a Bradley-Terry style objective, but are deployed as pointwise scorers that assign a scalar reward to each prompt-image pair. Large-scale preference datasets and models such as PickScore [ 15 ] enabled automatic ranking of generated images according to human judgments. Subsequent reward models, including ImageReward [ 32 ] , HPSv2 [ 29 ] , and HPSv3 [ 20 ] , further improved visual preference modeling by capturing visual quality, aesthetics, and text-image correspondence. Recent work also explores alternative reward formulations, such as generative reward modeling in RewardDance [ 30 ] and UnifiedReward [ 26 ] . Despite their effectiveness, scalar reward models compress multi-dimensional human preferences into a single implicit score. This makes the learned reward difficult to interpret and vulnerable to reward hacking: a T2I policy may exploit superficial visual features such as brightness, contrast, saturation, or aesthetic style while ignoring prompt-specific semantic constraints [ 2 , 8 ] . AutoRubric-T2I addresses this limitation by replacing an opaque scalar reward with an explicit weighted set of natural-language rubrics, allowing the reward signal to remain interpretable. 2.2 Automated

Read the complete text