Scaling Multi-Reference Image Generation with Dynamic Reward Optimization

Wenwang Huang 1,∗ , Yusen Fu 1,∗ , Junjie Wang 1 , Mengfei Huang 1 , Yulin Li 1 , Gan Liu 2 , Jing Cai 2 , Yancheng He 2 , and Zhuotao Tian 1,3,† · 2026

Read Scaling Multi-Reference Image Generation with Dynamic Reward Optimization by Wenwang Huang 1,∗ , Yusen Fu 1,∗ , Junjie Wang 1 , Mengfei Huang 1 , Yulin Li 1 , Gan Liu 2 , Jing Cai 2 , Yancheng He 2 , and Zhuotao Tian 1,3,† (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26947.txt.

Introduction

Wenwang Huang, Yusen Fu, and seven colleagues scale image generation that follows several reference images at once. Personalised generation from one reference works well. Mixing many references of mixed types is much harder. Existing benchmarks barely test those complex cases. Their new OmniRef-Bench covers rich combinations and large reference counts.

Mainstream open models struggled on the new benchmark. Performance fell sharply as mixed-type references multiplied. Their training framework, DyRef, attacks the problem in two stages. Supervised fine-tuning first teaches the basic multi-reference skill. A second stage reweights learning by difficulty and stretches reward differences within groups. Hard cases with many references thus get more training pressure. The authors report clear gains on complex generation tasks.

Catalog record

CREATORWenwang Huang 1,∗ , Yusen Fu 1,∗ , Junjie Wang 1 , Mengfei Huang 1 , Yulin Li 1 , Gan Liu 2 , Jing Cai 2 , Yancheng He 2 , and Zhuotao Tian 1,3,†
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26947

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scaling Multi-Reference Image Generation with Dynamic Reward Optimization Wenwang Huang 1,∗ , Yusen Fu 1,∗ , Junjie Wang 1 , Mengfei Huang 1 , Yulin Li 1 , Gan Liu 2 , Jing Cai 2 , Yancheng He 2 , and Zhuotao Tian 1,3,† Abstract While personalized image generation has achieved remarkable progress, multi-reference image generation (MRIG) remains a challenging task. Most existing benchmarks fail to adequately evaluate complex MRIG scenarios, hindering further progress in this area. To better assess model performance on complex MRIG tasks, we introduce OmniRef-Bench, a benchmark that covers complex combinations of reference image types and a large number of reference images. Evaluations on OmniRef-Bench show that mainstream open-source models struggle in complex MRIG scenarios, and their performance deteriorates significantly as the number of mixed-type reference images increases. To address this issue, we propose DyRef , a two-stage training framework. In the first stage, supervised fine-tuning equips the model with the basic capability to handle complex MRIG tasks. In the second stage, we introduce Difficulty-aware Advantage Reweighting (DAR) and Discriminative Reward Scaling (DRS). DAR dynamically adjusts the optimization objective to improve performance when handling a large number of mixed-type reference images. DRS enlarges intra-group reward differences for more effective policy optimization. Experiments demonstrate that DyRef significantly improves the performance of open-source models on OmniRef-Bench and single-image editing benchmarks, demonstrating the effectiveness and generalization capability of our approach. Keywords: Personalized Image Generation, Reinforcement Learning Code Repository : https://github.com/Weistrass/DyRef = Datasets : https://huggingface.co/datasets/Eason0438/OmniRef-training = Benchmark : https://huggingface.co/datasets/Eason0438/OmniRef-Bench = Contact : [email protected] 1 Introduction Figure 1 : Versatile samples of the proposed DyRef . Given complex samples that involve mixed reference image types and varying numbers of reference images, DyRef consistently generates high-quality results in accordance with user instructions. The emergence of diffusion models [ 33 , 38 ] has substantially advanced personalized image generation [ 53 , 26 , 19 , 29 , 46 ] , enabling high-fidelity synthesis closely aligned with user instructions. However, among various personalization paradigms, Multi-Reference Image Generation (MRIG) [ 57 , 51 , 58 , 44 , 31 ] remains challenging due to the difficulty of effectively integrating multiple reference images of diverse types (e.g., subject, pose, and style). Addressing these challenges carries significant implications for fields like professional visual design [ 50 ] and advertising applications [ 5 ] , where coherent integration of varied visual elements is crucial. Insufficient Evaluation for Complex MRIG. Despite its potential, MRIG research remains constrained by insufficient benchmarks. Existing benchmarks [ 2 , 3 , 31 , 60 ] typically involve simple tasks with limited numbers and types of reference images, neglecting complex scenarios that require integrating numerous and diverse references. Additionally, these benchmarks lack fine-grained and multi-dimensional evaluation metrics to accurately assess model performance across various reference types. These limitations hinder meaningful comparisons between methods and impede progress towards practical MRIG applications. Figure 2 : Performance of DyRef . On both MRIG and single-image editing tasks, DyRef (Ours) consistently outperforms open-source state-of-the-art methods and achieves performance comparable to leading closed-source model Nano Banana Pro. The Proposed OmniRef-Bench. To better assess MRIG performance, we introduce OmniRef-Bench, a personalized image generation benchmark comprising intricate combinations of diverse reference types as well as a large number of reference images across synthetic and real-world scenarios. Furthermore, we propose a hybrid evaluation strategy combining objective quantitative metrics with Multimodal Large Language Models (MLLMs)-based assessments, and design fine-grained evaluation criteria tailored to each reference type, facilitating a robust and multidimensional evaluation of complex MRIG tasks. Key Observations. Using OmniRef-Bench (Tab. 2 ), we find that mainstream open-source MRIG models [ 53 , 58 , 10 , 54 ] struggle in complex MRIG scenarios, exhibiting a huge gap compared with state-of-the-art closed-source models [ 8 , 40 ] . To bridge this disparity, we conduct further experiments and observe that while open-source models generate high-quality images with a few references, their performance deteriorates rapidly as the number of reference images involving mixed reference types increases, as illustrated in Fig. 3 . This observation raises a critical research question: Can mitigating this degradation effectively narrow the gap between open-source and closed-source models? Figure 3 : Motivation of our DyRef . (a) Qualitative results: open source models yield high-quality results with a limited set of references, but suffer from significant artifacts and semantic loss as the number and complexity of reference images increase. (b) Quantitative results: on the OmniRef-Bench, CLIP-I scores between generated images and target images confirm that the performance of mainstream open-source models significantly drops as the number of mixed-type reference images increases. Our Solution. To address this issue, we propose DyRef , a two-stage training framework. In the first stage, we use Supervised Fine-Tuning (SFT) to equip the model with preliminary capabilities to handle complex MRIG tasks. In the second stage, we introduce Difficulty-aware Advantage Reweighting (DAR) and Discriminative Reward Scaling (DRS). Specifically, DAR dynamically adjusts the optimization objective to encourage the model to focus more on underperforming samples with a larger number of mixed-type reference images, while preventing overfitting to well-performing samples with fewer references. DRS enlarges reward differences within each group for more effective policy optimization. The combination of DAR and DRS enhances the model’s ability to learn from samples with a large number of diverse reference images, thereby improving performance on complex MRIG tasks. Experimental results (Fig. 1 , Fig. 2 and Fig. 3 (b)) demonstrate that DyRef substantially improves the performance of open-source models on OmniRef-Bench, achieving results comparable to state-of-the-art closed-source models. Moreover, DyRef further enhances the model performance on single-image editing benchmarks, highlighting its effectiveness and generalization ability. In summary, the contributions of this work are as follows: \bullet To better assess model performance on complex MRIG tasks, we introduce OmniRef-Bench. Using this benchmark, we observe that open-source models struggle in complex multi-reference settings, and their performance declines rapidly as the number of mixed-type reference images increases. \bullet To address the issue, we propose DyRef. Building upon SFT, we employ DAR to encourage the model to focus more on underperforming samples with a larger number of mixed-type reference images, and utilize DRS to enhance reward differences across samples for better policy optimization. \bullet Experimental results show that DyRef not only substantially boosts performance on OmniRef-Bench, but also improves results on single-image editing benchmarks, demonstrating both its effectiveness and generalization. 2 Dataset and Benchmark In this section, we first introduce the motivation of OmniRef-Bench in Section 2.1 . Next, Section 2.2 describes the composition and construction pipeline of the benchmark. Finally, Section 2.3 outlines the corresponding evaluation strategy. Table 1 : Comparison of OmniRef-Bench with existing benchmarks. Eval Granularity indicates whether a benchmark conducts tailored assessment for every reference type (fine-grained), or simply conducts a general evaluation (coarse-grained). Max Combs denotes the maximum number of reference image type combinations. Obj refers to objective quantitative metrics. MLLM indicates evaluation conducted using the MLLM. Benchmarks Max Combs. Expert-Annotated Eval Granularity Eval Perspective XVerse [ 2 ] 1 ✗ fine-grained Obj PSRBench [ 49 ] 1 ✗ fine-grained Obj + MLLM MultiBanana [ 31 ] 3 ✗ coarse-grained MLLM DreamOmni2 [ 58 ] 2 ✓ coarse-grained MLLM OmniRef-Bench (Ours) 4 ✓ fine-grained Obj + MLLM 2.1 Motivation As shown

Read the complete text