UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward
Yufeng Cheng Wenxu Wu Shaojin Wu Mengqi Huang ∗ Fei Ding † Qian He · 2025
Read UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward by Yufeng Cheng Wenxu Wu Shaojin Wu Mengqi Huang ∗ Fei Ding † Qian He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06818.txt.
Introduction
Custom images containing several specific people, each keeping the right face, are the goal of a 2025 image generation paper led by Yufeng Cheng, with five co-authors. Humans are acutely sensitive to faces, and models that customize images often mix identities up when several reference photos are given. Scaling to many identities has lagged.
Their framework, UMO, recasts multi-identity generation as a global assignment problem: references and generated faces are matched many-to-many, and a matching reward drives reinforcement learning on the diffusion model. A scalable dataset with multiple reference images, part synthetic and part real, supports training, and a new metric measures identity confusion directly. Experiments show stronger identity consistency and less confusion across several existing customization methods. The authors report a new state of the art among open-source systems for identity preservation.
Catalog record
| CREATOR | Yufeng Cheng Wenxu Wu Shaojin Wu Mengqi Huang ∗ Fei Ding † Qian He |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-06818
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward Yufeng Cheng Wenxu Wu Shaojin Wu Mengqi Huang ∗ Fei Ding † Qian He Abstract Recent advancements in image customization exhibit a wide range of application prospects due to stronger customization capabilities. However, since we humans are more sensitive to faces, a significant challenge remains in preserving consistent identity while avoiding identity confusion with multi-reference images, limiting the identity scalability of customization models. To address this, we present UMO , a U nified M ulti-identity O ptimization framework, designed to maintain high-fidelity identity preservation and alleviate identity confusion with scalability. With multi-to-multi matching paradigm, UMO reformulates multi-identity generation as a global assignment optimization problem and unleashes multi-identity consistency for existing image customization methods generally through reinforcement learning on diffusion models. To facilitate the training of UMO, we develop a scalable customization dataset with multi-reference images, consisting of both synthesised and real parts. Additionally, we propose a new metric to measure identity confusion. Extensive experiments demonstrate that UMO not only improves identity consistency significantly, but also reduces identity confusion on several image customization methods, setting a new state-of-the-art among open-source methods along the dimension of identity preserving. Code and model: https://github.com/bytedance/UMO ]UXO Team, Intelligent Creation Lab, ByteDance † † footnotetext: * Corresponding author \dagger Project lead. \checkdata [Project Page] https://bytedance.github.io/UMO \correspondence Yufeng Cheng at 1 Introduction Image customization, which aims to create images that simultaneously adhere to the semantic content of textual prompts and the visual appearance of reference images, has garnered significant research attention in recent years. Among various subjects, the customization of human identities ( i.e. , ID) has attracted particular interest due to its broad range of real-world applications, such as personalized film production and virtual avatar creation. Different from other subjects, humans are exceptionally sensitive to identity customization, i.e. , even subtle discrepancies in appearance can lead to a noticeable loss of identity fidelity, thereby making human ID customization significantly more challenging. This challenge is further amplified when multiple identities need to be customized simultaneously, as it requires the model not only to preserve the unique characteristics of each individual ID, but also to maintain clear distinctions among them within the generated images. Figure 1 : Showcase of our UMO model in different scenarios. The detailed prompts are listed in Table 9 . Figure 2 : Our UMO unleashes multi-identity consistency and alleviates identity confusion. Existing image customization methods suffer low facial fidelity and severe identity confusion, while UMO can tackle these problems with results in blue boxes . Existing multi-identity customization methods primarily focus on constructing improved multi-identity paired data to enhance the consistency of multiple identities. For example, DreamO [ 18 ] and OmniGen [ 33 ] build large-scale training datasets for identity customization, including multi-identity paired data derived from either video sources or in-context image generation. Meanwhile, some recent approaches aim to mitigate confusion between multiple identities by employing identity masks to explicitly constrain the location or position of each identity. For instance, MS-Diffusion [ 29 ] introduces a layout-guided mechanism to explicitly control the generation location of each identity. RealCustom++ [ 13 , 17 ] further proposes to explicitly separate the influence masks of each identity in order to disentangle their respective generations. In summary, existing methods mainly adopt a one-to-one mapping paradigm, in which focuses on learning a direct correspondence between each identity in the reference image and the corresponding generated one. In this study, we argue that the existing one-to-one mapping paradigm fails to comprehensively address both intra-ID variability and inter-ID distinction , leading to increased identity confusion and reduced identity similarity as the number of identities grows. On the one hand, intra-ID variability refers to the inherent variability within a single identity, where the same individual may present different attributes ( e.g. , poses, expressions, etc. ) between the reference image and the generated output. On the other hand, inter-ID distinction underscores the importance of not only accurately capturing the distinctive characteristics of the target identity during generation, but also explicitly suppressing the features associated with other identities, thereby ensuring clear separation and minimizing identity confusion in multi-identity scenarios. As the number of identities increases, the risk arises that the distinctions between different identities may become less salient, potentially approaching the degree of variability observed within a single identity. Therefore, by focusing exclusively on the one-to-one mapping between each identity and its corresponding reference, existing methods overlook the increasing overlap between intra-ID variability and inter-ID distinction as the number of identities grows. This limitation fundamentally restricts their scalability, as they are unable to effectively preserve clear identity boundaries in large-scale multi-identity scenarios. To address the challenge, we propose a novel multi-to-multi matching paradigm, whose key idea is to reformulate multi-identity generation as a global assignment optimization problem, aiming to maximize the overall matching quality between multiple identities and multiple references. Therefore, each generated identity can be paired with the most suitable reference, which maximizes inter-ID distinction while minimizing the impact of intra-ID variability, thereby enabling accurate and scalable multi-identity generation. Technically, to ensure that our method remains concise and readily applicable to various customized models to improve identity consistency, we propose a U nified M ulti-identity O ptimization ( UMO ) framework, which operationalizes the multi-to-multi matching paradigm through a novel Reference Reward Feedback Learning (ReReFL). Specifically, UMO begins by defining a lightweight and reliable single-identity reference reward based on the cosine distance between identity embeddings. This is then scaled to a multi-identity context by formulating a bipartite graph between multiple references and generated identities, and optimizing the global matching score to achieve the optimal assignment. Moreover, to support the effective training of UMO, we develop a scalable customization dataset with multiple reference images for each identity, along with a new metric (ID-Conf) designed to precisely evaluate the extent of multi-identity confusion. Our main contributions can be summarized as follows: Concept: We highlight that existing customization methods fails to address intra-ID variability and inter-ID distinction, as their one-to-one mapping paradigm leads to decreased identity consistency with the scale of the identities. For the first time, we propose a novel multi-to-multi matching paradigm that maximizes overall matching quality between multiple identities and references. Technique: We propose UMO, a novel Reference Reward Feedback Learning framework featuring scalable multi-identity reference reward optimization, which can be easily integrated into various customization models. Metric: We propose ID-Conf to accurately evaluate the extent of multi-identity confusion. For a reference identity, ID-Conf is defined as the relative margin between the two most similar generated candidates, given the observation that confusion occurs with several indistinct generated faces. Performance: We conduct extensive experiments on XVerseBench [ 4 ] and OmniContext [ 31 ] , including multi-identity scenarios. Our UMO significantly boosts identity similarity and mitigates confusion simultaneously on various customized models, leading to the highest ID-Sim and ID-Conf scores. This demonstrates its strong and general promoting effect on identity consistency, showcasing its capability to deliver state-of-the-art (SOTA) results with high fidelity identities as shown in Figure 1 . 2 Related Works 2.1 Multi-subject Driven Generation Text-to-image models [ 24 , 22 , 7 , 16 , 26 , 1 ] experienced explosive growth in recent years. Despite the increasing