MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation
Wenqing Tian, Hanyi Mao, Zhaocheng Liu, Lihua Zhang, Qiang Liu, Jian Wu, Liang Wang, [0.5em] New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, [0.25em] Equal contribution · 2026
Read MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation by Wenqing Tian, Hanyi Mao, Zhaocheng Liu, Lihua Zhang, Qiang Liu, Jian Wu, Liang Wang, [0.5em] New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, [0.25em] Equal contribution (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21937.txt.
Introduction
Tian and co-authors release MultiBind, a benchmark for a sneaky failure in multi-subject generation: traits landing on the wrong person. Users feed several subject photos plus long per-person prompts, and models often swap attributes. MultiBind is built from real multi-person photos, with ordered subject crops, masks, clean references, and an inpainted background. Its scoring matches generated people to the true slots and checks identity, appearance, pose, and expression separately. Subtracting true similarity patterns splits quality loss from real mix-ups: drift, swaps, and blending.
Catalog record
| CREATOR | Wenqing Tian, Hanyi Mao, Zhaocheng Liu, Lihua Zhang, Qiang Liu, Jian Wu, Liang Wang, [0.5em] New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, [0.25em] Equal contribution |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79128 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21937
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation Wenqing Tian, Hanyi Mao, Zhaocheng Liu, Lihua Zhang, Qiang Liu, Jian Wu, Liang Wang, [0.5em] New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, [0.25em] Equal contribution Abstract Subject-driven image generation is increasingly expected to support fine-grained control over multiple entities within a single image. In multi-reference workflows, users may provide several subject images, a background reference, and long, entity-indexed prompts to control multiple people within one scene. In this setting, a key failure mode is cross-subject attribute misbinding : attributes are preserved, edited, or transferred to the wrong subject. Existing benchmarks and metrics largely emphasize holistic fidelity or per-subject self-similarity, making such failures hard to diagnose. We introduce MultiBind , a benchmark built from real multi-person photographs. Each instance provides slot-ordered subject crops with masks and bounding boxes, canonicalized subject references, an inpainted background reference, and a dense entity-indexed prompt derived from structured annotations. We also propose a dimension-wise confusion evaluation protocol that matches generated subjects to ground-truth slots and measures slot-to-slot similarity using specialists for face identity, appearance, pose, and expression. By subtracting the corresponding ground-truth similarity matrices, our method separates self-degradation from true cross-subject interference and exposes interpretable failure patterns such as drift, swap, dominance, and blending. Experiments on modern multi-reference generators show that MultiBind reveals binding failures that conventional reconstruction metrics miss. 1 Introduction Multi-reference image generation has rapidly evolved into a practical workflow where users rely on fine-grained, entity-indexed prompts to independently control multiple subjects in editing, design, and content creation [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 ] . In many real-world use cases, users provide several subject reference images and write explicit “Subject A/B/C” blocks to specify distinct attributes, actions, and relations for each entity within a single scene [ 14 , 15 , 16 , 17 ] . As these systems become more capable, a central question emerges for the community: how do we reliably measure fine-grained controllability under long, structured instructions in multi-subject settings? In this paper, we focus on multi-reference, multi-subject image generation. Users provide several subject reference images, often alongside a background reference, and a long, entity-indexed prompt that assigns different attributes, actions, and relations to specific subjects in a shared scene. The goal is not merely to generate a globally plausible image, but to compose all subjects into one scene while preserving the identity and unspecified attributes of each reference, simultaneously binding the requested edits to the correct target subjects. This binding requirement is exactly where current systems often fail. When per-subject controls become detailed and intertwined, visual cues and textual directives can leak across subjects: a jacket intended for Subject A appears on Subject B, a smile lands on the wrong face, or apparel cues are averaged across references. We refer to this failure mode as cross-subject attribute misbinding . Closely related to the “binding” and “leakage” errors studied in compositional generation [ 18 , 19 , 20 ] , this failure mode yields outputs that may look globally coherent at a glance while still violating specific user intent. In other words, each subject must satisfy two requirements at once: follow the requested edits, and retain unspecified attributes from its own reference without absorbing cues from others. Evaluation in this setting therefore has to be both subject-specific and attribute-specific. Fig. 1 illustrates the setting and several representative failure modes, including drift, dominance, swap, and blending across subjects. Figure 1: Multi-reference, multi-subject generation takes multiple subject references, a background reference, and an entity-indexed prompt as input. Correct binding respects slot-specific instructions, while failures include but are not limited to drift (a subject deviates from its own target without being confused with other subjects), dominance (one reference dominates multiple subjects), swap between subjects, or blending where attributes are mixed across subjects. This diagram illustrates the patterns through variations in clothing across different subjects. However, existing evaluation protocols lag behind these practical capabilities. Many works and benchmarks emphasize global similarity signals, such as CLIP-based alignment and distributional fidelity [ 1 , 21 , 22 ] . Some personalization methods explicitly study subject confusion, but quantitative evaluation is typically limited to face-identity self-similarity or pairwise matching in an identity embedding space [ 7 , 8 , 23 , 24 , 25 , 10 , 11 , 26 , 20 , 27 , 28 , 29 ] . While such scalars may correlate with overall fidelity, they provide weak diagnostics for complex controllability: they cannot answer who confuses with whom , nor can they provide quantitative indicators to distinguish generic self-degradation (drift) from cross-subject interference [ 26 , 20 , 27 , 28 , 29 ] . Recent VLM-based protocols can improve overall prompt adherence assessment [ 30 ] , but they are largely reference-free and do not expose per-subject correspondence errors. A second limitation concerns benchmark construction, particularly the choice of target images. Some benchmarks synthesize targets by prompting a generator to compose multiple subjects into a single image [ 31 ] . While scalable, generating targets from synthetic prompts creates an inherent dilemma. Because these scenes are unanchored from real images, complex prompts risks generating internally inconsistent images, whereas overly simple ones fail to rigorously test multi-subject control. Furthermore, other benchmarks do not provide a paired ground-truth target image at all, relying instead on reference-free (often VLM-based) judging for scoring [ 13 , 30 ] . These issues motivate our shift toward benchmarks grounded in real targets, which naturally guarantee both rich, consistent details and explicit correspondence supervision. We introduce MultiBind , a benchmark designed to stress-test long-prompt, multi-subject controllability with explicit subject correspondence supervision. Each instance is anchored to a unique real target image, and provides: (i) per-subject ground-truth crops with instance masks and bounding boxes, (ii) canonicalized subject reference images and an inpainted background reference, and (iii) structured attribute descriptions compiled into a long, entity-indexed prompt. Grounding each condition in a real target enables attribute-rich prompts that remain realistic, diverse, and internally consistent, while also supporting reproducible similarity-based scoring. Beyond the benchmark, we propose a confusion-aware evaluation protocol that makes subject-attribute misbinding directly measurable. For each attribute dimension, we first compute subject-to-subject similarity matrices between generated subjects and ground-truth subjects using dimension-specific specialists. To isolate the changes introduced by generation, we subtract the inherent similarities between ground-truth subjects to compute baseline-corrected delta matrices. This step effectively disentangles generic quality degradation (a subject losing its own features, reflected on the matrix diagonal) from cross-subject interference (a subject absorbing features from others, reflected off-diagonal). The resulting diagnostics expose whether failures manifest as drift, specific swaps, dominant confusers, or blending. Fig. 2 provides an overview of the full MultiBind pipeline. Contributions. Our main contributions are summarized as follows: Benchmark. We establish MultiBind , a robust benchmark for multi-subject and multi-reference generation. Unlike previous datasets, it is grounded in real target images and provides exhaustive annotations, including per-subject masks, bounding boxes, and background references, alongside structured captions rewritten into entity-indexed prompts. Evaluation. We introduce a dimension-wise evaluation protocol that leverages specialist representations to produce confusion-aware similarity and delta matrices. This framework