Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignment
Bac Nguyen, Yuhta Takida, Naoki Murata, Chieh-Hsin Lai, Toshimitsu Uesaka, Stefano Ermon, Yuki Mitsufuji · 2026
Read Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignment by Bac Nguyen, Yuhta Takida, Naoki Murata, Chieh-Hsin Lai, Toshimitsu Uesaka, Stefano Ermon, Yuki Mitsufuji (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01224.txt.
Introduction
Bac Nguyen and six co-authors improve object-centric learning, where a model must split a scene into its separate objects without being told where they are. Their base method uses slot attention with a pretrained diffusion model, but slots often tangle together or drift from the objects they should track. The new method, CODA, adds two fixes.
First, extra register slots soak up leftover attention, so object slots interfere with each other less. Second, a contrastive loss pulls each slot toward the image content it represents. Together, the changes act as a practical stand-in for raising the shared information between slots and input. On synthetic video sets and real photo sets, CODA improves object discovery, property prediction, and image generation from parts. Register slots cost almost nothing to add. The result is a sturdier way to parse busy real-world scenes into objects.
Catalog record
| CREATOR | Bac Nguyen, Yuhta Takida, Naoki Murata, Chieh-Hsin Lai, Toshimitsu Uesaka, Stefano Ermon, Yuki Mitsufuji |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-01224
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignment Bac Nguyen, Yuhta Takida, Naoki Murata, Chieh-Hsin Lai, Toshimitsu Uesaka, Stefano Ermon, Yuki Mitsufuji Abstract Slot Attention (SA) with pretrained diffusion models has recently shown promise for object-centric learning (OCL), but suffers from slot entanglement and weak alignment between object slots and image content. We propose C ontrastive O bject-centric D iffusion A lignment ( CODA ), a simple extension that (i) employs register slots to absorb residual attention and reduce interference between object slots, and (ii) applies a contrastive alignment loss to explicitly encourage slot–image correspondence. The resulting training objective serves as a tractable surrogate for maximizing mutual information (MI) between slots and inputs, strengthening slot representation quality. On both synthetic (MOVi-C/E) and real-world datasets (VOC, COCO), CODA improves object discovery (e.g., +6.1% FG-ARI on COCO), property prediction, and compositional image generation over strong baselines. Register slots add negligible overhead, keeping CODA efficient and scalable. These results indicate potential applications of CODA as an effective framework for robust OCL in complex, real-world scenes. Code is available at https://github.com/sony/coda . 1 Introduction Object-centric learning (OCL) aims to decompose complex scenes into structured, interpretable object representations, enabling downstream tasks such as visual reasoning ( 2 ; 12 ) , causal inference ( 49 ; 63 ) , world modeling ( 30 ) , robotic control ( 20 ) , and compositional generation ( 51 ) . Yet, learning such compositional representations directly from images remains a core challenge. Unlike text, where words naturally form composable units, images lack explicit boundaries for objects and concepts. For example, in a street scene with pedestrians, cars, and traffic lights, a model must disentangle these entities without labels and also capture their spatial relations (e.g., a person crossing in front of a car). Multi-object scenes add further complexity: models must not only detect individual objects but also capture their interactions. As datasets grow more cluttered and textured, this becomes even harder. Manual annotation of object boundaries or compositional structures is costly, motivating the need for fully unsupervised approaches such as Slot Attention (SA) ( 39 ) . While effective in simple synthetic settings, SA struggles with large variations in real-world images, limiting its applicability to visual tasks such as image or video editing. Combining SA with diffusion models has recently pushed forward progress in OCL ( 26 ; 59 ; 1 ) . In particular, Stable-LSD ( 26 ) and SlotAdapt ( 1 ) achieve strong object discovery and high-quality generation by leveraging pretrained diffusion backbones such as Stable Diffusion ( 48 ) (SD). Nevertheless, these approaches still face two key challenges. First, as illustrated in Fig. 1 (left), they often suffer from slot entanglement , where a slot encodes features from multiple objects or fragments of them, leading to unfaithful single-slot generations. This entanglement degrades segmentation quality and prevents composable generation to novel scenes and object configurations. Second, they exhibit weak alignment , where slots fail to consistently correspond to distinct image regions, especially on real-world images. As shown in our experiments, slots often suffer from over-segmentation (splitting one object into multiple slots), under-segmentation (merging multiple objects into one slot), or inaccurate object boundaries. Together, these two issues reduce both the accuracy of object-centric representations and their utility for compositional scene generation. In response, we propose C ontrastive O bject-centric D iffusion A lignment ( CODA ), a slot-attention model that uses a pretrained diffusion decoder to reconstruct the input image. CODA augments the model with register slots, which absorb residual attention and reduce interference between object slots, and a contrastive objective, which explicitly encourages slot–image alignment. As illustrated in Fig. 1 (right), CODA faithfully generates images from both individual slots as well as their compositions. In summary, the contributions of this paper can be outlined as follows. (i) Register-augmented slot diffusion. We employ register slots that are independent of the input image into slot diffusion. Although these register slots carry no semantic information, they act as attention sinks, absorbing residual attention mass so that semantic slots remain focused on meaningful object–concept associations. This reduces interference between object slots and mitigating slot entanglement ( Section 4.1 ). (ii) Mitigating text-conditioning bias. To reduce the influence of text-conditioning biases inherited from pretrained diffusion models, we finetune the key, value, and output projections in cross-attention layers. This adaptation further improves alignment between slots and visual content, ensuring more faithful object-centric decomposition ( Section 4.2 ). (iii) Contrastive alignment objective. We propose a contrastive loss that ensures slots capture concepts present in the image ( Section 4.3 ). Together with the denoising loss, our training objective can be viewed as a tractable surrogate for maximizing the mutual information (MI) between inputs and slots, improving slot representation quality ( Section 4.4 ). (iv) Comprehensive evaluation. We demonstrate that CODA outperforms existing unsupervised diffusion-based approaches across synthetic and real-world benchmarks in object discovery ( Section 5.1 ), property prediction ( Section 5.2 ), and compositional generation ( Section 5.3 ). On the VOC dataset, CODA improves instance-level object discovery by +3.88% mBO i and +3.97% mIoU i , and semantic-level object discovery by +5.72% mBO c and +7.00% mIoU c . On the COCO dataset, it improves the foreground Adjusted Rand Index (FG-ARI) by +6.14%. Figure 1: Image generation from individual slots. Top: slot masks. Bottom: generated images. Both methods can reconstruct the full scene when conditioned on all slots (last column). However, Stable-LSD (without register slots) fails to generate images from individual slots. Our method yields faithful single-concept generations, demonstrating disentangled and well-aligned slots. 2 Related work Object-centric learning (OCL). The goal of OCL is to discover compositional object representations from images, enabling systematic generalization and stronger visual reasoning ( 12 ; 2 ) . Learning directly from raw pixels is difficult, so previous works leveraged weak supervision (e.g., optical flow ( 33 ) , depth ( 13 ) , text ( 61 ) , pretrained features ( 50 ) ), or auxiliary losses that guide slot masks toward moving objects ( 3 ; 4 ; 62 ) . Scaling OCL to complex datasets has been another focus: DINOSAUR ( 50 ) reconstructed self-supervised features to segment real-world images, and FT-DINOSAUR ( 9 ) extended this via encoder finetuning for strong zero-shot transfer. SLATE ( 51 ) and STEVE ( 52 ) combined discrete VAE tokenization with slot-conditioned autoregressive transformers, while SPOT ( 28 ) improved autoregressive decoders using patch permutation and attention-based self-training. Our work builds on SA, but does not require any additional supervision. Diffusion models for OCL. Recent works explored diffusion models ( 54 ; 48 ) as slot decoders in OCL. Different methods vary in how diffusion models are integrated. For example, SlotDiffusion ( 59 ) trained a diffusion model from scratch, while Stable-LSD ( 26 ) , GLASS ( 53 ) , and SlotAdapt ( 1 ) leveraged pretrained diffusion models. Although pretrained models offer strong generative capabilities, they are often biased toward text-conditioning. To address this issue, GLASS ( 53 ) employed cross-attention masks as pseudo-ground truth to guide SA training. Unlike GLASS, CODA does not rely on supervised signals such as generated captions. SlotAdapt ( 1 ) introduced adapter layers to enable