Multi-Part Object Representations via Graph Structures and Co-Part Discovery
Alex Foo Wynne Hsu Mong Li Lee · 2025
Read Multi-Part Object Representations via Graph Structures and Co-Part Discovery by Alex Foo Wynne Hsu Mong Li Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18192.txt.
Introduction
Alex Foo, Wynne Hsu, and Mong Li Lee at the National University of Singapore tackle object centric learning, where a vision model discovers objects and parts from images on its own. Most methods encode part whole relations only implicitly, through indirect training goals. The learned objects then fall apart when parts are occluded or scenes shift out of distribution. Their 2025 method represents parts explicitly as a graph.
A co-part discovery algorithm finds the parts and how they group into whole objects. The team adds three benchmarks testing recognition of multi part objects under occlusion and distribution shift. Across simulated, realistic, and real world images, it beats leading alternatives on discovered object quality. It recognizes multi part objects accurately in the hard occluded and shifted settings. The learned representations also predict key object properties better downstream, supporting explicit structure as the way forward for robust vision.
Catalog record
| CREATOR | Alex Foo Wynne Hsu Mong Li Lee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-18192
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-Part Object Representations via Graph Structures and Co-Part Discovery Alex Foo Wynne Hsu Mong Li Lee Abstract Discovering object-centric representations from images can significantly enhance the robustness, sample efficiency and generalizability of vision models. Works on images with multi-part objects typically follow an implicit object representation approach, which fail to recognize these learned objects in occluded or out-of-distribution contexts. This is due to the assumption that object part-whole relations are implicitly encoded into the representations through indirect training objectives. We address this limitation by proposing a novel method that leverages on explicit graph representations for parts and present a co-part object discovery algorithm. We then introduce three benchmarks to evaluate the robustness of object-centric methods in recognizing multi-part objects within occluded and out-of-distribution settings. Experimental results on simulated, realistic, and real-world images show marked improvements in the quality of discovered objects compared to state-of-the-art methods, as well as the accurate recognition of multi-part objects in occluded and out-of-distribution contexts. We also show that the discovered object-centric representations can more accurately predict key object properties in a downstream task, highlighting the potential of our method to advance the field of object-centric representations. 1 Introduction Human interaction with the environment relies on the comprehension of diverse visual scenes, powered by a robust visual system which decomposes each scene into abstract and manipulable objects 24 ; 40 . Each object’s features can be explicitly described, and different objects are recognized via the appearance and relative pose of their constituent parts 19 . Emulating this part-whole compositional system in the form of multi-part object representations can improve robustness, occlusion-aware perception, generalization to out-of-distribution images, and interpretability of machine learning algorithms for vision 17 . Figure 1 : Comparison of existing methods. (a) Information-bottleneck approaches face challenges with multi-part object recognition, while (b) methods that rely on learned features struggle to compose part clusters into object wholes. Our proposed Explicit Compositional Network (ECO-Net) consistently and accurately composes parts into object wholes. Recent work utilizes an implicit object representation approach in which scenes are decomposed either via: (a) information-bottleneck deep learning, where high-dimensional inputs are encoded into a fixed number of low-dimensional object representations then recombined for reconstruction 12 ; 10 ; 38 ; 39 ; 21 , or (b) similarity clustering of pre-trained image features 37 ; 35 ; 42 ; 41 ; 15 . Information-bottleneck approaches face challenges with multi-part object recognition, while methods that rely on learned features struggle to compose part clusters into object wholes (see Figure 1 ). The fundamental limitation of these methods stem from their assumption that object part-whole relations are implicitly encoded into the object representations through indirect training objectives such as input reconstruction 46 or self-supervised contrastive learning 6 . While it is challenging to develop structures that consistently compose parts into their respective objects, this lack of part-whole structure in the unsupervised learning of objects leads to failure in recognizing objects in occluded and out-of-distribution contexts despite many training samples and iterations 17 ; 8 ; 22 . These limitations impede deployment of these methods to real-world applications where robust recognition of multi-part objects is often essential 45 . To address the limitations of implicit object representation, we propose Explicit Compositional Network (ECO-Net), a framework that leverages on explicit graph representations for parts and learns to discover multi-part objects via a co-part object discovery algorithm. Here, objects are defined as graphs, where the nodes define the features of each constituent part which make up the object, and the edges define the spatial relationships between the parts. Based on the intuition neighboring parts that recurringly appear together across samples should belong to the same object 3 ; 28 , we design a co-part object discovery algorithm which clusters parts together to form object wholes based on the repeated occurrence of parts and neighbors with similar positional relations. Discovered objects are processed and integrated into an object memory module which is used for downstream tasks. Figure 1 shows that our proposed Explicit Compositional Network (ECO-Net) effectively composes parts into object wholes. Extensive experiments show that our framework outperforms state-of-the-art methods in discovering multi-part objects from simulated, realistic and real-world images in a fine-grained manner without supervision. We also introduce three benchmarks to evaluate the recognition of multi-part objects in occluded and out-of-distribution contexts. Our solution excels in accurately filling in the missing parts of objects in the occlusion context, while also accurately recognizing learned objects in the out-of-distribution context. Furthermore, our method predicts key object properties in a downstream task with greater precision. Our contributions include: (1) a framework that utilizes explicit graph-based multi-part object representations, (2) a co-part object discovery algorithm as an alternative to indirect unsupervised learning objectives, (3) introduction of AbsScene-O, GSO-O and AbsScene-C benchmarks for evaluating robustness of object-centric methods on multi-part objects in occluded and out-of-distribution contexts, (4) experimental validation of our solution’s superior performance on object discovery, occlusion-aware perception and out-of-distribution generalization, and (5) demonstration of the usefulness of learned object representations for predicting key object properties in a downstream task. 2 Related Work Numerous works have shown remarkable success in segmenting real-world images, using supervised signals for image segmentation and object detection 18 ; 5 ; 27 . Unsupervised methods like SLIC 1 employ a modified k-means algorithm to cluster pixels, while Normalized Cut (Ncut) 36 and Felzenszwalb’s method 14 use graph-based modeling on hand-crafted features for segmentation. Others use co-segmentation to jointly segment objects across images, but rely on intra-class objects being common to all images in the ground-truth 30 ; 7 . All these methods do not focus on obtaining useful object representations for the segmented components. Unsupervised methods to obtain object-centric representations can be categorized as follows: \bullet Information-bottleneck deep learning. This approach uses two forms of attention to decompose scenes 46 . Sequential attention methods uses RNN inspired models to attend to different image regions. MONet 4 and GENESIS 11 employ a deterministic network to perform the attention and sequentially discover and represent objects in the scene. These methods may neglect smaller objects as they tend to produce a weaker signal during the attention process, leading to incomplete or biased representations of scenes with objects of varying sizes. To overcome this, GENESIS-V2 12 uses stochastic stick-breaking to perform attention randomly. Iterative attention methods randomly initialize a set of object representations and refine to bind these objects to different image regions. Slot Attention 32 utilizes cross-attention along the object dimension. While it is fast and can be extended to handle videos 26 , it may fail to discover objects when the training set is diverse. BO-QSA 21 initializes the slots of Slot Attention as learnable embeddings and uses bi-level optimization, resulting in more stable training. However, the number of clusters are fixed a priori which limits the applicability in real-world scenarios where the number of objects is not known beforehand. \bullet Clustering of learned features. This approach clusters similar regions into objects based on learned features. DINOSAUR 35 utilizes self-supervised pre-training to obtain learned features before using iterative attention for feature reconstruction. MaskCut 41 extracts image features using DINO and applies Ncut for segmentation. OC-Net 15 leverages learned feature connectivity to discover objects and designs two object-centric regularization