Graph Cut-guided Maximal Coding Rate Reduction for Learning Image Embedding and Clustering

Wei He, Zhiyuan Huang, Xianghan Meng, Xianbiao Qi, Rong Xiao, Chun-Guang Li ∗, · 2024

Read Graph Cut-guided Maximal Coding Rate Reduction for Learning Image Embedding and Clustering by Wei He, Zhiyuan Huang, Xianghan Meng, Xianbiao Qi, Rong Xiao, Chun-Guang Li ∗, (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18930.txt.

Introduction

Wei He and co-authors gave a unified method for image embedding and clustering in two thousand twenty-four. Clustering pipelines usually extract features with a pre-trained model first, then cluster them as a separate step. Splitting the stages can yield weak clusters.

Their framework joins both stages under a coding rate reduction principle, which favors compact, structured features. A graph cut module supplies partition information that guides compression cluster by cluster. Better partitions then improve the embeddings in turn, in a feedback loop. Tests on standard and out-of-domain image datasets validated the approach. The authors argue joint learning beats treating feature extraction and clustering as separate problems.

Catalog record

CREATORWei He, Zhiyuan Huang, Xianghan Meng, Xianbiao Qi, Rong Xiao, Chun-Guang Li ∗,
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-18930

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Graph Cut-guided Maximal Coding Rate Reduction for Learning Image Embedding and Clustering Wei He, Zhiyuan Huang, Xianghan Meng, Xianbiao Qi, Rong Xiao, Chun-Guang Li ∗, Abstract In the era of pre-trained models, image clustering task is usually addressed by two relevant stages: a) to produce features from pre-trained vision models; and b) to find clusters from the pre-trained features. However, these two stages are often considered separately or learned by different paradigms, leading to suboptimal clustering performance. In this paper, we propose a unified framework, termed graph Cut-guided Maximal Coding Rate Reduction (CgMCR 2 ), for jointly learning the structured embeddings and the clustering. To be specific, we attempt to integrate an efficient clustering module into the principled framework for learning structured representation, in which the clustering module is used to provide partition information to guide the cluster-wise compression and the learned embeddings is aligned to desired geometric structures in turn to help for yielding more accurate partitions. We conduct extensive experiments on both standard and out-of-domain image datasets and experimental results validate the effectiveness of our approach. 1 Introduction Image clustering, as a fundamental problem in computer vision and pattern recognition, aims to group images without annotated labels [ 18 ] . Image clustering is usually addressed by two successive stages: a) learning representation to generate features from images at first, and then b) finding clusters from the learned representation. State-of-the-art methods for image clustering, e.g., subspace clustering algorithms [ 47 , 7 , 29 ] , demonstrate remarkable performance on simple datasets such as MNIST [ 25 ] and COIL [ 33 ] when proper features (e.g., Scatter Transform [ 3 ] ) are provided. In general, different clustering algorithms implicitly employ different assumptions about the geometry of the clusters. For instance, in k -means algorithm [ 32 ] each cluster is modeled as a standard Gaussian distribution and thus characterized by the mean vector; in subspace clustering [ 45 ] , each cluster is modeled as a low-dimensional subspace; in manifold clustering [ 42 ] each cluster is modeled as a (low-dimensional) submanifold. Clustering performance will dramatically degenerate when the data distribution deviates from the assumption on the clusters. When dealing with challenging and complex datasets, such as CIFAR [ 23 ] and ImageNet [ 9 ] , the learned features play a more crucial role, as clustering based on conventional feature extraction typically fails to achieve satisfactory results. The key ingredient among the recent remarkable progress in clustering is learning features by pre-trained models (e.g., auto-encoders [ 40 , 22 ] and contrastive learning [ 5 ] ) that are suitable for the downstream clustering task. In the era of pre-trained models, deep clustering methods are designed by learning features via visual pretraining and then learning the cluster membership from refined features [ 44 , 26 ] . More recently, when the large-scale pre-trained models, e.g., DINO [ 4 , 37 ] and CLIP [ 39 ] , are prevailing, deep clustering methods achieve impressive clustering performance by leveraging the rich representations produced by large-scale pre-trained models [ 1 , 27 ] . While promising clustering accuracy has been reported, these deep clustering methods are usually designed to learn the pseudo labels without leveraging the potential effect from the structures of embeddings. There are a few attempts [ 28 , 11 , 8 ] to learn the embeddings with desired structures—a union of orthogonal subspaces , by leveraging the framework of Maximal Coding Rate Reduction (MCR 2 ) [ 48 ] . Nonetheless, none of these methods have developed a principled joint optimization framework to learn both the structured embeddings and the clustering. In this paper, we present an effective joint optimizing framework, termed graph C ut- g uided M aximal C oding R ate R eduction ( CgMCR 2 ), to learn both the structured embeddings and the clusters in principled way. Specifically, in CgMCR 2 , we integrate both the normalized cut based clustering learning and the MCR 2 based structured representations learning to form a unified optimization problem. Moreover, we design a two-stage training procedure, which consists of a one-shot initialization and a fine-tuning, to effectively train the proposed joint learning framework. We conduct extensive experiments on benchmark datasets to validate the superior performance of our proposed approach and also provide a set of ablation studies to evaluate the effect of each component. Our code is available at: https://github.com/hewei98/CgMCR2 . 2 Relate Work Pre-trained Vision models. Typically, pre-trained models leverage self-supervised pretext tasks to learn representations from unlabeled datasets. For example, Autoencoders [ 22 ] use an encoder-decoder architecture to learn latent low-dimensional representations by requiring the decoder to reconstruct the encoder inputs; contrastive learning (i.e., SimCLR [ 5 ] ) exploits the self-supervision information via data augmentation and learns representations that maximize the agreement between positive pairs and the disagreement between negative pairs; and the following studies are proposed to improve the contrastive learning by enhancing the training stability (e.g., MoCo [ 15 , 6 ] ), reducing the requirement for negative samples (e.g., BYOL [ 13 ] ), or avoiding the collapse solution of learned features (e.g., VICReg [ 2 ] ). More recently, large-scale pre-trained models based on large models such as BERT [ 10 ] and Vision Transformers (ViTs) [ 12 ] have showcased the capability to learn rich representations from diverse data sources. For instance, MAE [ 14 ] leverages the ViT as the backbone of auto-encoder and uses the input images with a high mask proportion for training. Inspired by self-supervised pretraining tasks in natural language processing, DINO [ 4 , 37 ] implements a self-distillation framework with ViTs without using annotated labels. CLIP [ 39 ] pretrains a vision-language model on image-text data pairs to learn visual concepts from a text-guided contrastive learning task. Although the pre-trained models mentioned above successfully learn meaningful features, the structure of the clusters in the learned features remains unclear. Clustering via pre-trained models. The success of pre-trained models has led to breakthroughs in image clustering. SCAN [ 44 ] suggests a three-stage clustering pipeline: learning embeddings from a pre-trained model, using a clustering module for label prediction, and fine-tuning the clustering module using pseudo-labels; RUC [ 38 ] and SPICE [ 35 ] enrich the pipeline of SCAN by utilizing robust training, refining network architecture or adjusting the fine-tuning strategy; CC [ 26 ] and GCC [ 49 ] propose unified frameworks for feature learning and clustering by optimizing the instance- and cluster-wise contrastive loss and graph contrastive loss, respectively. IMC-SwAW [ 36 ] integrates a discrete representation into the self-supervised learning via a classifier net to simultaneously learn the cluster membership. MiCE [ 43 ] introduces a probabilistic clustering framework that combines contrastive learning with a latent mixture of experts. ProPos [ 17 ] combines prototype scattering and positive sampling using EM-like steps to learn uniform, well-separated representations. More recently, TEMI [ 1 ] proposes a self-distillation clustering framework by leveraging large-scale pre-trained models; and TAC [ 27 ] brings in the pre-trained CLIP texture embedding as external guidance for image clustering. Unfortunately, the intrinsic structure of representations learned or refined by the deep clustering methods mentioned above is

Read the complete text